{"id":"KYzuLKhafwM","title":"DiPaCo paper: Decentralized Model Training","description":"The DiPaCo paper from DeepMind, highlighting its challenges and potential in decentralized model training. This video sheds light on the future of machine training and how it may reshape the landscape of AI research. #DiPaCo #DeepMind #DecentralizedAI #MachineLearning #AIResearch #ModelTraining #ArtificialIntelligence #TechInnovation #ComputationalModels #FutureOfAI #NEARai #NEARprotocol","channel_name":"NEAR Protocol","published_at":"2024-12-09T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/KYzuLKhafwM/hqdefault.jpg","transcript_text":"yeah so so there's a follow-up paper from Deep Mind called deaco which tries to solve the problem with models not fitting on a single machine and that one is way more involved it's not just you know a small change in the optimization step and the experiment section is uh um if you try if you like try to extract Apples to Apples comparisons you will notice that daku is not doing as great which is probably sort of something you would expect when you um because it's only using a subset of parameters on pass so but it is it is a great forward first step right and like from here again in in a form of a competition we can push it forward and hopefully figure out a way to train a massive model in a in a decentralized manner","transcript_synced_at":"2026-11-03T05:12:21.437Z","segments":[{"start_seconds":0,"text":"yeah so so there's a follow-up paper from Deep Mind called deaco which tries to solve the problem with models not fitting on a single machine","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=0s"},{"start_seconds":8,"text":"and that one is way more involved it's not just you know a small change in the optimization step and the experiment","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=8s"},{"start_seconds":15,"text":"section is uh um if you try if you like try to extract Apples to Apples comparisons you will notice that daku is","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=15s"},{"start_seconds":22,"text":"not doing as great which is probably sort of something you would expect when you um because it's only using a subset","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=22s"},{"start_seconds":29,"text":"of parameters on pass so but it is it is a great forward first step right and like from here again in in a form of a","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=29s"},{"start_seconds":37,"text":"competition we can push it forward and hopefully figure out a way to train a massive model in a in a decentralized","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=37s"},{"start_seconds":44,"text":"manner","url":"https://www.youtube.com/watch?v=KYzuLKhafwM&t=44s"}],"youtube_url":"https://youtube.com/watch?v=KYzuLKhafwM","url":"https://near-pedia.xyz/videos/KYzuLKhafwM"}