{"id":"L_8N5G-43wc","title":"How Expert Parallelism Works in AI Models","description":"Expert parallelism breaks down AI computations into efficient steps—attention, expert assignment, and dispatching—ensuring each machine handles its share of inputs seamlessly. Here's how it all comes together! #AI #MachineLearning #DeepLearning #NeuralNetworks #ArtificialIntelligence #TechExplained #AIModels #ParallelComputing #TechTalk #FutureOfAI #NEARai","channel_name":"NEAR Protocol","published_at":"2025-02-25T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/L_8N5G-43wc/hqdefault.jpg","transcript_text":"uh to understand what they do let's uh let's quickly revisit all the steps with expert parallel right so first thing you do so so you have the input Vector uh sorry input batch uh from the previous layer or from the embedding it's on every machine right all of them do attention let compute uh then after attention they assign experts let compute once experts are assigned they dispatch dispatching is networking now every machine has their share of the every expert has the share of inputs they they compute the the expert the MLP","transcript_synced_at":"2026-11-03T05:06:25.573Z","segments":[{"start_seconds":0,"text":"uh to understand what they do let's uh let's quickly revisit all the steps with expert parallel right so first thing you do so so you have the input Vector uh","url":"https://www.youtube.com/watch?v=L_8N5G-43wc&t=0s"},{"start_seconds":8,"text":"sorry input batch uh from the previous layer or from the embedding it's on every machine right all of them do attention let","url":"https://www.youtube.com/watch?v=L_8N5G-43wc&t=8s"},{"start_seconds":16,"text":"compute uh then after attention they assign experts let compute once experts are assigned they","url":"https://www.youtube.com/watch?v=L_8N5G-43wc&t=16s"},{"start_seconds":24,"text":"dispatch dispatching is networking now every machine has their share of the every expert has the share of inputs","url":"https://www.youtube.com/watch?v=L_8N5G-43wc&t=24s"},{"start_seconds":31,"text":"they they compute the the expert the MLP","url":"https://www.youtube.com/watch?v=L_8N5G-43wc&t=31s"}],"youtube_url":"https://youtube.com/watch?v=L_8N5G-43wc","url":"https://near-pedia.xyz/videos/L_8N5G-43wc"}