Mixture of Experts: How AI REALLY Works
Nothing is loaded from YouTube until you press play, and then without tracking cookies.
About this video
Unlock the secrets of Mixture of Experts models! This video dives into the DeepSeek V3 model, exploring its architecture and implementation. Learn how this powerful technique, used in GPT and similar models, processes information and generates text. Join us to understand the inner workings of this cutting-edge AI technology! #MixtureofExperts #DeepSeekV3 #GPTModels #AI #ArtificialIntelligence #MachineLearning #DeepLearning #TransformerModels #NLP #NaturalLanguageProcessing #NEARai
Transcript
116 wordsCaptions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.
- 0:01
uh okay now the next thing is that the uh the actual model is the mixture of experts uh we've seen mixture of experts before I think uh it's pretty well known that all the GPT gpts a mixture of experts mixol is a mixture of experts um okay so so very quickly what is mixture of experts um here I have an example where you feed the token corgis and it goes through the whole trans Transformer the Transformer
- 0:30
consist of embedding which embeds the token into the space um in which the model can can reason then it goes through multiple layers and it does un embedding which gives you potential tokens right and