{"id":"tFm2FsxdDYk","title":"Scaling Intelligence and Generalization - Lukasz Kaiser","description":"Lukasz Kaiser from day 1 of NEARCON on why generalization, not raw scale, determines whether modern AI systems become usable, reliable intelligence in the real world.\n\nWebsite: https://near.org/\nNEAR AI: https://near.ai/\nIntents: https://near.org/intents\nChain Abstraction: https://www.proximity.dev/\nSharding: https://nearone.org/\n\nFor the devs\nGitHub: https://github.com/near \nDev Docs: https://docs.near.org/\nDevhub: https://devhub.near.page/ @NEARDevHub\n\nConnect with socials\nTwitter:   / nearprotocol  \nBlog: https://near.org/blog/\nMedium:   / nearprotocol  \n#NEAR #nearprotocol #NEARAI","channel_name":"NEAR Protocol","published_at":"2026-02-26T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/tFm2FsxdDYk/hqdefault.jpg","transcript_text":"I assume you all came here because AI is quite exciting these days. We have quite beautiful things and I want to rain on the parade a little bit. So, we have agents. We also have self-driving cars. You may have driven one around here and they're quite amazing. I use them every day. There is one little problem. Have you tried to go to Berkeley in one of those? It's not going to take you there. It will take you to to South Bay now, but it can't cross the bridge. So, a car that has driven millions of miles, that drives thousands of people every day, can't go to Berkeley. We need to think, why is that? Why does our amazing AI not have the ability to cross the Bay Bridge? Well, it doesn't because in contrast to a human driver who can spend 10 hours and drive you anywhere in California, it does not generalize the right way. So, the answer to all of these questions, why can't Whimo drive me to Berkeley? Why can't you talk to LLM in a low resource language? Why can't LLM really cure diseases? Also, why is your agent so annoying? Why do you need a harness of 20,000 prompts to run anything? The answer to all of this is one and the same. Namely, that our AI has not yet truly figured out how to generalize from little data. So I want to take you on a little tour [snorts] to show you that in my opinion this is the one problem that has been driving machine learning for all this time and that we have actually made great strides in this problem which also means we may know how to make even more strides. So my journey with this starts a long time ago about 12 years or 13 um where my friend the other Ilia h had a very powerful talk about RNN's. I don't know if anyone still remembers what RNNs were. These were like the first models that could do something like translate from English to German. And back then there was this picture like there are powerful models that can compute a lot like RNN and then transformers and agents and so on and there are models that we can teach and back then people only knew convenets and they were not quite powerful enough. So the RNNs were like the first powerful models that you could actually train to do something. So that was wonderful and back then people thought, okay, this is the end. We have a powerful model that's learnable. It's going to solve everything. Well, it didn't quite work that way because it turns out you need to push these circles. You need to make it more powerful. So I will not go right now into the details of what RNN's were, but the point is that they could only hold a constant number of bits in their state. So I imagine them as this snail that goes over token by token. They it goes through these tokens and it carries its state and it updates this state. It eats a token, updates its state, throws out some token that it produces and that goes on but it never grows, right? It can only do so much. So it's computationally powerful in some sense. It can do n steps on n tokens. Back then it was a lot. But it has a fixed state size and cannot do much about that. So that was cool. It it's it's very trainable though, right? You can teach it on arbitrary data. Well, the thing that came after and these models were more powerful and still very nicely learnable were transformers. Um, so we had the pleasure of working with Ilia on on this little architecture that that became quite well known. Um transformers are better in the sense they can look at everything that happened in the past. So their state grows and grows and grows and grows. Now nowadays we have a little bit of a problem with that because you know we run out of tokens and need to do compaction and so on. So, so it's a blessing and a curse, but it's mostly a blessing because uh this unlimited space allows this thing to look at everything it has done before and it becomes much more computationally powerful while still being learnable in the sense you can train it on arbitrary data. It can learn from anything you give it. And so, has this improved power to learn given it anything? Well, yes, it's given it literally everything, right? We had chat GPT. It's in self-driving cars. Actually, it can generate you images. Once you have a model that is more computationally powerful that you can actually train, amazing things happen. So, this was the first chat, right? The first generation of amazing things. Um, and many people thought, and this is also true, that this is all because you have more layers and more data. Now more layers and more data is amazing. Don't get me wrong. It it's you scale up and up and up, but it doesn't generalize better. It only makes you learn things that are already known somewhere on the internet. Now, once you start attacking problems that are not known, like you need to drive to a place you didn't have in your training data, you need to run your agent on code that it has not seen and you wanted to not do stupid things. This is not enough. It still helps if you do this on the internet scale because chances are you've seen something similar. So you absolutely should still scale but it's not enough if we you want your agent to really run things that they have not seen before. So what is the solution to make an even more powerful model? Well, reasoning reasoning models are models that have a chain of thought before they give the answer. I I assume everyone here has seen that. So why are reasoning models um much better at generalizing? Imagine the this simple question. When does the zoo in San Francisco open? So chat GPT 3.5 would answer you this question. It would probably say 10 10:00 a.m. Why would it answer it? Because it has been trained on every single website in the world 10 times. So, it has memorized that the zoo website usually says opening 10:00 a.m. But then if you, you know, if the zoo changes its time, of course, you need to retrain it. If you're asking about zoo in Beijing, it needs to memorize that zoo's website again. Now, that's not a generalizable strategy. That's not how we want these models to work because if it encounters a new zoo, it has no chance, right? So reasoning models are much better because they have this chain of thought and they're trained with reinforcement learning to learn a strategy to give the correct answer. What is the strategy? The strategy is go to a search engine, find the Zeus website, bring it in context, read it and ex extract when it opens. Now that's a much better strategy. That's exactly how we do it. That's how reasoning models do it. And that's why you can train them on only like a thousand on 10,000 or a million tasks instead of trillions and trillions of tokens and they start solving problems like telling you when the zoo opens or running your code or solving math problems and so on. So these models have a much higher computational power. It's basically unlimited. It's as long as the chain of thought and they can use external tools. So in in in the sense of computing power, they're great. They have a learning problem though because they can only learn from tasks where you know this is correct or this is wrong. At least currently this is how most of these models are trained and that's not such a nice way as the usual learning where you just throw arbitrary data and say learn. So these models have brought upon us the new era of agents and problem solving models, math code, but their training is not yet great and they have another problem. Okay, so math I mean by now this is even old, right? We we can solve all math competitions but we need one more step. That's what I want to tell you. We need models that do reasoning but that are much more parallel because current reasoning just goes token by token by token so slow. We need models that reason in parallel and that learn reasoning from arbitrary data. And that's still something to do but I believe this is the last big obstacle and something that we can actually find out in research. And when we finalize this, these models will do like real world research not just in mathematics but in all domains. They will be amazing agents. They will hopefully allow our AI to really step out and do truly new things uh in amazing capacity. And I believe this is research that is already happening in many labs. There is a whiff in the air. there's like 10 new labs every month because I think people feel that that there is another paradigm change in the air on top of the amazing capabilities that we already have and I think that will bring truly amazing things to [snorts] both our agents and AI","transcript_synced_at":"2026-11-03T04:40:33.294Z","segments":[{"start_seconds":9,"text":"I assume you all came here because AI is quite exciting these days. We have quite","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=9s"},{"start_seconds":16,"text":"beautiful things and I want to rain on the parade a little bit. So,","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=16s"},{"start_seconds":24,"text":"we have agents. We also have self-driving cars. You may have driven one around here and they're quite","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=24s"},{"start_seconds":31,"text":"amazing. I use them every day. There is one little problem. Have you tried to go to Berkeley in one of those? It's not","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=31s"},{"start_seconds":39,"text":"going to take you there. It will take you to to South Bay now, but it can't cross the bridge. So, a car that has","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=39s"},{"start_seconds":46,"text":"driven millions of miles, that drives thousands of people every day, can't go to Berkeley.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=46s"},{"start_seconds":52,"text":"We need to think, why is that? Why does our amazing AI not have the ability to cross the Bay","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=52s"},{"start_seconds":60,"text":"Bridge? Well, it doesn't because in contrast to a human driver who can spend 10 hours and drive you anywhere in","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=60s"},{"start_seconds":68,"text":"California, it does not generalize the right way.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=68s"},{"start_seconds":73,"text":"So, the answer to all of these questions, why can't Whimo drive me to Berkeley? Why can't you talk to LLM in a","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=73s"},{"start_seconds":80,"text":"low resource language? Why can't LLM really cure diseases? Also, why is your agent so annoying? Why do you need a","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=80s"},{"start_seconds":88,"text":"harness of 20,000 prompts to run anything? The answer to all of this is one and the same. Namely, that our AI","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=88s"},{"start_seconds":97,"text":"has not yet truly figured out how to generalize from little data.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=97s"},{"start_seconds":103,"text":"So I want to take you on a little tour [snorts] to show you that in my opinion this is the one problem that has been","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=103s"},{"start_seconds":111,"text":"driving machine learning for all this time and that we have actually made great strides in this problem which also","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=111s"},{"start_seconds":118,"text":"means we may know how to make even more strides. So my journey with this starts","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=118s"},{"start_seconds":124,"text":"a long time ago about 12 years or 13 um where my friend the other Ilia h had a","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=124s"},{"start_seconds":133,"text":"very powerful talk about RNN's. I don't know if anyone still remembers what RNNs were. These were like the first models","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=133s"},{"start_seconds":142,"text":"that could do something like translate from English to German. And back then there was this picture like there are","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=142s"},{"start_seconds":149,"text":"powerful models that can compute a lot like RNN and then transformers and agents and so on and there are models","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=149s"},{"start_seconds":157,"text":"that we can teach and back then people only knew convenets and they were not quite powerful enough. So the RNNs were","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=157s"},{"start_seconds":165,"text":"like the first powerful models that you could actually train to do something. So that was wonderful and back then people","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=165s"},{"start_seconds":174,"text":"thought, okay, this is the end. We have a powerful model that's learnable. It's going to solve everything. Well, it didn't quite work that way because it","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=174s"},{"start_seconds":181,"text":"turns out you need to push these circles. You need to make it more powerful. So I will not go right now into the details of what RNN's were, but","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=181s"},{"start_seconds":191,"text":"the point is that they could only hold a constant number of bits in their state.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=191s"},{"start_seconds":198,"text":"So I imagine them as this snail that goes over token by token. They it goes through these tokens and it carries its","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=198s"},{"start_seconds":206,"text":"state and it updates this state. It eats a token, updates its state, throws out some token that it produces and that","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=206s"},{"start_seconds":214,"text":"goes on but it never grows, right? It can only do so much. So it's computationally powerful in some sense.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=214s"},{"start_seconds":222,"text":"It can do n steps on n tokens. Back then it was a lot. But it has a fixed state","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=222s"},{"start_seconds":228,"text":"size and cannot do much about that. So that was cool. It it's it's very","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=228s"},{"start_seconds":235,"text":"trainable though, right? You can teach it on arbitrary data. Well, the thing that came after and these models were","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=235s"},{"start_seconds":242,"text":"more powerful and still very nicely learnable were transformers.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=242s"},{"start_seconds":247,"text":"Um, so we had the pleasure of working with Ilia on on this little architecture that that became quite well known. Um","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=247s"},{"start_seconds":256,"text":"transformers are better in the sense they can look at everything that happened in the past. So their state","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=256s"},{"start_seconds":262,"text":"grows and grows and grows and grows. Now nowadays we have a little bit of a problem with that because you know we","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=262s"},{"start_seconds":271,"text":"run out of tokens and need to do compaction and so on. So, so it's a blessing and a curse, but it's mostly a blessing because","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=271s"},{"start_seconds":280,"text":"uh this unlimited space allows this thing to look at everything it has done before and it becomes much more","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=280s"},{"start_seconds":289,"text":"computationally powerful while still being learnable in the sense you can train it on arbitrary data. It can learn","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=289s"},{"start_seconds":296,"text":"from anything you give it. And so, has this improved power to learn given it","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=296s"},{"start_seconds":303,"text":"anything? Well, yes, it's given it literally everything, right? We had chat GPT. It's in self-driving cars.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=303s"},{"start_seconds":311,"text":"Actually, it can generate you images.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=311s"},{"start_seconds":313,"text":"Once you have a model that is more computationally powerful that you can actually train, amazing things happen.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=313s"},{"start_seconds":321,"text":"So, this was the first chat, right? The first generation of amazing things. Um, and many people thought, and this is","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=321s"},{"start_seconds":328,"text":"also true, that this is all because you have more layers and more data. Now more layers and more data is amazing. Don't","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=328s"},{"start_seconds":335,"text":"get me wrong. It it's you scale up and up and up, but it doesn't generalize better. It only makes you learn things","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=335s"},{"start_seconds":344,"text":"that are already known somewhere on the internet. Now, once you start attacking problems that are not known, like you","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=344s"},{"start_seconds":351,"text":"need to drive to a place you didn't have in your training data, you need to run your agent on code that it has not seen and you wanted to not do stupid things.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=351s"},{"start_seconds":359,"text":"This is not enough. It still helps if you do this on the internet scale because chances are you've seen something similar. So you absolutely","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=359s"},{"start_seconds":367,"text":"should still scale but it's not enough if we you want your agent to really run things that they have not seen before.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=367s"},{"start_seconds":377,"text":"So what is the solution to make an even more powerful model? Well, reasoning","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=377s"},{"start_seconds":384,"text":"reasoning models are models that have a chain of thought before they give the answer. I I assume everyone here has","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=384s"},{"start_seconds":391,"text":"seen that. So why are reasoning models um much better at generalizing? Imagine","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=391s"},{"start_seconds":399,"text":"the this simple question. When does the zoo in San Francisco open?","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=399s"},{"start_seconds":404,"text":"So chat GPT 3.5 would answer you this question. It would probably say 10 10:00 a.m. Why would it answer it? Because it","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=404s"},{"start_seconds":412,"text":"has been trained on every single website in the world 10 times. So, it has memorized that the zoo website usually","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=412s"},{"start_seconds":419,"text":"says opening 10:00 a.m. But then if you, you know, if the zoo changes its time, of course, you need to retrain it. If","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=419s"},{"start_seconds":426,"text":"you're asking about zoo in Beijing, it needs to memorize that zoo's website again. Now, that's not a generalizable","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=426s"},{"start_seconds":434,"text":"strategy. That's not how we want these models to work because if it encounters a new zoo, it has no chance, right? So","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=434s"},{"start_seconds":442,"text":"reasoning models are much better because they have this chain of thought and they're trained with reinforcement learning to learn a strategy to give the correct answer. What is the strategy?","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=442s"},{"start_seconds":452,"text":"The strategy is go to a search engine, find the Zeus website, bring it in context, read it and ex extract when it","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=452s"},{"start_seconds":459,"text":"opens. Now that's a much better strategy. That's exactly how we do it.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=459s"},{"start_seconds":463,"text":"That's how reasoning models do it. And that's why you can train them on only like a thousand on 10,000 or a million tasks instead of trillions and trillions","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=463s"},{"start_seconds":472,"text":"of tokens and they start solving problems like telling you when the zoo opens or running your code or solving","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=472s"},{"start_seconds":480,"text":"math problems and so on. So these models have a much higher computational power.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=480s"},{"start_seconds":486,"text":"It's basically unlimited. It's as long as the chain of thought and they can use external tools. So in in in the sense of","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=486s"},{"start_seconds":493,"text":"computing power, they're great. They have a learning problem though because they can only learn from tasks where you know this is correct or this is wrong.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=493s"},{"start_seconds":502,"text":"At least currently this is how most of these models are trained and that's not","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=502s"},{"start_seconds":509,"text":"such a nice way as the usual learning where you just throw arbitrary data and say learn.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=509s"},{"start_seconds":516,"text":"So these models have brought upon us the new era of agents and problem solving","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=516s"},{"start_seconds":522,"text":"models, math code, but their training is not yet great and they have another problem. Okay, so math I mean by now","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=522s"},{"start_seconds":532,"text":"this is even old, right? We we can solve all math competitions but we need one more step. That's what I","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=532s"},{"start_seconds":539,"text":"want to tell you. We need models that do reasoning but that are much more parallel because current reasoning just","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=539s"},{"start_seconds":546,"text":"goes token by token by token so slow. We need models that reason in parallel and that learn reasoning from arbitrary","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=546s"},{"start_seconds":553,"text":"data. And that's still something to do but I believe this is the last big obstacle and something that we can","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=553s"},{"start_seconds":561,"text":"actually find out in research. And when we finalize this, these models will do","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=561s"},{"start_seconds":568,"text":"like real world research not just in mathematics but in all domains. They will be amazing agents. They will","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=568s"},{"start_seconds":575,"text":"hopefully allow our AI to really step out and do truly new things uh in","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=575s"},{"start_seconds":581,"text":"amazing capacity. And I believe this is research that is already happening in many labs. There is a whiff in the air.","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=581s"},{"start_seconds":589,"text":"there's like 10 new labs every month because I think people feel that that there is another paradigm change in the air on top of the amazing capabilities","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=589s"},{"start_seconds":599,"text":"that we already have and I think that will bring truly amazing things to [snorts] both our agents and","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=599s"},{"start_seconds":607,"text":"AI","url":"https://www.youtube.com/watch?v=tFm2FsxdDYk&t=607s"}],"youtube_url":"https://youtube.com/watch?v=tFm2FsxdDYk","url":"https://near-pedia.xyz/videos/tFm2FsxdDYk"}