Open Weights and the Future of Model Building

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:10

   Thanks everyone for coming. I'm really excited about this fireside. Ashish and me have been working at Google before. For those who are not familiar, Ashish is one of the co-authors of attention is all you need. The last time we were on stage was actually with Jensen and the rest of the co-authors except Nikki at GTC in 2024 and uh I've been trying to pull him in ever since.

2. 0:35

   We have we have to we have to schedule our catchups better. It shouldn't just be on stage all the time. But now that you're in San Francisco, welcome. Uh we can do that more regularly. Exactly. And so I will let Ashish introduce Essential and kind of what he's been working on. But yeah, really excited to talk about open weight models. Why is this important? why it's important for us to have uh this in the world as well as uh how do we really

3. 1:00

   kind of continue advancing the science here. So Ashish, take it away. Um well uh I'm I'm the C co-founder and CEO of Essential and um as mentioned and very relevant to the to the conversations here. We care about you know making open source bringing open source models to the frontier openweight models and our focus is coding and stem and uh our first model release was last year in December we released range one

4. 1:27

   named after uh uh one of my heroes uh and favorite mathematicians Romano and uh Romano is an 8 billion parameter model that it aces uh coding and stem uh in in coding and stem capabilities and it punches above its weight for the weight class. So I think uh to the extent that it's it's important and and I think and I believe it is I I I think it was the best American open weight model in its weight class and uh we've gotten great feedback. Uh lots of uh

5. 1:59

   people have been uh coming back and using it in their applications very recently actually uh and and this is where open weights are actually important. Uh telco has been it's open weight models are very important for telco. So we've been finding some nice uh accuracies there on their tasks. So yeah, great to be here.

6. 2:16

   Yeah. So I think one of I mean agree that like you guys effectively released one of the few I would say open weight models that coming out of effectively not China like right now you know if we if you're using kind of open source open models they mostly coming out of Chinese labs and although you know they've definitely been benefiting community there's challenges like a lot of businesses cannot use them for a variety of reasons there's you know challenges

7. 2:42

   with like not knowing the provenence of data that went in and uh you know we just had an article from Entropic saying that like there's massive scraping happening. So one question is what got you to say hey we're going to like open source this model like what got you to on this path and and you know I I think it's so this is a great question and and and some people are idealistic about open source and open weights. I I don't even think that we need to be

8. 3:13

   idealistic about it. If this this is provably unimpeachably going to be the most important technology that we ever create as a species, we barely understand it today. And and I think we need more and more models out there, controllable um um to be able to for us to be able to interrogate them, probe them, and understand them, understand the extent of their capabilities to fully adopt it in society. Right? So I

9. 3:37

   think that even practically if practically it's important that there be openweight models for us to understand how AI works and how we can improve it for everyone. Um obviously as a scientist and that's how our work started. I think that we all stand on the shoulders of giants. We have stood on the shoulders of giants. That's how science has advanced over you know millennia and I and I think openweight models is just another means of

10. 3:59

    continuing to advance the craft. I personally believe the craft must be open. There is still a tremendous number of open questions remaining. You know, like 100x efficiencies waiting to be squeezed out and open weight models and open source AI is a way of advancing that craft and it needs to be open.

11. 4:17

    Yeah. Yeah. I mean, obviously the transformers when they came out were open source. Yeah. Imagine if it was kept in a safe somewhere, right? Well, I guess you and I wouldn't have so much opportunities to catch up, I suppose. But yeah. Yeah.

12. 4:28

    Yeah. Yeah, I mean the interesting thing is obviously like if it wasn't a safe, nobody would actually even know this is a thing because it wasn't really part of any kind of product at the time, right? It was a research project that was bringing together effectively like four different like teams working on it. And so the opportunity of you know really collaborating really opening it up and really everyone contributing is just tremendous to

13. 4:54

    enhance how we understand it but also how we develop this technology and really bringing more on board who can actually do this as well. I think you raised an excellent point if if you think about it. I mean one big one big difference in how research changed in the last I would say maybe now I guess close to a decade right is that there were young brilliant startups that actually did some of the most advanced

14. 5:19

    research and now they're they're and they're they're some of the most sort of they're some of the iconic companies of this generation right but to your point about is there an institution is there a single large institution that is actually doing frontier work in the open that allows us to you allows the entire world to you know like allows you know uh not just closed but open models to move lock step in our understanding to advance I don't think there exists such a lab that is doing

15. 5:49

    frontier work today there doesn't exist a single institution and I think we would benefit from that I I I think we would benefit from it yeah yeah I mean I think that this has been the challenge of the incentives right this this work requires immense amount of resources computing first and then obviously people um second and so there's no like there's no for-profit company that would be like willing to open up their frontier and then on the other side the academia just

16. 6:20

    doesn't have resources and it's not able to raise resources because doesn't have kind of a profit machine underneath and so I mean this is where in theory crypto is actually the instrument to really facilitate that uh but obviously you know it also So there is a divide between kind of AI researchers and and really crypto world that hopefully we can you know bridge this uh with you

17. 6:43

    know this conference and community that we're building and and I think that I I I think I think we shouldn't forget that there is there's still a ton of work to be done there's still important work to I mean this is no more just an number one this is no more just a single architecture it's a system right like the Wright brothers were I mean it's funny the analogies to how the right to to to how to to building the first planes is is I I I think quite closer than we might think for instance everybody could look

18. 7:14

    up and realize that yes physics allows for flight but then all the physicists of the time didn't think that you could have practical planes right but what did the the Wright brothers didn't come up with new theories in physics about flight they basically solve a systems problem in fact from what I understand they they built a um wind tunnel they actually simulated they they simulated flight right So I think that there's so many aspects of this system that that can be that can be improved and there is

19. 7:39

    good work being done and there's a lot of great work to be done. I think the resources question is critical. I don't expect us to I don't expect us to collapse the in you know the gap in intelligence per dollar within in a month or something. It's going to take some time. So I do agree that you know finding uh some something like an organization or or or a mechanism to organize a resources uh organize research and resources is important.

20. 8:04

    Yeah. Along with the the the system the productionizing and bringing it onto bringing it these production systems into applications as well. Yeah. getting applications getting actually monetiz monetizing them to feedback and like create a system but that promotes science promotes kind of open weights and to some degree we're we're obviously we are we want to do that at essential I mean and at the same time we have to create a business as so you know we will

21. 8:27

    let we we our goal with putting open weights models is both adoption also getting contributions back from the world understanding how understanding how our models are used in in in environments and we've learned we've learned quite a bit with our release and then bringing that into our bringing that into our fold that advances our research roadmap. So this is how we aim to balance it. I think there's other ways of doing it I'm sure you other ways of balancing these two worlds as well. Yeah.

22. 8:52

    So maybe switching gears I think what's really interesting is what are there open areas right because I think you know there is obviously right now first of all there's like a meta camp which is meta camps which is is this is AI good or is AI bad right for society right removing that for now.

23. 9:09

    Thank thank you. Yeah. Yeah. Yeah. But also like hey is is the frontier companies which you know are they just like on the drum beat and just like they're going to continue dominate and it's just more scale right? you know there's like just more of the same at larger scale maybe cleaner data and that's all or there's actual science there's actual innovation there's new ideas that coming in and I mean the question I get asked a lot is is transformers the final model or we going

24. 9:41

    to have another one right that is better um so I'm curious uh to hear your thoughts here I'm I'm surprised you don't get you don't get asked if transformers is all you need uh but Uh uh I I think it's I think there's truth to both, right? I and this is my and this is somewhat a sobering thought, but I I I do believe that there is the the path of pumping in more resources in the existing protocols or the or or the uh in in in the um putting more resources

25. 10:14

    in the in in the existing way of doing things will give us benefits as it continues to. We put miraculous amounts of resources and we got miraculous results in coding, right? I I think that that that's a very I mean some of the the coding models are very impressive today what they could achieve visav last year, right? So I and I expect that to continue but at the same time like I said I I you know there's important there's advancements we we continue to

26. 10:40

    make in all the areas that you mentioned right I I mean so yes fundamentally we're trying to predict the future from the past right that's kind of what we do with sequence prediction I think even and and and and I and I I think that's a very general very general and important framing of the problem um I I I I I believe that You know, we're very for we're very fortunate that we have like we have algorithms like gradient descent that automatically learn from data the

27. 11:11

    patterns and the programs that explain that data right and uh and then there's another constraint that we must keep in mind which is physics modern accelerators right and that has played a role in how architectures have advanced but I I I think there is I I I believe there's still important advances to be made in architectures right you know the transformer was very effective at mixing at these having these short wires that mixed information locally and these long

28. 11:36

    wires that mixed information spatially and globally right and that and they're there turned out to learn patterns more effectively right for at least some class of problems and now if you scale it up nearly like all class of problems um there's there's I I think understanding for instance even on the data side what is the value of a particular signal like a document what is the influence it can have on the behaviors that you care about is another open problem right Um and and you can

29. 12:01

    you can think about interpretability as some part of this. Um there's enormous work to be done in in and and and I don't know why this isn't happening as much as it should but everybody should be building environments right where open weights models can can participate and learn from and I I think there's an there's a lot of important work to be done there evaluating model behaviors.

30. 12:23

    Um so you just you know a and long context long context is obviously I think the holy grail it's a fundamental problem and depending on how you depending if you're a model maximalist where you just believe that we should we should figure out how like we should extend memories unto infinity or we should have like you know mimic what modern ar what we have done with architectures have hierarchies of

31. 12:46

    memories right so there's like just in this conversation in this last minute there's so many problems that have come up I I I I don't I I think that uh yeah there's limitless amount of interesting problems to work on. Still a lot of good work to be done. Yeah.

32. 12:59

    And that's why we're optimistic at essential. Yeah. Yeah. Yeah, I mean I think there's all of the all the things you've named right and even even the gradient descent right which is indeed seems like a magic but at the same time it's highly inefficient from perspective of physics because it effectively requires global like global computation over the whole model which means you have a limit on how big models can be you have a limit on how much compute you need to actually do this

33. 13:26

    memory and so yeah I think there's opportunity at every level of scale and we we know the systems are highly inefficient, right? You need to show it a piece of data, a piece of pattern thousands of times to really remember it, which is clearly not like clearly not how we learn, right? I mean, I think every researcher that we've met has had a dream about data curricula, but has not but has scars from it as

34. 13:50

    well, right? And again, every many researchers that we met have probably dreamed someday I will defeat gradient descent and they only come back with scars. So, you know, just remember these words. But I I I I agree with you, you know, like I I wonder why can't we just look at data once, just all the data once and we've learned and and and and and we would learn we would squeeze out as much information as possible, bring it down to its irreducible entropy, right? And and and

35. 14:14

    maybe another way to say this is okay, yes, we fed the model the internet. Have we compressed it completely? Because ultimately that's what we're trying to do when we're doing next token prediction, right? We're trying to compress all the knowledge, all the skills, all the reasoning on these models. Have we compressed it to its irreducible limit? I don't think so. So there's a lot more work to be done there. Yeah. Yeah. I think that the

36. 14:30

    other interesting thing that's happening is the internet is and effectively becoming AI generated, right? And at the same time it's shutting down like the websites are starting to rate limit being able to scrape them. And so there there is actually an interesting and like problem that's happening is that the labs who started earlier have scraped internet before the internet shut down and created a lot more AI

37. 14:54

    generated content. because they actually have human data that uh new labs and new approaches may not have and so how you think of dealing with that and I mean obviously you guys need to actually deal with that as well 100%. uh you know like at one point of time we were simulating one point of time we were simulating what humans thought and now we're going to simulate in our models what models

38. 15:18

    thought right and then I I and and I I think but by by the way I this is probably already true of GitHub right how much of code that if you train a model on GitHub today how much of it is actually just going to mimic uh mimic the ram the the sort of uh imaginations of claude for instance right um so so okay so the good and the Right.

39. 15:41

    Technically it's it's a you can you can take this point that you made and reduce it to a question of what is the composition of natural that is human generated data and synthetic data that you actually that the right composition to the right composition to create uh to create the blend of capabilities that you care about right so fundamentally it is a question of that I think the issue that you're pointing out is can we distinguish between these two right uh I

40. 16:03

    mean if you didn't if you're not able to distinguish between these at all then I think we've reached uh we passed the touring test so we're good to go we're done. But I I I think we can and and and ultimately it's a it's it's a technical question of how much of these of this inorganic or synthetic data along with natural distributions creates the right composition of abilities for the behaviors that you care about. Um but but here I I I want I want to say

41. 16:28

    that I deeply I I I I cannot I cannot explain how how many how much I appreciate and revel or or or or or um how how much I appreciate human co human cognitive accomplishments. And I and I I I I perhaps I I might feel a little unhappy if you know our trace or the the human the the pulse of human cognitive thought is being wiped out and and that does disappoint me a little bit. I I I I

42. 16:58

    I think there's a lot of beauty in how there's a lot of beauty in in in what we have in the knowledge and skills in our knowledge and and information that we've produced and and and I and I hope that you know Yeah. Yeah. We will continue to enjoy it and the models will also benefit from learning from it. Yeah. So are you more pro uploading our mind into the into the net or uh uh

43. 17:22

    effectively keeping the I don't know guarded only humans allowed. Um that was a very abrupt transition to uh to to spec very abrupt speculation but uh if you were asking me I think the hybrids are an interesting strategy. Yeah. You you referred to extremes. Uh I I'll leave it at that. Yeah. Yeah. I leave it at that.

44. 17:45

    Yeah. So I I I agree. I think the there's a lot of opportunity right now. I mean the the current model training is extremely efficient, extremely resource intensive. I think this is where it's it's a really weird state right now where you like I mean you've guys built 7B model that's still pretty resource intensive, right? It's not something that even academia is able to do. But

45. 18:05

    this is this is starts to be the sides where you can actually do like actual science because this is where the some of the emerging uh kind of phenomena are coming. Do you think there is an opportunity to do science on smaller like on smaller models and with small a smaller compute to really get the properties out because I think that that's really been the the challenge. I think I think this is a fantastic

46. 18:29

    question and and you know when we when we shared the when we shared our journey for uh uh journey through range one in our blog I we wrote that trying to pick out these invariant these signals that are invariant at different scales is a is a is an extremely important and difficult problem and we succeeded to some degree but I believe that you know we we haven't completely succeeded there

47. 18:52

    I think that there is still I mean depending on the kinds of behaviors that you want to detect like at smaller models there's too much there's there's very little signal over noise and uh that's a huge that's the benefit of actually running at running experiments at scale but then maybe let's say we want to measure an abil a mathematical reasoning ability and and and the one a 1 billion parameter model is inept at showing any sign what we

48. 19:16

    need to figure out is a better way of measuring it and getting these traces early on so that we can uh so that we can significantly improve the uh researcher iteration and cost of scientific iteration as well. I wouldn't say that we have an answer to this, but I think you rais a good point. That's one of the most that's one of the questions that we do care a lot about at essential and we do benefit from by getting more more getting applying the scientific meth method first principles thinking and being very rigorous in how

49. 19:39

    we iterate over our ideas. Yeah, I mean like the first transformer, right, was ran on CPU and we could see some some signal from the noise, right? That was the but it it Yeah, it's pretty dramatic now that like yeah, beyond just basic language modeling like if you want to see actual capabilities, you need just massive models, you know. I I actually calculated this.

50. 20:01

    So I believe on um on I mean I guess on a on a the last generation of of GPUs we could probably do the English German experiment. Oh no no on the latest generation of GPUs on a single device we could probably do the English German experiment in a in an hour or so just by just by the flops itself. Right. So, so, so I I think you're I think relative to 2017, we have enormous resources.

51. 20:27

    Enormous resources. And maybe this is the worry that we're relegating a lot of our intelligence to AIS and noting a lot science. Sorry. Yeah. So, no, I really appreciate you coming. Uh this is all time we have as as you know we can speak for hours about this stuff but uh thank you very much for for uh listening to and hopefully you picked up some of the ideas where to where to invest in the future with your

52. 20:52

    time and efforts in building this uh amazing future. Thank you. Thank you.