Layer 1 Stage | AI x Blockchain - Illia Polosukhin and Mike Butcher in Conversation

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:01

   welcome back IAH have a seat you should have great thank you so much for joining and just to remind you again my name is Mike Butcher uh editor at large at TechCrunch the small little blog you may have heard of that writes about uh startups and uh Venture Capital um that was a fantastic speech what I'm interested in doing is I'm going to wind you back a little bit because right at

2. 0:31

   the beginning you as we say in journalism you buried the lead effectively the lead or the lead I can't remember I can never remember but you definitely buried it because right in there you said that you were one of the original authors of the 2017 AI paper attention is all you need surely love is all you need but let's move along you were obviously one of the co-creators this Transformer model um how does it feel to be not Sam

3. 1:04

   mman right now um well I don't know how it does it feel to be cell moldon so it's hard to answer to that directly but I think from my perspective it's uh on one side extremely exciting to see the evolution that we've seen right in in past uh five years that this technology has brought right and obviously there was a lot of work that went in into making something that was a research project that was

4. 1:35

   cool that you know first started to answering questions and translation then uh kind of become more and more of a tool and then actually turned into chat gbt and M journey and all the technology and like there's a ton of work that like Community with so far too you're far too humble but what it it it really is quite fantastic that that actually happen let's just but that's sort of dig in a

5. 1:57

   little bit there's a reason why you you wrote that paper um and really very much about AI of course but I know that there's a story about what happened next because you wanted people to work with you on these some of these ideas and you couldn't pay them tell us that story yeah so we pretty much kind of started neari we we did not have you know that much data or uh compute and so we've tried to be smarter we tried to generate better training data for models and to

6. 2:29

   do that we engaged people around the world uh to actually do small tasks for us and Alex skidanov mentioned some of that before and to do that you need to pay people right the students who usually would do this work they are in China they are in Eastern Europe they are in kind of developing countries PayPal doesn't work some of them don't have bank accounts you you know in Ukraine uh like some other system and like you cannot actually have even good

7. 3:00

   automated system to do that uh even now like this is uh still a problem and we kind of wanted some Global Payments Network that actually just works and we can program uh to do this payments for small amounts to students around the world and blockchain seemed like an answer right 2018 we started looking at it and it was not it like the fees that you would pay on bitcoin or ethereum were way higher than we were paying for the for some the work that people are doing and so we kind of started to try

8. 3:33

   to investigate what should it look like and kind of arve to near design to really build something that's easy to use highly scalable kind of accessible to everyone and really easy to program well is it really is quite a Visionary point that would you all so many people remember that time that era people talking about proof of work proof of stake but you came straight out with the

9. 3:55

   proof of stake model and of course the rest is history we we all know about that but let's let's dig in now a little bit more about the kinds of things you want to get into now now we saw open ai's announcements uh only a few days ago it's clear that what's happening and you referred to it in your speech about this agent model these uh you know mini gpts that they're going to build why is

10. 4:20

    it that you think it's so I mean you did talk little bit about about it but let's unpack it a bit more why you think web 3 and AI the future of AI are so inextricably entwined the futures of those two two issues yeah I think so there's two sides of this right one side is this idea that you definitely don't want one company or even like a handful of companies to be controlling kind of the intelligence of the whole world right and that's what is happening these models are bringing kind of intelligence

11. 4:53

    Tools in our disposal to build you know this different business or operators or assistants or interns or kind of this uh instruments and so you don't want them to be deciding like you know for example Nate trying to do a demo and it's going down for a few hours right you don't want them to be able to switch the model behind the scene for you without you even knowing and especially you don't want if you're in a business or

12. 5:19

    especially health care or any other use case where it really matters you don't want it to be kind of possible to for somebody to influence and change things as you're actually interacting with it imagine you have a financial system and you inter and you're kind of using it and somebody behind on the other side is able to like well if you know Goldman Sachs using it then let's just tell them

13. 5:44

    to buy all this stock right but opening I talk um like you don't want that to be happening you want a way to ensure that the results are correct you want to ensure that results is the same and kind of um have an audit Trail have have a way to uh kind of Prov provision and request um kind of new models for your specific use cases so you want that part and on the other part as I mentioned the AI agents themselves they cannot interact like they can only interact

14. 6:15

    with API right that's and at the same time they cannot you know do kyc they cannot like on board into a bank they cannot actually um transact as a business for example uh because they cannot have a legal signature right you need cryptography you need a way to interact with kind of the world and finances and to do that blockchain is a way it's interface the um you know I'm here to ask the dumb questions being a dumb journalist but why would you uh what's the incentive for a foundational

15. 6:47

    model like open AI to in introduce this extraneous uh layer over the top where is there how are they going to be beaten into it is it going to be a carrot or a stick so I I mean there's a carrot on the side of using blockchain right again the there's already plugins directly in chpt you can use to interact this blockchain right there's people who are have built some systems around that as

16. 7:11

    well uh and that's more of a carrot because it allows to have more functionality that is indeed Financial I think the open source part is going to be more of a stick and like there will be kind of a battle between regulatory cture the kind of centralization powers and open Source decentralization kind of open economy right the the standard battle that we have you know for thousands of years right is the centralized you know model better where you have full control and full ownership

17. 7:43

    and you're able to dictate to everyone else what should be done or you want a more open model where everybody can contribute where everybody can participate and you know actually economically be incentivized the problem that Sam mman a very clever man has decided you've seen what's happened to uh other platforms how uh in web 2 you had this sort of unbundling you had you know a company would would launch famously there's the the the famous graphic of Craigslist and out of

18. 8:14

    Craigslist came Airbnb and uh you know car sales and other kinds of services and unpacked and unbundled those sort of early web One models into web 2 um the unbundling is already happening with open AI they're already unbundling themselves as per their their announcements of only a few days ago we're literally seeing it happen before our eyes so they're they're leaping forward into the future a to create this app store ahead of all of us trying to innovate and create new new kinds of

19. 8:46

    startups and apps and and and other kinds of services how are we going how are we going to fight that I don't think we need to actually fight that I think the the the interesting thing that happens here with transitioning from needing to write a lot of code Building Services and systems to this new natural language interface and so through that like the those gpds right that they

20. 9:10

    announced are just a set of natural language instructions and kind of prompt engineering and you know maybe a few like Integrations API Integrations and you can actually replicate that very easily and cheaply on top of a open source model right you don't need to like to lock it in like the I think the bigger challenge is there's actually no business model for building those period that that it's more of a problem of startup land that we actually need to

21. 9:36

    have like in same as open AI they incentivizing them just based on on their revenue in the future and actually the blockchains have pioneered that model actually openai interestingly very similar structure to a blockchain foundation they have a foundation that is nonprofit that indeed wants to redistribute the revenue to the participants and contributors so like they're not fully that different they're just doing it in a very centralized manner right but like structurally I

22. 10:08

    think we will see either them actually going in that direction themselves or projects like ritual and others who are trying to kind of build this more open source open you know participant uh system where everybody can actually benefit from contributing and then that becoming a bigger kind of economic driver for people to start and participate unless like and kind of compete with open a models I know that you are quite excited about the

23. 10:40

    potential for zero knowledge proofs um and uh their application you know in in AI um and on on these new models tell us a little bit more about your thinking about that and the you know the kind of wider implications zooming out yeah so zero knowledge proofs is a really Innovative technology that allows to pretty much go from what we usually have which is you know servers and uh kind of back ends that run and it's it's guarantee of

24. 11:15

    correctness is just based on you know that this company is running it to have a mathematical proof that it's exactly executed the the way it said it would and this transition allows a lot of new things I mean there's a lot innovation in blockchain space and scalability but in machine learning but also in data processing in kind of data acquisition this allows to have pretty much in you know potentially a ton of compute potentially running the model and have a very small

25. 11:44

    proof that's really cheaply to verify that this output is correct now Z zero knowledge although it's not completely new technology it's very fast evolving and in like past four years it you know went up in speed thousand times times right but it's still not there to be able to prove like large models but you know we can see that progress continue chipping away but at the same time you

26. 12:08

    can already do something on data side right you can have data Providence you can have uh kind of understanding where things are coming from and how they were used and have some additional proofs for payments for example uh if some data were used or not used uh for training the model you were originally at Google and obviously uh went on to create near um you must have quite high level contacts what's some of the things say they say to these the platforms say to

27. 12:39

    you uh I'm sure you're amongst friends you can reveal to us uh what what those private conversations but more generally speaking um what sort of vibe and what sort of you know signals do you see coming out of uh those the B the bigger platforms and how they might because they probably agree with you about some of the things you've just said on stage they know that probably it will happen in the way that you've characterized it and yet we don't see those signals

28. 13:09

    coming out but do you think they're open to that conversation I think there's probably two sides there's one side which is the hey this is important technology but it's like the consideration is that it's too dangerous for people to have it right and this is where like a philosophical discussion is is like is it too dangerous to have technology available to everyone or not and you know we can have like a separate

29. 13:34

    discussion on like the philosophy of that and how at least I believe you know whatever like information wants to be free That's my kind of main kind of um philosophy and so through that like whatever people build will become open righted and so like there's no point of trying to hold it back right it's like trying you know yes we we have built this but like one way or another either somebody else else or you know it will

30. 14:00

    leak or somebody else will build it and kind of knowing that it's possible but sorry Contin yeah so that's I think the biggest kind of decision Mak because I agree like for open AI I think they actually are very much aligned with that's why they build this the company structure and everything is very like kind of blockchain ask from a incentives perspective it's because of this but

31. 14:23

    they they they have this fundamental like oh this is too dangerous for others we are the ones who know know best how to do it right and this is like I don't agree with that premise but this is kind of their main thing I think for many other companies right now it's more of a survival game because they see that this technology coming in actually will uh disrupt their business model completely

32. 14:47

    right go for Google in many ways you know if you just go into a chat and asking questions you don't need to go uh and visit Google search you don't need to go to pages right yeah and if people don't visit pages and go to and see the ads right they're disrupted so I think that's the other transition for them it's like they're trying to they fight for survival in the way and obviously Google has been AI Powerhouse so they

33. 15:11

    also a little bit the kind of the bright thing there too so I think that's where you know if you look at the forces and then you have you know meta which is open source and they go in full on open source and they really trying to be The Driver there and kind of provide and work with a lot of actually external companies to uh be The Driver of Open Source Innovation well of course it's not in it's in meta's interest for the

34. 15:36

    for for over source to be llms to be out there because it's it's not their bu search is not their business model uh whereas of course it is uh Google now that you're CEO as per the recently the announcement only the other day um are you going to be positioning near in that direction to work with these uh the big Tech sector I mean I mean the the idea as I mentioned like AI is a kind of big theme for near and you know Alex kov mentioned like us investing actually

35. 16:07

    even before this hype started into more data we have you know been building uh kind of capabilities around foundational models we've been working with projects for decentralized inference we work you know we have hackathon and and projects in the ecosystem building tooling around this and we going we want to work with kind of projects and companies in that are aligned with us on how we see this

36. 16:30

    and so uh so yeah I'm I'm sure there will be some interesting announcements in coming weeks do you think that there's oh oh that's interesting okay I'll take not of that um do you think that there's something in a sort of a structural problem in the way web 3 the web 3 Industry if you want to want to call it that to want to a better phrase has in that because of this you know sort of competing layer layers the layer One models there's the uh sort of decentralized approach there's the uh

37. 17:00

    the sort of the open source movement that's often underlying it that it hasn't had that centralized Lobby voice that the big Tech platforms have say in Washington or in Brussels um how do you think you're going to be able to insert yourselves into those conversations about the future of AI uh in the future yeah I think I mean some of it is true that I think there's hasn't been like there's actually a bunch of love Lobby from B 3 in various places but

38. 17:32

    they also kind of decentralized in a way right there's actually a bunch of them and they have different aspects and shapes although share a lot of the same values and structure um I think on the I side uh kind of this AI blockchain intersection is just forming it's just coming together I think the missing piece that you know I uh like to point out is that The Regulators need to start

39. 17:57

    using blockchains itself and that that's probably the missing piece around kind of the lobbying of blockchain it should not be like explaining blockchain per se but actually explain how to use it for their own use case and then as they start to use it as say start to use it to record the you know what are the addresses for example what are the bank accounts or people who are you know

40. 18:20

    violating the sanction list the ones that record you know registered Securities and all the metadata around them like this becomes a source of information that everybody else can use transparently and easily and like this is literally the point of Regulation is to have this source of information that everybody can rely on and so it allows to have the protection for the users now

41. 18:42

    implemented across the stack everywhere else I think that's a missing piece not just not just the voice but actually starting to use it it's like you know everybody in like trying to regulate Computing without owning a computer that's current state do you think the Trojan Horse that will be able to be inserted into that conversation will be the protection of citizens I I owe it well I think yes because you know in

42. 19:08

    other words you know the governance the protection the ability to allow citizens to own their own data that will finally win the argument for governments and Regulators especially from the aspect of like kind of reputation and content because what happens is right now it's not the the misinformation is actually not the problem with AI this is a problem that existed in the world for

43. 19:32

    thousands of years since you know people started talking right we have blockchain itself is built on you know bisle to and full tolerance because we assume there is some percentage of people who are maliciously trying to attack the system right the challenge with AI is that it gives a tool that can scale up the misinformation can scale up the manipulation scale up the like and make it individual how you for example

44. 19:57

    generate made a way to for example convince a person to vote for someone right so like in politics what we'll see next year is that individual ads generated specifically for individual person for a political candidate that have that know exactly what are your political interests are and fine-tune the what candidate is saying for that person right that's possible right now it's not that hard and so if you want to actually have a way to protect from that you need to use cryptography you need to use a way to

45. 20:29

    record and uh kind of like have a ledger of this information well it's all um but obviously these mer the merging of these services are already starting to happen I mean the UK just only couple of a week ago announced a week or so ago announced that open AI would be powering the new um chat bot for the National Health Service in the UK so these things are already starting to be rolled out well I mean this is the other side I'm saying like somebody maliciously trying to attack and say like I am your doctor and

46. 21:00

    you know or I am the candidate you should vote for and by the way I really agree with you on everything you say right and you should vote for me right and like literally deploy that to to like everybody in Iowa and I want to throw a couple of things at you in the last dying minutes of the conversation and i' love love to hear your thoughts Sam mman and Johnny Ives ex ex Apple obviously the famously creator of the uh the iPhone the well obviously with Steve Jobs but the the designer of the iPhone

47. 21:31

    are rumor to be working on a hardware device powered by AI presumably open AI what are your views on that I think we'll see the open source models running on edge on mobile phones uh and this will be the way to actually like have the private data used and at the same time will promote the open source a lot more and get get it kind of to be a main driver for this not not openi models um you said I think in your tweets uh tweet stream the last couple of days you said the future of open web

48. 22:04

    is multichain not maximalism all of us in web 3 need to work together to help Founders and developers deliver the best possible experiences to users to achieve mainstream adoption uh are you buoyant are you optimistic about mainstream adop adoption for sure yeah I mean we have you know biggest apps in web3 consumer apps in web 3 that are built on near we see kind of the next wave of

49. 22:28

    applications that are really hiding the blockchain coming and kind of enabling new experiences and I think that in itself is I mean already approved but I think we'll see kind of a wave after that which will be kind of the experiences that kind of we're not possible in centralized right something where it's more about the kind of how the community coming together versus uh kind of individual kind of qu conversation that

50. 22:57

    right right now is when you're using an app right and I think that is where we'll see a lot like kind of social uh interactions coming in that will be really exciting and will drive more and more kind of uh effects but it will not come if you need to install wallet and if you need to like know your zerox address right it will come when you can use this really straightforwardly um I was looking about back at the history of of how we covered

51. 23:23

    uh near in the old days on Tech Crunch and some some of the articles in the early days you were build as an ethereum killer like many other uh projects out there are you happy with the how things turned out I mean when never were're trying to kill ethereum I think like I've I've said that even in the beginning like ethereum you know and with any decentralized network you cannot kill them they will continue running they will you know if people want to build on them if there's value on using them it

52. 23:53

    will continue and we always thought of us as adding more functionality making things easier really unboarding developers and users who don't want to know the exact details of how your blockchain works but actually want to abstract the blockchain and leverage it as a um as a primitive for their applications and so for that we you know we're on a good track obviously there's a lot more to do and a lot more projects to bring um but the same time you know as I said like we we actually started by

53. 24:26

    talking with the CDM Foundation we started by talking with uh folks across the whole ecosystem we have worked with a lot of folks in various ways as well as uh you know now we're actually introducing some of the features to really open up and kind of collaborate in in active way well as somebody who's observed you from a far for the last couple of years and and seen how n's uh

54. 24:50

    grown I think it's it's been absolutely fantastic to to watch uh the ecosystem develop and it seems to be quite an exciting time but afraid we're out of time but for now Ilia pukin thank you so much and for me Mike Butcher from Tech crunch thank you for everyone thank you see you again thank you