NEAR AI Office Hours #6 EQTY Lab and Theoriq

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   custom field hack the world and make it TI blockchain flows near the one we're close to won't stop us now together we need Define the game when near we'll find the flame empowered Minds rise and sore use your own AI we explore hold the future hand in hand in this Tech driven wonderland hello everyone uh this is near office hour is number

2. 1:02

   fiveish um uh today we have a pretty cool set of people here let's begin with a quick round of introductions uh Ron do you want to start sure nice to meet everyone I'm co-founder and CEO of theoric uh we're an AI agent based layer super excited about how agents are really going to be the next wave of AI and we're passionate about a future where the community owns and creates agents collaborating together to solve

3. 1:34

   problems rather than big monopolists controlling the AI agents that everyone has to use so we're enabling that future and super excited to be collaborating with near on that awesome uh and Jonathan great hi everyone great to be here nice to see you again Ilia and great to meet you Alex Jonathan founder of equity lab and we work on AI Integrity so these are endtoend tools around provenance and governance of AI

4. 2:06

   so be excited to chat with you got some fun ideas and big fans of the ecosystem so excited to do more I hand it over to Ben awesome yeah Benedict you want to say a couple words about yourself yeah sure uh Benedict uh I'm calling in from Toronto uh CTO at Equity lab uh have been have been working with Jonathan for a long time uh worked on data Provence on human rights data have been the ipfs in fcoin ecosystem for a long time really excited about uh opportunities

5. 2:38

   that we can explore today um yeah chat more cool yeah and I guess we can we can start by um setting setting some stage so I remember a couple years ago when everybody was saying hey guys we're doing metaverse nobody really knew what metaverse was uh so so I think it's a um uh uh now it's slightly different but still every everybody thinks of Agents differently so Ron when when you when you speak of Agents what do you how do you guys Define

6. 3:09

   agents yeah of course Alex you know what we see is that agents are uh increasingly autonomous software that uses AI models uh but can also use tools access data in real time and you know what we see is that allows for you know a rich range of tasks right it might be things like governance in a dow to help understand uh the conversation and Empower people to make better decisions it might be doing project research to decide if you're investing or

7. 3:43

   collaborating with a project you know to understand more about it and its underpinnings how it works it might be for developers to better uh analyze code for for bugs or simplify integration with a protocol right these are all things that agents are are doing today and what we're seeing happen is is the foundation models that are out there these uh llms are getting more powerful that they're really good they're getting

8. 4:11

   good and much better at planning right they're getting much better at tool use to be able to call out and take advantage of tools um they're they already now have massive context windows so you can do what's called augmentation and pass large amounts of information at runtime to say like here's a cool Corpus of relevant information summarize it and use it so that's letting agents work you know increasingly autonomously at more

9. 4:36

   and more sophisticated tasks so that's what we're talking about what we're seeing and I think we're just entering the era where that's possible with the models that came out in the last year um but we think that that pace of progress is uh head spitting and that the kind of models that are going to be coming out over the next years are going to enable much more Rich powerful agents uh that

10. 4:59

    the Community can create and collaborate on do you think there's a risk that uh proprietary models will will become much more sophisticated than open source yeah I think I think that um the proprietary models have a lead right that there's probably U at least a year lead between you know if you look at gp4 came out and it took about a year for any openweight models that were similar

11. 5:30

    quality right so you're talking it probably like about a year um the other question is going to be you know to what extent will you have the capital to train the the largest state-of-the-art models right so you know we're now seeing the race to the the the gigawatt data center you know with hundreds of thousands of h100s to train you know maybe the generation models after the one that's coming out next year right

12. 6:01

    and and spending a billion dollars plus to build clusters like that is quite expensive right there are people looking to do that for open models but um I think I think that in some ways uh as a community being able to build on uh different models and to let Builders integrate and build useful things whether it's commercial or open makes a lot of sense um at the same time you know I think you're going to continue to

13. 6:32

    see this the advance of open models that will be a year behind and often that you you get good enough performance right so like in agents that are helping analyze social discourse they can use an open model like like la3 and uh do just fine right it could be fast and perform well instead of having to rely on you know let me call out a commercial endpoint from a proprietary model yep yeah uh yeah it's what

14. 7:04

    I'm uh interested in is How likely it is that so so it took a year to catch up with gp4 or like it's it's arguable if we actually managed to catch up but once you know GPT 5 comes out uh it's interesting if if we will be able to catch up as quickly or if the Gap will be increasing but I I guess Yeah you mentioned that for agents as of of today that might not be important you know for for many agents I

15. 7:35

    guess yeah no I think most agent Builders are um pragmatic right they're going to sort of assess the performance and quality of different models I mean you always have some concern about censorship and capricious change from a commercial model right like you know when we were testing some agents early on using o uh gp4 uh we were having the agents assess the quality of output of one model by saying you're an expert Professor analyzing a a

16. 8:08

    response please give it a grade and suddenly gbd4 started saying do your own homework I'm not going to tell you the grade on this you know so they had ad adapted it to avoid misuse of chat GPT and it broke our application right so this is where commercial endpoints can be you know you don't have control right and they they they might shut things down because of some other reason or you know deploy a random change you have no visibility into right with uh so there

17. 8:38

    are reasons why having you know open models is really nice when you're building agents but uh everyone building agents is like pushing the envelope of what can be done with the models um so yeah I think I think pragmatism does make sense but also you know many developers are say hey like we're not not going to be completely locked in with one model we'll use a variety and you know where can we want to use open models and only use commercial where we

18. 9:10

    have to Y and and you mentioned the model I'm not even sure if they intentionally broke gp4 you knowf uh the the way to train is there's a model of the world that is being trained and then the model is trained to to to perform well again against it uh and the model of the world is trained on annotations and if a lot of annotators you know uh uh if a officient number of annotations refuse to answer then the model of the world says oh as a matter of fact refusing to answer is actually quite beneficial right and then the

19. 9:47

    model learns to to refuse it might it might even be you know unintentional but still still break the uh the flow so so I'm Equity lab you guys are uh looking at ads as kind of problems uh as you are looking at kind of integrity and uh observability yeah so I think the first thing to say is that we think about most solutions that we're focused on are in the Enterprise and um therefore they're

20. 10:20

    already looking at compound AI systems or like what we think of like as ensembles and so part of the solution nilia I think begins with an acknowledgement that that these endpoints can actually be orchestrated together to accomplish different goals and that we might want to think about routing queries depending on you know how they come in the prompt um is able to be directed towards different knowledge graphs different pre-processing of the prompt to actually

21. 10:45

    get you the right embeddings and then send them off to the models that we think are going to be the best suited to give those answers and increasingly I think you know the combination of models is something that we're seeing is actually very beneficial uh that said so sorry I say that from the perspective of control and our task therefore from a provenance standpoint is to make sure that the handoffs between all these different

22. 11:07

    systems is one that you know can be seamlessly tracked and that we have this ability to have a portable interoperable uh unified Vision basically of of the whole workflow uh now that said uh I agree with Ron um you know the the largest of the large language models are capital intensive open an continues to outpace others but what we have seen um is that in spite of that there are still a lot of things that specialized models can do

23. 11:37

    really well in pre-processing coming up with um what is emerging as like a notion of like certified answers which is where other models can start to do various forms of classification or um human defined um glossies Etc that are like actually really important for doing Mission critical work um examples like that of course are like in regulated indust Industries where you don't have the same kind of margin of error as you

24. 12:02

    can with like let's say a conversational AI um you really need to have something if it's going to turn around reliable answers that it sticks to uh some fairly tight rules so I think it's a combination of those um while acknowledging that yeah the the close Source AI at the moment seems to be winning the battle for uh the largest language foundational models do you guys have insights into how close we are to to being able to prove that the a particular model was trend on particular

25. 12:37

    data you know I I was actually hoping that we could have that discussion today I think that you know I without getting um right now I'd say that our best work has been done by creating verifiability around systems of rag um that's where if you really want true attribution we can do it on the inbound to the llm and we can talk to you about like if te work that we've done and some of the

26. 13:01

    pre-training continuous training where we have full control over that but with respect to kind of Downstream if you have a fully baked model rag is going to be at this point your best opportunity right to create attribution and even then there are challenges with rag in terms of hallucination things like that so um actually Alex was hoping to have that conversation with you guys because we're seeing that you know there's glimmers of things from anthropic and

27. 13:28

    others that are are starting to get into questions around attribution but I'm curious like when do we arrive at that moment where we have a true fmri of the model you know where we can see what what kind of data has lit up essentially to provide the inference it's a big question so yeah I mean so historically to debug this models back back in back in uh in the day we would backrop from the wrong answer or whatever something you want investigate and you would

28. 14:01

    score uh the training data to identify which training data contributed the most now the problem is on the 15 trillion token data set that is probably the most expensive thing you can do right because it's pretty much proportional to the amount of training you need to do this so and that and that's like really fundamental challenge here is that this like adds this levels of training data is just uh too competition expensive there's also like there a paper on um

29. 14:32

    also using an approach pretty much like how much information uh information that that this specific training example contributed uh during the inference uh like um got the proper definition like sh Sharpie or something yeah the shly value yeah we've we've looked at those but they they suffer from the same problems that you're describing yeah it's super it's super computationally expensive computation expensive and also like there's um there's a there's a

30. 15:02

    particular challenge with shapley which we which we have found which is that um the interaction between the data sets is like really actually what you want to understand as well so just um shapley does a a reasonable job of approximating some of those things with like a regimen of testing where you try different combinations essentially to see like you know what if I take something out but I have to do all the different kind of the Matrix basically of of the interaction

31. 15:26

    between the different uh dams and um you know I think that one of the things that it it just it's very wanting from a computational standpoint plus it doesn't really recognize that there's a lot of other things that go into value creation for uh the processing of data and um that exists kind of beforeand it's it's fairly yeah I mean I'm I'm kind of like I at some point I was thinking about this and I arrive to a conclusion I don't think it's practical and so if it's not practical

32. 15:59

    we should figure out a different way to do to achieve the same goal without trying to solve this problem right so in the way what AI does this AI models they we transformed from a like kind of if you look at the history of humanity right we had a oneto one right we have the conversation um you know the the wise the wisest and old eldest would pass down to the you know this next Generation then we had you know gurg

33. 16:33

    press and we had books and this got us to one to end we got to kind of I mean I would say like social media is actually a transformative medium of exchange is like end to end but now we have like a you know kind of a compression model that takes all this end to one and then you can get anything out of it and so I think from this perspective I we need a new model of rewarding creators new model of kind of um operationalizing

34. 17:06

    that the same as we had the new model when Gutenberg Press came out right and you kind of needed to reink I mean almost completely Society how it operates what is the loss you know copyrights Etc just because like that was such a fundamental shift in technology and so so that's where like I I think of more okay well assuming you actually can not backtracked to like a specific piece of data or whatever that

35. 17:32

    contributed to this what are the important parts well you still want reward creators uh you still want a reward for new information because either way like this is something the system doesn't have um and so you need you need kind of systems for that uh and at the same time you want to understand uh when the model is indeed hallucinating which by the way you know sometimes hallucinations are good sometimes they're bad like that really depends on the context like if it's answering a factoid question versus you know you're

36. 18:07

    asking you to write an essay uh or write a science fiction novel can be very different and so and that's where potentially a new methods right with search and with like additional uh techniques and top can help uh make that more you know solve for the problem you're actually trying to solve versus trying to go back and like figure out you know oh this piece of data is actually affected everything um because indeed like it's just such a massive statistical model I mean I think it's

37. 18:39

    very true that like it's very hard practically like conceptually it's easy to measure how much uh input added value to a model but it's so costly right like you would do ablation you test with and without a data set and you say like how much did it improve my loss function right that tells you how much the data helped but nobody has time to do ablation with all of the different input

38. 19:03

    corpuses like it would cost a fortune right so while it's theoretically easy to measure the value of an input data set at least against benchmarks nobody's going to spend the price I think the good news though is I think you know what the uh Jonathan you and the team are doing in terms of measuring the value of data at runtime with Rag and as we generalized feeding data into tools

39. 19:28

    you know this approach of like most of the value of data for AI agents is going to really be the runtime access to the data I mean you see the similar thing like even in open AI uh deals that they're doing with content providers you know the primary focus seems to be around runtime access um I worry that we end up with a world though where big Tech does these Walled Garden deals and

40. 19:53

    there's a lot of exclusives right like we've locked up this information and no other agents can access it right it's only available to our users right so that would be a really bad outcome I mean we're already seeing some of this like in the earlier days of machine learning Twitter data was easily accessible you know Reddit data that were public archives it was great for building models everyone benefited now

41. 20:17

    they've all thrown up giant pay walls and they're all like you know the only way you could access the Twitter data is through you know super priced API or the grock uh llm integrated into it as a service right so you know I sincerely hope and I think there's a lot that the web3 community can do to create open alternatives to these these commercial data sets yeah I'll just jump into ad I think

42. 20:46

    Ilia you're the way you framed um this and Ron I think you're picking up on the themes of like the dilemmas around this I the Gutenberg Press like opened up not just simply a new way as you were saying of um disseminating information but it was in it of itself a new form of media like it is it was a dramatic new way of communicating things and therefore the communication itself changed Society changed as a part of that new type of communication right so it's not just that you have new methods of

43. 21:18

    Distributing information but the information itself changed as a part of that and I think that AI presents exactly that opportunity um for a massive transformation of a a new form of media um and with that of course compensation becomes really uh up for discussion to say like what exact how do we compensate people with this new form of communication um we've been working with Creative Commons uh from like the very beginnings of kind of

44. 21:44

    2020 uh two late late 2022 when this stuff started to go mainstream asking him saying well like copyright questions around like a cc license for instance which is a very important part of you know creating the internet Commons um what what does that look like in an age of Ai and I think that you know I don't want to speak on their behalf but I think the conversations uh that emerged um really required us to think about as a society what do we want to still be

45. 22:08

    fair use because that assumption that we've had so far over the last like 200 years of copyright you know uh standards has basically come down to this idea that um we want copyright to be out there so that creators can have the right incentives to create that's like that's the handshake we give creators the copyright in order to ensure that they can like feed themselves and they can create things but you want the net

46. 22:33

    result is that you have more communication that's what we want as a society what you're describing around with wal Gardens sits kind of in contrast to that in which now people would start to get worried or like well if I publish my stuff on the internet if it's just going to be scraped up by some sort of agent uh and then used and commercialized and hurting me then maybe I won't communicate and so we have this like dilemma that's emerging about like

47. 22:56

    what do we want for the public comments how much information do we actually want out there and we're at this risk in which there could be this chilling effect if we're not careful um the rush to pay creators presents even more problems because as you're saying there's a wal garden and also by the way you could be paying people in ways that actually violate um fair use so as an example um we saw this play out with

48. 23:20

    ebooks um in a prior generation like I have the ability if I buy a paperback book to go and sell it to Ben and I can do that at any price that I want well if I buy an ebook Jeff Bezos has decided through a state contract which is enforcable through the terms of use of my ability to get a Kindle that I wave my ability to sell the book that I purchased why they decided to do it that's it it's like there's no they didn't consult the Constitution they didn't affirm any real rights around me

49. 23:53

    and my my second was called the second sale Doctrine so um we're very concerned that we could emerge in a world where there's less communication because people are worried about this kind of notion of compensation is we've like a traditional notion compensation and secondly um that companies that have the the Deep Pockets could end up rewriting uh the rules of fair use so we're in a very delicate time um so just just throw just yeah to throw one more thing which

50. 24:24

    is like if you if you design computation a wrong way it will also lead to like you know we have a private chat I'll also throw it into the data set because without asking you because like the reality is a lot of the communication is multi-way and kind of the the whoever's owner of that is also becomes more complex yeah I mean there's a lot of like you know this documents and whatever else right um I mean the reality is like yeah so sorry there's a

51. 24:55

    little latency please no I just want to say like I think that one of the realities is like the current the current approach where you know we need the new old every single piece of data in the world right to train one model is also just not the kind of the approach we're gonna go forward with so uh part of it is also going to be interesting how it evolves with more uh you know synthetic data as well as a general generally more approaches

52. 25:26

    where you know we can use already trained models to kind of and level them up instead of uh you know trying to find an of 10 trillion tokens that potentially don't even exist right now totally um I wanted just to share one thing and bring Ben in to to talk to you guys uh at near U we're excited to enter into the ecosystem now and I think part of what was fun sitting down with Cameron uh this last week was actually learning how near got started was very much to create the proper incentives for

53. 25:58

    democratized AI I didn't realize that it was the protocol was was actually created after the initial um intention uh to democratize AI which is amazing and um we've looked at areas of open science as one of the first reference implementations for us we created this thing we called climate GPT which was the fine-tuning and then also the groundup training of a Tas specific model around the impact of climate

54. 26:23

    change we took one of the largest independent scrapes of the web and um created a a custom if for this as well as 300 billion tokens to train a model from the ground up working with our friends at Erasmus and and aptech and um I think this is a really interesting area because open science is something that most people have good convictions on and there's a lot of broad agreement

55. 26:47

    at least within the scientific community that you want open science and there the compensation uh is very different because people are typically not making money off of their research papers right they're getting a salary and they're they have like a stable income from Grants and things like that the the royalties on the papers are are minuscule and I think there's a general belief that the Publishers are somewhat

56. 27:11

    extortive and and not really adding tremendous value for for the price that they're charging so in that respect um you know we we find that that's a really interesting area because a lot of these copyright questions just simply evaporate like people just want a really smart model on the impact of climate change that's like a global priority everyone can agree on that that's like probably a good thing for it to not only

57. 27:36

    be open but for it to be transparent as to what data went in and no one's sitting there and talking about compensation they're talking about the benefits basically of the model rather than like the commercialization of the model um from a society standpoint um I'm curious like how with near you guys have been thinking about kinding a public AI initiatives of that kind and seeing if we can create um momentum

58. 28:02

    around those types of foundational models that that should remain open right they shouldn't be um they shouldn't be private for their own I think my my and our position in general is all models should be public uh because at the end it's kind of you know it's a human intelligence that we are summarizing and kind of making available as a tool and it indeed doesn't make sense to you know you leverage addent billion people multi-billion people uh kind of

59. 28:34

    interactions to then create a for for profit product uh on itself right and then the other problem is that this for-profit products then plugged in into very much attention based business models which creating a huge negative flywheel of better model getting people you know uh pretty much manipulating people to spend more time on the platform making more money invest in a bigger model and kind of just like get

60. 29:05

    this FAL going where you know it actually attacks directly the only resource that actually is finite in our life which is time and attention so which you know at the end attention is all you need and so the challenge is yeah like how do we how do we turn this models upside down where they don't benefit the company that build them but actually benefit the people both using them and uh kind of maximize their success their wellbeing their uh economic kind of

61. 29:37

    opportunities and in the communities around them that for and I think that that's kind of how we frame Ed was usually there yeah I think one I think one big open question is um how do we deal with the long-term stewardship of a model um how do we make sure that we we have the right incentives to compensate existing contributors and allow things to move forward to progress that we can have new

62. 30:07

    models that come out out of something and still recognize the contribution of past uh past based models and past data contributors and and I think this is a big open question we will have a lot of experiments on how to do this right over the next years but I also feel we're at a at a point right before that that we need to be able to capture uh what went into a model what is the what is the data how is it a segment of some data how do we represent data uh and and then

63. 30:40

    also like what are the governance decisions what are the human decisions that went into these are we securing the computation can we be sure that um one data set that gets transformed is actually uh we show what what what computation happened there um I I think verifiable Computing is big is a big part of this um secure builds the same reason why we don't want to like just grab a untrusted software package why would you want to grab untrusted bottles untrusted data so I think uh

64. 31:13

    cryptographic uh proofs is a big part of the uh the story to enabling massively Federated collaborative AI um so I feel the work that we're doing at Equity is is is to capture this uh data supply chain adequately and then once we have this once we know how to represent data represent model represent computation and uh human decisions who the Labor uh who who who who are the laborers of of enabling a model then we can start experimenting with new new governance models and new steip models

65. 31:53

    for that and I would I'd offer I think yeah the the route is is ultimately going to be in this particular case of the climate GPT so my understanding is that that is a rare circumstances where a lot of people actually want to contribute their information uh for free right so in general that wouldn't be the case right so we do need so a lot of things we were doing before we started building the Proto was around

66. 32:22

    how do we build systems where a lot of people can work and get uh rewarded for brand new contributions um yeah but I think it is interesting to uh to think of other examples where uh a lot of data can be contributed by people purely based on the you know on like working on a common goal and not and not you know team uh reward for their work yeah I I think I think Ron was saying something at the same time as Alex uh may be back to Ron yeah thanks Ben yeah so I I was just

67. 33:20

    saying I think in addition to the Integrity of um the computation right that you know some sense uh what what we really need is a proof of alignment to like know that an agent is doing the thing we wanted and it's working well right so even if you sort of faithfully compute the output of a model um if it's hallucinating that's still not good right so like a lot of what we're working on in theoric is like how do you work at that next layer to have Community signals of this is a high quality agent it's producing good output

68. 33:53

    that I would want to use and how do you so proof of contribution can you assess is it useful output and then proof of collaboration how do you put several agents together that are useful to solve a user's problem now I think you know to understand to sort of have confidence in the compute there's a range of options people are going with right whether it be you know it's low stakes and I'm

69. 34:18

    going to trust a centralized provider or you know optimistic approaches where maybe I can go back and challenge a computation or maybe you know more you know a sampling based approach where sometimes you you have a consensus of independent computation or trusted execution environments there's a lot of approaches that people are experimenting with I think it won't be one siiz fits all right like um I think in the the web 3 Community there's a lot of excitement around zero knowledge proofs but you know today you're talking

70. 34:49

    about orders of magnitude more expense and I don't know anyone who's thinking that today's AI models are so cheap and asked that they'd like to to pay 100x tax on that right so I think they won't be used for llms anytime um probably anytime in in my lifetime um but I do think they're going to be used for other things like small models for fraud detection so I I think there's layers with computation Never Say Never that one one thing we've learned is that you know all all of this has like exponential

71. 35:24

    Improvement uh hard between Hardware know binary Fields like all kind of stuff just like it's fair but but will the exponential of the compute demands of AI continue to run ahead of the exponential of capacity in ZK that's the question will the lines cross well I've talked with somebody who's like they're building Hardware Accel like ZK acceleration Hardware which I mean not completely different

72. 35:54

    from actually the matrix multiplication hardware and matrix multiplication proving is actually n squ versus the multiplication is Cub so you actually potentially can verif like actually generate proof for matrix multiplication while doing the matrix multiplication on the same Hardware it's not going to be probably state-of-the-art like M tensor core but it can you know spit out you a proof at the end as well so like I mean this is not happening tomorrow it's not

73. 36:24

    happening next year but I would say like this five years I would I would assume we'll have some Hardware that actually uh interleaves ZK proving uh with uh some of the I mean if if the car models and car structure stay right was the say with kind of the architecture of the model itself so like no no exra laty if there's enough money if there's enough money so so conceptually you could get to no extra latency and it would just be a small

74. 36:54

    cost but you would it's more it's about energy yeah it's going to take more energy to compute but yeah it's going to be like pretty much will be in parallel well one thing we can be sure is there's going to be different technologies that come into uh representing their uh verifiability uh proofs of computes um we have we have worked with CK proofs representing CK proofs we have worked with te proofs uh so trust and execution environments are actually super performance uh the the

75. 37:28

    overhead is Tiny um and I think we need to to agree on a way to represent these different types of computation proofs and make make people understand uh what do they mean like when you're when you're representing uh a verifiable compute uh what what risk there are are you trusting this 100% how hardened is is your is your is your training or build platform um and I also want to pick out

76. 37:58

    one thing that Ron said earlier that um it doesn't matter your benchmark scores right like like at the end of the day like how how how performance how how it relies on community feedback people who are actually using this to tell like whether whether uh that this this AI is of high quality and I think these are actually Community inputs and these people are actually working contributing labor to improve the model and and so I'm just trying to illustrate how broad

77. 38:32

    um uh the collaborative efforts is that people who are giving feedback to the system is also contributing to perfecting the system definitely agree on that and I think it's why you know giving people incentives for contributing whether it be through feedback or creating agents or even you know providing data that gets used right I mean obviously one of the promises of blockchain and crypto is that you can actually give micro payments and reward people for their

78. 39:07

    contributions in a in a reasonable way so I was curious on IIA one thing that we have been thinking about specifically on chain signatures which we were actually quite intrigued about is this ability to start to orchestrate payments across deep in and really getting to the point where you could scale computation scale storage and but collectively manage those types of efforts and you know I find it kind of

79. 39:38

    an irony in a sense that um the people that are enjoying at this point the most amount of success as an Insurgent Hardware solution or I say infrastructure solution is is cor weave and previously they were mining ethereum um and really good at it um and they turned their attention to high performance compute they have you know enjoyed this kind of Cinderella run of of success now and you know to the point where you know they're

80. 40:03

    kind of been anointed by Nvidia as like the Challenger in the space and I think it's interesting like how different a world would have been if they had actually allowed for the community to own what is now you know a 10 plus billion dollar company right that's and so there's actually a few projects doing exactly this now some of which are coming very much from the same background being miners kind of operating mining uh infrastructure and

81. 40:33

    now turning it not just into like hey this is a company that's gonna you know provide a compute but actually tokenizing compute and offering people to really participate in that as well or bring their compute or bring money and then they are able to access it um and operate it and then offer it to others so I I I actually yeah I've been thinking in general this that's actually a better business model by the way they compute marketplaces on their own

82. 41:03

    because compute marketplaces are raed to the bottom um like the margin going to get compressed but the you know whoever whoever Works directly on hardware and kind of brings that as an asset class uh for people to participate in the economy in the future uh of compute that's that's where and and you know you Branch off to non GPU Hardware non Nvidia Hardware over time as well uh to add the diversification and you create kind of this you know compute index that people

83. 41:32

    can participate on while also if you have if you have that you are able to access Compu as well so like for companies and you know projects that are that are actually consuming compute it makes sense to participate in that both kind of as a way to also acquire resources so I've gotten a lot more bullish on these kind of deepin compute providers you know people like you know IET and a and render and NOA because uh

84. 42:00

    I think the bottleneck in the next few years is going to be power right the ability to have a data center where you can power racks of gpus and and other accelerators for AI which will mean that those who can stand up and operate data centers wherever they can get a good power supply can actually be very profitable right so if that's the bottleneck resource then it's actually quite bold for decentralized compute providers yeah I think that in our

85. 42:32

    experience and having spent time in the decentralized storage space for about half a decade with fcoin um one of the things we recognized was that you still need to acknowledge that um scale is important and you want to make sure that you know the fragmentation of the landscape where you can kind of have this Utopia in which like anyone can just kind of Airbnb their GPU um that is a bit fanciful um in so as far as like

86. 42:57

    when you really want Enterprise grade reliable Solutions we're not yet at that point in which that type of distributed system makes sense that doesn't mean you have to have a centralized solution it just means that you need to build tooling and reliability um both in terms of the companies and the offerings that they have and that doesn't just spontaneously emerge by creating a a protocol it actually requires software

87. 43:20

    it requires uh a variety of kind of economic incentives to create collaboration of that kind um we see possibilities of creating new new types of financial incentives as well um to uh to allow for various forms of payment and securing that type of infrastructure Etc so I was curious like within the near ecosystem um you know where what what was what what are you thinking about in terms of um as a layer a coordination layer potentially for deep

88. 43:50

    in we love the chain signatures as like a a way of beginning like treasury management and payments of that kind we think that's actually really great but I'm curious like what what else do you guys have in store or what what are you what are you guys excited about specifically for your protocol to ad yeah I mean well protocol is obviously very generic uh but we have a whole host of kind of AI applications

89. 44:14

    that are coming uh that we I mean we GNA announce in a bit uh so I'm not going to front r that yet um and at the same time you know obviously we're talking with even more folks right see how we can partner up across chain signatures across other already kind of applications building um as well as some of the tech that we're building at neari and I think the you know like the main

90. 44:41

    challenge that we hear over and over again is actually not the infrastructure at this point the the main challenge is the demand uh right now in the three kind of um environment I think a lot of people building in infrastructure there's a ton of it but it's not actually used anywhere uh productively and so I think even bigger uh kind of um focus will be like how do we link all the pieces together with the you know

91. 45:15

    massive user base that we have in consumer apps such that the agents the you know the agent to agent payments the data you know various data acquisition data crowdsourcing the mod decentralized and compute all of that actually fit you know makes uh and produces value for the users and kind of some of the businesses as well that are participating this and I think that's that's where at least like from our perspective I mean then kind of from product thinking and like ecosystem building where we're starting

92. 45:46

    to spend more time uh just because yeah like the infrastructure plays in crypto that's you know there's tens of thousands of those as usual uh but uh yeah how do we how do we really bring value to users in like uh in a way that indeed benefits them that you know maximizes their uh success metrics while also uh you know ideally is better product than what uh web2 has to offer and to give you an example so King is this app that's it is number one by you know monthly active users in web three

93. 46:27

    now it's because it's pre-installed on operating inside operating system on Android for o and phones but what they're working on is bringing the layer of personal data right right now all this data you know every app sends it you know Google constantly sends your your location to Google servers you know every other app is also trying to like send some data to their servers there some gdpr violations happening left and right and so what they do is actually they brought in

94. 46:59

    a platform that have all your data your you know your location data your shopping data your uh views data Etc all on local device all available and it never leaves and what they do is they offer now way to run machine learning models on top of it on the device as well and keep the inference on device so this is a new SDK that you as a a publisher or Advertiser can use that will do that will do this on the local device and so the idea there is actually

95. 47:31

    this is better product this the advertisement or or recommendation model that knows everything about you that doesn't need to try to like as Advertiser trying to reconstruct who you are from like five signals but actually has all your information has a better like recommendation than not knowing all of this none of the data ever leaves your device this is recommendation shown to you in the app or in the website uh all fully local as well and they see 3x CTA Improvement right clicks through

96. 48:03

    ratio Improvement um just from that and so that is like what better product that a user owned are it's products that are delivering privacy bring everything local be able to use more of your data to give you a better result and make you more successful and so that's kind of the mindset right where we want to see more of product like this where uh it really links together all those pieces into end user kind of success while also

97. 48:35

    is better product than do it in F2 style yeah I I'll add to that one thing which IIA we're seeing on the Enterprise side where we're focusing is that um the interaction of data um that exists like with different data owners is really important and forming collaborations while still maintaining trust and privacy is also a big part of the solutions that people want because like my surely my data might be great that I've got but I know if Ben has complimentary data or potentially ways

98. 49:06

    of enhancing my own data set um then you know our ability to collaborate is really important for the success let's say of our joint business the challenge is that I can't for variety of competitive reasons or even regulatory reasons I might not be able to just ship my data over to Ben or for that matter I might not be able to even see see um even uh very granular results of the

99. 49:30

    fusion of our data we might need to abstract quite a bit um in order to create like proper Rules of Engagement and today those things have been done with centralized clean rooms primarily right that's the adtech model that I think you know has been fairly successful at for data enrichment um but what you're talking about I think is um if you can get the right Primitives the next chapter in that of providing value is not just monetization at let's say an atomic Lev of one user but in terms of

100. 49:59

     figuring out cohorts and other other ways of understanding predictive Behavior across more than one user and the question is how can you could still compensate people fairly through that process maintain privacy Etc so we've we've been thinking about Primitives that allow for that type of thing to exist we deal with it more at scale so Ron we'd be excited to understand like how an agent could be deployed like from let's say knowledge graphs that we help

101. 50:24

     people prepare that's that's kind of a nice handoff but um but it feels like you know where web 3 has such a cool opportunity is to write the rules about like why we might want to collaborate in that way and how can we form you know proper compensation methods because to date the only people writing those rules are the companies that have hoovered up all of our information and they have no incentive

102. 50:53

     to create more collaboration or properly compensate people it's it's antithetical to their business model so I'm really excited IIA to hear about stories like that because I feel like if those companies that can provide that type of agency for an individual user imagine like what if they thought about networks of users like that could be really really cool I mean to build on that you know we're certainly seeing like this

103. 51:18

     opportunity for people to build agents that can contribute and help others right so that you know if you can package access to some data through an agent and it can be part of a collective that users and exess to find Value that's really compelling you know so unlike open AI where you could create a GPT in their GPT store but it doesn't get used by default it can't call other gpts they don't compose and you're relying on them to not shut you down like they did

104. 51:52

     plugins and how how you would monetize is like well maybe someday we'll we'll figure out we'll give you permission to do something until we decide to turn it off right so the web 2 collaboration model is already very tilted I mean it's kind of shocking to see a company as early in its journey of adoption as open AI already going so far down the path of really not empowering developers in

105. 52:20

     their ecosystem but this is where you know the web 3 uh Community can do so much better and really Empower all the stakeholders and retain choice at all the levels and certainly Jonathan to your point you know we we'd love to explore how your knowledge graphs can help agent Builders be build better agents that'd be awesome just like we're excited about collaborating with near I'll offer one last question because I feel like I'm sitting with uh IL and Alexon you guys have been at this for so much longer certainly than I have

106. 52:54

     in um in the AI front um I'm interested in um the specialization of knowledge and I'll I'll Alex I'll build off of a question that you had with the the internet lag um which is we when we built the climate gbt model we had the foundational model but then we started to fine-tune it based on certain preferences or to make sure that we could incorporate certain views I'm thinking like of um communities that

107. 53:17

     might be significantly underrepresented in data sets uh like indigenous Elders as a really good example they often have like a wealth of data and in fact You could argue like maybe even more important information about the environment that than we would ever have in written form they have it in verbal form so we've been thinking of ways of starting to expand uh the documentation

108. 53:38

     of that but I'm curious um is another approach where you could have human intervention super high quality data and some form of prioritization of that um layered on top of a foundational model do you guys see the specialization of these models through tuning or other methods as promising because we if so then that allows you to think more of open AI not as by analogy not as like the next Google search algorithm but

109. 54:11

     more as like maybe an operating system that maybe they get smart about like doing like an Android like approach where essentially people could start to build um knowledge on top of it to create like you know the C more customized and important use cases than you could imagine people subscribing to you know various forms of kind of specialized knowledge stores do you see that possibility there

110. 54:34

     architecturally for AI to start to incorporate like highly specialized data sets they're small so yeah it's it's it's definitely a possibility uh and F tuning generally shows very good results though in this particular case if we find tun open AI models open will still keep all the right uh yeah all the benefits so it's a uh yeah I don't know if your idea was to fine tune open AI models using their API

111. 55:12

     or if you know uh oh I see I see you're saying you're saying finding open source models to compete against like a uh one model to fit them all yeah yeah that's something that's uh uh definitely a uh a very promising Direction I often think of of benchmarks already like a fine tune smaller models are better than gbd4 kind of generically right um but it the challenge with all of this is always like the okay well now you need to string together a lot of like I mean this agentic framework is how you string

112. 55:52

     together now a bunch of different models and like specialized agents to do things and kind of route between them but uh as part of the fine tuning you may like mess up stuff so this is where you need to make sure like it doesn't lose some of the capabilities that you actually need uh as part of just like you know you kind of think the weights right toward the distribution of data that you've been showing it versus kind of a

113. 56:19

     generic distribution so so like you kind of you know think of it as like you have this cloud of knowledge right you kind of cond ing it closer to the um to the fact to the fact and knowledge that you want like specialized but then it gets Spar everywhere else so you know it's kind of starting to mess up everywhere else yeah we ran in the climate GPT we we did run specialized U benchmarks so we created like a clima bench augmentation we created a miss and disinformation on climate Benchmark um

114. 56:50

     but then also we ran when we did the fine tuning against the standard benchmarks to ensure the model wasn't getting degraded um so we see I I guess I'm I'm I'm heartened to hear that because oftentimes I I wake up and kind of think well it's kind of hopeless in a way that because the foundational models are that much better um but in the end if you see promise with specialization then there's certainly a lot more Economic Opportunity for for people to I think the way to think about it is

115. 57:16

     economically the specialized model like it can be smaller for the same performance on specific right and so it's more economical like even if forget about closed or open it's economical to run a specialized model that's smaller on Specialized task than to run a gigantic model everything for like you know nailing the uh hammering the nail so that's kind of the the logic here so that like from just efficiency and energy perspective right that makes

116. 57:46

     sense now like where that's what I'm saying like then you need a framework around like how to route specific requests but that's uh that's indeed a solvable problem kind of there's a lot of innovation happening there all right I think we're over the hour now so I just wanted to kind of say thank you for coming uh great to see you and hopefully the audience enjoyed the conversation I definitely did uh Alex

117. 58:11

     really needs to get uh maybe it's better internet and uh yeah I I hope uh people will tune in for next time as we GNA continue this series uh with more amazing people to join thank you thank you thanks