NEAR AI Office Hours #1 The Importance of Transparency and Reproducibility in AI Models

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   done doing a start. Just a quick intro. What are you working on? So. Yeah. Good to to meet everyone here. some of the familiar faces I definitely seen. And they are definitely the first time I have ever met you, previously just came across with you on on, you summer. So. Hi, guys, I'm Dylan pond.

2. 0:15

   so we are building the, a chain, using blockchain on chain data. So we see it as a foundation model, because in a day, you know, the most crypto native data structure, 100% not language data view center, the entire object space, entire chain data.

3. 0:28

   It's an open graph network. And we are the locked in a model and also, model technique, to connect different other models. So for example language model or tree models. So decentralize this model development process. So adding a data song kind of a what I developing at a moment. Don do you want to go next. Sure. Hey everybody.

4. 0:46

   Some of you I know pretty well, some of you first time meeting you, but I'm Don. I'm the co-founder CEO of. Never mind. We're building, I payments protocol. So the way we see things, we believe we're witnessing the rise of a new consumer, trillions of AI agents, and that these agents are going to change commerce forever.

5. 1:13

   so much so that we're branding this new evolution of commerce, AI commerce. and the way we see ourselves is sort of as the, PayPal of AI and that engine, it's going to power AI commerce going forward. really looking forward to the conversation today.

6. 1:35

   Thank you for inviting me. And also looking forward to collaborating with near, as we push the envelope on AI and Web3. Jasper. Hey everyone, I'm Jasper, I'm the CEO and co-founder of hyperbolic. I graduated with, from Peking University with the highest honor student of the year.

7. 1:55

   And then I finished my math in Berkeley in just two years, which make me the fastest person to finish a PhD in the history of Berkeley, and also gave me the idea of the the name of my future company, hyperbolic, because I study hyperbolic geometry and topology.

8. 2:10

   I won several gold medals in various mathematical competitions, and after graduation I work at Citadel Securities and Ava labs. Before founding hyperbolic, I have Bolick. We have the mission to build a decentralized AI cloud to enable open access to AI, where achieving this goal by solving three major problems.

9. 2:29

   First, system. We're building a scalable system to aggregate global GPU compute and develop AI services to maximize the performance of any kind of GPUs, including Nvidia GPU, AMD GPUs, and other GPUs. Second, verification. We want to make sure the outputs coming from the network are run on the correct AI model. And lastly, privacy.

10. 2:50

    We want to protect users data and privacy, even when the requests are handled by a random third party. No. Awesome. So yeah, I mean, Amelia, we have Alex here as well. we've worked on some of AI stuff before. but, yeah, really excited to have you here.

11. 3:07

    And, Yeah, I mean, I kind of wanted to kick it off, to talk a little bit about, what we're seeing in the news. You know, this week been kind of, you know, wave of new announcement, curious. What what do you guys think? And, I think it's interesting to contrast that with what we were working on. Right. And and a more open source AI space. Right.

12. 3:31

    how do you guys think about it or what questions do you think are, you know, we should be we should be discussing the intent of, considering. Yeah, actually, I have a I have a thought on, the comparison of open source and closed source. So, on one hand, I feel like, you know, we criticize opening.

13. 3:49

    I put a lot because we have a closed source, model architecture for end of the day. I'm also thinking on a from a business perspective. So pretty much all the, you know, business model I've seen, you know, for example, all the open source models like Lama three or etc., you know, the main focus of them, the reason why they can, open source is because their business model is not for the model itself is really for,

14. 4:10

    you know, maybe you might have to want to onboard as many developers, a contributor to the, Facebook system, for example, like so that's in that case, it makes sense for them to open source the, their model. But on the other hand, I've been thinking like, you know, SDR, like when we are talking about open source on a closed source, what we are really talking about, is there is a kind of like a amount of political correctness, you know, correctness or is more like something we feel like this entire Web3 needs that this will be open source so we can provide the transparency.

15. 4:43

    And also, I understand lots of different projects that are working on, you know, providing verification of, closed, source models, for example. I'm generally just curious, like to learn more about, different people's opinions because, you know, as a model business for OpenAI, you know, this entire business model is built on top of model.

16. 5:02

    so the model, they have to protect, just in case if. They're open source, it they probably. You know, the evaluation will go to zero immediately overnight. so this is kind of my, thoughts on this is I've been thinking this quite a lot, like when we are talking about open source and closed source, is this just a political view or we think, this is the right approach to open source everything?

17. 5:25

    Yeah, I mean, I can I can kick it off, because I've been thinking about it a lot as well. I think there's, few, few kind of dimensions to think about. Right? One is, I call I mean, there's actually a scientific paper, called Sleeper agents, but the one is like, what is actually open source?

18. 5:45

    Yeah. Right. Actually, what we have right now is not really an open source. It's open parameters. It's actually as if you got an executable file from a producer and they said, you know, we we did everything correctly. This is a good model.

19. 5:59

    Just use it. Right. So, I actually think there is a need for knowing what data went in into the model to really be able to, kind of verify that this is indeed, the model that kind of represents the, decisions that you are looking for.

20. 6:18

    And this is spans from kind of when I work at Google, we would make decisions on what data to not include. Right. because like on the internet, there's all kinds of crap, you know, spam, but also just misinformation, you know, conspiracy theories, etc..

21. 6:35

    And so there is a decision somewhere to like, oh, we don't including Q and on websites or we not including Alex Jones websites or, you know, do we include true social or not and stuff like that. Right. And like as you make those decisions, you kind of affecting the outcome. And, and the other way works as well. Right?

22. 6:51

    If you want to affect the outcome, you will start, you know, poisoning your data in such a way that when somebody uses it in a specific context, the model will output something. And it's really hard to test set. So so yeah, encourage to check out the sleeper agent paper actually, camera and share it with me.

23. 7:08

    So so that's kind of piece number one is like the problem is because you don't know what data went into it, even if the model is open, but especially if it's if you just, you know, using an API, you have no idea what you're actually interacting with.

24. 7:22

    Now, on a business model side, I mean, I don't think it's, true to say that if you open source your code, then immediately your value goes to zero, right? I mean, for example, blockchain source code is open. Anybody can for kids like you can spin off and near fork right now while we talking, but that doesn't make neopixel go to zero because there's brand.

25. 7:46

    There's kind of users adopting, there's network effects, etc.. And I think what we see was open AI similar thing like right now even though there's alternatives and you know, like it's kind of hard to say now which model like yes, GPT four generally is better.

26. 8:02

    But you know, in some things cloud is bad. In some things, Gemini is better. It's it's hard to say that that. Okay, well, now everybody's switched away from it because there is such a like brand awareness of this. And I mean, this is hypothesis obviously.

27. 8:20

    But if they have open the model potentially there would have been a lot more people building on top of it. And kind of contributing back, to it. Right? What we've seen was Lama so that's kind of my opinion. one of the reasons they kind of, sometimes use is that like, oh, it's dangerous for people to have this model.

28. 8:41

    And I think that the specific reason that I want to call out because, like, I think it's kind of a weird way to, to justify your business decision again, like, as a business decision to the, you know, it is a decision to for them to make.

29. 8:56

    but the dangerous specifically is seems misguided because a, I mean, they're not a military operation, right? If somebody wants to break in and actually steal the weights, it's not like that hard and be like at the end and, you know, you're able to kind of get to this level if you really need to, raise enough investment.

30. 9:17

    And so and I think kind of the this is where we get into Web3, I think the we should be preventing whatever the AI models are possibly can in a dangerous way, do we should be creating systems that are protecting against that versus trying to pretty much limit what you can do as, as models.

31. 9:38

    But I mean, just for you working on verifiable compute. So I'm sure I'm thinking a lot about this. Yeah, I actually just posted Twitter post yesterday talking about this. So. if you look at the news like Gemini, right, like people criticize about the pictures that the AI model generated, like it's about races, it's about genders.

32. 9:59

    And, and the same thing happened on OpenAI and other AI models. So I actually think, if you don't open, you don't you don't make all the, all these I like transparent and verifiable. It's really hard to see what kind of data or like what how people what bias that people inject into this model, starting from what Elliot mentioned.

33. 10:24

    Right. The data like if you have, you can basically poison the data. And then because AI model is a black box, you never know what kind of data is, are fit into this model. So, transparent data and like maybe some verification method to make sure the data is correct is necessary.

34. 10:45

    and second, I think the Chinese process also matters because, everyone, if you train a model, you basically are, training, changing the weights, and it's also undetectable. So, so you need people need to figure out a way to verify that the training processes actually follow the standard process that people try to train the models.

35. 11:07

    So I, I call improve our training. And then also, if you. Have feedback. That human provide, you also want to make sure that the feedback are correct. And we can call it improve our human feedback. And when you run the model, you want to make sure that, the results are actually correctly run on the model, which is something that we're working on right now, and we call it like proof of inference.

36. 11:32

    And in general, this is more like a proof of intelligence concept. You want to make sure that the intelligent that the AI, artificial intelligence is actually correctly trained and correctly run, by following the instructions that the AI models have.

37. 11:51

    so I think speaking of open source, closed source, I would tend to think like open source as not only just as, like open source. The wait is more about open source, the whole process. And once you make the whole process transparent, you can reduce the bias.

38. 12:12

    You can make sure everyone have access to AI. That's the the second part, because we think everyone should have free to enough, artificial intelligence, just like everyone, have the freedom of speaking. Everyone have the freedom to publish in a book you remember before. Like people are allowed to bring books. Everyone need to read Bible, right?

39. 12:35

    And now everyone can have different opinions. And that's kind of motivate the society to move forward. So the same for AI. If we only let one company or like a few companies control the AI models, then all the inputs that other people will have will come from these.

40. 12:53

    I wish you basically come from the values from those companies, and that's not good for our human society. So in general, I think we need to make the whole AI pipeline more open, transparent and verifiable. And the second question for this business model, I totally, totally agree with what it is that like the, open source doesn't mean, you basically give out all the values you have, is more about the mindset.

41. 13:21

    Like if you closed source, then basically you are building a SaaS model. So you need to sell the service to people. Right? But if you open source, it is more about that. You try to build an open ecosystem. That's the reason why blockchains and decentralize ization makes sense.

42. 13:40

    Because, when you open source the code, people can build and can help you to improve the security of the blockchain and, like, make sure that it's more performant and everyone on piece of this ecosystem. So everyone has the motivation to stay and keep improving it.

43. 13:58

    So if we can use decentralization to make, build a decentralized AI ecosystem so that everyone will own piece of this ecosystem, that everyone have the motivation to keep improving this ecosystem so that we can achieve AGI on decentralization.

44. 14:17

    So that's how I feel about open source. And, close source. Yes. It's my turn to add something. Yeah, not a ton more to add. I mean, I, I agree with what Jasper just said. I don't think, an any coherent, cogent business model is predicated on being closed source.

45. 14:39

    you know, I think there's, thousands of examples of highly functioning, extremely productive organizations that are built on open source technology. I mean, two that come to mind off the top of my head, Databricks with spark, confluence of Kafka.

46. 15:00

    You know, there's these are organizations that are highly profitable that have, you know, they're revolutionizing even the AI space with Databricks. Right. so, I mean, I'm also super biased. I am like open source through and through, screwed closed source.

47. 15:17

    you know, to your point, Jasper, I think the point here is, you know, we're all in the Web3 space because we recognize the power of community and networks, and that's what open source does. Right? It is that rallying point around the technology side that brings in people capital, effort to build out these projects.

48. 15:42

    And by doing so, you know, raises up these communities that, through thick and thin are gonna, I think prevail. industry has demonstrated that. I mean, I'm going to date myself, but when I started working in enterprise, almost 25 years ago, I remember, being in a meeting where we were talking about implementing and, new database architecture and bringing up open source and being laughed out of the room as a junior guy.

49. 16:16

    Like, what an idiot this kid is. There's no way we're using open source in a in a enterprise setting. And now, you know, that is the predominant architecture and software in use at most enterprises. because. Take taking risk if they are using closed source, because if that company goes under.

50. 16:37

    Exactly that vendor lock in is now what used to be perceived as an advantage is actually now perceived as a risk. And I think we all recognize this. Right. So, I'm of the opinion I don't know why, as an AI, especially on the model side, when, you know, the reality is, regardless of the sophistication of the model, you know, this GPT four or have or the GPT theories have, a performance benefit over like, the Lambda series?

51. 17:12

    yeah, maybe a small one, but the reality is, given the same context, the inputs, they're going to converge to the same type of inference. you know, there's closed sourcing that that training capability really provide a massive benefit over the long term.

52. 17:28

    Probably not. so and I don't know if you listen to, like, Sam Altman now, he seems to maybe be changing his tune on the open source code source side, at least in some of the the conversations more recently. Time will tell, with how open AI goes.

53. 17:47

    But yeah, I think my opinion open source will dominate. I think there's also the aspect of, decentralization. Right. Because they're not, they can't be mutually exclusive here. I think this is an avenue that we need to figure out.

54. 18:06

    And to your point, Ilya. Yeah. On the question, like, what are we seeing, especially this week that, OpenAI etc. are doing they're making a very concerted effort to package all of this stuff together and make it super easy to use.

55. 18:26

    And to product play. Right? I mean, in a way like the models are becoming more of a commodity, right? Yeah, I mean, exactly. There's a race. On on it, a. yeah. Yeah. But at the same time, the products that you build on top of it or the way you create the experience is what becomes really important.

56. 18:48

    I actually, I mean, zooming in into this because, I mean, you guys are all kind of building in a little bit different, dimensions. Right? And as well, what Jasper was talking about. Right. We want a way to get kind of a reproducible, you know, open source model from sources.

57. 19:09

    The thing I've been thinking about is, like, how do we test, how do we test the models? How do we test the training of this model? How do we ensure that indeed it's, you know, correct. or like if you retrain the new model or if you fine tune it, did I get things?

58. 19:29

    So I mean, I'm curious kind of diving into this because I feel it's probably one of the more important topics as the space is evolving, and especially in blockchain interactions where you have adversarial kind of, you know, especially if you're dealing with payments and money, like, you, Don Ryan, like, it's, you know, people will try to find ways to kind of exploit it.

59. 19:55

    So I'm curious what you guys think. I have some thoughts of all, I mean, Alex also in work and some of the stuff. I mean, on that point, maybe it's less with respect. Our focus is less with respect to like, the training and, and more related to the utility of the inference.

60. 20:16

    And obviously there once you start paying for these things, in particular AI agents in their outputs, there's going to be questions around, was the model that was advertised actually run? Is the inference correct? so in that case, to, to Jasper's point, having fidelity and transparency on what those inputs are, or at least, an ability to reconcile, the inputs, either, you know, from a, from a training or a context point of view to the output I think is important.

61. 20:54

    both with respect to what am I receiving, but also with respect to attribution. Right. Like if the source information was scraped from the internet, consumed, used in training purposes and then, is output it in the likeness of a particular artist or something along those lines.

62. 21:16

    I think having the fidelity on that, provenance trail to the point where you can properly, prescribe attribution to those inputs so that when you commercialize that output, it may or may not flow back to in some capacity that original input and, and that creator of that input.

63. 21:37

    I think that's something that also needs to be reflected, as maybe not part of the training, but definitely part of the output. and then from our point of view, you know, things that are important, we look at like, why do we need a decentral AIS protocol for the the payments piece?

64. 21:58

    Right. as we move towards this state, of semi autonomous and autonomous agents, I think there's going to be like there's potentially going to be a mandate from those agents that this payments piece is decentralized because I'm thinking about, you know, the the censorship aspect of this, from a strictly payments point of view, if the payments piece is controlled by, you know, a handful of organizations or, you know, OpenAI, Microsoft and Google, Facebook,

65. 22:37

    and those independent autonomous agents are outputting inferences that are incompatible with the business model of those underlying organizations that control the payment space. The best way to censor those agents is to just shut off the payments piece.

66. 22:57

    Doesn't matter if the rest of that AI agent is built on, you know, decentralized technology, if they lack the ability to transact with other agents, that payments piece becomes the the ultimate choke point. and so, you know, I think it's a complicated question.

67. 23:16

    La, you know, because there are aspects where you can, you know, try to discern the inputs versus the outputs, etc. but when you cobble all of these things together as, an agent, there's a whole bunch of different components that you need to answer with in conjunction with the verifiability and the veracity of the output. But yeah, I can share my thoughts as well.

68. 23:43

    So I technically, I think, what you try to solve is like, let's say you, you train, you train on this model and then you want other people to reproduce the same training process. Right. And I think there are two, so this is basically the verification mechanism.

69. 24:04

    You want to make sure that the training process is verified. there are two things. So one is like that reproducibility and determination of these floating point calculation or even like, like GPU, calculate concurrent computations.

70. 24:20

    And second is more about like the high level verification mechanism design, which kind of mechanism you want to use to, to verify to and punish the people that don't, don't do the right things. so first for the reproducibility problem, we did some experiments currently and we can show that, if you use the same GPU, if you use the same Cuda version, actually the result will be the same.

71. 24:48

    So the floating point issue won't be a problem. But if you use different Cuda version, if you use different GPUs, we and the result will be slightly different because of, the floating point calculations. I know people trying to do like quantization to make sure that the results are deterministic.

72. 25:08

    But however, in terms of relating to training, you need floating point. So, maybe people can come up with a way to use, like, integers to, to train a model. But currently most people just use floating points. So, it needs some work to actually make sure that the floating point calculations, it's, is deterministic, on different GPUs.

73. 25:35

    we're working on that. I know a lot of other research groups are looking at that, and hopefully we can solve that very soon. second is more about, verification mechanism design. so, so there are like there is a few mechanisms like consensus, SSC maybe like optimistic mechanism.

74. 25:58

    we also invent a new one. I will share it a little bit later. so for consensus, I think, in theory actually work, right. You basically just ask ten nodes and then each node will train the model on the itself, and then they will they just vote on, if majority of them are honest.

75. 26:17

    And by having the reproducibility, then the majority of the AI models will be the same. Then you just vote it, just pick the majority votes of the the result. Right. And then for optimizing mechanism, you can say like okay, I like one node to train is model.

76. 26:36

    And then other people can challenge them within like a challenge may not let's say seven days. So if another node retrain the model and then figure out, okay, these two models are actually not the same, then they can break down the training process into small pieces and then you just compare piece by piece, and you can even break it down, further and finally generate a fraud proof that can adapt blockchain to execute, to arbitrate.

77. 27:06

    so, so these are like the two to mechanisms I think probably makes sense. But it, it that's like for consensus. It caused a lot of overhead. So I don't think it's very practical, especially for the big models. You probably can train a random forest, like you can train a linear regression model. by doing this. I think that makes sense.

78. 27:29

    for Optimus and Optimus mechanism, you need to, the they doesn't have pure strategy. Nash equilibrium. Which means that the incentive that everyone will need, at least one person to to verify that, the probability is actually not 100%.

79. 27:48

    So it means that sometimes, when you train a model, actually, no one would verify, although your model training process is wrong. so we invented a new one called proof of sampling. So and, so in general is more like, one general result.

80. 28:05

    And then with a certain probability, the system were randomly assigned another note to check the result. And then if the results are different, it will initiate an arbitration process and the dishonest all will be punished. so, so I think, like you need to, in order to actually solve this, like transparency and verification, verifiability of chaining process.

81. 28:29

    You need these two things like reproducibility and verification mechanism. I think I can share some source from the model perspective, on my own. So pretty much I, you know, the way I think in this day. So on one hand, we are developing our own model. You know, this is the entire blockchains of graph data.

82. 28:46

    So we assign properties to different wallets and, different interactions, you know, nodes and edges. Also we're trying to this model, from learning the behaviors and learning the consequences of different behaviors. And it our perturbed behavior is based on the, the current behaviors of the market or the world.

83. 29:01

    So on one hand we are developing model ourself on and a hand is we have a technique to connect with different other models. So verification means a lot. And on the other hand we'll be thinking like developing a very generalized approach can be very hard. except, you know, maybe potentially working with Jasper, for example.

84. 29:18

    But you know what we are thinking, what our thinking process is, you know, rather than verifying, because I end of the day, averaging, it's a large model, rather than verifying what we do is how we learn from the results. Because in a day, like, we know how powerful the, for example, is or how powerful Arma three is, is from using it is found a result. Right.

85. 29:39

    So, for example, what we're doing with our model is because we are producing behaviors and we develop some metrics that are learning some consequences. For example, the price change of the on chain tokens is is from the a behavioral change. So we learn like how accurate we can predict a token price.

86. 29:55

    And what's a correlation. accuracy that we can do a lot from our model. So if you know, from a period of time then the model performs pretty well. It means that the model, the states. Right. because it can, you know, develop, accuracy in that way.

87. 30:09

    And also for, other models like, you know, I think there are 1000 possibility to, to prove that, people are doing things right. But, you know, this is just one possibility, only that, if the result is one, which means a model is wrong. So, this is a kind of our thinking process.

88. 30:28

    Yeah. Of approaching things at the moment. That's that's pretty cool. I, I think like this, this kind of echo with what I think about verification because I believe, the world of verification mechanism are the world of trust is heterogeneous.

89. 30:46

    People think used to think consensus is the only way to to build trust in decentralization. Actually turns out that like there's a, a tons of different methods that you can build trust and do verification. It incentivization. And in the future people will come up with different ways to verify the result, verify the models.

90. 31:05

    And as a user, you basically can do the trade off between the performance and security. Sometimes you want better performance. You would choose like verification mechanism that's more performance, focused. And then if we really want security, maybe like it's involve money transaction, then he would definitely want the high security verification mechanism.

91. 31:27

    Yeah. I mean, just to play devil's advocate that I mean, this is more of a question. I don't know the answer to this, but that's a you raise a good point, Jasper. There's the heterogenous nature of trust and security and the world that I'm looking at right now with these models, people are actively paying for them. Right?

92. 31:49

    I think it looking at the requirements and demand from the consumer side, there's just this innate trust. And what I'm getting is, you know, trustworthy and validated by OpenAI or whoever. But but that's kind of the current state where at least OpenAI is themselves. Don't expect you to move money around based on decisions of the model.

93. 32:15

    Right? Like there's. Yeah, I mean, in the loop there. Who decides kind of on that. But so I guess my question is like, when do we think we're going to cross this real threshold where that that driver will be there? Right. Because the way that we're doing things right now is we're kind of separating church and state.

94. 32:35

    We make sure that the payment, the settlement piece for the component that you use to interact with the agent has high fidelity to it. But then there is this abstraction, because, you know, the compute piece is taking place off chain, where that verification and validation is, opaque.

95. 32:57

    Right? Yeah. and it doesn't seem to be huge to turn at the moment. However, at some point we're kind of, I think, all trying to front run this scenario where that does become a requirement. I'm just wondering, you know, if the if everybody on here is thought about what's the timeliness for this, when does that requirement actually manifest at least manifest at scale too.

96. 33:23

    Yeah. I mean I think it's it really depends, right. Because it's the same as, you know, as a company you trust that, you know, your Silicon Valley bank will be there and you started money there, that account. And then, you know, it's all fine for I mean, 20 years until it's not.

97. 33:42

    And so I think the challenge here is kind of similar, except I mean the I everything on steroids. So I think this kind of stuff will happen on steroids here as well. I mean, it's everything is from a disgruntled employee or like, you know, it's, it's a one if statement in the, in the code base.

98. 34:01

    Right. And now, it's, it's making different decisions based on, you know, the, suffix of your email. Yeah. Or a disgruntled Devin that you're, you know, paying a hundred bucks for 15 minutes worth of dev time all of a sudden, you know, starts, maliciously editing your code.

99. 34:22

    Yeah, but but I think, like, I mean, the way I think about it and, because there's like, few different aspects of this. Right? Which is like, how do we know the model is good and right, which there's like few pieces around that.

100. 34:36

     And then when you get an inference, how do you know it came from this model. Right. And so it starts indeed was like, okay, well what data goes into this switch right now again for ChatGPT even for Lama, we don't know. and so one of the things that like, it's actually really hard to debug these models because if you don't know what data went into them, like, the way to debug them

101. 35:00

     properly is to have a backdrop from the output that you consider incorrect on training data, to see what training data have affected the most, those kind of outputs. Right? And so without training data, you actually are not able to debug. Like what are the, kind of most offending data points that led to this.

102. 35:19

     And you definitely not being able to like, kind of, tune it. For example, if there's some, some specific examples that you want to kind of hit unlearn, there is a then there's okay, well, let's imagine somebody does put out a data.

103. 35:35

     How do you actually validate it if it takes like, you know, hundreds of thousands of GPU hours to build this model, you definitely cannot expect, you know, X number of other participants to reproduce that run. And then, you know, you probably don't want even a single other party to reproduce around because so expensive.

104. 35:55

     so one of the ideas is indeed, you know, chunk it in like so right now, people can not just release the final parameters, but actually a checkpoints, intermediate checkpoints when you're training, right? For example, every every hour, or every ten minutes and then and then now you just need people to validate like each independent party can validate one checkpoint.

105. 36:18

     Right. And greenlighted. And so you kind of validate every transition. But it doesn't need to be that every kind of single party validates all of them. Right. You just need enough people to validate different transitions. And this is actually where I'm interested.

106. 36:31

     If anybody is building a, economy around that, because this can actually be incentivized to, you know, so there's different parties contributing validation for specific checkpoints. It does require solving the reproducibility of, of GPU computation.

107. 36:48

     But, I think that's an interesting kind of, nugget. Then what you mentioned, Dylan, which is like, okay, well, I got the model, but like, is it actually working toward my expectations? Right. So what is the evaluation? And this is one, one thing that I'm kind of also struggling with is like there's a lot of kind of benchmarks, right? There's more you there's all this stuff.

108. 37:09

     But like the all kind of first of all, they're all static right there. Like they're not evolving. They're not adding. They usually have like, I think I saw one of the paper had like 600 different benchmarks, right. Which is like, it's actually very expensive to evaluate a model on that on a benchmark takes like I think six hours to, to do just evals.

109. 37:34

     So how do we actually like have benchmarks in such a way, and how do we do benchmarking across all those models like is 70 B model like I'm assuming to be better than, I don't know, some deep coder AB coding? Like how do I know this is it's three benches, this bench like so many benchmarks.

110. 37:53

     So I think that's a really interesting I know people who are building different products, just building their own benchmarks for their own use cases. Right. And usually but then there's no like good tooling around that. So like how do we how do we build that.

111. 38:08

     This is actually something where, we're looking at right now is Alex. And then and then you go to what Jasper was looking at especially is like, okay, well, when you do evaluate the model, how do you know this is the right, the right output from right model.

112. 38:24

     And again, like there's so many things that can go wrong right from, you know, it's just different parameters that get loaded to, you know, somebody maliciously just like related tokens to blacklisted tokens to all kinds of stuff, right.

113. 38:39

     To even, you know, additional additional injections in the context that you didn't have. Right? I mean, literally, you're sending to API, it can do whatever you want. They can inject context for you that, you didn't you didn't have in mind. So, like, how do we identify and kind of, manage that?

114. 38:56

     I mean, the idea of, like, you know, there's always probability that you will be check. So, so that kind of gives good guarantees unless there's like a massive collusion of partition and network. so I think like, that's kind of the whole flow needs to be indeed like kind of thought through. We have eval so we have kind of probability.

115. 39:18

     We have checking evaluate like kind of a checking protocol in a way for even open source models, fully open source with training loop. And then we going to. Yeah. Like when somebody is trying to maliciously now attack this now at least you can debug like, okay, well, you know, this is why it led to this.

116. 39:39

     Like there is a training data and Reddit that somebody is like specifically posted so that later, you know, when it became into training of the model, it actually affected the decision making. So you can actually start, understanding where some of this data kind of injections happened, which like I'm assuming people are doing them because like, it doesn't cost you anything to do that.

117. 40:03

     And again, if you don't know what training data is, you're not going to be able to like detect that. Totally, totally agree. And, I totally I think, evaluation is definitely, necessary. And, a lot of people come up with, ad hoc benchmarks, but the, the data like, because it's fixed.

118. 40:26

     So there is always a way to train the model. To fine tune it on the training value. Yeah. So, are you guys familiar with our AMCs and chatbot arena like the. Yeah. So I think their philosophies, the, the best way to evaluate and model AI model, is based on human feedback.

119. 40:47

     So every time when the user send a prompt, the, the app will randomly select two anonymous models and then generate the results and say same back to the users. And the users can pick which model is better. And once you have you have collected like millions of users data.

120. 41:06

     Then you can basically, do a iterating to decide which one is the best, which one is not. So that's how GPT for all, found out that they actually are the best right now on the chatbot arena. I think this is something that we are very, very interesting as well, because like when, when you try to as because we try to, solve the inference problem and when, when you want to, like, provide these AI models, we feel people actually

121. 41:37

     want to know, like which model is the best for marketing, which model is bad, best for co-writing, which model is best for like math, etc.? So having this evaluation service is is a way to kind of let people know what models they want.

122. 41:53

     we actually, partner with LMS. This was supporting their, like model, running support, then to run models, to serve people. And we think a better way to improve this is through a token, because right now users cannot make any money by providing the feedback to that.

123. 42:14

     Right. And, I mean, the philosophy is if you contribute, then definitely you need to own or make some money out of it. so if we can use token to incentivize people to provide feedback to a model, to, to this platform, then it will have we'll have more data than what we have right now, and we can have a better idea about how the, how great the model is.

124. 42:41

     And even further, if you can actually evaluate the performance of an AI model within a few hours, then basically you will like speed up the iteration loop. And every day if you come up with the idea and then you actually know a, I didn't know whether it training method makes sense or not, and you can accelerate, the improvement of AI towards AGI.

125. 43:06

     and I have a question. So, you talk about the data, Ilya, what is the best way to check the data quality at the moment? Because, like you said, you want to make sure that no one will, try to poison the data, try to poison the model. And what's the best way to check?

126. 43:24

     Right? Yeah. I mean, I don't know if there's, like, a best way. there's like, few different heuristics you can use, to analyze the data set. the one interesting heuristic is actually by taking a model, let's say you take lemma three, and you actually run it over the data set.

127. 43:47

     So pretty much what you would do during training. But you can now see and look at the loss and identify documents which have like high loss. This is usually either spam or like it's pretty much can be one of three things. It's spam.

128. 44:07

     It's something that's like completely new to the model or it's something that's like weird, right. And so so this is like things to look at is like, what are the things where model is like really surprised by like already well trained model surprised by there's also things like where the loss is very low, which probably means it's, like not informative at all.

129. 44:30

     You can clean that up because this is not the tokens you're going to benefit from. it's empty calories. So yeah, you can use literally the model now to score your data and understand where the, kind of information is. There is also like on the document level itself, this things that are just like unpredictable. Right.

130. 44:50

     So an example is like if you're looking at at like, New York Times article or whatever, we're not using New York Times. Right? So whatever I read an article or something, you cannot predict username of the of the user. Right. That's writing it.

131. 45:05

     You cannot predict, you know, few few things like that. So it actually makes sense to cap them or, placeholders and, or like to pretty much modify data set to not included or, you know, in the loss, cap it out and stuff like this.

132. 45:23

     It's just like there are places where model will always stumble, because if I give you this, you will not be like, even with all the knowledge in the world, you'll not be able to predict it. So. So there's a lot of just things like that where as you dig in into data, you can actually start understanding those how to make models better, how to, reduce some of the hallucination because like you forcing the model to like decide on predicting something that it will never have an option to do.

133. 45:51

     So you can give it option to like, give, you know, a placeholder, for example, that when it doesn't know, for things that you, you don't expect it to ever know and like that. Now there's more basic stuff. I mean, there's variety of like PageRank style, information that you can use, right?

134. 46:12

     if you have a common crawl, you can like, you know, parse out URLs, build a form of PageRank, and then, this is a pages that nobody links to probably are not as interesting. although probably coming crawl has less of that. and then there is an interesting question of like trust trustful like trust and, truth.

135. 46:36

     Right. and this is where things get complicated, right? so the, the example I like to use is Barack Obama is born in Kenya. If you Google that explicit phrase, right. there's a ton of websites which talk about it because this was a big conspiracy and people love talking about conspiracy.

136. 46:56

     And so how do you decide if you want to include that or not? Is that's becomes really complicated. And this is where I actually think the data curation should be at the community level, where, you know, some people will decide to include this subset of, you know, information about this and you kind of have non-uniform distribution of truth fullness in the way or like subjective view if this data should be included, not, and so I think those are interesting.

137. 47:28

     Like again, from a, from a decentralized like community based perspective, you can imagine people like kind of almost, curating different data sets, right, to train on depending on the country, depending on community, depending on worldviews and then having kind of different modifications of the model.

138. 47:49

     That's, have different biases in the way, based on that. Just, dig into this a little bit. I mean, looking at this, there's the intricacies of what we're talking about, but then there's the application of it all and the consumption of the output, as well as I think Jasper, you you bring up a really good point again, around the attribution side and like collectivism and the community helping to train these solutions collectively.

139. 48:30

     my concern is like the Web3 community, especially at this AI cross section, is doing itself a bit of a disservice. the, the the web two AI space hates crypto. I think that's pretty you know, you talk to people that are building AI soon, as I mentioned edge crypto based you they're like I yeah, you know, hang up the phone.

140. 48:59

     so I think, you know, we need to make a real concerted effort to bridge that divide. Right. And I think that some of this stems from the architectural decisions that we make as well as like the application decisions we make. So from our point of view, one of the things that's, you know, of significant importance to us is, account abstraction, right?

141. 49:25

     And things like gas payments, snow pop ups, things like that. I think, what would be incredibly beneficial as a community is if and I think, you know, this is why we're all here, having an L1 that is like entirely focused on the success of AI and supporting the, the, the functionality side, the the ease of use component would go, I think.

142. 49:57

     Yeah, I mean, it would pay massive dividends in both the mid and the long term. la Alex Cameron I don't know if there's stuff that you guys can talk to on these points. I know, you know, there's there's network abstraction and things like that.

143. 50:13

     But are you guys seeing the same sort of thing. And then what can we do collectively to like try and minimize this point of friction, of trying to get the non Web3 AI builders to start to look at what we're doing, because I think we bring a lot to the table.

144. 50:32

     we, we've discussed a lot of it here that's going to be massively beneficial to the overall AI ecosystem. Yeah I mean, I think like from our side there's like few things. One is I mean, the protocol itself is designed to be kind of as user friendly already as humanly possible.

145. 50:49

     I mean, that's why Nir has kind of the most consumer users, on our side at the same time, you know, we obviously open to work on the requirements that people have to to keep adding that kind of and making. It more. Accessible. And then on the other side is pretty much as you said, it's it's kind of for us right now to prove out that this is not just, you know, regular crypto, but actually, like, hey, here's the real use cases of

146. 51:22

     using this to, to benefit the, like AI developers. So as a whole. Right? You know, either, for example, you know, Jasper's trying to expand how much compute you can access, right? And like, what types of computer can access, or, you know, how do we coordinate more agents, etc..

147. 51:45

     So like, how do we actually bring real, value like we have, you know, crowdsource data sets that are unique and not accessible. And we can do this because we can coordinate like, payments a lot more effectively, and have people around the world actually participating in data labeling.

148. 52:02

     so a lot of it is just like show the real value. I think crypto has generally been like sell sells, sells, devalues, sells the vision, right? And then not show the not actually show the actual value. Right. And like, have, you know, a thousand users. But now it's time to like show the value and then explain the values.

149. 52:23

     Right. So so I think that's kind of, you know, what we should be what we should be leading ways is like. Hey, hey man, really. Really awesome data set here is like, you know, you need compute to do inference. This is actually cheaper per token, right?

150. 52:37

     Because we use all of the compute humanly possible or you know, you need just better like way to interact with there's all you need like micropayments or you need, you know, spin up this, like, system and, you know, you know, it's going to be expensive.

151. 52:54

     How do you get users to pay for it? Well, if you set up all of the payments and everything yourself, you're going to like, be, you know, it's super complicated. This is easier to do here. So I think that that's kind of the leading here.

152. 53:05

     And then like we pave the crypto way underneath. where we're almost at time. So, I want it to start, rounding. Yeah. But I mean, thanks for thanks for joining. hopefully this was good. Again, we're testing the format, but, I was pretty excited to see you and to chat about this. hopefully.

153. 53:30

     Yeah, we'll continue doing more of this and we'll have, more of, kind of, conversations, as well and the more technical side as well. So sounds good. Thanks for inviting me. Awesome. Let's ship it.