The Real Cost of Intelligence

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:10

   Oh, hey. Uh, hi everyone. I'm Jasper. I'm the CEO of Hyperbolic. Uh, we're building the, uh, the biggest GP marketplace and market maker in the space. Hey everyone, I'm Jordan from Public AI, uh, co-founder and chief marketing officer. And what we're doing is building the human layer of AI. So connecting human intelligence and expertise to AI systems. And we do that with a tasking site where people can come and upload their data for training

2. 0:38

   models. And we make it equitable for them so they get paid. And our clients love it because everything is consensually collected. So it's all opted in. And no legal red tape to train on our data. Hey everyone, I'm Ben, co-founder and CEO of Jensen. Uh we're building the network for machine intelligence. uh we connect all the resources that machine learning needs basically to run uh entirely programmatically. Uh so the idea is instead of having to go through human world infrastructure like legal

3. 1:06

   contracts and trust and things like that uh reputation you can do all of that programmatically access compute access data access information etc. Um this is kind of built obviously for humans to use but ultimately for machine learning models themselves to use. They can't really sign a contract with a human but they can do it programmatically. So as long as you connect up all the resources they need, they can start accessing them without any humans in the loop essentially.

4. 1:30

   Good stuff. All right. So the way we'll structure the first part is I have a question for each of our panelists, but I'd love to have the other panelists tune in to that question as well as they see relevant or fit. So the first question is actually for you Jasper. Um starting with the basics of how do you expect the cost of compute to evolve in the future and is there going to be a

5. 1:53

   sufficient supply of compute in the coming years for all of this increasing demand. Yeah, my answer is no. I mean GPU shortage will always always be there at least for a while. Uh the GPU requires like a lot of different uh like different components, right? Like the when we talking about GPU, we're not just talking about like the GPU hardware. is actually like access to GPU which require electricity, land, power uh and like construction and also like

6. 2:21

   the GPU components itself. Um so like there are so many bottlenecks uh in order to actually like set up the enough GPU resources for people. uh at the beginning it's like you can't get uh the amount of GPU that you want and later on like I think Jensen somehow solved that later on it's like the electricity and the land and now it's like memory and I think we're going to see so many

7. 2:45

   different bottleneck in the future uh besides that there's like a something called like Jav's paradox so like the the lower the cost of GPU actually people will start uh spending spending more resources on that and so naturally like uh the consumption will be much higher than what we can ever.

8. 3:03

   Any followup on that of how you expect the cost of compute to evolve in the future? Yeah, I think I I agree with Jasper. We're just going to keep kind of ripping through the compute that we have access to. Like the scaling laws still seem to like hold. Like the more compute you give to something, as long as you've got enough data, the more you can increase the power of the models. Um I think what we'll see is a really big focus on efficiency. We've seen it a lot already.

9. 3:27

   Um obviously, as Jasper mentioned, Nvidia are focused on efficiency of chips. etc. Um our view is that you can do a lot of efficiency at the like protocol and software layer. There's a lot of inefficiencies in the access to the compute itself. And so like for example accessing compute on a time basis in our view is a lot less efficient than a task basis. So when you execute a machine learning model ultimately you're running a set of

10. 3:49

    operations but right now a human has to kind of guess how long those operations are going to take and then they rent access to a GPU for the amount of time they think it's going to take. There's huge inefficiency in that. you don't need to do that estimation process if you can build a taskbased system. So if you can you can actually squeeze that efficiency really heavily and then get more out of the current compute but ultimately again as Jasper mentioned I think the the more we bring the cost down the more we make it more efficient

11. 4:13

    Jeb's paradox kicks in and the more we want to use until we hit some like limit on the scaling laws which I don't know that doesn't seem to be happening anytime soon. Jordan, anything on the cost of compute? We have a question coming up for you afterwards. So, I mean, I agree on the like compute will always be used if it's there. Like 100% people tear through as much as possible.

12. 4:36

    Hopefully, you can do more and more with it as you as we get cheaper compute and as open source makes it more possible to do like more with less um on that compute. So, we see the bottleneck is data, but we can go to the next question. And on that question, Jordan, I mean, you you've done a lot of work on the need for human input of data when training specific models. Is the real cost of intelligence, right, in the future bottleneck by compute or by the

13. 5:04

    need to access more original types of data? Yeah, when you're talking about getting models to do more um new things and to do old things better, um data is essentially the substrate of intelligence that allows those models to learn new things and do them for less expensive um and do them more reliably.

14. 5:23

    So that's where we see the real bottleneck is around data. And what we're doing is making sure that clients are getting what they need to go from, you know, a word error rate of 10 10% down to 2% for their real-time translation devices. And we do that by going to people around the world. We have a network of over 3.5 million contributors who can show up and get paid for speaking their native dialects

15. 5:47

    um and bring that intelligence into the models so that they can do the tests that they need to do. I think on that data topic, a lot of times people throw around words like synthetic data or they like to say things like most of the models are trained on Reddit or things like that.

16. 6:04

    Do you guys have any strong opinions on how close are we to needing significance amounts of synthetic data or uh you know are we going to be bottlenecked by scraping the internet in the future and when will we reach that? We'll start with you Jordan and then we can go over to uh uh Jasper and Ben. Cool. Yeah, I think we're we're past the era of scrape and prey where you can just like get everything off the internet and hope you don't get sued. Um there's over 30 active lawsuits at the moment where AI

17. 6:34

    labs are getting hit and are hitting that legal red tape. So I think that we're we're at the limit of like the scraping. um it still happens and it's useful when you're doing foundational model training just to get like the baseline set for a model, but when you're trying to get like eek out performance out of a model and training to do new things, that's where synthetic and scrape data really fall flat and the

18. 6:58

    real fresh human data makes a huge difference. Yeah, I think um I I definitely agree with the kind of hitting the limits of the scrape data. The way I like to think it of it is that we've bootstrapped machine learning on basically the knowledge that humanity's been curating over the course of the creation of the internet. But that's like a fixed amount of information. And then after that you end up with this long tale of like currently created information that you have to start using. Um and the nice

19. 7:26

    thing about the internet was it bundled in our view resources data and information. Data the definition I would give is like raw data. Like if you put a sensor in this room and it just like captured data, that would be data like a a video or an audio or something like that. And then information is something that some intelligent being adds to that data. So you could label it, you could categorize it, you could kind of add just like overlays to it, whatever you want to do. You you've added something

20. 7:50

    to it. Um and the internet has both of those things. Going forwards, we have to be able to capture both of those things somehow from like human interaction with the world. And so we need all these like passive data collectors and all these ways of hooking into human interactions with technology in order to gather both of those things together. Um so I think it it's going to change how you build applications with machine learning built in. You have to move to some kind of

21. 8:13

    continual learning model where you're actually training a model alongside your users. You're capturing that information and data live from their interactions and then you're making the model better. Um, but I think we've hit diminishing returns or we're very close to hitting diminishing returns from just like pre-training another model on the internet. You can get efficiency gains

22. 8:30

    and bring down the size of the model, but ultimately I don't think that's going to be the big differentiator for people using these tools anymore. It's going to be UX and like product improvements and that's all going to be baked into that continual learning.

23. 8:42

    Yeah, I totally agree. Like I think uh reinforcement learning is like uh become more and more popular like people starting realizing that um having just like all the internet data is not enough. You actually want the AI to to interact with their environment and generate new data for AI to improve its uh performance and quality. Right?

24. 9:04

    So I think that's uh like there there will be different ways to get different data in the future and uh I think we're already past the the stage that just scraping the internet. I I think I add one thing you mentioned synthetic data. I think synthetic data is one of those weird things where like in on the one hand it's just a pure augmentation technique. It's just a trick basically like yeah you can get some more training performance out of it. You can't create new data from thin air but as Jasper mentioned reinforcement learning in a way kind of

25. 9:31

    does because it allows the model to like generate the information itself and then train on it and you're basically doing one enormous search for new information which is kind of it's kind of synthetic data but it's a bit more interesting but I have a bug bear with synthetic data.

26. 9:44

    Right. Just like it's an augmentation trick like you're not creating anything. Well, the next question is actually for you, Ben. We've or yeah, we've talked about um compute. We've talked about data. I'm not sure how familiar the audience is on collaborative training, but I think it would be helpful if you could kind of explain a little bit your thesis on how collaborative training

27. 10:04

    might provide for more cost-effective outcomes or not um with the amount of compute that's being used for it. Sure. Yeah. Uh it's a nice segue from from what we were just talking about. Um, so people might have seen an application we launched called RL Swarm.

28. 10:17

    Um, RL Swarm was basically uh a completely open-ended network where you could connect a machine learning model and you could do reinforcement learning collaboratively with other models all around the world over a decentralized network. underneath essentially all that is doing is a very large distributed search process because within reinforcement learning essentially you're you're getting a model to kind of interact with this environment and to

29. 10:41

    essentially test outcomes against a reward function. Uh that testing of outcomes requires you to constantly generate new ones. You're basically trying to find a good one so that you can then train on it and you can improve yourself and the reward function tells you whether it's good or not.

30. 10:55

    Collaborative learning is just a way of saying let's just distribute that massively like we can parallelize that process because the process of generating rollouts which is what the model is doing is embarrassingly parallel. So I could have one model that I run that just like linearly generates rollouts one after another and I could try and find the best one. But if every single person in this room did that concurrently and then shared them with each other we could all decide which are

31. 11:20

    the best ones. We could all train on the best ones that we see. Uh that's really powerful and actually uh it's a training technique that allows you to just scale like as far as you want but at the same time it's a kind of leveling technique for the entire world because it functions as a form of mass distillation. Basically every model in the room that is doing that process is learning from the other models as well.

32. 11:42

    And so in a kind of open- source context it allows you to continually improve all of the models together. It's a kind of equitable way of allowing machine learning progress to be shared amongst a crowd. Um it doesn't really map that cleanly to open source but that's like the closest thing you could think of uh in the kind of traditional world. Um there's an interesting analogy that you see in the centralized world right now like people probably saw anthropics announcement that DeepS etc are

33. 12:07

    distilling their models and like they're getting their best performance because they're just distilling anthropics model. That's basically what RL Swarm does, but it does it intentionally. And so you can see that as a form of like bit torrent essentially like you connect up your model and you're just benefiting from every other model in the swarm.

34. 12:23

    They're all able to like teach each other essentially. We think this is the path forward for local open-source like privacy preserving sovereign machine learning. It's kind of the like Linux version of like the current Microsoft that you have with OpenAI, Anthropic, etc. You can all train a model and keep it current. It won't just be a model that you took the open source checkpoint and then like 12 months later it's no longer any good. There's nothing you can do about it. Actually, you can all keep

35. 12:47

    them current and up to date without depending on a closed source model. Very nice. Any uh comments on that on collaborative training? Um let's see where the humans fit in the loop. All right. So, the next three questions are just going to be open questions. Uh would love to hear kind of your honestly your candid opinions, right? We all like honesty. So just tell us exactly what you think, right? Um it's it's probably

36. 13:13

    fair to say that decentralized networks have struggled with real world adoption uh you know over the last five or six years. Is there a place for tokenization on this frontier of artificial intelligence whether that be with compute whether that be with data whether that be somewhere else? Do you think tokenization is relevant in this conversation and where?

37. 13:35

    Yeah, I I can go. Um I I definitely see a place for it. You know, public has our own token and for us it's a way of giving people not just that opportunity to earn but also to own the shovels with the tools that they're earning with.

38. 13:48

    They get to own part of the platform that is public u by earning the token. Uh of course they can also sell the token and get USD out of it. Um we have a revenue buyback program. So when clients come and pay us revenue, we issue public for rewards. And then as people sell it, we support the price of the token with our revenue. Um, which puts us in a um a small category in web 3 where we have like a web 2 business and revenue that is supporting uh the price of our token. So it's not so speculative. Um, which makes it less

39. 14:17

    exciting for some people, but um it makes it more of a sure bet for people who want a piece of AI infrastructure. Um, also, you know, we have a community of like 3.5 million people. Token is something for them to rally around. We also have a database NFT. So, this is something that's fun and playful and it's more on the marketing side of things, but tokens work for that as well, the the marketing side of things.

40. 14:40

    Um, and right now we've got a database staking pool going. So, you can get your piece of 35k public if you go and stake a datab with hot. So, so Jordan, I'm your answer, I guess, is yes, tokenization is relevant still. Yeah, make a token. Any any thoughts, Ben, on that question?

41. 14:57

    Yeah, I have to agree. Um, I think probably three reasons. Uh, one, security. Um, so in our world when we're providing programmatic access to all of these resources, you have to game theoretically secure those resources. If you don't use game theory to secure them, you have to establish trust in some other way. And typically those trust mechanisms are always centralized, like we have legal contracts or something. Without that, in an entirely programmatic world, you have to have

42. 15:23

    people staking. You have to be able to slash stakes, etc. In order to do that programmatically, you basically need to use tokens. Like, it's going to be incredibly difficult to use fiat to do that. Uh, you don't necessarily need your own token to do that, but you need a token. Um, the second thing is programmatic payments. Uh, so in this world where resources are able to be accessed programmatically, if the entities accessing them are machines themselves, they need a kind of machine native payment system. Um, and crypto

43. 15:49

    tokens are a machine native uh, payment system. And then finally, probably the most like mcurial like one I strongly dislike usually is like incentives. Basically, you can kind of bootstrap certain things if you have a token. The way token values work, uh, it allows you to uh, get things off the ground initially by pooling capital and then like pay that capital back in the future. I think this has gone wrong like infinitely many times in the crypto

44. 16:14

    space. I still remain optimistic that if you for example tokenize a model and then build it together, you could actually have shared ownership of a model and there could be revenue flows back to that token in the future. Um I think we'll get it wrong a bunch of times before we get it right because the crypto space always does that but it doesn't mean that it's not a good technical mechanism for doing that. Like ultimately I think that is a good

45. 16:36

    mechanism personally. Um so yeah to wrap up security programmatic payments and then kind of shared ownership incentives etc. Although that last one is the thing that we always get wrong in the crypto space. So expect a bumpy road before we get it right.

46. 16:50

    All right. We've we've talked about the financial cost of intelligence. We've talked about the compute cost of intelligence. We talked about the data cost of intelligence. Where does privacy fit into this picture? Is there room for privacy or is the real cost of intelligence sacrificing our privacy in order to get really good access to, you know, certain AI systems? I'd love to

47. 17:12

    hear all of your thoughts on how that conversation is going to evolve in the future. Starting with you, Jasper. Yeah. So, uh there is something called confidential computing. Uh I would say it's still in the early stage like when you try to like do fully private in terms of like running AI models across like a a GPU cluster. uh there like there I think there's still attacks that you can do but I think uh Nvidia AMD and all these like physical chips company are making it better and and there are a

48. 17:42

    lot of like more uh like more and more secure ways to run AI models and so there's uh I think there's like potential there like people if if like there I think like if people starting care more and more about the privacy so um we probably going to see like uh inference providers starting in providing AI models running like an enclave and that will that will makes the the information much much secure and uh in terms of cost uh it probably won't like add too much cost because uh

49. 18:15

    usually the like the latest blackwell chips already have the enclave and so uh it should be it won't impact the cost that much. Yeah, for us uh the privacy boils down to transparency. We have people who are sharing their data and we have clients that need a guarantee that that was like consensually collected and that we have the attribution trail back for each and every user that's contributing to our

50. 18:42

    platform. So the privacy is um something that needs to be kind of like upfront and not hidden from people for it to be equitable and for people to want to show up and share data as opposed to the traditional models where the data is like you you trade your data for a service or something like that and it's not so transparent and it's not clear exactly where the value is. Um and a lot of people don't care but as Jasper mentioned they're starting to be more cognizant of it.

51. 19:13

    Yeah, I uh I agree with Jordan on that consentbased piece. Um I actually before Jensen, I co-founded a privacy company, ran it for a couple of years, and we tried to retroactively bring in that consentbased data access like paradigm to the sort of social media world and retroactively bringing that in just does not work. Like there's no incentive for anyone to shift their business model around that. Um we were building this at the time of GDPR, CCPA, etc. those like

52. 19:38

    GDPR gets a lot of like hate because it's really annoying but ultimately if you read it it's actually a very well-ritten regulation like it it lays out a view of the world which would be fantastic for a consumer like it is a good way of doing things unfortunately you cannot regulate it into existence it's just not going to happen um and so I think we we sort of failed in the last wave of the like attention economy internet we have a new wave now which

53. 20:03

    centers even more around extracting information from us and our actions etc. Um, and I think we have to get it right within the tooling rather than trying to put regulations in in like 5 years and then just not being able to like it's not going to work again. Um, my view is the strongest way to do this is to ride the open-source wave and the local execution wave. Um, so there are other reasons why local execution of machine learning models is beneficial than just privacy. There's efficiency reasons,

54. 20:30

    there's latency reasons, the chips are getting better. There is going to be a drive to have execution happening on device. And I think we should build tools right now that allow users to get similar or better experiences using ondevice machine learning than they can get using those cloud-based tools. And if we build that alongside kind of open-source style technologies to allow those models to improve together, um, to

55. 20:54

    continue to stay uptodate, etc., then we can actually have a legitimate alternative to centralized model providers. But it requires us to like build that tooling, opt into this like collaborative style of learning, this like new wave of open-source which is technically possible but requires like some social change. Um, so I think that's how we establish privacy. I do think people care, but what I learned in my last company is people care up until

56. 21:18

    it comes time to pay and then they don't care. Um, so you can't put any kind of payment or friction in front of them. You have to build tools that are just as good, if not better, and get them to adopt them. All right, we're going to go full circle.

57. 21:30

    This will be the last question and it's going to be on the human cost of intelligence. And I think the simple version of it, you probably are seeing versions of this around the internet, is are we cooked chat, right? Um, how do you guys see, you know, rapidly improving AI impacting human employment of white collar jobs? And I just want you each to conclude on how you see, you know, the human cost of intelligence either benefiting or taking a hit in this new era.

58. 22:01

    First, sure, I can go first. Um, I think there's going to be a lot of change from AI. I think two things. one, I don't like the kind of agentic wave in the way that like most kind of people talk about it as this like anthropomorphized thing where you take a machine learning model and it just replaces a person and like our jobs in the future will just be instructing models as if they are people

59. 22:26

    to go and do things. I don't think that's a very efficient way of doing things. I don't think humans are naturally very good at that. I think some are, but there's a reason why the kind of like management class is small. Like it's not actually that efficient.

60. 22:37

    Not that many people are that good at it, etc. I think what people are good at is having a goal in mind and then trying to progress towards that goal which involves using a tool typically. Um and so I think the real wave of AI progress as it impacts humans will be vast improvements in the tools that we use allowing us to continue doing the jobs we were doing before but more effectively rather than becoming task managers. Like I don't think that's natural. I think there's going to be

61. 23:00

    this wave where we all think we have to be task managers until people are like this is awful. I hate it. I want to go back to like creating again. and then they realize the tools have got better and they actually just kind of use better tools now. And so I think our interactions with technology will get more efficient closer to how we interact with other humans, but it's not going to

62. 23:17

    become this like taskbased instruction uh system. It's going to become more just like using better tooling. Um along the way that does change the efficiency metrics of labor like you have a bunch of people doing this this task. If somebody can suddenly do it 10 times faster then do you need as many people etc. Every technical wave the world goes through this. this is just something we have to go through. Um, ultimately when we come out of it, you end up with a

63. 23:40

    kind of mixed up world of like different tasks that humans can do. But the lump of labor fallacy basically says that we're not going to kind of run out of tasks for humans to do. There's always going to be new things that we discover that humans can do efficiently. And so my view is that technology just progresses us way faster. It makes us more effective. It makes our lives better. But it will in the kind of initial stages just shake up what we do

64. 24:03

    and what we think we should do. Um, but I think that's natural and I think we should embrace it personally. All right. So, Ben says, "We're not cooked." I'd love to hear from Jordan and Jasper to close this out. Hot in the room. I'm feeling kind of cooked. Yeah, I I think it's not it's not a question of what work the AI will take, it will take some work, right? Um, and as it gets better, the real question is, do the people who are enabling that

65. 24:27

    work, who are part of that knowledge substrate, who are providing data, are those people paid and do they get something out of making these systems that can do more? So, that's what we're building, that's where our mission and vision sits is the human layer of AI where we make sure that the value that's going in gets rewarded.

66. 24:45

    Yeah, I think AI is like a much more power tools than what we've seen in the past. uh I view it as like the new indust industrial revolution and so every time when there's a revolution happens like lot people that don't adopt the new tools will like probably don't get used to it but then like I mean most a lot of like if you see like how fast AI move right like chachb just came out

67. 25:09

    three years ago but now like almost everyone like interact interact with AI in some sense and like many engineers like software engineers are using like clock code and many AI AI coding tools to do it and I think I would like uh go to any many other other like uh industry as well. So I'll say if uh you're someone who want to like embrace the the change then probably you should start

68. 25:33

    learning how to use AI tools and uh that will 10x your productivity and u and I'm for sure there was some professions that will be like outdated and no one need to do that like like in the past like you have type writers but now like no no one ever do that and I think uh that will happen again as well.

69. 25:53

    All right. Well, thank you to our panelists for giving us insight into the real cost of intelligence.