[REDACTED] 2024 | A New Era for AI Compute

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   hey everybody uh my name is Cameron Dennis uh I'll be interviewing this wonderful panel today about the future of decentralized compute uh just a little bit of background on myself before we get going is I've been building the near ecosystem for about four years in crypto for about eight um right now I'm working on near AI to help stand up the ecosystem uh sort of different parts of the whole stack uh so these guys are all focused on the

2. 0:22

   compute layer in some way shape or form or decentralized inference and so um really excited for the panel today I think these are some of the best guests that we could brought and so just to get kick things off we love to start with a quick round of intros so Matthew you want to go first sounds good uh first of all thanks for having me I'm Matthew one of the co-founders of open gradient uh prior to starting open gradient I was a research engineer working on quantitative modeling and Engineering

3. 0:47

   for an equity options trading desk at two Sigma which is a Quant fund based in New York um you know I guess just a little bit more context I you know started getting into defi was fascinated by what web 3 could offer but we realized that there were no outof the-box solutions that would allow people to you know host or integrate ARL compute directly into their applications and Empower these applications with more

4. 1:13

   expressive forms of computation and uh that's what kind of informed the initial Vision behind open gradient which is to build something where people can just like drag and drop upload their models and you know inference it securely within their smart contract directly awesome thank you thank you for having me as well um my name is Matilda I'm currently the head of strategy at G um my background is a more on the VC

5. 1:37

   side so I started off at binance Labs the vcma binance spent about two years there did a bunch of deals and then always wanted to move on to the operator side and here I am here I am um at G what we're trying to build is the economic and financial layer of computational assets particularly GPU um so you all know that there's already super high demand for professional grade gpus right now in the real world in the offchain world and the demand is only getting bigger and bigger so these uh

6. 2:08

   cloud and data center what have started to do right now is to collateralize their gpus uh with asset managers like Blackstone for example to borrow out to borrow out loans and use that borrow funds to uh buy more gpus and scale their business so what we're trying to do is essentially bringing these kind of deals on chain we tokenizing loans back by gpus or the future cash flow of gpus so that cloud and data center can get

7. 2:33

   funding in a more in an easier easier way um and investors no matter retail or crypto whales or institutions can invest in a new type of yield bearing assets um yeah that's about it yeah I think it's really cool um I just like the idea of actually being able to invest in the infrastructure layer right we can't go and buy AWS shares right you can't buy Google Cloud shares right you you're

8. 2:59

   you're your ability to actually invest into that infrastructure stack as a as a retail consumer is is pretty limited to be honest right you you either go Nvidia or you know something kind of uh smaller but uh you really can't kind of Hit the the the the cloud stack very effectively as a as a retail buyer but um yeah Mark here uh from Aether we build decentralized GPU infrastructure for for AI and gaming companies you can think of us like uh kind of an AWS but uh the

9. 3:29

   difference being we're entirely uh GPU focused uh we're 100% Enterprise grade right a lot of deepin U is kind of built on consumer contribution we're not uh a consumer grade uh ecosystem everything in our network is is Enterprise grade sitting in a data center or connected to uh Enterprise grade Network infrastructure but uh yeah we like to think that we're making it easier for uh builders in the AI space to access critical resources that they need uh to

10. 4:01

    to build cool stuff and uh just quick question for you mark uh the builders in the AI space I've talked to a ton of them all the big research Labs uh you know independent Founders a lot of them are a little bit skeptical about this whole decentralized compute Market they think it's riddled with low latency it's you know not incredible uptime how do you guys address this I mean the the

11. 4:25

    first challenge uh or the first thing that you kind of need to uh look at right is is uh you have to have Enterprise grade slas to serve the vast majority of companies that are looking to engage with your infrastructure right which is why you can't build a network on the back of consumer equipment right because you cannot guarantee that a gaming PC is going to be um you know kept on at night right maybe the RGB stops you sleeping right you you you you can't hit Enterprise grade slas with with consumer equipment so uh first of

12. 4:59

    all if you're going to be serving that type of customer that cares about that type of thing then you're going to be needing to kind of uh limit what contributes to your network in a way that uh ensures that you can kind of maintain those those slas I think um the other uh I guess challenge you mentioned like latency right people uh we educate our community and and and hopefully the broader uh community that um decentralized doesn't necessarily

13. 5:29

    mean not collocated right um a big concern in in um in the AI space when you're when you're training or even even running inferencing um is how effectively how how effective is the communication between all of the gpus doing the work in your ecosystem right to the extent that you you really need these machines in the same room as each other right the the the instantaneous communication between these devices is so critical that you need them located

14. 6:00

    right in the same Data Center and a lot of people hear decentralized and they immediately assume that means distributed that means that like all of the equipment can't be adjacent to each other uh but that's not always the case right so we have a large number of big collocated clusters within Aether right um because that's what's necessary for for large training runs uh but um yeah long story short AI needs a certain type

15. 6:32

    of compute certain customers need a certain type of compute and we've built our Network specifically to serve uh those customers cool and uh for Matthew how does your permissionless approach uh affect pricing Dynamics compared to you know traditional Cloud providers yeah so the way we take this is kind of like a staggard approach right initially let's kind of build a network ensure that

16. 6:54

    there's sufficient supply and demand and eventually build in some module that allows the free market effects to really take you know take control of how we price compute on our Network um at open gradient um and this is tricky because we have what we call like a heterogenous Network which means it's not just you know one type of node doing like the same type of work we have gpus providing

17. 7:17

    computes we have um tees as well you know also CPUs um and depending on the specific requirements and demands for your compute either it could be you know tees generating you know computational ad stations it could be you know gpus generating ZK snarks in addition to running inference or it could just be CPUs running inference on smaller classical machine learning models the the cost of compute really does vary

18. 7:43

    between you know all these different use cases and kind of we want to build this like flexible module into our Network that allows you know supply and demand and you know the varying kind of you know fluctuations in supply and demand to you know take effect and allow the network to uh you know bring in more suppliers of gpus and compute when demand is higher and and vice versa so

19. 8:07

    that's kind of like our approach again today we're more focused on ensuring that we're able to get traction and actual grounded real world use cases of this compute within the web through ecosystem and then like you know gradually focusing on optimiz optimizing for cost and I I do want to hear from you but real quick what are those you know tangible real use cases that you're seeing today because I feel like sometimes in in web 3 it's a lot of talk not always a lot of delivery but you

20. 8:33

    guys are delivering so what is what is happening yeah I think that's a great question right now like a lot of people when they look at the crypto AI space the most common thing we get is oh you know crypto AI is just purely mimic you have these people building like AI agents that are tweeting but realistically they're not generating a ton of you know grounded value so to speak the kind of use cases we've looked at in terms of Applied modeling are kind

21. 8:56

    of they're they're not entirely novel or new they're kind of of borrowed Concepts from traire web 2 and some examples include kind of building machine learning models to optimize amm trading fees to help reduce net impermanent loss faced by liquidity providers and the models that we built which are all available on our AI model Hub show that we can for pairs such as BTC usdc you know if you incorporate these models into amm design we can increase fee

22. 9:21

    collection by 18% which which is like a sty number we're building risk models for Lending protocols and CDP protocols we're also building um these spot forecasting models for vaults as well as prediction markets so these are again very grounded machine learning use cases that have seen use in trafi um especially coming from two Sigma like people build these models all the time

23. 9:43

    and we're simply building infrastructure that helps support s use cases are you sorry are you targeting more web 3 customers for your service than than kind of web2 I mean we're very web to uh from a from a customer perspective the vast majority of the revenue that comes through through from web 2 companies you're more focused on kind of the web 3 customer um so the the kind of approach

24. 10:06

    we take is we don't necessarily I guess uh discriminate between web 2 and web 3 it's infrastructure that provides seamless and secure AI workflows um so web2 customers are free to you know we have a a python SDK that allows developers to you know abstract away the blockchain complexities and run secure a workflows that generate snar and AD stations directly from their command line interface or from python um so in this case you know we cater to web 2 as well but I think presently you know being very realistic today if you look

25. 10:38

    at you know web 2 AI demands and web 3 AI demands web 3 has a much higher requirement for verifiability and security just due to the way blockchains work you know they're replicated State machines right we need to ensure that every single node is able to verify some computational proof to really trust uh the results of the computation and use it in some protocol um so I guess like long answer short uh

26. 11:02

    a lot of our customers are web 3 today but we actually do have some web web 2 customers we're talking to notably in uh you know Healthcare actually as well so that's interesting to us super coola very curious about guib because I think computer is this neish asset class that I think will out compete even oil maybe eventually uh and you guys are pretty much creating the I kind of see it as

27. 11:28

    like derivatives economy around her like real world assets that obviously like can be trayed in almost like the the hyperf financialization of comput as an asset class um so how are you seeing the sort of like inflows either from uh one the interest from compute providers but then two the sort of like demand side uh are these people primarily like Wall Street Traders are they just independent Traders um yeah walk us through a bit

28. 11:53

    more of like the customer segment of G yeah yeah for sure great question actually um um starting with the GPU side right the demand is already very big and it's growing bigger and bigger um a recent case is a deal between um Blackstone and cor weave cor weave is one of the biggest hyperscalers these days as well um so they cataliz a whole bunch of gpus with Blackstone in the private credit fund and borrowed out $7.5 billion um meanwhile the investors in that private qued funds are enjoying

29. 12:25

    roughly 15% of Interest which is quite attractive um however like these kind of deals it takes a lot of you know it takes time to process it takes a lot of like due diligence um you will have to be as big as like core weave in order to strike this kind of private qu deal with Blackstone there is actually a lot of like medium and smaller uh cloud and data centers that also have this kind of demand they've got a whole pipeline of like clients wanting to use their gpus and they need Capital to scale up their

30. 12:57

    business so that's what we're tting at the beginning um we're targeting basically uh cloud and data centers that have like roughly a few hundred million to a few billion a in terms of assets uh they're the ones that would be bringing like meaningful traction to us uh they have like um preol solid professional grade GPU to Cloud toize meanwhile they are not necessarily being served by big asset managers like blackstones right

31. 13:22

    now so that's on the supply side right now we've already got a pipeline of $1.8 billion worth of uh G GP assets starting with uh our co-founder actually our co-founder Alex he has a cloud company called GMI Cloud uh which is serving like Alibaba xiaomi it's got a whole bunch of h100 and the procurement contract of h200 as well with Nvidia so we're starting with that we've also in

32. 13:46

    the middle of striking deal with a data center in Thailand um some data centers in Taiwan and uh in the US in Japan as well so i' say the demand is actually very big um and um on the supply okay I will say actually that's more on the supply side on the demand side uh on the capital side I would say what we're targeting essentially um retail investors crypto institutions regardless or crypto whales on the institu institution side like crypto funds no matter like Venture funds liquid funds or like um Family

33. 14:21

    offices that have a crypto exposure for them like um the yield is quite attractive right um because if you go to the real world and invest in a private credit fund obviously you have to go through a lot of processes and especially for retail like you can't really do that because you're not an credit investor by this you know defined by these funds um but you can participate uh and earn yield on chain um and for institutions like es especially crypto funds they wouldn't

34. 14:50

    have to worry about any kind of UNR and off-ramp right so because they can just immediately invest on chain and um by kind of eliminating a lot of uh intermediaries we can bring the yield of our bond model it's kind of like a bond for AI um to somewhere between 15 to 20% and if it's an equity model it could be like even more than like 40% so it's quite attractive yield to them as well we we are already getting a lot of interest um yeah so I think we're pretty

35. 15:20

    confident on uh the interest on both sides right now uh it's just a matter of execution to actually like bringing all these deals on chain and like going through all of the legal proc processes what is that missing part like you say bring those deals on chain like what needs to actually happen what does bring deals on chain mean um well it's literally just like uh legal work right cuz like I mean we obviously write the AI wave and everything but essentially

36. 15:43

    we're also a bit more of an rwa play and you know like the most important uh factor in any rwa project is pretty much the regulatory you know uh aspects the legal setup of everything um so essentially we need to like talk to all of these like data centers in the will fly around but uh and to meet them like in real life not not online and then like um and then like really sit down and flesh out all of the details of these deals we need to sign agreement

37. 16:13

    with them and um and we need to make sure that legally speaking like um for example if it's a loan structure they will pay us the interest and then will distribute the interest on chain for example to all of the investors and so it's really just like ining these deals yeah in many ways as well because we we're we're quite close with with the Gabe ecosystem um in many ways it's also

38. 16:38

    having the the infrastructure on chain too right because you're You're Building these investment structures that are tied to uh these gpus right so ideally you want those gpus on chain too right you want the ability for that kind of uh investment contract to be linked to the infrastructure on chain um just I guess from a from a transparency perspective from a from a connectivity perspective but um yeah I think it's it's for me it's exciting just uh the the

39. 17:09

    opportunity that it provides for for the average kind of non uh non-sophisticated more retail investor yeah I mean it's it's interesting to me I think the the real world asset world has been around for quite a while um and I think the main blocker has been regulation and Regulatory Clarity and so I'm I guess I'm kind of curious uh Mark like are you concerned at all about these things

40. 17:33

    maybe even be defined as a security and has this you know is this actually a concern because I I'm close with like the SF compute team uh and and they're like a big kind of web to compute provider and uh they're going they're jumping through so many Hoops to try to get regulated properly and it's been taking them years and years and years and then I see all these like decentralized solutions kind of launch much more quickly um and I'm curious I asked them like why don't you just do the web 3 thing and they're like yeah

41. 18:02

    we're trying to you know get listed on the CME and so uh I guess is this a concern which which piece which piece I guess like the tokenizing and then from there like you are you able to like Whit list Blacklist in order to like prevent any sort of like misuse of compute I mean yeah we we I think compute is is a is a it semi-regulated it's politically incredibly Sensi exactly right semi-regulated geop geopolitically very relevant um we for example have to gate

42. 18:35

    uh compute coming from certain regions right to for example h100s or like flag current Flagship or or maybe you know h20s are out there now but you call it Flagship um GPU from Nvidia right shouldn't be in China right shouldn't shouldn't be there but uh we have had compute try to unboard from China right and we need to be really cognizant that uh if we were to um accept that onto our Network that would be you know a potential liability so that's the challenge with permissionless ecosystems

43. 19:09

    right you want true decentralization you want permissionless uh networks but at the same time there's a there's a risk there um that uh that we need to we need to manage you were just talking about this go ahead please yeah I was just going to say um like you're bring up like the idea of securitization and like coming from trafi like we use this thing called how we test to determine whether something's a security or not um I think the risk of you know regulation for

44. 19:33

    compute like I guess like gpus in this case being a security is relatively low just because there's not like one unified Enterprise which is like one of the um you know principles of the how we test and the other one I think is like you know investment or you know you're expecting some kind of profit which I think you know is is uh maybe a thing but there's no shared Enterprise right so I think it would probably be classified more as a commodity if anything um yeah similar to gold or or

45. 19:58

    these other like precious metals and typical Commodities that you know CME Traders might bid on but uh I do agree like kind of to your point um you know right now these powerful gpus are you everyone's looking at them you know especially since we don't want them to fall into you know the like our our political opponents' hands and I think there is whole can of warms you can unpack there but um yeah I I just like wouldn't really classify them as

46. 20:22

    Securities today I mean we only have about 30 seconds left but Matthew real quick uh the intersection between compute and agents like I see this these agents actually eventually purchasing compute themselves training other agents uh via this compute I guess you're all of you really are focused like doing this uh quickly quickly what do you think the future of this intersection looks like so I think permissionless compute or permissionless in general is just like powerful for agents because it

47. 20:46

    gives agents a lot of agency to do more things um in this case right like you might imagine uh once agents get to the point where they're able to actually make sense of of data and figure out how to train their own models they can deploy their own agents and really gain a lot of sovereignty autonomy and agency by deploying agents that can they can you know have do other things on their

48. 21:10

    behalf um that's like something that we're particularly excited about uh just like to throw in something that we're working on um so we have it's not so much an agent deploying another agent but we have um this agent project we're working on where the agent is actually able to query other machine learning models that can help it you know inform it uh a for instance in this case we're able to have the agent query a spot forecasting model that can give the

49. 21:33

    agent some idea and notion of what assets might appreciate based on that machine learning model and that's able to allow the agent to make much more precise and accurate decisions in terms of what to buy um and this is like a powerful use case of how composability and permissionless compute can really make agents you know or can get agents to the next level it's going to be wild um I wish we had more time feel free to find us in the back after uh thank you

50. 21:57

    for having us thank you thank you