NEAR AI Office Hours #21 with Phala Network and Super Protocol

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   we'll find the flame empowered Minds rise and S use your own AI we'll explore hold the future hand in hand in this Tech driven Wonderland okay hello everyone we're live uh that's episode I I never remember the number of the episode but that's episode number next of near office hours uh today is a topic that I'm particularly excited about uh it's a trusted execution environment uh uh it's uh it's it's something that a lot of people in the AI space and web three space are excited

2. 0:47

   about and potentially one of the very few practical ways of achieving certain uh features that are extremely desirable so we have um two different uh companies with us today we have F protocol and uh super protocol oh it's F Network sorry uh and uh uh yeah do you want to introduce yourself uh NRI you want to start yeah uh thanks Alex uh having me uh I'm nucle and founder of super protocol um about 14 plus years uh in uh area of decentralized confidential

3. 1:26

   high performance Computing served uh this is my my first startup in in this area and uh yeah thank you and and they want like to give a quick very quick overview of what of what super is super is a um combination of uh Marketplace for AI ml uh agents uh and uh decentralized confidential uh Cloud uh I believe this it's this uh combination uh will be successful uh to resolve many

4. 2:06

   tasks we see today in eii and C area Julia you want to give a quick uh background on yourself yeah sure sure sure yeah I'm actually Intel actually yeah so I've spent uh 18 years more than 18 years at Intel uh where I led large scale projects um developed incentive um programs for Intel consumers and uh championed and enabled confidential Computing and within while I was still at Intel uh over the last 10 years last ones I also worked closely with nuk and his core team uh in order to support and

5. 2:51

   drive various projects related to secure decentralized Computing and it Technologies um and now now I am see o at Super I oversee operations and business development and uh work with different partners um to extend the whole ecosystem for confidential and AI um and self- suran Ai and specifically I also own the direct relations with uh major te manufacturers which are Nvidia Intel AMD andm so yeah excited to be

6. 3:25

   here awesome okay and H do you do you they want to give in on yourself and the what F does sure uh yeah so I'm H in the co-founder and also CTO of the fa team uh and Fa is actually a very old project probably uh close to you know how old near is yeah we have been building uh in the te space for five years since 20 uh 20 uh uh 2019 we started the project aiming to solve some of the problem you know the the blockchain always have one of that is that you cannot really deal

7. 4:06

   with you know the confidential uh uh confidential compute or confidential uh input data on the blockchain because whenever you put everything on the blockchain it's transparent so uh another thing is you know lot of limitation like you cannot do the heavy computation on the blockchain or you cannot you know access any data outside of the blockchain so all those kind of limitations are nonexisting in traditional Cloud so that's uh one way were thinking why not to bring something

8. 4:37

   similar to Cloud Pro uh Cloud uh Computing into you know the blockchain space and uh after find uh checking all the related uh Solutions te is the only uh practical solution we have found so far so that's why we got so passionate about building on te so uh in the first few years we focus we have been focusing on bringing up the Cloud Supply so we have a network with more than I think 40K noes joining the network we know none of the noes so all of those noes

9. 5:10

   are provided by you know the individuals or some miners as long as they have the hardware they join the network and uh you know the developer can hire can hire those miners by paying them and the miners are going to run their workload on their decentralized Cloud this is what we have been do in the past few years and this year this uh I mean it's it's never you know more make me more

10. 5:35

    excited because now we have very solid demand uh especially from the decentralized AI space and we have never you know uh so you know a a good uh supply of the te technology I mean the next generation of the te uh probably we can we can cover uh in this uh this podcast as well so yeah uh combine them together I think this year is going to be very uh you know uh excited for uh the project using

11. 6:08

    te so so I think one one uh a good way to start would be to explain what exact guarantees tees give right because I think people talk a lot about tees but not everybody understands what are the precise uh guarantees you get right so like for example do can it be certain that a very particular code is being run right how do I know you know like if I'm encrypting my data how do I know that it's only decrypted within the kees who

12. 6:38

    who controls the key for decryption Etc right yeah so so H do you want to give like a quick overview of how of you know how ke is uh work and what what guarantees they provide yeah sure um yeah I I think a good analogy to think about te is like black box it's just like a black box you can put anything in the black box then it's just a right the Black Box itself has the characteristic to make sure everything running in the te is secure it's temper proof uh it's privacy preserving uh and

13. 7:18

    you know because it's a black box so from anyone outside of the Black Box you cannot really do anything with uh you know you cannot really uh manipulate anything inside the black BX say you you cannot steal the data inside or manipulate how the program is running inside so this is the mental model I usually Des describe how te look like to uh the people are not familiar with te but uh this uh just sounds a little bit like too good to be there right why can

14. 7:54

    we have such a thing that's so easy to use and has so good uh ristic especially combine uh compare with you know the cryptographic solutions like zero knowledge uh proof and MPC fhe they usually have a very high overhead like if you put something running in zero knowledge proof usually you get overhead of around maybe one one 100 times or even 1,000 times for FH is even uh higher like a 10,000 times but for Te

15. 8:27

    it's ju it's just like a CP but a CPU in the black box so uh there's no really very high overhead it's just like a 5% 10% each um so why it can be so you know nice and uh is there anything wrong with that uh the answer is uh te actually has a completely different uh uh security assumption and uh uh threat model compared with uh ZK MPC and other cryptographic solution so so uh there are a few things first we

16. 9:01

    need to make sure T itself is done at the hardware level and it's done in a right way there can always have some bugs in the hardware but of course we people can also have bugs in ZK implementation but that's just the same there could be bug in the implementation and second whenever you put something running in the te you want to make sure uh uh once you interact with the te you can make sure okay this is the right program and the right program is put in

17. 9:33

    the right environment which means that uh the program you are interacting with is not you know some fig program this is the first point and the second point is uh the program you're interacting with is really running in a authenticated te instead of a fake te otherwise you know if you put it in a fig te anything is lited there's no protection at all so the way to do that is something we call remote testation uh I believe we can cover this a little bit later but basically you can take remote

18. 10:04

    attestation as a proof so you put program run in the te so the te give you a proof you can verify the proof uh so that you can uh be convinced that this is the right program is running in the genuine T Hardware but I think couple things that go ahead yeah I can adjust T isn't it like something super Noel Tech um it's well known many many years uh and when you use iPhone or you keep your wallet

19. 10:37

    ler Ledger or you know uh just Apple announced they're developing data center T based data center to keep uh private data for AI purposes everything and everywhere T will be used uh same time this a t is a gold standard to say trth because all welln uh manufacturers who are provide who developing uh CPUs gpus or t compliant uh you can use uh large CHS high performance CHS with with zero let's say degradation uh um on inference and you

20. 11:20

    can you can use it uh easily but same time T has uh uh has uh implementations a lot of implementations in our previous projects uh project region we worked uh using te we worked with High um with uh top copyright holders to prevent and to distribute intellectual property through huge chain of resellers Distributors we worked with uh big data I I I will say later why we why I'm mentioning but uh this is very important

21. 11:59

    and we served 150 million users unique profiles including all personal data uh for digital collaboration ecosystem collaboration purposes between telecommunication companies retailers process about in PE about one billion transactions daily of personal data using decentralized distributed uh confidential Computing network uh to prevent leakages to prevent business collaboration and what

22. 12:28

    is the most important um when we talk about t or another alternative uh Technique we always should remember that uh confidential Computing or in different implementations is not only about how to prevent um data Lage or to prevent to keep privacy it's always about how to run applications how to run virtual machines how to at resources how to make it scalable who will be responsible I mean in alternative tech tech stack who

23. 13:07

    will Who will provide this deployment or Network Solutions which will uh guarantee to your user scalability and as general basic SLA and there are always uh open questions NT YT is additionally YT is looks very interesting and promising there are a lot of uh uh instruments developed for Te by major major vendors and open source uh communities and you can easily deploy it and run your T based

24. 13:41

    infrastructure uh with uh no limitations with no degradations regarding scalability maximum up time or log or any other uh integration and um using Integrations with uh external drive seal systems and so on so on and using popular development tools it's in complex it's a very important thing when we are talking about mature technique uh to provide our services uh on through ecosystem I wanted to quickly uh CL

25. 14:22

    couple questions that I personally found interesting couple weeks ago when I started digging very deep into tees uh well I guess just deep let's say uh so F first one that was not obvious to me but but I want to uh to make sure that people who listen to it uh understand that this is the case is that uh with T is uh I get I get the guarantee that the code that is executed within the te on my data is indeed the code that I believe it is right so so like the the the the

26. 14:54

    owner of the hardware cannot swap in the code inside the te right and run uh something else right which which I think is a like like that's not like naturally once you start thinking about the use cases you understand that this must be the case right but I don't think it is necessarily obvious immediately that this is the case right so so like there's an attestation of the code

27. 15:17

    executing in the te is a very particular piece of code uh which I think is very important uh and the second thing is that the way the security is established is by uh uh ke internally has a private key right that is not that does not exist outside of it uh and then uh the manufacturer of the hardware has a way uh to confirm that the public key is in fact uh you know the public key matching that private key is in fact something

28. 15:48

    that that I should be trusting right is is that is that more or less the mental model of the of the t's uh yes exactly uh how t first of all I said yeah I mentioned that t is a well-known technique you are using everywhere every day uh including your iPhone and uh how it works when manufacturer uh is um let's say developing this Crystal of silicon crystal uh they Implement function uh specific function for each processor and

29. 16:23

    this function collects uh like uh like dark uh uh um few parameters from this specific uh CPU or GPU and create this function and burn out uh this uh bridge between uh cheap outputs and your internal Crystal uh uh part of te and in in this way uh in this function creat this pair of uh public and private key and you can use uh and of course this is to say it's not like it's not Black Box to say truth

30. 17:03

    it's a verifiable uh Black Box I will I want to say uh it doesn't work like we put something and we don't know what happens inside no you just have Simple Solution you can create using your private key encrypted uh Enclave secure Enclave managed only by this price private key non- rejectable private key no one can reject this private key but same time you can you can make this process fully verifiable it means that you can uh

31. 17:41

    aestate this key uh CPU or GPU that's why uh GP uh T based server could not be faked because you can't fake uh this uh process I mean te same time you can create uh uh open- Source uh software which will could be uh deployed on on your uh on your on your CPU or GPU you can create these verifiable enclaves and you can create smart contracts which will govern these processes inside of this verifiable ATT estated uh uh environment it means then

32. 18:22

    using T this is just for a general overview it's a just pair of private public key and private is non- rejectable but same time you can manage all processes inside of this let's say box and that's why uh it it it it works very well for distributed computing because when we talk about uh distributed environment distributed uh suppliers who will Who are providing your uh your infrastructure for your project do I trust them I mean distributed and well-known deams right

33. 18:58

    should they trust someone I mean jnes who is from Texas who is owning server where I should deploy my business no from my point of view uh te is a very important part of this decentralized infrastructure because you don't need this trust uh to John or someone else one side from one side from Another Side this part of uh public private key based infrastructure non rejectable private

34. 19:28

    key infrastructure B te uh would be managed by smart contracts only because no one has access inside of this and if you are deploying your open source solution where all services could be managed by smart contracts only you can guarantee to all participants uh few important things first of all no one can reject or manipulate with your data and services because everything works in inside of this ankov or group group of

35. 19:59

    ancles combined in clusters easily by the way the step back in my previous uh uh uh previous sentences about readymade solutions to create clusters or pools and same time you can guarantee to your users regarding collaborations data Integrations data flows Services integrated with each other that no one can can manipulate with this uh uh Services too because everything is based

36. 20:30

    on guance by smart contract these two important six give us very interesting possibilities for AI specifically for AI area because today AI is a we are developing AI GS right we see a lot of projects who are developing vertical Solutions they're collecting a lot of data and from my point of view uh web 3 uh crypto AI uh industry has uh super chance uh to make uh to make something super interesting in area of

37. 21:03

    combining infrastructure desize uh uh governed by smart contract infrastructure self sering infrastructure with crypto narratives in Ai and these possibilities is are very important from my point of view and comes just from this simple simple idea of non rejectable private and public Keys Alex if I may add if I may just that because on the one side we're talking about te and this is some kind of a common standard on the other side me is working a lot with uh the

38. 21:37

    ecosystem with GPU providers with miners with AI projects what I actually see is that there are people who uh knew uh and learned about te in some time ago when there was no Nvidia GPU te with there was no Intel TDX for example with s amdv was not C paired with um with h100 in order to drive confidential mode and for these kind of people they believe that this is intx for example and it's it has

39. 22:09

    its pros and C but this is all what te can do and now we're in the mode when combining h100 uh h200 the and and blackw these kind of gpus if they are enabled in t mode compared with the right CPU could bring up absolutely different approach to how we can treat AI how could we treat te in general and what I witness is that still uh a lot of ecosystem members still miss the capability of what Nvidia CH gpus can

40. 22:45

    offer if they are not used just as an accelerated um GPU but really uh use the the the best part of it the confidential mode itself and so these kind of things that for example you buy the system with four gpus of h100 and literally you cannot use it in confidential mode because it does not offer the needed configuration for example right so these kind of things that we constantly work with the whole ecosystem order to explain these t standards and what they

41. 23:18

    can offer only if they are offering to the market the gpus for example instances in a proper way if it's like one single GPU h100 plus specific CPU either AMD SCV or Intel TDX with fifth generation so these kind of things and until we um make them understand that there's much more opportunity if this knowledge becomes much more available to the market the more opportunities would be really happening on the both side on

42. 23:51

    the supply side from the GPU providers miners uh Cloud providers Etc and on the top side on the demand side when people could really start using the confidential Computing in the way that they really needed in confidential and self- Survey mode yeah so you mention go for it go for it oh No Just say totally agree with that so as a person well I have I have two immediate questions so so let's begin with the with the one more

43. 24:23

    relevant to the uh to this awareness right so let's say as a person I want to buy a machine that can be used uh you know like as part of you know one of the clouds with te's right how do I know if my machine is properly configured like is there an easy way like other websites that will tell me which CPUs work with what gpus or like how do I do that how do I not accidentally commit a lot of resources

44. 24:47

    into a machine that will not be uh appropriate for this use case yeah this is going to be hard yeah we actually uh recently we have build the feel bare metal servers so as like your question mentioned this is like you you buy some Hardware then how can you make sure this Hardware can work and it turns out is really not something very easy first you need to have the right Hardware which is uh for CPU you you need a CPU for like Intel

45. 25:19

    CPU you need it to support Intel TDX so usually you you will need the fifth generation of the Intel CPU uh and for for GPU there are only two GPU supporting uh uh uh te right now that's Nvidia h100 and Nvidia h200 uh and if you put them together usually it should work but it's not the end of the story then you need to deal with you know the operating system uh the BIOS Set uh configuration uh and uh even for the operating system you're

46. 25:51

    going to twe a little bit in the Linux kernel because everything is super super new h100 was just a ailable since last year but there the the the the the uh the CPU you have to use was not available last year I mean the Intel CPU AMD is a little bit earlier but Intel CPU was only available this year that means when you are using the combination of the two Hardware now it's like you

47. 26:17

    are the uh early birth tester for Intel and Nvidia but it worked pretty well the only point is that you will need to figure out how to to configure that some customization in you know the Linux kernel uh the the Nvidia driver and also how to build the virtual machine image to boot it up so it's a little bit pen on the S we have spent roughly three uh three weeks to figure out the configuration for

48. 26:48

    h200 uh yeah we haven't tried you know AMD yet but Intel with Nvidia GPU is good I I just words uh yeah h100 h200 they play the current AI market right and all uh providers wanted to support confidential Computing because I don't know I I to say truth I don't know uh server chips I mean supporting h100 h200 without confidential Computing possibilities I mean this is a this is a trend and not only let's say infrastru I mean Hardware

49. 27:27

    software is in place menting we see a lot of integration uh support by by the way Aram and other players who are developing a lot of integration software to support This Confidential the huge players like Google Microsoft uh Apple Driving confidential Camp that's why it's it's not like local play Let's play something yeah with confidential C no guys uh huge data centers are developing it that's why we be we are believer uh of this uh uh

50. 28:01

    tag and I will just add from the uh well like super protocol sources actually so what you need is just you need to have the right hardware and these are very like of course these are very easy to get identified but once you have this Hardware in place then what you all you need to do is to unboard and then you will have the uh super protocol image that will uh that will allow to you to forget what confidential Computing is you just plug in right you have the

51. 28:33

    right image on board and then uh the whole super powerful system helps you to offer confidential Computing without you knowing and being an expert in confidential Computing space in general and for example in June we had work work workshop um driven by our CTO who uh demonstrated how to use h100 with uh in confidential mode with smart contract so it was really demonstration of what super protocol can do and it's available

52. 29:04

    on YouTube channel of our super protocol side right so just to have an image what can be done right it's not some Theory it's just like it's already available to see how it works and uh we are planning to uh soon start unboarding um the GPU providers and miners interested to be part of super protocol ecos system specifically for for driv This Confidential s in AI in reality yeah exactly all miners are running uh h100

53. 29:38

    h200 there are a lot of suppliers on the market and we see a lot of requests how to involve uh H1 this chips in AI in AI crypto AI infrastructure Y and I just to to support your your comments regarding how how users can actually identify I do believe that still there is a gap the knowledge gap between what is te what's confidential Computing and how the mass Market should know about that for sure so we do have already plans of writing a

54. 30:15

    joint partnership articles in order to reveal this kind of like to to close this gap between what needs to be purchased what what is applicable what are the technical requirements why what kind of capabilities they can open up and even if you do not unboard the hardware at the moment if you buy the right Hardware now you have this kind of a long-term you know opportunity to un

55. 30:38

    boid whenever you are ready to go in case you have already uh you know write uh hardware and specification in place so definitely there is a gap between I'm working with with with our partners and with te manufacturers as well because uh we we uh will we share with them the knowledge of what we we all is permissionless trustless all these kind of things that they actually do not know

56. 31:03

    right to some extent because they were always about Web Two world and because AI it's not about web 3 or web two it's about the opportunity to drive confidential self Sur in Ai and that's where actually confidential Computing comes in so no matter from where you are coming from the more you know of web 3 and confidential Computing if it's cooked together properly that's how uh

57. 31:26

    the the the community it's self will learn much more and start using it and that's where we are closely working with the ecosystem to do that like as soon as possible yeah um just a one ler uh trying to add up something here like uh for the user on boarding Journey no matter it's B2B Enterprise level customers or it's very retailed uh customer we find it like uh there's a the barrier for people to understand te

58. 31:57

    start is the the educational you know Journey you you mention you besides that there's another barrier which is the uh reability of um this uh chain of trust in Te technology so basically chain of trust means that it's not blockchain right means that when you see something you can verify this information based on algorithm to The Next Step as step by step people can actually maybe they get a hush but from the har and M tree they

59. 32:30

    can verify eventually where the root of trust is coming from and in this steps there's some maybe security assumption to uh uh to verify the trust so we we find it out uh no matter is for web two clients or the web three clients there's a big barrier for them to understanding and there's no tooling for them to the testing of course you can for example in n h 100 you can do remote test a uh for these uh inference results coming from

60. 33:01

    the model running inside GP but uh the problem is like con them verifi them onchain and and the retail people they don't uh even be able to uh get approv on blockchain and use it to check back where's the verification coming from so we see this as another barrier and uh you know we are our team is trying to figure out that um problem too yeah so I have another uh sort of interesting um another thing that that was not obvious to me before I started looking more into the kees so so I think it would be a nice thing to cover is

61. 33:41

    that when we talk about CPU I I think like the mental model the way I see it right and it's a very very simplistic model is that there's a key inside of the uh I mentioned it before there's a key inside of the chip right I can always go to the you know to Intel and say is the public key you know valid or like right and then uh anything that is encrypted with that key I know that was computed inside the key right so so like as as long as everything else around that I verified I made sure that

62. 34:13

    the code was the code I wrote or like the code I trust uh you know then then I know you know first of all uh you know the data I sent if it was encrypted with that key it would be decrypted only within the key you know and any data encrypted with that key was actually computed there right so with GPU it makes a lot of sense now with GPU GPU is not a general uh it's it's much less

63. 34:35

    generic Compu device right so how does it I I cannot send code directly to the GPU right so so how do I as a user so so I'm sitting in front of my laptop somewhere in Texas there's an h100 how do I how do I know that the code on h100 is the one that they have does make sense yeah yeah uh actually uh I think probably I can share the screen because I just have a right uh diagram to show that yeah let me quickly yeah let me do

64. 35:06

    that very quickly so here we have here we go yeah can you see that we never tried sharing screen on this te before maybe Jared it's not screen it's not available yet okay okay all right so uh yeah let me just simply describe that in CPU uh te um yeah you have the key in the CPU the CPU signed the proof so it can make sure okay this is the right program it's deployed on the right Hardware uh and in

65. 35:39

    GP on the GPU te uh side uh the thing working this way oh it's showing that's nice so uh you know at the left side there is the CPU so-called root of trust it's basically the the key mentioned by Alex this there is a key here uh and um on the right side this is the user and the user gets something called remote testation this and uh this execution uh this proof is signed by the private key

66. 36:06

    in the CPU and because only the right CPU has the key so everyone can verify okay this is the uh right proof proving that your program is running in the uh in the right environment and of course another related thing is uh how to verify the code you deployed is exactly the code running in the T it involve something we called reproducible build but uh to make it simple it's just like you have a smart contract deployed on maybe ethereum or near then you're going

67. 36:37

    to check the source code on the blockchain Explorer and if the code matches the one running on the blockchain in the smart contract then you can be convinced okay this is the right code nobody has manipulate the bannering code uh you Deploy on the Chan uh and to move it forward then uh yeah let me quickly show you this one uh we also have a uh yeah let me share this so yeah I'm going to uh skip the

68. 37:09

    other things yeah here uh uh yeah can you see this one oh yeah should be okay so uh when you are running the cput with the GPU uh actually the GPU become a co-processor for the CPU and you put them together this is how it works on the uh on the center side this is the uh cput and basically you put your program like your Lama three codes uh in the CPU environment which is just the virtual

69. 37:39

    machine and uh on the right side you have the GPU but GPU just follow what CPU tell is to do so on the CPU side you have the driver the driver will tell GPU okay now you can you can boot up with the te mode enabled and once it's putot up then uh the CPU and GPU can establish some kind of uh endtoend communication encrypted Communication channel so nobody else uh can see what's happening between the CPU and the GPU so here now you have actually two black boxes one is

70. 38:13

    the CPU and another is the GPU and the communication between them are encrypted so like if you're a hardware operator you cannot see anything running inside because everything you can access is already in so so the important thing here is that so GPU te requires the CPU te to work right exactly and there is no as I said popular gpus like h100 h200 there is no this is server GPU based right

71. 38:45

    there is no uh CPU without confidential Computing I don't know yeah that's why it's it works like standard you can buy h100 and 99% it will be uh Supply it in with te based CPU and but just imagine Alex uh what possibilities gives us this architecture you can easily run any containerized application inside of this virtual machine you can allocate resources right you can uh get detailed matric you can

72. 39:23

    uh promise to your uh cl to your clients all tolerance you can able to restore to run restore operations uh you can provide stable performance uh you can you can use all open sourced uh uh software uh for your for your clients inside of this uh uh with with no additional development because my my question to uh to any uh let's say web3 protocol who is providing uh Crypt based narratives for AIML purposes who will develop these I mean 99% of welldeveloped infrastructure for

73. 40:05

    web to who will Who will run same because from from the box uh this uh what I mentioned uh software doesn't work on uh on on ZK or fully homomorphic encryption text who will Who will develop this things right this stuff and in in in in in in uh using te you can easily migrate everything from Web Two to web 3 just cize your application on Amazon and run it in confidential Computing mode that's all zero barrier for web to businesses that's why I am

74. 40:41

    always uh my uh my my goal is in in super protocol my goal is to bring first of all web to businesses on WB through rails and using this AI narrative as that I said the AI is a nice is a great chance maybe last chance for web3 to be common and popular for everyone because AI is always about private data AI is always about how to train how to verify models using a lot of private or valuable data who will provide this

75. 41:14

    verifiability for for your customers for businesses and collaboration is a is a silver bullet and using te we can we can run and we do it uh uh with do it every day uh using this TCH that's why yeah this scenar works well yeah one one sentence summarize so you put Web Two in you get web three out so Web Two in web three out exactly so so I I wanted so so there are two things which I think

76. 41:46

    interesting uh uh among the things that were mentioned in the uh in the last couple minutes one in the last like 10 minutes one is that uh the draw back of uh well I guess there are two two potential drawbacks of the of the tees one is that we do trust Intel in Nvidia right so like at the end of the day if they are malicious those private Keys might have been not you know might have

77. 42:12

    been known right and right so we assume that Intel and Nvidia are not malicious and those private Keys indeed only exist yeah inside of the right so so that sort of like if not for that that would be you know the perfect solution right uh but I think the other interesting aspect right is that your slide mentions sgx fails and I know that n has a very strong opinion about that website about the vulnerabilities found in sgx right so so like like many people don't trust sjx as much anymore and you

78. 42:45

    know for many people sjx is TE right like people change right so so yeah what is the how should we communicate to people uh that is GX but fail does not make the whole technology worthless yeah I think this this is a great point I just pasted the link of sx. fail in this chat room uh and uh yeah maybe uh only few guys in in the space know that actually this uh this website is created by Andrew Miller uh he's one of the major contributor and he's the man you know te advocate in the

79. 43:25

    space so although you know a lot of people people say sgx sgx is insecure there's even a even a website called sgx f you know and but actually the people who create this kind of uh website are the people who pushing the industry forward so uh you know sgx is awful I admit that sgx is awful in terms of both you know the security is the first commercial uh te available has a lot of bucks uh and uh also it's very hard to

80. 43:58

    use Flash B team has been building on sgx since last year they have spent I think around half a year just to put the gas running in sjx still with a lot of performance uh you know uh uh regression but uh in this year we have the Next Generation te Intel TDX and amdv and you know also like AWS natural so for this new this kind of new generation is much better because Intel has already learned

81. 44:30

    a lot of lessons so most of the problems happening in the past was already uh fixed uh I think just two months earlier there was a hack on S from sgx but turns out it was only targeting you know sgx uh built in 19 uh 20 uh 17 that's very early and all the related bugs has already been patched so that's why I mean uh te is getting much better and the whole industry is improving so yeah

82. 45:03

    I think this is much better and and also you know for in terms of the developer experience uh in the Next Generation T te now you can just spend I don't know maybe a few days or even a few minutes to put your workload into the te because now you don't need to change any code specifically for the te so this is a huge Improvement this is one side another side I want to mention this that my mental model to think about you know the security problem of te is exactly

83. 45:33

    the same as the zero knowledge proof say uh for zero knowledge proof there is a theoretically you know ideal uh model to a mathematically correct model to make sure everything is uh is uh you know verifiable you know it's the mathematical proof and as long as you give it the right implementation you get this property so this is super you know appealing on the te side you know we can also have the ideal model for the te you

84. 46:05

    can imagine say there's ideal te where there's no you know back door in the te design and the te itself can be fully open sourced anyone can try to verify the implementation of the te then if say we can somehow Implement that then the next question is how we Implement that there are a lot of technical challenges but so does the technical challenges in the ZK space you know they have the performance problem they have the you know the generic purpose Computing

85. 46:38

    complexity problems on the te side we need to solve other sites of problem including like how can we make sure the hardware design can be fully open source the answer is yes we can do that and how can we ensure the manufacturer of te can be fully maybe not fully open source but fully verifiable by anyone who can you know afford to do that uh and the answer is we're not there yet but we have the hope so there are just a few things we need to figure out so that we can have

86. 47:11

    some te that has the fully open source design fully verifiable and everyone can even manufacture manufacture that there are a few you know Hardware researchers putting their passion into building such uh a fully open source te and that this is very promising I think probably we can see the fully open source te happening in maybe three or five years fully agree go ahead this Tech I mean te based producing uh chips uh

87. 47:45

    could be fully verifiable there is no let's say limitations to do that especially when fobs and and uh architecture designers are different businesses and yeah they can uh easily ver verify this process by the way and in general terms yeah we we do projects based on te and other Tech uh many many years and always security is uh in most in web three systems relies on cost benefit ratio right of course you can

88. 48:17

    attack uh in ethereum protocol but cost of attack is extra expensive same te when you because from article to article there are many many years circulating absolutely old school narratives where were patched and fix it many many times like relay attack for example you know what happens inside of te I can uh I can uh I can make State reset but same time you you can easily use trusted time from te to prevent this attack and why uh

89. 48:54

    this is this is what I'm talking about first of all any Tech technique should be used uh uh should be used and developed by uh by uh by uh experienced teams uh I mean regarding security and any system could be hacked including based ZK fully homomorphic doesn't matter because if algorithmical part could not be hacked something on top of that could be and we see a lot of hacks of traditional

90. 49:29

    protocols every day uh regarding te I never heard uh any uh any exploit or uh successful t attack from use on enterprisers uh like um huge enterprisers like apple try to hack iPhone how many attacks uh on on te based iPhone were successfully resolved I never heard about that I mean publicly yeah download and hack this is to say truth impossible but same time million of hackers trying to attack it same te te is a very popular uh including in all

91. 50:10

    devices and of course we see we see attacks every day on T but how much are successful and how much are uh are effective I mean in terms of uh expens and result uh and cost efficient uh that's why T is always and security is always balance between a price of attack and results you have uh and yeah yeah yeah oh I want to add one quick Point uh recently there is a very interesting debate on Twitter I think maybe two months ago uh some uh

92. 50:49

    some people claim that as long as you can have like a uh like a scanning electron microscope yeah absurd absolutely absurd yeah and in terms of I think quick answer is no of course no because it's very uh it's a very you know comp complicated uh process but another from another point of view so say a microscope will take roughly 100K to uh to you know just uh uh well it's 100K yeah 100K to uh to to to to buy one right so what if we just put you know

93. 51:30

    two Bitcoin into the te make it available for any hacker to hack so as long as you you you know you you you're good enough you can even buy a a microscope to extract the key and try to hack the 2 BTC from you know the the te by yourself but on the other side if say the T the the 2 BTC has been there for I don't know maybe half year one year and it's not being hacked that means well hackers you know they're not capable to hack it with just the SEC the economic

94. 52:03

    security of 100K us did someone do that already or that's that's for now hypothetical hypothetical to say Tru Han sorry uh I have to say uh using charged beam you can't read memory it's impossible physically yeah sure when you when you're trying just imagine Alex you are how Works electronic microscope you use uh charged beam of particles uh to read uh structure physical structure I mean of

95. 52:35

    chip but it's impossible to read memory because you are using charged beam it's impossible that's why it's generally from ground stage it's absur overestimate my knowledge of physics yeah exactly that's why a lot of absurd things but realistically to say truth t is the most common Tech uh regarding conf uh privacy and security uh preventing leakages of of data in the world every iPhone every Ledger every I don't know cloud is using te based uh

96. 53:12

    confidential Computing and of course a lot of markers trying to attack and we grow with with this Market together yeah yeah yeah and regarding the could you please just add about um tunelling that super protocol is also implementing because uh no matter how secure the any technology is always an opportunity to add something on the top and by the way yeah it's really interesting because we we work closely

97. 53:43

    with armm you know they are providing chips not only for servers for personal devices too and uh starting from uh version N9 or version eight uhal Computing will be uh available in arm in personal devices on personal devices just imagine what possibilities and perspectives we could have using this part because unification of these protocols andal Computing because all vendors were closely with each other just imagine what possibilities we have we talked with

98. 54:16

    major vendors I can't uh let's say name them but are they have plans to implement verifiability attestation in all popular uh uh let's say uh uh browsers and this this verification will be available from from the box uh I mean uh confidential Computing and just imagine your data will be prevented from leakage and will be protected starting from your touch on your mobile device or on your laptop up to uh server uh where

99. 54:51

    your data is uh uh computed and everything will be verifiable just imagine this these perspectives are super interesting regarding AI hype regarding data collection hype and we wanted to follow uh this uh this trend I mean in super protocol I I yeah that's super cool H you also wanted to say something right oh yeah actually that's the previous uh Point uh I want to add uh since you know when we were talking

100. 55:29

     about the sem the scanning electron uh microscope challenge so I would like invite every te project to c-u you know uh the maybe is Can it can be like a CTF project so we just put a few Bitcoin into te and make it a live challenge just put it there and uh we can even like build a prediction Market on top of that so if say You can predict it's going to be hacked this year then it's effectively become a hatch of your risk

101. 56:02

     right it become a insurance for T usage but if you believe it's not going be going to be hacked then you can even win some money by putting the bat on the on the on the prediction Market over you know the the the result of the CTF challenge great idea great idea H yeah let's work together to make it happen but but to say Truth uh are a lot of let's say requests coming from the market to use confidential Computing but there is a less number of

102. 56:35

     cryptocurrencies uh uh and more about AI uh prevention of uh cannibalization of business leakage data leakage and um we talk with projects uh how to regarding you know there are popular modular blockchains they wanted to collaborate between different uh Pro protocols they need oracles inside of that and confidential Computing could be one of way to resolve this problem infrastructure to scale web three uh AI oriented blockchains and their subnets

103. 57:07

     uh crypto AI payment protocols everywhere when you try to scale off something I mean AI base database big database uh you have only two choice you you have to go to Amazon or any centralized Cloud which not absolutely uh which is out of web3 scope in 100% or you can use uh confidential Computing for for for to resolve a lot of problems uh and I today I don't see any other Solutions readymade solutions to provide these Services uh that's that's we are and cryptocurrency of course is a very

104. 57:49

     important and idea of to keep to to to put uh few blockch few Bitcoins in Te it's a nice nice story we are ready to support it by the way it's interesting we will do it exactly yeah I think that's because inherently web stre has a higher standard than web 2 because in web 2 uh say if you check the confidential Computing provided by gcp Asia or AWS basically everything they offer just a it's just a way to convince

105. 58:20

     your client now you do need to trust the cloud but on the web stream with this is another story because now we want to convince everyone you know every literally every user I'm not going to cheat you because I make everything open source and there is a you know verifiable Hardware so there's no way to cheat anyone but you know this is already a higher standard compared with web2 web 2 just need to prove the uh the good face of a cloud provider themselves

106. 58:52

     yeah uh yeah in addition to that is uh early this year also went to the uh uh confidential Computing uh Summit in San Francisco I'm not sure if you were there or not but it was really uh nice because uh in the past people are struggling were struggling about how to find the real world use case of Intel sgx because sgx was the only available tech technology at that time uh but only until this year everyone I think maybe

107. 59:24

     90% of the people are thinking about using uh te to solve the uh solve the uh you know AI related problem it's both from coming from web two and web three uh I think is also inherently because uh AI is data inent uh data in uh intensive application and it involve a lot of trust and a lot of you know privacy related uh problems and this is when you know the te solution can shine so I I I think uh yeah until this year we didn't see a huge demount of te before but in

108. 1:00:02

     this year everything is changing yeah so speaking with confidential Computing Consortium sorry just um yeah with we are part of confidential Computing Consortium and last month we actually uh made the um super tech talk where our CTO actually unveiled like on a high level but not just our architecture but also um specific components that we have developed in order to ensure that super protocol is like what it is but at the same time to explain to confidential

109. 1:00:36

     Computing Consortium all the members what is how can actually web three and confidential Computing work together and how super protocol specifically implements this technology it's also available on our YouTube channel so if anybody wish to look at that please welcome as well cool okay I have last lowlevel question which I find interesting which is so so I think today uh one inter one interesting use

110. 1:01:07

     case that that is definitely easily implementable in is inference right so you you run your model like you have the orchestrating code running in the DX uh the model is running on h100 within the secure Enclave you get the but but that uh that use case in my head at least right it's it's all enclosed in a single machine right so let's say I want to run training that is that is running on the te right so then I need gpus to communicate across

111. 1:01:38

     machines is it something today or is it something that like achievable in the future where are we in terms of a great question Alex Alex it's a great question because te is a standard and there are a lot of applications supported from the box you can use VM very well-known solution and run it inside of in Virtual machines based on confidential Computing with no to H1 like the way I understand the security model the root of trust is that

112. 1:02:11

     the GPU uh is attesting to the CPU that it's valid CPU is attesting to me right but if two machines have to talk to each other they have to talk over infinite band over in Envy link that will be like where where is decrypt where does decryption happen where where does encryption happen like when I exchange when gradients flow the the Envy link how do I trust that uh that is not

113. 1:02:36

     intercepted yeah uh we actually uh dig into that a little bit uh So currently I believe just from what Nvidia has already released in their uh driver now there's only one to one mapping that means uh now there's only one CP or one virtual machine you can actually have multiple uh confidential virtual machine in one CPU but now uh they only support to attach one GPU to one virtual machine uh the reason behind I think there's no any technical challenge you can all you

114. 1:03:10

     can of course uh uh Implement something like we have multiple GPU even cross the server to share some session key so that they can incr and decry together but now from what I un uh from what I can see is that uh they have only implemented the uh key agreement between one CPU and one GPU you're going to pass through the GPU to a virtual machine uh on the CPU side but you can only uh pass through one if you pass through more then uh yeah I

115. 1:03:42

     didn't check but uh from what my uh from my understanding is it's it's not implemented yet but it's just a problem on the software side it's not on the hardware side and uh essentially once the GPU and CPU are paired they're going to do a key exchange it's pretty similar to https like you you you use https to browser your website so you have an endtoend encrypted Channel between the server and your browser and here uh CPU is the browser and the GPU is the server

116. 1:04:15

     so they have a endtoend encrypted Channel they're going to agree on some session key so that whenever the CPU give any input to the GPU it increas cry it first and then it transmits the saer message to the GPU and once it's arrive the message will be decrypted inside the GPU first before it get executed in the GPU and this also involve another very interesting uh observation which is uh the GPU can actually run at the full

117. 1:04:48

     speed but the communication between the CPU and GPU has some little bit overhead uh we did some uh test say the bandwidth between CPU and GPU can be uh 40 gab bits per second uh and if you enable the te mode then it will reduce to uh s uh I think it's uh it's 14 bits uh gigabits so roughly uh oneir of the band WIS this is because now you need to do the encryption and decryption this is a little bit overhead the bandwidth but beyond that the GP itself doesn't have

118. 1:05:29

     any performance overhead and if you apply this model to multiple GPU case then there are enve link between the GPU it can even cross the servers and then the problem becomes how you know those gpus can negotiate and agree on some shared key so that uh the one the the the Seer tax encrypted by one GPU can be also decrypted by another uh effectively so I think it's just a matter of the software but media also mentioned that

119. 1:06:01

     in the Next Generation GPU B 100 they're going to have this ready they also announced that in the confidential Computing Summit uh early this year and just yeah few minor in in super protocol you can from Marketplace you can use tunneling or overlays uh to provide this security network uh between independent uh Nvidia chap I I want to say that and the link is not let's say is not you can use independent

120. 1:06:34

     server providers to train your de your model but but the tunneling will it go will it go through the CPU or is it gpus talking directly to each other yeah I I think uh if you go with some uh uh some uh I mean third party solution instead of M link then uh is I'm pretty sure it's going to go to the CPO first do the encryption then you can do to the CPU or not uh of course to say to Alex uh uh in in in in decentralized

121. 1:07:12

     networks uh to say truth we are not oriented to serve high performance training because training from our point of view is more about uh centralized teams and Solutions how to beat uh 100,000 gpus bought by Elon Musk or hundreds of gpus bought someone I we oriented first of all to serve Market of usage where uh privacy and security for for individuals start uh data and AI is is important well but but I I I do want to

122. 1:07:50

     train a large language model on yeah you you always have this Gap I mean independent servers and low latency of course yeah uh quick addap to that uh yeah although just running the unmodified codes in the te mode with multiple te will be very challenging because uh I think it's just because Nvidia is too busy to work on that so we don't have native solution yet but on the other side I know two projects uh

123. 1:08:23

     one is prime uh rhyme in uh intellect and another is non research uh they are pretty hot on crypto trator recently they have some algorithm to reduce the training bandwidth between uh between the I mean the noes by from like a 100 fold to 900 fold so this is going to be huge although it's not comparable with you know just using the EnV link to do the collocated uh do the training on

124. 1:08:53

     collocated servers but it actually make the this distributed training somehow you know possible so I think uh this is also something uh I'm very willing to see how it plays out and because you know uh if we can actually distribute and even decentralize the ownership of the GPU we don't need to put them in a physical uh you know data center actually means that now the ENT barrier to train the model can be lower down a

125. 1:09:26

     lot and I think this is also the vision of near AI yeah okay we a little bit over time so let's do some closing notes if you guys want to say anything about how you envisioned the future of T something you're excited about or any other closing thoughts n you want to start yeah to say Truth uh first of all this year in super protocol we wanted to roll out about fully functional toonomic based uh test net uh and everyone is

126. 1:10:01

     welcome we have uh uh we are we are will be happy to collaborate with projects with confidential Computing needs uh and in q1 we wanted to roll out main net for everyone that's why we are absolutely open for collaboration specifically in AI uh ml industry and Big Data collaboration and regarding perspectives uh I believe that uh te just starting uh Market uh I mean TT just opening Market uh and specifically AI Trend uh

127. 1:10:43

     generates uh because a year ago T was uh was silent I and we use it in in local let's say projects for local needs but today T is uh T is a gold standard and uh we will be happy to collaborate with everyone who who are interested in Te maybe yeah jul you want to add anything yeah just uh just maybe on the other side like just recently we had uh one of the calls with one of T manufacturer and we were discussing their plans like not even for you know where we are now

128. 1:11:30

     in like quarter two or one year from here but they were really like talking with us about their plans in like and the challenges around te that they believe needs to be solved within 10 or 20 years from now and this is really in exciting and inspiring that like we live in this world right when we can collaborate with the the team manufacturers right and Ensure that we are on the next on the right level to incorporate the the focus areas the

129. 1:12:01

     availability the features anything that will apply to the whole web three Community once we are unleashing this all but also keeping always in mind what's ahead of us and like having it all together sounds really great so follow us in the space we will yeah we are happy to share and really be Community owned super protocol oh yes so I I can also go first and then H I'll be try to be quick so uh yes I think as theic mentioned earlier

130. 1:12:34

     uh in his blog um uh confidential compute orable offchain compute Technologies like te can be a perfect glue uh layer or glue uh component to connect what's happening in web two and what's happening in web three and uh you can't Den the demand in the market and we can definitely see how the demand is increasing rapidly in the market but of course the current Solutions are not perfect and for example the thingle Reliance on single Hardware root of keys

131. 1:13:13

     so this is something uh we are trying to S and in the meanwhile uh as a followup perspective we're super open mind to collaborate with all Technologies we are not just a fan of te we are also a fan of npcf zkp so whatever in cryptographic based Solutions can help to do something we are open mind to help it integrate it and maybe working as a defense in depths and another uh super growing case for our offering now is for AI especially

132. 1:13:49

     the decentralized AI companies and teams if you want to bring agent SDK or AI models into your onchain infrastructure uh working with f is definitely something can help to boost up so uh yeah this is what we are looking at now thank you yeah so yeah my closing comment on that is uh I think this year and the next year will be the the I mean the super cycle for Te actually so T is

133. 1:14:19

     very very promising uh and it's for reason first we have the demand uh now you know decentralized AI needs to me needs to bridge needs to NPC needs to decentralized the front and also needs te so we have a lot of demand and we also have the right we also get the right Supply because now we have the next generation of te we have the GPU te for decentralized AI so yeah I think all you know the Primitive all the building blocks to make t to the sky is already

134. 1:14:53

     there so now it's our time to make it really to the sky so uh on top of this I think uh there are few things uh I will call in action first is that te is still something new compared even compared with ZK te is really new ZK has been researched for I think a few tens of years so we need more research on te we need more security research we need more education let everyone know what is te what te is good and what is not good at

135. 1:15:26

     and how can we mitigate all the risk bring by the te because we just need to make sure everyone has the right understanding so that we can use T in the right way so this is the first thing I would like to call for Action the second thing is we need better accessibility to the software thinking about sgx we still need to spend like half a year to build something in real with sgx but now with TDX it's much better maybe we can reduce it to one months and uh is it possible to create

136. 1:15:58

     some kind of soft software so that even average developer who don't know te can actually just uh run something in the te enjoy the benefit in a few minutes if is possible and I think the answer is yes but that means we need to have some kind of high uh I mean high standards uh uh open source SDK maybe with all the best the security practice enabled and with all the boiler plate coats already there to reduce the burden for everyone to access to so second thing I want to mention is better

137. 1:16:36

     accessibility to the software and the third one Hardware like super protocol is providing the hardware and we are also providing the hardware and I believe more and more projects is going to provide the te hardware for the community so with more Hardware uh Supply then uh the accessibility uh of te to the uh developers will be much better in the future so I can see probably in a few months or one year almost all the web stre almost all the

138. 1:17:07

     Web stre projects is going to use te as a part of their solution and I think this can also bring a lot of benefit to them uh and uh in terms of of f we are trying to work hard to solve all this problem we're collaborating with like Flash B team and we're looking for forward to collaborate with near team uh to bring te very accessible to the AI Builders and the web Rebuilders okay awesome exciting times

139. 1:17:40

     uh thanks everybody for for joining us today h thank you guys for coming thank you Alex thanks for having us I