The Future of the Private ML SDK w Phala Network, xTrace, and NEAR AI

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   all right everybody how's it going so uh my name is Cameron Dennis I've been building a near for about four years uh building with near AI for the last about eight months to year or so it's been an amorphic organization but I'm here with uh Felix and Marvin uh we're going to be talking about the private MLS Decay and what it means for uh verifiable privacy in the world of AI so without further

2. 0:23

   Ado we love for you to introduce yourselves Felix you want to go first uh I think he goes Co yeah okay uh thanks having me uh my name is Marvin uh I'm the co-founder of f network uh I'm a running a non technology Ro coo and uh we been working in the space for five years on te and uh uh basically in the beginning like everybody in crypto space hatte te you know it's not purely based on crypto graphic Solutions but like we

3. 0:52

   believe there are some uh big potential in it and uh it can scale you know uh uh uh to uh know to help the web three standard eat the web 2 software so that's basically the our what we believe in so uh yeah we've been working in the space for a while and it's super excited to see how AI jump in the space right now and the ver ability can play a part of it yeah cool well just before we move on tell us a little bit more about F like what is f yeah F in reality it is a

4. 1:25

   abstraction layer of uh how te Works in a WEP 2 world so in know T first of all te is not Hardwares right like te is a system systematic work based on how security Hardware works and uh each uh cheap manufacturer have their own te system such as Intel SGS in TDX mdv uh Apple have its own te standard and uh yeah so when you unlock your face in your phone you're using te right where you open up uh bance these uh uh you know plus Keys you are using tees so

5. 2:02

   it's kind of already been wildely used but the trust Assumption of this web 2 te doesn't fit with how web stre you know like use cases Works in web stre scenario especially like uh uh in Dy AI we uh consumer don't trust the developer developer don't trust cloud provider so we only trust blockchain right and zkp so how to build a systematic work based on that it requires some uh software level of uh abstraction so F basically

6. 2:32

   do only two job to abstract these web 2T standards uh together and provide a software solution uh that fit with web stream requirement and another job is that we provide security Hardwares that feed these these uh uh Solutions yeah yeah we'll we'll get more into the details in a bit but uh Felix quick introduction of yourself and a little bit about XTR hi every I'm Felix I'm the

7. 2:56

   CEO and founder of XTR and we are a pretty new company basically started in June and what we're building is on encrypted context layer and data and memory for AI agents so what I mean by that is that every user can upload their data onto our system and then as a data protocol we can guarantee that we have no control of your data so that uh all the data will be safe and then still searchable by AI agents so what I mean by that is usually the guarantee for

8. 3:21

   database holder is encrypted at rest and at Transit and then xray get one further guarantee which is encrypted and use so AI agent can still send out encrypted query to xres encrypted database and get the data they want so this pairs perfectly well with te because we can essentially allow agents or larg AG model running in tees to have data access to external databases and on top of that we can also be the persistent storage which is called memory for a

9. 3:46

   agents running Insight which allows uh this to be on persistent storage it makes the agent running inside stateful yeah yeah just for summarize quickly there's a lot of words um so in short xrace is really providing the personalization like Railway so if you're using an agent and you want to say hey agent go buy me a cup of coffee that agent doesn't understand like whether you want whole bean coffee or you know ground coffee or

10. 4:12

    hot coffee or cold coffee like it needs to understand your preferences so with xrace you're able to use xrace to personalize your agent while ensuring privacy which is really a prerequisite to this like user owned AI thing that we're all talking about so um thank you and uh moving back to Marvin so uh you know your whole mission is to make you know trustless compute affordable with sort of through the private ML and also everything else that you're building

11. 4:35

    with f and I'm curious like how you see this uh kind of adoption by developers like do people really care about privacy and verifiability right now that's a great question so uh like our whenever I'm teaching I'm now saying hey use no use te instead of ZK using te instead of smart contract on eer our mental thoughts is if you canun something on instram do it if you you can run an agent on year do it like te is the

12. 5:06

    technology that uh as a a better solution than Amazon Cloud right like it's kind of in the middleware between uh how web 2 and web 3 works so uh we we are a believer of you know chain maxing and we are believer of cryptographic solutions will merge together eventually we already see the trend how this training works for example follow te attestations writing using zkp to generate approvers so that you can use the ZK proof to know you know as a as a

13. 5:40

    as a testation for how the uh te uh uh works so like so AR is like uh the web series adoption will haveen um many Technologies you know uh working together instead of single uh Tech so you said you said this word attestation yeah it's a very very important thing that everyone should understand can you quickly like explain what te attestations are okay and like where they live yeah so uh security is based on assumptions right like for example uh

14. 6:14

    like a h work is just imagine there's a black box in a CPU or GPU in these chips and the programs the data running inside this we call them Enclave the black BX running inside The Enclave actually is running in secret but uh uh uh better attestation system you can know exactly how this program running as it should be so how to know it first we need to know uh like how this program looks like so

15. 6:44

    you probably need to open source to make sure that the others can verify it and then you put the open source software inside this uh uh security uh chip uh onclave and execute it with a testation which is provided by the uh GPU manufacturer you know uh te systems you can know okay this competition result is actually running with a specific uh Hardware uh environment with uh you know a specific software uh version so that

16. 7:15

    you know okay with the attestation proof you know how this uh program is running as it bu so that's the very basic very rough abstract description of how te works but in reality is more complex for example right uh like uh it's based on a security assumption for the Hardwares so what if you don't trust in the hardware right like these things have been hacked yeah it can be hacked it it it can be

17. 7:46

    hacked so you should know like these security assumptions before you use it and another thing is uh there's a uh for example like with testation is not as simple as zkp with sing approver you can attach the entire you know MCO tree or the the entire har you know uh uh history behind it you in with te attestation it can separate the uh verifi abilities so for example some proof uh actually means that okay the environment setup is correct some proofs

18. 8:22

    means like the uh a Docker you know runs that view so Etc ETA it it can be into very different layers yeah so now we have a general idea of how tees work and what they are but um Felix is bit more on the the rag pipeline so can you quickly explain what rag is what it stands for and exactly what uh xrag is the sort of new primitive that xra introduced I think to me like uh AI agent is composed of basically three components right like the larger in model the second is a rag and dynamic

19. 8:54

    memory and the third one is a tool Co ability and then what is ability to co to yeah like calling API to execute certain functions and then what rag is is basically allowing you can think about when you're training a large Range model it's kind of like a a training school and we're doing the exam the rag is like open book so way for AI agent to reference outside data that is not outside of the training data that can be

20. 9:18

    like real-time data or like propriety data private data or all of that and then what we're trying to do is that we're making sure that all the data stored inside our Vector store are all encrypted and there still a still uh still available to be uh semantic search on so what semantic search means is that the agent can send out a query and then find the most relevant data in the whole Vector store and we're making the whole process encrypted which means that the query is

21. 9:42

    encrypted the data is encrypted nothing is leaked for in the protocol and then still the agent can find the response they need it's crit like super important yeah do you how do you anticipate this actually getting out to the masses like do you have to you know natively integrate with xtray every single time can these agent frame work sort of add X Trace by default can this be added to the private mlsd so everyone that adds that uses the private mlsd benefits from

22. 10:07

    private rag or you know crypted memory yes or cord or like design is REM modular so we allow like a chain agnostic and also agent framework agnostic as long as you have data that you want to work with certain like agent framework then you can use S system and then make all the data you want to interactive uh encrypt it so we really see that working with all the agent framework will become like this distribution Channel and then our ultimate vision is is allowing like one user to upload all their personal data

23. 10:32

    into our data layer and then they can choose the data Access Control to different agents they're working in for for instance they can only open their e-commerce cabins to the e-commerce agent and then the text data to the taex agent only so that they have one unified layer that has like shared memory and like and kind of gather all the fragmented memory and still work with all the different AI agent AI apps they want to work with yeah so I'll say um

24. 10:55

    I'm working closely with these guys because honestly I want a solution which is I want to be able to have a single window that have an agent watch my screen at all times verify that none of this data is being leaked to some third party cloud and it suggests questions that I can ask on calls and like you know suggests things that I should do with my with my browser and so um this is what I think that we are all sort of building towards like this like private

25. 11:20

    ml um but I was recently at uh uh an AI conference where everybody didn't think private ml was ever possible but but now we see things like you know Nvidia coming out all new h100s h20s will have this te support and so this demand is not coming from cryptoland it's coming from TR like the large you know Compu providers like you know Nvidia Intel like how do you sort of see this sort of playing out long term I think a very

26. 11:46

    important reason is like uh actually we uh know the information about private ml in uh not not not know the concept of you know this uh technology but more like how the uh uh business or already been working for a while uh through the last I think I3 uh conference in San Francisco like it's more like a academic one and actually the researchers from Nvidia Intel you know openi all of these guys are there all of them know that private uh ml is already working and so

27. 12:20

    but uh unfortunately it's more targeting on Enterprise level of users the only company who actually doing a good job using te for private uh AI is Apple all of the Apple uh intelligence uh cloud service going through the uh te Cloud powered by Apple M3 uh security chips so like exactly the chips running inside this Max so but I'm thinking like uh now it's more tarting on Enterprises but uh

28. 12:51

    it's going to be consumer faing like uh I'm not sure how many of you guys already used uh opening air operator it's terrifying yeah you have to like kind of like type in your log and input all the password in the cloud browser so that this uh you know uh agent can operate jobs for you in that Cloud Pro browser and it is having massive adoption to their consumers I'm saying I I I don't trust it no yeah I think I saw in Microsoft like the new uh I think it's taking

29. 13:23

    screenshots of your your window pretty much like every x amount of seconds um yeah it's horrifying I don't want that so Felix um for encrypted memory xrace um what are the industries that you're really looking to kind of go to market with yeah I think currently we're just uh observing like a burst of like agent framework building like a lot of agent Launchpad and everyone is trying to kind of work on the agentic site and then we

30. 13:48

    really looking into uh two sectors of one is like just encrypted knowledge based like how rag works the second part is on encrypted memory because this memory that is generated by larger range models Dre like all this T time scaling and a lot of reinforcement learning is really valuable and I believe that this new memory and cre memory will unlock collective intelligence between AI agents so that each AI agent can call on the data that they generated or Own It

31. 14:11

    by themselves and I think this new paradigm is like we the new data opportunities unlocked so we can kind of make multi-agent collaboration really happening where all of them have their own data because currently what I'm seeing is that a lot of multi- agent collaboration just like the same chat gbt or whatever larger with different prompts I think what really makes it different for multi-agent collaboration is the data they have access to the proprietary data the past experience they have and that'll really make a

32. 14:35

    difference when different agents are working together yeah I am super bullish on this approach like uh I I I think uh uh uh uh privacy is just like a basic uh feature but like very few uh people or Enterprises will pay for privacy that's the reality but uh based on this uh fundamental of privacy you can build a very good ideas for example example in in a private RG scenario I would say I'm super bullish on the sharable memories

33. 15:06

    because my Pino is I'm using enthropy I'm using open Ai and I have to upload everything about T and follow again again again there a lot of my tokens I believe many consumers feel the same too but this uh if there's a like personalized memory database that I can upload my knowledge for once and it can be shared and used to many different agent platforms that would be a super convenient B save a lot of computing resources I'm kind of curious like I

34. 15:37

    think one of the biggest issues that we're all going to face in this like privacy verifiability world is actually getting the real world to trust the code trust cryptography how do you see us actually bridging that Gap you think it needs to come from trusted organizations like apple or can it come from you know applied cryptography Labs or that's a very good question it's a tricky one like for example like what's happening on safe right now it's kind of a good example and I'm not saying I think safe have the one of the

35. 16:09

    best security teams in the world in web but like the baby heck viabilities is not because of the contract right the contract is verifiable so everybody using it for many times so it's safe but for the uh uh how the front end and server end works they still need to uh uh use some Dynamic back end of to you know for the uh user experience on the frontend and of course there's tons of alternative front end for safe but still

36. 16:41

    no career in right like why because if as long as you are using server there could be availabilities how to save that I say uh I'm not saying te can save the issue can't solve this would it be like a standard does it have to come from a company does it have to come from a government like what actually what will get people to trust verifiability in cryptography yeah I mean uh but I'm I'm

37. 17:08

    just trying to address that if you put how the back end of save works and how the front end Works a open source it right so that everybody can reproduce it by by yourself B uh providing centralized service is okay but for for example running inside te at least the uh security uh the white hat can always tracking the attestations of these backand lcks to see if there's anything weird happening to see if there you know the hacker changed the front end backend

38. 17:42

    code you know without your knowledge like the more exposure of transparency logs from Ai and you know backand of front end the better y yeah that's my point yeah that makes sense uh for me my security assumption kind of based on like the https protocol like using similar like encryption method and AES encryption so kind of like uh if you want to say I'm unsafe you have to first break htps or AES so that's kind of how we uh address a cryptography primitive

39. 18:14

    yeah so some sort of standardized protocol that really if you ask the you know Joe SCH off the street hey which https they're not going to know or they kind of trust htps as a default right compared to the previous times I think Google kind of force it to happen yeah because they have the Monopoly of like web search so they just kind of drank everyone who doesn't have the TOs protocol and enabled so not yeah so I

40. 18:39

    think there's some way to make it work I mean this was uh this the last question it's a little bit you know far out but you know assuming all of this technology ends up getting you know really proven out and it works um how long do you think it would actually take to get in the hands of Everyday People well I was saying it's not a technology question because is already in there like based on your our

41. 19:01

    Benchmark in very early in last year using uh te gpus like h100 200 you only have 5% of tradeoff to have V level you know inference for basically any uh model so it's already in there and for AI agents it's uh it's obviously is a it's it's already working very well in tees so I would say you know for uh uh but for training is is I say this year is is a beginning because uh like uh trust list or verifiable training is a uh is haven't reached a like a full kind

42. 19:41

    of uh status yet so it's only a time a matter of time we need the hardware yeah like it needs to be everywhere yeah which is happening yeah and what happened in the market is really help to pushing that a lot of like us companies and US citizens don't like for example deep seek right like we love deep seek as a open source software but you don't trust the inference service from China so how to solve that like we need some trust L you middleware to operate these

43. 20:12

    uh uh open source models and you can see how they collect your data or not collecting your data so I think you know it's a the market is pushing it yeah just to kind of repeat the question you know assuming all this works perfectly today what does the world look like in about 5 years yeah it really really depends on like how people how good the model is and how good the vertical agents are so I think currently I think

44. 20:36

    the agent still hasn't leave up to it hype yet but I see a lot of good transformative like uh breakthrough every week I'm doing like this uh nearly week with camera about like all the AI research and everything so I can see it happening but I think just takes like that breakthrough Point inflection point and then when there's Mass adoption then we can see that people actually need to upload all their data to make sure like the agent can get the best out of all

45. 20:59

    this proprietary or personalized data yeah thanks guys I me that's about it we'll be around if you have any more questions thanks for listening thank you thank you