NEAR AI Office Hours #18 Bagel Model Fine-Tuning and Data Privacy

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   light feel the custom field hack the world and make it tight blockchain flows near the one we're close to won't stop us now together we redefine the when near we'll find the flame empowered Minds rise and store use your own AI we explore hold the future hand in hand in this Tech driven Wonder okay hello everyone hope you're having great uh time I'm sure there's a

2. 1:04

   ton of people at in Singapore right now but uh for those who are with us welcome and uh we have here ban from Bagel I'll let you introduce yourself and we can kick off with some questions from both sides yeah sounds great um hello everyone and um um thanks uh n foundation for having me for this uh session with you today my name is Ban I'm the founder of bagel bagel uh we explain Bagel as a research lab we do research on AI and cryptography and we recently launched a product which is called the bakery you

3. 1:45

   know Bakery where bagels are made uh which is a decentralized model fine training platform with privacy and monetization for the models and data sets uh my background is uh both uh kind of in the machine learning and cryptography space I used to be the AIML infra lead for Amazon Alexa so my team was responsible for delivering the Wake word model to more than 100 million Alexa devices around the world and my journey in crypto kind of started back in 2017 where I was in the founding team

4. 2:20

   of the Bitcoin trading platform of cash app and for them I was doing some anti-money laundering machine learning but then I got to know about Bitcoin so like hey this great thing you don't need really any bank to send money and then learned about ethereum learned about near like all of those other different options kept going down the rabbit hole never looked back and yeah and recently

5. 2:42

   before doing bagel I was the ecosystem lead for a layer one uh blockchain awesome well great to have you here and excited to learn more kind of uh how you're approaching uh what you building a bagel and bakery as well as I know you have some questions for me uh so maybe we can kick off yeah like I'm I'm curious yeah how you see the space on kind of web threei side obviously you guys I think doing

6. 3:19

   more go to market on AI side first right to then yeah bring web three to them yeah which I think is is very important as well um but yeah we can just kick off there and kind of go from there yeah totally so um we see ourselves as AI first and blockchain side of the things are more of an implementation detail so we see I personally see blockchain as a coordination technology like why which allows multiple people who are probably not collocated at the same space who

7. 3:58

   probably uh don't trust each other who probably haven't even met each other in any way like haven't shaken hands to build any trust between them they can come together and collaborate with each other in a permissionless way so that's what blockchain enables fundamentally and that's how we enable uh that's how we see the usage of blockchain within the product that we're doing and you're right like we are doing the goto Market

8. 4:22

   in the web 2 AI space first because we feel like that's where the most of the demand would come from like we are a decentralized model fine tuning platform who will fine tune the most models people in the web to community who are like smaller startups we want to support the longer tail of startups who are using their are fine tuning models for various use cases so that's the crowd we go after and there has been some interesting challenges you know when you we try to go pitch them a blockchain

9. 4:53

   based solution some of them love blockchain some of them not so much we are being creative in how to solve that issue and but to summarize that's uh how we see it uh we see most of our users coming from the web to space not the web Tre space yeah I mean totally makes sense I think that the audience there is definitely you know dramatically bigger uh and I I think there's an interesting aspect that is somewhat underappreciated in web 3 is that web 3 companies

10. 5:25

    actually have very little data uh of Their Own right not I mean there's obviously a lot of like onchain data and and and you know Twitter and other public info but compared to web two companies who actually like in the business of collecting data the there's less data available on web3 side kind of by Design and so there's just less possibility to do interesting things there as well at least at this time I think I think we can we'll get there as we building out a little bit more of a

11. 5:57

    framework like in this user own dat side but okay sense sorry go ahead go ahead uh no I I totally agree like whole point of this user and AI is that like all the resources are not owned by the platform itself in web to if you're doing something you have to first start with by finding the resources and own it yourselves that's how you give the uh service to the users but in web3 we have

12. 6:24

    the superpow is that we can let users own the resources themselves and coordinat with each other and we become the coordination mechanism in the middle right so yeah I totally agree with you on the sentiment and so on the kind of under if we if we pick under the hood of the bakery how uh how does the coordination like are are you guys doing like multi multi- node fine tuning or like you just kind of Distributing load between uh individual nodes

13. 7:00

    yeah um so we are doing actually a really interesting thing like we're doing like Edge compute so uh also we are kind of like almost in stealth still so happy to like go really super deep with IIA like when we talked like one to one sort of in public but I'll just uh glance over it a bit uh we have uh this technology which we called GPU reaking it's a kind of like a flashy name to get attention but what it is actually we have partnership with decentralized networks um for for example we have

14. 7:33

    partnership with filecoin Foundation where we are getting all the miners from the minor Network to provide their Hardware to do the compute on our uh on our platform and we call it gping because the hardware is being used for mining and being used for the AI workloads from the bakery at the same time so it's like used in multiple places so and what that enables us it enables us to do Edge computer because if it's a you know uh data center which has the storage and the

15. 8:04

    computer we can make sure the storage like the data sets and the computer collocated that's kind of the deal we have all with all of these data centers and then that gives us way better performance in terms of the AI workloads that we do instead of like let's say AWS because on uh AWS you have something on one region and all the other resources in another region they're not collocated so there's this latency that's going on

16. 8:30

    right so that's how we uh basically do it under the hood in the bakery in terms of like the uh compute side of things and yeah so that's like the performance side and we also uh provide other advantages over centralized Solutions like you know hugging faces of the world is that we allow the users to monetize their resources too so it's like there's a tradability aspect of the models and the data sets on the platform and the third part of it is privacy so if you don't have any privacy of the

17. 9:05

    resources you cannot monetize like on hugging face you share your data set it's open for the public nobody's going to buy it from you right and it doesn't also incentivize high quality resource owners to provide the resources so you also enable privacy and how we do the Privacy we uh use Solutions like differential privacy we are working towards using Solutions like fully homomorphic encryption which is little

18. 9:29

    down the line because the technology is still not ready for prime time but that's how we see the future nice yeah no I mean those are different definitely important Concepts and and pieces uh as well as you know how do we do kind of Federated training or more distributed training o over like a uh kind of just very decentralized Network all all of the topics that people like for Federated learning we have seen some challenges is that um so we have like you know in our

19. 10:09

    web to customer Discovery like I personally have talked to like thousands of people I'm pretty sure you have as well I mean you have been in the space for way longer than I have but um so what we saw is that in Federated learning you actually have to reveal the model to a big amount of users like you're sending the model to all the data owners right and the data privacy is preserved to certain extent but I mean

20. 10:33

    that also is questionable somewhat there's like model distillation attack where you can get the data out of it anyway but um at the same time a lot of the users that we were targeting they were not open to you know sending their proprietary model to all this massive amount of data owners right so you preserve the data privacy in Federated learning but you don't necessarily preserve the model privacy there and

21. 10:58

    that was one of the walkers from our end to do the Federated fashion like something we are not exploring um Federated learning works great when it's all open source model right you send them you get the weights you aggregate them uh in centrally but it a lot of the web to AI companies are not like using totally open source they have some some of the proprietory solution Bak in there m makes sense yeah yeah I mean all of

22. 11:27

    these they definitely like depends on the use case and on yeah kind of do you want to reveal the input do you want to R the model do you want to um kind of ccate compute or distribute compute and and so you get like different parameters depending on the choices yeah and we wrote a like really um I mean quite popular by now is that uh research blog post on our blog blog.

23. 11:54

    b.net it's called with great data comes great responsibility like is kind of a pun to Spider-Man I'm a big Spider-Man fan myself and in that we kind of like did in-depth research U like what does it mean to use differential privacy what does it mean to use Federated learning what does it mean to use zkl what does it mean to use fully homomorphic encryption tees MPCs like all the fancy things that's also now coming and what

24. 12:24

    we realized is that like in short term lot of the solutions have to work together like there's no all be all solution right uh like you have to mix ZK with differential privacy some sometimes like some of the things that we are doing or you have to do Federated learning in uh some fashion like but no one solution can support the end to end privacy that you're looking for so yeah um but in the longer term I personally see a lot of value in homomorphic

25. 12:54

    encryption that's I kind of see the future when it's scalable yeah well it's an interesting question like how scalable can it get uh because definitely the current performance is still pretty far from uh from what people are looking for Curious what's your uh opinion on this on the Privacy side of things at least for AI well I think it it all depends on indeed the use case and kind of the requirements right um I kind of the way

26. 13:28

    I think of it is from like a user type more so like if an individual consumer user right like my data is probably not that big and like it sends on my device or kind of you know in my private CL like privately encrypted data um and so I want my assistant agents Etc to access to that I uh I think this is where you can either have you know e completely local stuff or uh you have

27. 14:03

    the fully homor encrypted like retrieval generation right so yeah the K kers neighbor over fully homor encryption is relatively performant operation right it's just a kind of uh interesting two Vector multiplication yeah so that that part is like possible we working with few folks uh who have been working on a solution for that um then if we're talking about like bigger uh either like a community or a

28. 14:36

    company right where you have like a group of people accessing some data and you want like a you know find you a model for that or model that can access to this uh that's when things that that's when things become more complex because indeed like ideally you don't want to move data out of your kind of private Cloud yeah so ideally you bring the model to it it and so then yeah like it's a model open source then it's easy to bring to it you just need to find now like is your private Cloud offers compute that's a

29. 15:09

    separate problem right like are you trusting in this case are you trusting Amazon or you like running you know on pram or you're running kind of in a distributed way in the first place so all of like those are all different like you know things that you need to handle I think right now again like if you're in private Cloud you probably are in the cloud already and so you can just

30. 15:30

    Leverage The compute that exists there um or uh I'm I'm curious about actually some of the hgx that uh gpus like h100s are going to be offering like how what is it what is the tradeoffs there um and then there's another uh so then you go to Next Level which is yeah like if I I want to kind of sell my model yeah without revealing it right that that's where things get harder um and I'm not sure that's like I'm I'm I'm not fully convinced that that's a use

31. 16:09

    case we should be optimizing more because models in the way are becoming more and more of commodity and uh I think I think keeping them open source is useful and kind of like building around that the models are open um but there is an opport to build something where potentially you don't have anyone having all the weights at once right and like nobody can like collect them kind of at the same time and you just have enough like you know

32. 16:41

    you you have some form of ratio coding right where like yeah data is like distributed enough and uh you can like you do maybe more computation than you would need if it was all collected once but then you kind of have this model as like a a a a a result of lots of machines kind of working together with some parameters around that that's more closer to what we usually consider like in file coins and not in like blockchains right where you have like larger number of replicas that are

33. 17:14

    together able to achieve some result I mean filecoin actually has yeah somewhat similar design right um yeah exactly and uh yeah so I think that that is like a uh an interesting like place in the tradeoff set where the model is accessible it's you know potentially like guarded by set of criteria but it is not like also owned by any individual part of it um and I know some people are trying to achieve

34. 17:45

    that so that's that's the so that's why like it's kind of a spectrum on one side and then like you know you go from I run it on my you know on my phone like Edge model to like I want to run it yeah you know in my cloud to I'm running somebody else's stuff maybe in sgx uh to yeah like I want to run it across like a uh kind of this decentralized network um that's interesting and then yeah and then you have like another

35. 18:15

    dimension which is like where data lives and like how it travels right and so you have kind of a like places in this in this Matrix like where you want to be yeah totally um curious to know about your thought on the inference time compute things that's like kind of came into you know limel even though this has been the case for me I feel like uh rag is an INF time Compu thing and we uh uh

36. 18:43

    in the bakery we provide Rag and fine tuning both like for me rag is like the shallow fine tuning thing where you don't change the weights and the Deep fineing we or change the weight so uh we as a bagel were one of the first uh players in the know decentralized rack space and uh we have always liked it but it kind of like lost to the Limelight but coming back with the inference time compute kind of conversation that are happening but then I also feel that even though model is open all the weights are open of the

37. 19:16

    model and the algorithm the same time if the mode goes to the inference time compute like whatever you do with like Chain of Thought or rag or whatnot and then some of this still stays like you know um kind of private and uh proprietary even though the model itself is open do you think uh curious to know your thoughts on that yeah I mean I think that there's like again multiple Dimensions here right I mean definitely inference time

38. 19:49

    compute in general is you know is a direction this is all been going right like I think everybody was pring with Chain of Thought um indeed I think think rag is also like a inference time computer is a good good way like how to reduce really large context right down to something that like you can process effectively um the I mean in a way like the gentic Frameworks right that's like re you know run multiple times as well it's the same thing that's like one of the things we've been looking at as well it's just

39. 20:24

    multi-shot the INF Compu is basically multi-shot instead of one shot well not not just multishot right it's you you can you know you can imagine it like editing its context it can do a lot of things right so like you have uh it can indeed like add it I mean it can like okay forget about what I was thinking that was wrong direction right Etc or you can have a search even at inference di where you like search different options and kind of score them

40. 20:51

    with a different model and then um so there's a lot of things that you know potentially can be done here in this this framework um but I think yeah I mean say same as we people like we don't try to you know immediately respond to the question so uh I think that the the interesting thing the interesting thing here is going to be now like for the models we kind of need to start evaluating them not just based on like how good they are answering questions but how fast they are or like how much compute they spend on answering

41. 21:28

    questions right yeah and so uh there's going to be like you know you're good at math at 90% in 5 seconds right versus at maass 90% at like 100 seconds so I think that would be interesting uh kind of Dimension that that uh we're going to add and and then I think the that's where a lot of the systems that actually have rugs and other system like other like information and structure and will

42. 22:01

    become very relevant because like that's actually how people think about this is like how fast can I get my answer how cheap can I get my answer and how correct it is um I also think the sorry go ahead finish no no go ahead um I also think it's actually a good way to democratize the space like right now it's I mean I'm talking about the hardware kind of right Nvidia has the almost like a monopoly on

43. 22:27

    the training side of things but if the more and more compute moves on the uh inference side of things where they have less of a mod I think that also opens up a lot of the hardware uh playground like a lot of different players can come together and um yeah so then the Monopoly is not there anymore right do you think it's the same way about this I well I don't know I think I mean

44. 22:56

    I think Nvidia is still there um I think we'll we'll see some interesting aspects around like if most of your training is spent on like unrolling like long Chain of Thought Etc right and kind of scoring them you don't need as much interconnect right like you can assume you know some parameter server model where you have a lot of replicas that are able to they just like enrolling the you know potential

45. 23:28

    candidate and then scoring them and then like only sending potentially updates from the ones that actually found like the right right response uh you can have a lot more kind of distributed setting and training setup and you can use Arm and you can use you know exit doing great job with like using arm for example and M3 yeah stuff like for inference so like I think there would be just like a lot more experimentation here but I don't know if if it's gonna like like I mean h100 is

46. 24:02

    just and and obviously black ho uh is just still generically a better tool for for a lot of like kind of stuff and so like and the you know the optimization between like how much energy it uses versus how much computer get because I think at the end like this is something people forget at the end it's also about energy because that's actually a base cost I mean yes there's like must imine of Nvidia but there's like also just how much like just to burning yeah and so um

47. 24:36

    and so whoever has like a good uh balance there and has good software and I think the other piece that's been kind of limiting is like nobody was managed to get really good software for those devices so yeah I'm not convinced that this Monopoly is ending yet but I think we'll we'll see more more setups and experimentations here that makes sense yeah how are you thinking about it yeah I also think the same but I also think it's like this community Hardware

48. 25:07

    comes into play and that kind of opens up the as you mentioned like EXO like they're one of those players in this space the Comm as the community Hardware comes into play with the inference time compute um I think that also opens up a lot of opportunities for the for coordination of different uh actors owning this community level Hardware at different locations and how do you let

49. 25:31

    them collaborate with each other in a trustless manner that's where more of the use cases of blockchain opens up as well right so a training historically has been even though a lot of people are working on the decentralized training part as well but historically has been like a lot of collocated things but inference uh opens up the playground uh in a lot of ways and I kind of foresee a

50. 25:54

    lot of blockchainbased products and use cases in that area yeah I mean well there is already a bunch right that's that's yeah exactly uh yeah so I actually I couldn't but um I couldn't help us notice your background I actually had the question for you before like what's the context behind the black dragon uh well it's just a like back from I was 10 that was my like username for video games and whatnot so

51. 26:36

    it just kept you know just kept kept staying with me as I was growing up and so at some point I probably had an opportunity to to change it on like Twitter and GitHub but um but then it passed right and now it's like now I'm just uh I'm just living the meme now uh no I like it black dragon is great and yeah I mean there might be some even interesting meme probability with black

52. 27:06

    dragon and maybe Transformer like mix it together some kind of pun because of your work uh in the Transformer as well yeah Transformer black dragon that's I'm I'm typing into chat GPT to generate me one right now yeah fair enough but also like um I guess if we're talking about the Transformer stuff um I mean I have read uh about your work uh in Transformers like a lot of hundreds of thousands of people around the world right

53. 27:41

    um then I'm curious that you know um if you had to kind of reimplement Transformers today in a decentralized fashion right like have you ever thought about it what did you do like let's say you had to yeah like change the architecture what would be the approach yeah that's a very interesting question because I mean one important part about Transformers is that it was optimized for the highly

54. 28:16

    parallel uh Hardware right gpus pretty much yeah and a lot of the kind of a lot of the motivation came from the fact that like hey we can actually do a lot of Matrix multiplications in parallel a lot more then um then you can do like if you just sequentially read words um and and so you can do a lot of this like let's just read everything in parallel and then kind of make sense of it over time right with like few layers of of the of the Transformer now if you assume this is a

55. 28:58

    question like okay when we say decentralized what does that mean right yeah so if we assume like what I was saying like hey we have this like decentralized cluster pretty much of machines that are like continuously learning and updating and doing stuff right well the architecture should be different right because like the the you don't have this interconnected parallelism anymore right you have a different type of parallelism yeah you

56. 29:22

    have you you have like almost like normalization that comes from the like said some nodes will go down and you know people see different like people feed different data from different sides so so you kind of need like a different architecture at the core um and then on top of it the the interesting thing has been that the GPT part right is is coming from this like hey let's just like language run a language model over yeah you know internet and and it's definitely you

57. 29:53

    know clearly very powerful concept but even now you can imagine you would take the you can take a gbd model and generate data set right and this is kind of what o01 is is like you can generate the data set now that is different that is like actually for more about like taking a question taking a problem and trying to solve it and getting a solution and training on that now not on predicting Next Step but on actually solving the

58. 30:26

    problem and so you're changing the value function uh and so here you would also potentially you know can consider changing value functions as like kind of distributed cluster now how exactly this would all work together I don't have like a exact details but those are just the concept that like okay we we we don't have like a immediately access to the whole context of everything but we have you know let's say let's say we have a million machines we have millions of contexts right yeah so how do you how

59. 30:58

    do you reason between them how do you interconnect them well you want a different again different architecture right uh to to to optimize for that but at the same time you have this yeah like you want any information to be encoded in multiple places in this you want to be able to potentially even recover some information you lost and like detect that some information got lost yeah um the I think the the thing that's kind of the

60. 31:30

    most I think the the most unclear part is just because like a lot of the basic skills right like I can kind of understand how the math works of the of the network to actually do right you know copying the answers finding semantic matching doing some things where it gets really unclear it's like some higher order logic and higher order reasoning that this mod like Transformer now show at like large parameters like like this transition that we see from

61. 32:03

    you know 70 billion to like 400 billion or above like that transformation is not very clear like like some of the scales that it's able now to do it's it's not very easily explainable just like hey you know even if it si the same amount of data or whatever right like why the the change of parameters allowed it to do like such such a better job at reasoning and so that one is a little

62. 32:27

    bit hard to to Think Through how to apply in kind of more decentralized distributed space so everything else I think is probably can map like you know like how to store information how to retrieve it because like those those we have as simpler concept we can reason about and kind of expand them to just St right um anyway that's I mean I I actually been thinking about this and with' uh we're planning to do like maybe a reading group in in one of the times

63. 33:00

    for uh this office hours to um to zoom in into like this different approaches around like distributed and decentralized learning yeah no that's fascinating I personally have spent a lot of time like at some point not recently like how to do a distributed attention mechanism let a like how how to make it like how can you distribute it over a geographically distributed set of nodes but still maintain a global context for that and I think there are some work on that but there hasn't it

64. 33:39

    hasn't been explored that deeply but the best solution I could come up with over during my exploration was like you do some kind of hierarchical tree kind of approach you get the nodes you get the attention and like get them one and at the center like whatever being coordinated that's the root of the the whole tree um yeah um that seems like that could be something that could work for the attention mechanism at least and which seems to be the hardest part of

65. 34:12

    this yeah um I mean in a way like the the rag is a you know a form of like very basic attention as well right it's like find I mean it's it's just not it's not a yeah you bring it back into to like the text space to feed it instead of bring the vector space and usually usually rag is done by like a a you know simplified version of a b ex right you don't have like this multi-step reasoning stuff but then rag itself can

66. 34:43

    be scaled a lot a lot wider um so yeah I think I think there's there's interesting Concepts there I think the the other one is just some form of mixture of experts right where you actually routing to different nodes and different um who may be experts in different uh aspects of of things right so you know maybe some node is more trained on medical data and some no is trained more on astrophysics data Etc you can imagine that setup as being kind

67. 35:16

    of effective way to like packaging data on different nodes um and then you still have like some more High LEL reasoning nodes that kind of uh take over from there um no makes sense sounds like you um you personally is um are very um bullish and interested in the rag framework itself like instead of like putting information into models and that kind of allows for better privacy as well because it the rag at least preserves a lot of information after the encryption stuff

68. 35:53

    uh like homomorphic encryption is that right yeah well I I think it's definitely part of the architecture right because I mean it's it's a little bit strange to assume that we can like plug in all data into a model yeah right because model is fixed level fixed number of parameters right so if you keep keep plowing stuff into it like ideally you actually want model to be the like just purely intelligence right

69. 36:23

    like it just does reasoning it does logic it does thinking the pro the problem is that there's no such thing as pure reasoning like you're doing like a lot most of the reasoning we have is like based on some common uh Concepts and the problem is common Concepts there's no such thing as common like there's it's it's all you know is Apple a common concept is Earth a common like it's a spectrum as well right like um like people are definitely not common

70. 36:54

    Concepts but like as you as you get close to you know kind of more like an object is a common concept and yeah like you should be able to reason about objects and so like where is that like line like it's same as you know if you teach teach a kid only about like you know sciences and then never expose them to anything else right they will have a very limited VI and ability to about things true so but then all of the data

71. 37:22

    like you know oh this movie star that did that Etc which right now this models actually know probably can live outside of the model right and be like fetched in the moment and so one of the things that I think I think some people done that but actually training with rock in the process so instead of like right now you let's say you and again with synthetic data you can actually do this more effectively yeah you can actually say like hey instead of like predict this next

72. 37:54

    word like beginning of the document predicts next word word right that's the current setup like you have no idea what's going to happen and then you like let's say you imagine you're trying to write them like you're trying to predict a Wikipedia article about like I don't know some Castle in Germany yeah right and you have no like model has no idea what the hell go is right so it's like just like a high level distribution of like cussley

73. 38:17

    Words yeah um but let's imagine you actually fetch some articles about this Castle from other pages and put them in a context and you say now predict those and you don't back props through the through the first part but you you gave it context on like what it's going to be writing about and now it doesn't need to learn the facts it just needs to learn how to fetch information and reason about information that came in the context properly right so you're training it to learn how to reason about

74. 38:51

    like more sophisticated contexts that contain actually answers and contain the information um and this is like you know what I was back in the day doing at Google with that is like you have Wikipedia article and a question and you're trying to answer we were trying train training that way instead of just predicting next token um and it's it's definitely more effective way of training uh but it's harder to get data

75. 39:17

    in that way right because uh like and and you may lose some things uh when you do that so so that's why like I think a lot more experience of can we like remove the need for it to memorize at least some like arbitrary facts and instead like learn mostly just the basic like thinking Primitives um yeah yeah it's interesting uh it's very related as well we recently like last week released a research post

76. 39:55

    on our blog it's called machine learning uh which is becoming a bigger and bigger kind of Hotpot of research nowadays like how can you delete certain information from a model without fully retraining it and um the sign we are seeing in the market uh is that like people are not interested in running massive massive models with like trillions of parameters even where a lot of this information are not really related let's say to the business that they're trying to build for right like they don't need

77. 40:34

    information about the film industry if they're trying to do some kind of astrophysics right and right now how we are seeing is that all of those information as kind of um hogging your resources and you have to run them you don't have any choice especially if you use the models from the open source ecosystem because they have a lot of information so from our perspective we see is that like this indiscriminate

78. 40:58

    data accumulation is probably not the future it's about like selective forgetting and figure out like which information you need specifically for the case you have and the architecture you mentioned actually very fascinating that also kind of falls into that criteria right kind of almost like right now rag is a comes after the output layer of a model right you kind of make it in the layers before that um it's very

79. 41:32

    abstract like how I'm kind of thinking about it right uh you kind of make it before that so that you don't have to have all those information in the parameters and like blow the size of parameters so high and then the cost of running the model is also uh dramatically lower in that way um so from our perspective the machine learning especially distributed machine learning is important but machine unlearning is becoming equally important as well as we

80. 42:06

    go yeah I mean I think I mean there's there's multiple sites here yeah like kind of can we indeed especially when you like fine tune for your specific problem yeah like it naturally kind of forget some stuff right that that's just like how statistics works so you controlling for what it forgets is a really interesting problem um I think I mean people people do a lot just like by generating a bunch of like

81. 42:36

    synthetic examples for their fine-tuning real data to like make sure it remembers the stuff they actually care about as well and so like there's all the tricks right to do it right now but I don't think there's like a fundamental approach which says like maybe it's like a almost like a generative adversarial setup right where yeah like after training a badge right you actually

82. 42:59

    check that it's like like it's scale Divergence from the original model is you know the like in this direction that like you care about and and and it's losing the things you care about like you don't care about them it's not it's gaining things you care so like maybe there's interesting side there um yeah but coming up with sorry sorry I was going to say like uh we are coming up with with uh new model architectures

83. 43:31

    live over Twitter yeah over Twitter yes um yeah I mean for me I I was actually always thinking more like can can we get it more um more than it learns like fundamentals and then you go and and you apply it to different spaces on top so like the kind of the core model is like as debiased in the first place and and you're almost like you almost punish it if it remembers like specific facts and instead like kind of can derive things from scratch that like that that type of model like it's almost

84. 44:10

    like follows you know this philosophy like basic principles versus being like right now because right now we're training it to be like you know speak like a human and and like have you know and you can like oh and pretend you're this like now you're able to do this like no let's just remove all that like get a basic like intelligence unit and then then you can like use it in different contexts on top right yeah and

85. 44:34

    so that's why I was like I think one way to do that is actually uh if you have like synthetically generated data that is about more about about using context information to answer questions or to write articles Etc versus trying to remember the facts so that like next time you write an article about the same CLE the same star you actually know it is as facts right so like the facts are always kind of given to

86. 45:06

    you yeah no that's fascinating um makes sense I have a lot of thoughts on that but actually have like a different uh also curious to KN your thoughts like outside of the tech stuff uh is that the you know business side of things like the web 3 AI web 2 AI and I have my personal experience in that too but want to know your experience well is that like what was the experience when you know you tried to kind of uh sell the vision of eer earned AI using the web3

87. 45:37

    technology to the mainstream web to AI people what was the reception from them yeah I mean I think there's like different type people have people in different camps maybe um there's obviously like AI researchers who are more interested in advancing the research and so like how it's applied it's kind of secondary for them yeah um and I think there it's it's it's more about like can we use the open source

88. 46:11

    can we use incentives to like do better research yeah um or you know like a very specific examples like crowdsourcing right using blockchain for crowdsourcing is just like way more effective like you can get a lot more done um for che for cheaper the and that like that specific use case for example people are excited because like hey you know I can get more data for less money faster because it can like scale up as well great yeah uh

89. 46:44

    there's a camp of obviously people who are in big companies who are like you know there's nothing wrong with the big company having all your data and providing you best best experience that it can provide right I think that that's a hard like argument to reason about because like the I would say a coercion that kind of being outside of that that you see is like really hard to like

90. 47:10

    present until until it's really like uh until it's very negative right like this like you know slow boiling of the Frog right like we give out our data you know Tik Tok is like prop you know keep sucking you in into it and you're uh they're better and better at like predicting what you're gonna spend more time on and you know and then um I think there's a camp that like kind of recognizes like that data privacy is important that uh but they not GNA like

91. 47:47

    particularly do anything about it and so this is where for us is like really about building really good products or better products that have privacy kind of in hnt to it yeah um also I think like the data privacy will just become like a cheaper way to run things I think just like the cost of of holding data will keep growing right the gdpr the California loss the like this yeah Hippa like it just keeps

92. 48:20

    growing right like so like at some point like it's just going to be like if you want to deal with private data it's like so expensive that like dealing it in a privacy preserving way is just cheaper and like it's it's like for us to to drop the cost like right now it's this cost is growing and this cost is just high and so for us it's like with technology and Innovation drop it under this that's when I think we'll see like

93. 48:44

    a shift and so I think like there's people who recognize this balance and right now it's just like there's no it's not like I can go to like you know privac data privacy.com you know use the SDK and done right um like there's no like whatever like a post grass but private for example like I want by private right there's no like offering like that right now that's like scalable and you know can serve million users so I think like as we're getting there on

94. 49:14

    different dimensions I think that's that'll become uh important and then there's people who are you know who joining we AI space because they actually see the value and excited about it and so uh I I I think um you know a lot of people recognize already recognize that like the our addiction with tech and kind of is is not healthy and we need we need a new way to approach this yeah it's interesting you mentioned that because I I read about this

95. 49:48

    somewhere is that like the whole reason you left Google to start near uh it was the AI company earlier uh and now it has become the combination of those two blockchain and but you and correct me if I'm wrong but I guess the goal was to build products which kind of consumes or Aggregates more data through the product itself and so that you can build better and better AI models is that um is that a correct information um well so I left because I

96. 50:23

    wanted to build a product that will have a feedback loop with user right that was the kind of fundamental part um and uh the the piece that was not clear uh like at Google like that you could build such a product and actually go to market with a yeah um and like there's a bunch of reasons for that including like I have I have tried to launch some products out of research and um the from a data perspective I mean when we

97. 51:00

    started Nei I like we definitely were not thinking about like data privacy per se we really thought more about like how do we change the user experience the consumer experience with this with AI pretty much right that that was like yeah pre you know pre- everything idea of like hey if you have an AI that can write code like the user experience will change dramatically you don't need SAS software you don't need applications you'll just build stuff on a fly yeah meanwhile you know let's build something

98. 51:33

    that like can build applications can build S software and like iterate to improve it so that we get there to that future yeah um the yeah I mean I I think we were just like too there was like research around like can we actually do that it was too early for like thinking about data privacy and stuff when we started exploring the blockchain side right which we like were wanted to use it for our own use case to collect data like the crowdsourcing was kind of our

99. 52:05

    main use case itself uh obviously the that became a lot more clear that this is like one of the fundamental offering the incentives is the one of the fundamental tools yeah and uh you can use all of this to like create a new new type of Internet new type of applications and services um and AI is is kind of your interface right it's it's a way you interact with it and kind of in a way

100. 52:37

     like you know do build functionality or or provide functionality yeah no yeah it makes a lot of sense and I'm glad you did like you changed both the a after changing the AI landscape you changed the blockchain scalability landscape as well which we in the community are grateful for of course um but yeah I totally kind of identify with your this data privacy uh stress on the data privacy like um that was my experience too when I tried to kind of

101. 53:08

     sell to a lot of these people you know who who are in the mainstream web to AI Community who are not necessarily you know like uh def Traders and doesn't really uh interface with blockchain in a day-to-day basis and they're like um so if uh the pitch was to them like hey if you have some AI which is built in decentralized way then the government cannot come and take your AI and they're

102. 53:33

     like get out of here like I don't care but government taking my AI I probably have bigger problems if the government is coming after me but one uh we kind of pitched them like hey but you can actually have your own data privacy you can have monetization or you can even interact with people to do Edge compete which gives you you better performance for your Solutions uh then you get like 10x better solution with probably 10x

103. 53:58

     less cost then all of a sudden they are very interested like hey what yeah how can we use it um so it's a like the technology that we have in the web3 space is like very very primed for disrupting the AI space it's up to us how we present it to them to the mainstream market um and if the presentation is the done in the proper way they are like really excited like I personally get more excitement when I talk to at a talk at a AI event like web

104. 54:29

     to AI event we talk like decentralized AI you can do this you can interact with people they're like oh this is like science fiction and the person before me they were Pro who were presenting they were probably building some AI customer chatbot so they hear that then then go to the decentralized AI like wow this is amazing you guys are doing really cool thing in the space right so I actually get like more

105. 54:54

     excitement from the web to AI folks when it's presented the correct way MH yeah I mean for sure I think like definitely PE people are interested in in seeing how because like I think the chat GPT kind of made it more cute like some of the like oh my data is used in weird way like then Reddit you know sold everybody's data for some money right and so like you're kind of starting to see like oh like all the

106. 55:24

     things that like you know and like file coin or RV like we're good examples of communities who like telling a lot like oh your data worth something right we also were telling that for a while but like now people actually see that oh actually my data is worth something right it just sold all my data yeah for this many millions right so uh I think that there's like just a lot more appreciation for that and kind of it's a lot more on you know in mass media as well uh and obviously for a lot

107. 55:56

     more people came in into AI where data is like one of the main resources so like now pretty much any engineer who is like has some curiosity is poking around AI as well right so uh so yeah I think it it's definitely primed and and companies corporations are also looking into this as well because for them it's pretty fundamental to if they mess up private data they'll like they'll have huge fines right so this is a um and like this landscape is changing very

108. 56:29

     fast and in a in a very unclear direction right between all this regions like you know what China does what Europe does what US does it's like completely different so yeah um so it's like as you actually the edge computer is really good example because if you just push the model to the user and then you can actually have a better product because you can have access to all of the data of like where they've been what

109. 56:53

     they visited what they doing what they at and you don't need to deal with it right because if you try to get all this data to your computer I mean to your servers like that's a massive issue but if you just send a model there and because you have all this data your model can actually be way better at predicting what this user will want or you know recommendation ads whatever whatever you do right so like Edge

110. 57:17

     compute is definitely like just a lot more uh effective because it has access to wider data the user and in turn indeed you can build better product and and better product is how you're going to win so yeah that's the whole idea of Apple intelligence right like they're doing this Edge compute on the private devices and you can do whatever fancy stuff with it and it gives really highly customized experience so that's great yeah I also like uh would love to know your thoughts

111. 57:51

     on like what's your priority right now with near I know like this Ai and intersection I guess especially for the AI and web 3 intersection and your work at near at the moment what kind of solution you're looking to see the most yeah so I mean the way we approached it is like how do we bring together almost a a stack of of projects and and offerings and so you know if you can imagine you have like a closed Source like Google open AI lab that has you

112. 58:27

     know a whole host of things internally is able to like effectively uh build and use that and and kind of people can collaborate Etc in open source right now we have like pretty much a Scutter plot of things right there's like you know hundreds of tools but they don't work with each other there's in web 3i there's you know hundreds of companies but they like all have different business models different tokens different like apis different ways of interacting Etc and so

113. 58:56

     and you have open source researchers who maybe don't even know there's like kind of this decentralized AI space and then they themselves or like building stuff in open source but then it's not like very reusable so how do we like bring it all together and really create like a striving environment that is competitive in velocity of research and velocity of product building that you can do of that

114. 59:21

     research by combining all of those different you know providers like you guys doing finetuning you know maybe somebody's doing inference somebody's doing like providing fresh data somebody's offering you know agent to agent payments and you know then going and bringing top researchers who are in Academia who don't have access to this compute to this data to some of the benchmarks like bringing them like oh

115. 59:45

     here's a platform you can build innovative solutions on so really kind of being the conduit of this like open source research and and and creating velocity in it yeah no uh makes sense love it we also love research uh we are a small team but we have done uh quite a few um uh actually fundamental research in the space um haven't gotten to the um kind of level of Transformer paper yet but we

116. 1:00:13

     hope to get there some point but yeah like we have this it's like a kind of plug for this two like we have this research blog like everyone please check it out blog. b.net uh yeah it's really popular among the research commun it's pretty new like we have been doing this for 10 months but yeah it's read over 112 countries worldwide still and aome yeah um and yeah I'd love for you

117. 1:00:38

     to check it out and know your thoughts as well yeah yeah I I just opened it so I'll check it out and and we let you know all right well awesome well thank thanks you for coming we're at the hour and uh been really awesome conversation so yeah everybody go check out uh the blog and I'm sure will see more things uh that we can work together on uh later on yeah looking for a thanks uh thanks for having me again