REDACTED Hackathon AMA with MASA ai

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:06

   T about to blow up it's time to testify CH signatures launched we reaching for the sky get locking the now sign change the his anytime anywhere I cat SEC chain technolog un the game chains over yeah t about to blow up it's time to testify CH just launched we reaching for the sky shcking e

2. 1:44

   come all right welcome everybody um glad to have you here today this is the redacted online hackathon AMA with Masa and uh if you don't know they are a project that does a whole lot in the AI space doing a lot of stuff with near and we're going to get to know a lot more about them uh first off I'm going to do a little housekeeping real quick before we all do introductions one the hackathon there's still time to sign up lots of bounties I'll pull the bounties up here I mean we

3. 2:32

   got over 600,000 in bounties aasa has some specific bounties that we're going to get into a little bit today uh we got a whole lot of bounties happening right now the deadline for submitting projects is October 25th so you still have plenty of time and you can stack bounties secondly this is the online heon then we also have November 9th through 11th the redacted event it's like an event it's a

4. 2:59

   conference it's an IRL movement and that will be in Bangkok just like very close to the Devcon venue so if you're going to Devcon this is a few days before the few days before it so catch them both and reclaim your sovereignty there all right housekeeping done let's introduce everybody uh first the co-host Joe Jump On In and give yourself an introduction yeah sure no I'm on a lot of these things H yeah no I'm Joe I'm representing near

5. 3:23

   devhub uh we handle the entire developer ecosystem on the near protocol and clenia Jump On In I'm super excited for you guys MAA is a great team hey everyone my name is Clen I'm co-founder of MAA first of all great intro and great opening the anime actually reminds me of avatar The Last air bender hopefully we have some fans in the crowd today um super glad to be here we have been working very closely within your team for quite some time very excited to Deep dive into what we're doing here at Masa especially when

6. 3:53

   it comes to developer applications and uh Brandon let's let's Jump On In yeah good to see everyone today um Brendan technical co-founder of Massa um really excited to be here to dive into what we're building at Massa uh give you some overview and insight into our kind of philosophy why we're building this who are we building it for and yeah thrilled to working with uh near on pushing the decentralized AI

7. 4:18

   ecosystem forward awesome man this is this is really exciting so before we jump in we know that like near has got three verticals blockchain chain abstraction and then of course the AI use your owned AI all that so that's where you guys definitely land and help out so first off if you can just kind of explain what your project is for like the audience who've never seen it like they just show

8. 4:41

   up yeah I I'll give a very high level overview of what Massa is so uh Massa was born out of uh both an ambition and a goal to democratize decentralize and give open and permissionless access to data scale breaking the Paradigm and the kind of strangle hold that large monolithic centralized organizations have on not just our personal data but data as a whole like we all know the Paradigm of you know Facebook now meta

9. 5:16

   um Google really owning billions of people's data to then monetize that using ad Revenue we're now moving into a paradigm where companies like scale AI who Source label and structure huge amounts of data for foundation models the likes of anthropic the likes of open AI even meta's llama model are all trained through a single institution having let's say 80% of the share of the market maybe even more I think they just hold back from saying 100% because

10. 5:46

    they'd probably be broken up for you anti-competitive nature and we decided that this Monopoly had to be broken up and it had to be put in the hands of individuals where any individual globally can both contribute data to the network and any individual or developer globally can access this data that means unlocking data from wherever it lives through a decentralized network of nodes that brings together web data Twitter data conversational data that's been

11. 6:20

    extracted from speech to text structures that and makes it available in a very easy to consume easy to use format for any AI or machine learning developer I do want to emphasize also machine learning because we often just bundle Ai and machine learning together and they're two unique disciplines um AI is where the attention is now so very much for agents using like rag fine tuning or

12. 6:44

    pre trining but then let's not forget this stuff is also hugely hugely valuable for um machine learning models so that's a kind of bit of background us giving high quality data to developers anywhere in the world okay that's great so if we think about the uh AI stack because I I for some people on here they might be we I I'll bring it way down to just like kind of like just learning about AI you've

13. 7:07

    got a stack right and it's kind of basically like data Computing models benchmarking and like agents right that could be a text deck that's kind of the near text deck and there's some more variations kind of it and you guys sit at the data area and you help part of the Tex stack and data oh yeah sorry go on yeah I was gonna say I really really love how you break down the AI infrastructure into a few pieces because sometimes when we see a headline like meta spending hundreds of millions of dollar building llama 3.2 model we only

14. 7:42

    see AI associating with model but the AI stack is much more complex and dynamic than that as you put very well Jared we have four layers of the AI Pac first we have compute then we have inference or model then we have data and then at the top of it we have applications that's the chbt the apps and the AI agents that consumers love and interact with on a daily basis at Masa we hold the view that data is the oil for the entire AI staff data is the most important

15. 8:13

    currency and the part that deserves the most amount of decentralization how get getting real time high quality data to developer is really key in not only building more affordable AI agents but also unlocking a whole world of possibilities when it comes to building new AI applications and building new AI agent and M we've really collated this massive network of 48,000 node workers for scraping and structuring data in real time and supplying such data to AI Developers for

16. 8:43

    them to build cool AI agents AI applications that we can dive into today that's awesome and then Joe I definitely want you to jump in here but so when I when I think about like AI the most important things are going to be like energy as you can tell like the world like how do we run the energy of all these like like you know warehouses of h100s and then two it's going to be data and so you have 48,000 which is quite a it's a massive number all scraping data

17. 9:07

    from all over the web that can then be funneled into specific ways into AI agents and then also machine learning but we might get into that as well right so that's sort of the yeah explain like I'm five high level of like data scrape data take data give data yeah exactly like if we think about um in the future agents becoming the like reasoning almost thinking machines these machines need eyes and ears to be able to interact with the

18. 9:37

    world those eyes and ears are data feeds or pipelines like we have data feed which is Optical and we have auditory as well um at the moment the predominant format for getting data into an agent is text based although that's changing specifically changing with you know open AR uh releasing their uh Speech engine which was like over the last couple of days so even in that instance the speech is converted into text and ultimately Finds Its way into the Transformer model

19. 10:11

    to to be consumed so we are building eyes and ears for AI agents and also creating a collection vessel almost like an archive or a library of all that data where it can be accessed by researchers by scientists by Machine learning by AI Engineers to in the future or today tune those models and we have two steps that we really go through we have a training of the model which creates in this instance a generalized Foundation model

20. 10:43

    that's very good at solving a general set of problems we release that this is what anthropic meta and this is what open AI are doing that's then released and within that if it's an open source open weight model they can be fine-tuned or or you can add additional data into the model at the prompt level which is which is rag retrieval augmented generation and that's really where you're in a sense creating an extended memory like let's say we all wanted to

21. 11:14

    have a discussion around hockey I'm I'm British I know nothing about hockey ice hockey um but I could quickly flip through a book put that into memory and then have a conversation that is what rag does it takes those pages from that like hockey manual and makes it immediately available and retrievable for the uh agent so it's not just eyes and ears it's also like how do you then transfer the data feeds which the eyes

22. 11:37

    and ears of a specific like topic like maybe that's a lecture we're listening to or in this case a book that we're feeding into the actual model or you know the most recent conversations around a topic on Twitter or the weather from weather.com or the price from coinmarketcap.com of of certain assets we feed that in through our pipelines and then we embed that in a memory or create a library or a dictionary that can be used for retrieval by the model or in the future for scientists to then train the model and I think that's a

23. 12:09

    really good way of thinking it thinking about it man dude this stuff is so cool like because I think AI agents are just starting to get into like the the the conversation you know for us for people into AI we're like yeah agents are the thing but for I think people are just starting to get into it uh Joe jum if you want to jump on in here we've got some yeah I I'd love to to help the audience better understand too a little bit of like why is you kind of touched

24. 12:34

    on a little bit like why is the decentralized piece of this though so important like what the argument might be just like I could just still I could still scrape data normally if I really wanted to these other companies are doing it why this model and what does the future really bring with this yeah I think there two Dimensions I think there's the ability to scale where um with centralized institutions like there's companies like bright data apathy

25. 12:59

    uh scale who does a lot of the data delivery for um AI companies is known for labeling and annotation but in fact they do have a procurement Department that goes out and gets a lot this data as well it's just not as well known they either buy or they Source themselves um and when we're thinking about acquiring data at scale in a timely fashion you're either crawling individually you know

26. 13:22

    maybe 50 100,000 web pages which takes a long time or you're spreading that load across a distributed set of nodes who are doing the work in parallel now typically the engineering uh Talent or the non or the expertise to run parallel kind of computer scraping jobs is quite technical so it's either going to cost you a lot of money or the developer resources be hard to acquire what we're doing is we are on a foundational level

27. 13:54

    building our infrastructure on a base of that distributed workflow so this is like typically what you've seen in in data jobs in the past like spark Hadoop have like spread workflows across many noes to speed up a process um and then allow the data to be coales May using map produce or another methodology to bring the data back together and you're ending up with a unified data set so we're

28. 14:19

    applying this model at scale and we're using either unused compute this may be on someone's laptop so you can run a node on your laptop this means you're not run running in the cloud with AWS or gcp it means you're able to monetize and like Leverage unused resources in your background so it's an opportunity for a revenue stream for a user which is adding a lot of value to different markets but then more broadly it's

29. 14:43

    actually a feature not a bug that this is uh decentralized the feature is you're able to spread a significant number of requests for a large data set over a distributed network of actors that are being coordinated and orchestrated using an economic model to uh maintain honesty and resilience in your network um that ultimately leads to a faster um more secure response and ability to acquire data at scale so it's

30. 15:13

    a feature not a bug and really this kind of data acquisition is much more suited to the permissionless nature or the decentralized nature of of blockchain like we have today yeah and I'll probably offer two cents from a non-technical perspective because I know our audience is a mix of technical and on technical folks uh here at near protocol and at Masa um we often are asked why crypto worldwide

31. 15:37

    decentralization for AI right isn't decentralized centralized AI great enough they have a huge amount of resources and a lot of Brilliant Minds focusing on building Central aai my answer is normally twofold especially in the context of what we're building here at Masa one is web free incentives as Jared mentioned up top um being able to build a network of 48,000 Noe workers in such a short amount of time it's really

32. 16:04

    impressive how do we manage to do that web 3 incentive plays a very key role in expanding and growing a network of contributors very quickly in a fairly easy way the second thing is really coordination as Brenan put in a more technical term we have designed this tokenomics and incentiv by incentive system to make sure that everyone is providing the highest utility of data and work to our decentralized network of data so if you take away two key words incentives coordination when it comes to

33. 16:37

    why AI needs crypto and decentralized Technologies yeah that's I think that's a really great point and a lot of discussion I hear outside of the crypto side of this is that the financial aspects right now of AI seem uh disproportionate and you can see if you you really think out long term you can really see how it's going to just widen that Gap even further and it sounds like one of the things that you might be solving is aware of like if I

34. 17:08

    have proprietary data using you as a network does that allow me to also be incentivized as that data provider it's actually even easier than that um so what we require from node workers is simply your bandwidth in most of the developed markets we would have unlimited data plan instead of just wasting your unlimited data plan you can very simply set up a node on your devices and then run a Masa node what you do passively on the mastera network is that you scrape and structure data

35. 17:41

    per developer request so it's almost like a set and forget kind of model you don't need to actively complete data work versus in the more centralized version let's call it the scale AI model they employ a huge amount of human beings across the globe to manually annotate data and label data and Masa on our decentralized Network you just set up the environment you forget about it and then you passively do work in the background and earn rewards on the back of

36. 18:10

    it okay let me just jump in real quick that's awesome so basically I'm sure there's a maybe a tutorial for that and we can put that in the link in the show notes for sure because that I think is something that like so for people hacking in this haathon we have over a thousand people already there's like I think 1,50 people signed up for this hackathon and growing they with they can hack but for people listening who aren't hacking this is something else that someone can do as

37. 18:34

    well is it something that like someone like a normie could do or maybe not norie someone who has a computer who's able to do this or does it take a bit of technical knowhow we're getting there um we're getting to the point where I would say in the next few months we will have uh an executable for both Mac and windows that you'll be able to run through like a a guey or or a UI flow on

38. 18:58

    on your computer simple as like installing uh an application like you normally do so that's coming very soon and that will massively lower the bar entry and allow that number to significantly increase um beyond the 50,000 or the 48,000 where we are today nice yeah i' I'd like to talk a little bit more about the con I don't even get this like technical but the concept of having the validator nodes set up so is this something though where you know it

39. 19:31

    does hit a point where uh you have too many like it doesn't like fully scale all the way like so if you become over almost oversaturated or is it something simply where if even if you hit a huge number everyone still is able to still earn rewards and you are continuing to distribute that all the way through we have a a very interesting way of Distributing curves to like add some mathematical terms here use a machine learning model right now to Define um a

40. 20:04

    ketosis curve that we then apply a cumulative distribution function to so what this ends up doing is it ends up normalizing outliers so these could be nodes that are for example scraping a disproportionately high amount of data for example um maybe there is a partition in the network and a particular subset of nodes gets a very high number of requests this normalizes those outliers into this curve uh and then actually in heavily incentivizes the top let's say 10 or 20% so you end

41. 20:37

    up with an S curve that represents the reward curve um and it actually incentivizes competition to get to the top of the curve so what this means is that workers uh on the network are constantly thinking about optimizing their infrastructure and the way their node responds to get to the top of that curve that's typically driven by the quality of data which is defining that as an interesting problem um along with the amount of data we we look at quality

42. 21:09

    as in what is in most high demand and obviously that's changing it's a stochastic system so what we're seeing in demand today may change next week and it's very important to be able to adjust dynamically the reward curve based on what is most in demand and what is most value to the consumers on the network to those people that are consuming this set of resources and to answer your question about you know is there a scaling limit not necessarily um I would say

43. 21:41

    that definitely geographically it benefits local clusters where you're joining the network you get lower latency higher performance with nodes that are in as we would call it your locality like maybe they're in your hash table and some are not so they'll get maybe preferential treatment if you're using the Network um but we typically see like work distribution as evenly as possible um across

44. 22:06

    nodes but allowing for enough redundancy where we're scaling horizontally with nodes that join the network so it's designed to scale horizontally it's slightly sublinear not fully linear scaling um but we uh are always looking for more nodes to provide more bandwidth um as demand is going up exponentially just you got actually almost answer my next question here about so as the consumer of it so as the developer of it it you know that I was going to ask how are you dealing with you know different

45. 22:39

    locations of as a global network but it sounds like again you are trying to both localize it as best as possible if if it's available so that I can I can lower my latency from there that's awesome exactly yeah and and then we have a a bit of mathematics or uh a a utility function that adjusts the fee that a developer will pay based on the amount of resources on the network so it's a little bit like how gas adjusts its

46. 23:13

    price the higher the demand the smaller the pool of resources the more you've got to pay to sort of get priority on the network and that's a that's a dynamic calculation cool this is this is awesome like the whole like nuts and bols Let's uh I'm going to Pivot just a little bit right I'm going to I'm going to I I like I like the mix of like let's get deep technical for people and now I'm going to take it to like okay let's talk use

47. 23:37

    cases you're you're you're building in this hackathon you've got some bounties there's some Masa bounties dealing with agents so I think let's talk about some use cases of Masa and how that works in this like area of near yeah absolutely so we're very fortunate to be part of the reacted hackathon um we're actually bringing the entire Masa team to Bangkok to participate and hopefully uh meet as many developers as possible on the ground that reducted for this haathon

48. 24:07

    specifically we have a bounty that would uh invite uh developers to build different kind of AI agents using Masa Twitter data as we all know Twitter data has many many use cases I'll probably share two today first of all I'm sure a lot of you love KS or key opinion leaders or influencers on crypto Twitter you can build a k AI agent using Masa free Twitter data so in the future maybe you're asking Elia from near or the AI agent version of Elia uh through your L

49. 24:42

    chat interface and then you'll be able to ask Elia question such as what are the latest developments from near what are the road map of near in the next uh couple month etc etc so that's number one k AI agent the second example I want to share is another topic that we all love nowadays which is mcoin you'll be able to build a mcoin AI agent rather easily with free Masa Twitter data and

50. 25:06

    then you'll be able to ask questions such as hey what are the hottest Bean coins um what are some price predictions you might have for me my AI agent etc etc those are only two examples but they're such a wild wild world of possibilities for anyone to be building AI agents using mafa data yeah let me follow up on this because this uh AI agent leveraging thing of the data so let's say like in um in in crypto right people are will into defi

51. 25:38

    so you could create an agent and you could you know they'll go to MSA and then uh tomorrow's Workshop you guys can actually show how like you leverage the data and like I maybe there's some parameters on you can say okay when you're looking at a Twitter data if you start seeing a certain like maybe I could ask the agent to have like a list of 20 tokens they could kind of keep an eye on it and then

52. 26:01

    if like sediment starts rising or lowering within it that agent could start doing things right is it like that does it work like that as well or do I always have to ask the agent and it retrieves the data or can I like set alerts yeah it's a good question I think there are two approaches um if you're the lowest hanging fruit here I believe is using a base model and like Clan was

53. 26:25

    saying with that base model um adding accurate context that is based on the prompt it receives so um using rag like retrieve augmented generation if I was to ask an agent you know what is the history of Bitcoin um it would create an embedding with that Pro with that particular input prompt that embedding uh uses natural uh language processing and it Vector like creates a vector which is essentially an association between words so the

54. 26:59

    association here is history and Bitcoin it will then use that Association to look up data that we've already put in a pre-populated database in a vector store and it will retrieve any data where there's a relationship to bitcoin or history from our data set structure that and then put it into the agent so it's like it's like looking on a index of a book Finding the pages that have particular topics on like let's say we're looking here for history and we've actually got three references in the

55. 27:30

    index we can then go and grab those pages consume those pages and then use that to respond intelligently as a human being that's how we look something up retrieve it memorize it and then put it into action we have that same process going on here with agents where let's say we through the Massa protocol scrape all of the keywords for the top three cryptos every single day um we actually

56. 27:54

    have data sets available for people on um it'll be on the near Hub also on hugging face you can just get access to these data sets immediately they're constantly updated and you can use them out the box you don't need to set up your own uh protocol node you can just use this data to get up and running really quickly you can then think about the best way of creating your like rag chain and what a rag chain is is the

57. 28:17

    chain of logic through which an agent acquires information to put into its memory like maybe it's going to go out and fetch that in real time like maybe part of that chain is is if I put a keyword in that says what is the price of Bitcoin now the fact that I've got now in there triggers part of the rag chain to go out and fetch the price of Bitcoin in real time retrieve that and

58. 28:43

    then put that into the model so as the agent can then consume that data immediately as opposed to what was the historical price or what is the sentiment over the last three days of Bitcoin which requires a very different data set so the most important consideration here is thinking what is your base data set for the product you're building is that product going to be an agent that speaks and talks like Jared well if so we would go out and get all of the historical podcasts that

59. 29:08

    Jared has been on convert those speech to text and then we' isolate Jared as a speaker and use that as context to maybe pre-train our agent or feed in to give the agent the ability to speak like Jared okay sorry this is kicking off ideas and then Joe jump in here because I could just this what I like about the AI and crypto space right like is my my thought is okay so now we're

60. 29:33

    going to create a Jared AI right like I want to make something that's similar to me so I can I can scrape all my Twitter I can upload podcasts maybe writings anything that you've ever seen right and say I've got enough that I can put that into an agent yeah is there a way because of the sort of decentralization and blockchain aspect that that can somewhat be gated a little bit so I have Pro like I have access to it but someone can't just instantly take me and clone me or have an agent clone a thous of me

61. 29:58

    and then send an army of me on like this is what I'm trying to figure out with like where does it stop and is that on your guys's end or is that not a big deal is it just like we're not there yet that's a very interesting uh theoretical scenario um I I think there's a lot of Builders out there that would like that Avatar or that digital twin of yourself to be owned by you right so you kind of

62. 30:23

    have rights to that synthetic version of you um there is nothing today stopping anyone from creating a synthetic Jared though um and masquerading with synthetic Jared which is an interesting problem right like how how do they deal with that so with the proliferation of this technology with these new features you're going to find that yeah on this hackathon somebody may make a synthetic Jared and me no one probably wants to

63. 30:50

    make a synthetic me but there's a lot of people out there like I'm just thinking like I'm making an AI agent content team right and this is my one use case in the Joe jump in with more Dev like off but okay and each one of these agents I'm going to do like a you know AI agent orchestration one of the agents I want to use Masa and use some sort of data feed in there yeah and I was almost thinking like you could basically could

64. 31:14

    make a list of 50 key opinion leaders or whatever people I like scrape their stuff get the sediment and what they're talking about and then have an AI agent write sort of responses of other things according to that is what I'm thinking it's an amazing way of summarizing a lot of data in a very concise way and that's in effect what our customers are doing and what our developers are doing with our data already so recently we published a case

65. 31:45

    study of a trading platform called Ola uh funded by a former binance data scientist what they have done is that they're currently monitoring a thousand crypto key opinion leaders where KO say tweets and then they have live realtime tweets from Masa streaming across their entire AI ecosystem and then they will be able to directly interact with the K agents so in the future it will be the meanie Jared the meanie clenia Brendan and Joe chatting with our community

66. 32:15

    instead of the four of us on the live stream now today totally like and people who are into trading I think this is where crypto people I believe that like crypto is going to you know crypto and AI I think they work really well together individually each really cool combined it's like going to change the way the world functions and so in this sense like I I just see it being like my AI agent are is going to be trading in

67. 32:40

    the future and it's like if people are GNA want to trade mem coins they can make their AI agents with enough data end points to like really compete and probably do better than human trading Poss yeah it's it's just fascinating because it's a way of projecting your opinion and your thoughts into another vessel that's really powerful and just going back to this sort of memory retrieval when you're having a conversation with this synthetic entity

68. 33:07

    so long as it has the right context loaded into it you can have a conversation about that topic you know and it's really powerful so you start off with this shell that you can then build more of like a a uh let's call it personality knowledge base uh environmental awareness you can sort of start injecting all these different things into it that makes it feel and seem very aware and very like you as an individual yeah I think for people in AI

69. 33:40

    like AI agents is the thing that we think is going to be the future of AI and user interaction in general but for people outside of AI they may not be aware of AI agent the better way to understand this is really your AI assistant or your AI friend or just a copy of your digital twin that can help answer questions can raise your mind can speak like you and can behave like you all right Joe how does this all work with near projects no I I want to actually though go back first for a second um you

70. 34:13

    talking about like the the real-time data piece of that so how how is that getting managed in that rag chain there is that are you as a developer is that incumbent on me to make sure that I'm retrieving that data in real time and then feeding it back in or do you guys have mechanisms within Massa to actually go and handle real time request yeah it's a good question um we're we're on the transition point we're actually doing um some work with a really big

71. 34:43

    Enterprise at the moment to refine the way that that logic gets executed I think you got your very good set of tools that you can use St Lang chain is like your your starting point if you want to have low friction get up and running quickly uh we even just recompiled a multi file agent into a single file agent using Lang chain plus uh near aiub and it will be deployed in near aiub there's one minor thing we're fixing with one of your engineering team and then it'll be available in there so hopefully during the hackathon you can

72. 35:16

    get access to this example bit of code pair it up with data in the near Hub and then you can kind of get up and running really quickly that's a good way of chaining together your rag logic however we are seeing that there are opportunities to um use an llm in this rag chain so for example allowing and llms have a characteristic where you can actually create functions within the llm where you can very explicitly Define a way in

73. 35:48

    which an llm will respond so maybe that is conditional logic that says if based on certain criteria you see this this kind of UT or input or response from the user structure this query and that query might be to go out and call a specific C request on an endpoint to get a specific set of data so weather might be one of these like functions that you're listenting out for parameters you're listening out for on the llm and if it hears weather it will have the knowledge

74. 36:18

    and understanding to construct a CO request to go out and call let's say a weather API or scrape weather.com through the protocol retrieve that then structure that response in a way where it can then use it in its chain of logic um but it has this almost like agentic flow where you can use a chain of agents to achieve an objective um that is outside of the scope of just a single

75. 36:43

    retrieval for example we're we're going to provide the tooling to do that where we see that logic deployed I think is still a big opportunity in the web three space I think there's lots of teams that are kind of thinking about doing that how that finally manifests itself um I think it will end up going from quite diffuse projects doing it to there being a winner um and if that Winner Is

76. 37:09

    Us that's great U we're certainly working on the problem yeah I was going to say as part of the larger strategy here is is it to become that One-Stop shop or is it to become more of a orchestr orchestrating layer for for this or both I think less orchestration for us because if we look at the way the web two stack has evolved you very clearly see a bifurcation between uh data aggregation data processing and

77. 37:35

    then the tooling for agent orchestration is two very separate businesses and I think they they require very separate thinking and talent um it in the event you become a multi-billion dollar valuable protocol I think you can do anything but in the first sense you focus on one of those two areas um I think the reality to today is there's some extremely good web 2 tools for orchestration that I don't know if they need decentralizing right now I think the most valuable part of the

78. 38:06

    decentralized stack is um the way the agents are deployed and owned you know to Jared's Point like how do I have sovereignty over this agent and own the IP of this agent and the reflection of me but then also how is the data aggregated and not controlled in a centralized way like when we think about um you know some of the examples you given here today about these companies going out even using like Mechanical Turk or individuals in countries to annotate data it's still a centralized identity deciding how those people get

79. 38:38

    rewarded and compensated there is no like to the to to what we're seeing right now in the ports in the US there is no Union of like annotation workers that can defend for themselves in a decentralized system you do have power as a token holder as a participant in the network to influence the way in which you get compensated the way you participate and I think that's incredibly powerful and those are the areas that will get decentralized first as opposed to the tooling um which Lang chain is by far and way I think the best

80. 39:09

    platform for doing this right now yeah um I totally agree what you're talking about with the decentralization aspect because here's where I'm like hoping that we like this is what I'm trying to do right now and I'm like Masa and I'm looking a couple other things so say I I I run content I want to make a team of six AI agents each with a specific focus of how I think of content maybe six maybe 10 who knows but each one of those

81. 39:33

    I'm going to Tool and specifically for what I think works the best right and so there's some sort of like I don't know my aspect that I want to own of that right because then if I can make that team of AI agents then I could leverage that and other projects could just like Ping my agents and then pay him a little bit of money and then go on and like basically use it in the same way that's kind of the hope but I don't know I don't know if that's like we're there yet but I'm going to try it

82. 40:00

    a lot with you guys like I think it's coming very soon I I I really do I think the way that they actually deploy that so I think there's two orchestration layers there's the the logic in which uh a single agent builds up its decisioning and reasoning um if you use 01 in GPT you'll actually see that Chain of Thought happening and that is increasingly moving to the llm so I actually think that there is quite a big

83. 40:26

    risk on platforms like lag chain Lang chain that you're using like a rag chain to do logic or reasoning because the existential threat is that's all going to end up in the llm and I think that's a very realistic possibility that's going to happen so again you know you've really got to think critically about what you're building today like what is the most valuable piece of the stack what has the most enduring value we know

84. 40:48

    for a fact that data is being generated every second of every day like there'll be people out there that say like we've used all the data in the world train our models that is factually incorrect because in the time that we've been on it we've just created new content today yeah there is not a single agent that's created on this that's trained on this podcast here every day human beings are generating more knowledge more information that has to be fed into

85. 41:12

    something there is always going to be the need for data pipelines there's always going to be the need to get this information into these models and I cannot emphasize enough the importance and the value of real- time data so I would encourage everyone to do this small experiment right now go to your chbt or Claud or your uh tool of choice and then ask the tool a question about

86. 41:35

    what happened yesterday I would bet you a Sal Masa right now that they cannot answer the question and then they will let you know that the data that's fed into our model is as of let's say April this year or December last year having real-time data is extremely important in the everchanging world that we live in right now yeah yeah interesting uh I'd like to talk about two things that I think we didn't quite touch on too much now so there's the

87. 42:07

    incentives for like the nodes and stuff but as a developer where am I paying a cost or how is that today and how's that going to be in the future yeah right now you're living in the rarified era of early adoption in a cycle um of of everything is either free or nearly free that doesn't just applyed to us but it applies to like the entire ecosystem um I think we've got maybe three or four

88. 42:32

    more years of that being the case um in this cycle so right now with Massa it's free um you could think of it in the future a little bit like bit torrent where you can not necessarily free leech you you'll have to pay a fee but you can also contribute back to the network so a developer can also run infrastructure to feed their own pipelines or indeed contribute to the network so there's both of those kind of positions but

89. 42:56

    right now it's completely free in the future you'll pay a minimal gas feere MAA uh in the same way that you would for a gas transaction on ethereum awesome and one of the things that we didn't really touch on yet is and I don't know this is the role that you guys are trying to play necessarily but a big piece of all of this data obviously is verification of sources and yeah is this legit data is that

90. 43:21

    something that you guys are taking on as part of the task of your network or is that a different layer somewhere in there yeah it's such a good question um to answer your question we are taking this on there is a a industrywide problem with decentralizing things that are not purely validating a lger or verifying an like a virtual machine's State like an evm operation um the way that we're approaching this and this is actually going to be it's not public um this is probably the

91. 43:59

    first time saying it publicly we're creating a secure execution environment in which we ensure through um an attestation that the code that a minor or a worker is running on the network is code that has been uh compiled uh to a certain standard so so what do I mean by that I mean that you can guarantee the execution of a request by a remote party and the request that that remote party is making is getting data directly from a source that Source might

92. 44:33

    be a web a web page that it's pulling in that Source might be data from a specific API that is authenticated by the node against it this is how we do it with Twitter we authenticate against Twitter's API user brings their credentials and then that node is able to deliver data but what is preventing that node from injecting a dictionary of a million tweets that they've synthetically created within llm and getting rewards to them the fact of the matter is we have this um uh secure

93. 45:05

    execution environment that's going to use a specific like Intel sgx chip you'll need to use to run this and it will ensure that the code that you're running is to a spec let's say and the binary that you're running is a authorized uh Network compatible version of code that enforces the collection the structuring and the acquiring of data so that's say we are going to roll out um ecosystem

94. 45:36

    wide and make available as part of the protocol so as other teams can also use this very like Bas primitive to verify that you know maybe compute is being done to standard like let's say I'm bringing a GPU how can I prove that GPU is in fact X me X gigabytes of memory y megahertz of power you could use this Enclave to uh do that in a remote attestation way interesting so is it fair to say that this is essentially like um is that

95. 46:07

    you're not you're not signing off on like the data itself is true what you're saying is that the what I expected to get from like the source I'm supposed to expecting data from is actually that Source it has not been exactly modified interesting exactly and using cryptographic signatures in the process so for example if I'm acquiring data from a remote API what's stopping me from routing that request or a proxy and spoofing or creating a an API that is a mock of the one I'm interacting with

96. 46:37

    well we verify within the container the actual certificate of the remote server as like a trusted source and if your certificate within your runtime isn't matching your Source server then that isn't valid data for example it's clever interesting yeah yeah dropped it it's clever like that I like I like when two technical people get going and they're like yeah yeah good yeah got that

97. 47:01

    question got it down you guys are soling problems yeah yeah it's not just me we have like a a really amazing team of people working on this internally and it's fun like this this kind of stuff is fun stuff to work on oh totally like I'm wondering too like okay so you have blockchain right the one thing about blockchain that I like think is the coolest thing is the digital ownership aspect like a true proof of anything really so is there a way I guess this is

98. 47:25

    a question for Joe as well like if someone's hacking in this hackathon to some way leverage that blockchain and this is going to be completely technically inaccurate but this is what I'm trying to think like there's an nft within that nft is all of your data therefore you kind of own it and people can't get to it and then you can leverage that into your agent or Joe does that make zero sense which I'm assuming it does but you know visually I'm trying to get there you say like

99. 47:50

    like it's a it's like a gatekeeper like the NFP serving as the gatekeeper yeah not like a ZK but you know something like that where like and I can constantly keep updating that I could I could upload writings that I'm into if I have a Blog if I do a podcast uh I have five streams of data I want to in in put it sort of like a big container of data that I own can someone hack that yeah I mean it sounds like absolutely I mean why not

100. 48:15

     you know so it the beauty of I I love like again like what mass is doing a lot of these other projects are doing is that they're opening up for you to be able to build on top of so however I then choose to leverage that if I want to build an interface that requires you to First hold an nft to see my interface then I can gate it that way and then the requests so on and so forth so I mean

101. 48:41

     and from a hackathon standpoint you want to actually like start chaining together bounties I mean here's how you start to do that where you know see how you can start to leverage multiple things but uh it sounds like from what mass is doing and then from other thing like you really not limited in how what you build on top of it if you need data like it's it's open there for you to to

102. 49:05

     use yeah exactly like if you want to bring a connector to personal data like maybe that's plaid to get your banking information maybe that's another source uh there's nothing stopping you from building a module and adding that package into the the network and offering it to a developer see this is go fundamentally what we're trying to transform is really the aggregation and structuring layer of data right how you get the data and then how you use the data is up to the developers but how it

103. 49:36

     matters in the MAA decentralized format is how to create a more permissionless and open way of getting data in the structure that is most suitable for AI developers see this is this is cool because Joe I just like I'm looking around if if you go to near catalog right um and I'll I'll put the link in the show notes it's a dev. n.org applications and it's got a bunch of like all the projects building on near right just a catalog of all the projects I'm just looking through them and I'm like okay sweat they could do something

104. 50:07

     with an AI agent where they like are scraping all the like they could get 20 different athletic K and every time something goes there that's like a running thing boom they could like pull that in leverage that on their app or something and people could ask the AI agent on their app what's the most trending thing of like athletic whatever I don't know whatever question you want to ask a bunch of athletes but then that's just one way to do all this then all the dii projects you could do that I

105. 50:33

     mean it's you could have meme. cooking or any of these like things they could just scrape the web and when some fun interesting thing happens that's like an outlier on Twitter instantly create a token on it or like a meme coin on it right like so you can start to see how you can gamify this as well and I I'm hoping I'm going down like the right thread I don't want to like throw people on like a no I love it I love it if you really really

106. 50:57

     want to experiment with Masa data during redacted hackathon I know there's um there are a lot of new mean coins around politics given we're in election season you can set triggers based on Twitter data if let's say a new politician's name is mentioned x amount of times you create a meme token based on the politician I think that's a clever idea well clever idea using Joe's word here

107. 51:21

     and but the point is you have unbonded uh imagination and possibilities when it comes to use M data dude Joe any okay so Joe this is any ideas like any use cases that you're kind of thinking from like a developer point of view that you could kind of share put you on the spot yeah definitely on the spot um well I wanted to you know so tomorrow we are doing a deep dive from technical standpoint on devhub live so renon will be back for that and I I want to kind of lead into

108. 51:52

     that and kind of uh introduce that in the sense of like if I'm a developer coming in from a web 2 model okay so I'm not I'm you new to this is uh M stepping in and to a place where like Google's big query is is it something like that like like what is the comparison that I would try to tell someone of like what is this centralized data and how I'm actually accessing data is it similar to something like that is

109. 52:22

     that how you would explain it to someone it's a very good question um the way that we typically kind of say it to um the rest of the world outside of web 3 is um scale AI plus um bright data now even with both of those terms you've got to be an AI to know what I'm talking about if you're anywhere outside of it that that doesn't really help you um so uh you know if you think about the the data

110. 52:53

     pipelines you're running in a daily like business uh um that's the set of the infrastructure we operate maybe clenia can add something that's a little bit more General and Broad outside of the AI space yeah as a non-developer normally if you need Twitter data where do you go to you go to Twitter developer API and then you pay anywhere from $5,000 to $40,000 on a monthly basis to scrape 1

111. 53:18

     million tweets and above with Masa you get it for free so put it in a very very simple way uh in instead of using very complex and very expensive Twitter developer API you use Masa Network directly yeah data bricks is another good example which is sort of more General yeah when you're thinking about like composing your sort of components of your of your data pipeline stack data

112. 53:46

     bricks is a is a really good idea of like that modular Lego brick type analogy of of building up the components you need to to get your data into your back end now if I want to put my own own data in though so I'm running a business I I want to put my data into the network is that now instantly accessible to everyone or am I do I have some controls around who can and can't access that in

113. 54:10

     the nft it's in the interestingly like we we we started very early on doing this notion of a like Locker for data um reality is that most Enterprises don't want to share their data they want to keep it internal therefore they have internal tooling in the cloud to do that Andor individuals that want to share their data struggle to monetize it because you're competing it's Google and Facebook basically like you have to as a platform get millions of hundreds of millions of users to to

114. 54:41

     make the data valuable enough with critical mass so for us we we typically have like trimmed out that functionality from the platform um because we've seen that product Market fit is is not there and the fit for us is more on the generalized data from the public or private web that's behind W Gardens Twitter being a really good example of that W Garden where it's extremely hard to work with the data unless you use a a partner Like Us and we' got a massive Edge by being a to

115. 55:12

     serve that datas people with SC yeah no that makes a ton of sense I mean so I think going back to like Jared's question of like if I was going to build something right I think the the simple path to getting out the door and getting used us to building with mastera would be of like how can I augment what I have right so if if I'm building a product that maybe I do use some personal data

116. 55:36

     that we do not want shared it's like how can I augment maybe like on user profiles right I want to bring in someone's Twitter feed you know based off their user profile and we can pull that into the app versus just trying to wholesale start from scratch right it's like okay let me Kickstart some of this stuff as well yeah your base substrate is your generalized llm and then each

117. 56:00

     sort of like plug that you're putting into it is is a new kind of memory feed or or or data feed and the the data feeds you can get from Maser can texture in like social Dimension being Twitter uh General web being web scraper so you can go out and get any information that's on the web um we also support Discord so if you're running your own Discord server and you want to for example do uh analytics on the

118. 56:25

     conversational information for example we actually use it for digesting product feedback so we have a bot in Discord that bot is constantly feeding the conversational messages back to us and we then can analyze that using an llm or an agent to tell us what are the most requested features what are the biggest bugs we're seeing now think about how that scales as a product to any company

119. 56:50

     operating in the space that wants to quickly and easily understand what Their audience is doing bang you've got a great product for this hackathon to build a way of understanding and interpreting what communities are doing what are they talking about this and this could be from the perspective of like Deens understanding what you know Community talking about or projects and companies wanting to understand deeply their users I even had a conversation with a VC last week who was like huh so I can use this to track all of the feeds

120. 57:16

     of my portfolio companies to understand at a glance what they're doing and what they've released or what they've been talking about by just saying you know what are my pfolio companies what have they done in the last week and immediately capturing that information making it easy to consume digestible and easy to understand in a conversational format um those are all the kind of use cases that you can use it for yeah I

121. 57:41

     think the juice of it is really that if you want to understand social sentiment across Twitter Discord telegram you get the data from Masa and any applications whether it's trading crypto we understanding your community levels of activity that would rely on social sentiment you'll be able to use masasa for so I kind of think okay I think pretty much every project on near should build an AI agent with their data scraping the internet whichever social

122. 58:11

     sentiment their marketing team or their Vibe is that's what I'm seeing right now absolutely yeah so I then I could almost say like anyone could do that so would an idea Joe that someone could actually make like a very simple sort of front end that like allows them to sort of walk through like okay you're going to make a an AI bot an AI agent for your project here's the five boxes you got to

123. 58:33

     input stuff into then it probably spits out some code that you then hook to your I don't know like sometimes I try to not get too Technical and sometimes in my complete lack of technical Joe and I do some pretty fun stuff but um so what do you think Joe is that something possible yeah again I I I love the idea of being able to like Kickstart something else that's like within your system right like I think every

124. 58:55

     developer you build a product it is hard to that's that cold start issue and sometimes about having users but also it's about having data and to be able to instantly have data sets that are available to me that have been verified is huge because now again one yeah it drives my cost down but it it simplifies that thought process of like what my app can maybe do and then I just tie it you know start tying everything back together yeah it it allows you to focus

125. 59:25

     on the product not the wrenching of the data pipelines which as as many people know is like just it's a lot of overhead um it really is I mean you can imagine taking a inventory of every developer in this cohort and tracking their socials and their conversational information and understanding what is the traction and the development that's going on in this cohort of hackathon developers there's so many different dimensions I think the key takeaway for me is

126. 59:56

     it's it's the ease at which you can break down and digest to consume very complex broad amounts of information in a very concise way yeah tomorrow you guys on the hackathon Deep dive on the workshop will you actually show how to make an agent is this going to be like something that people can tune in like myself like I'm not gonna I'm just gonna watch like because I'm really interested in

127. 1:00:21

     like I mean we could get just I I love the the the uh Theory with a I love just like the like uh when you just like imagine like one year two year five years out with any of these AI sort of like projects where it could go and I'm just seeing like multiple AI agents for each project focusing on different things you could have all your developer technical stuff put into one agent and anytime someone has a technical question

128. 1:00:45

     that agent could handle that you could have your marketing agent that's completely understanding crypto Twitter and then helping your project through its guidelines craft tweets that you can send you know it takes some refining but for the most part it could do like 80% of the work is what I'm saying it can do so much of the work it's um maybe 95% at this point it's crazy it's crazy I I think we lose track as human beings of

129. 1:01:10

     of how much cognitive load there is in synthesizing a complex topic and reducing its complexity into something simple yeah um abstracting away the need to do that and having trustworthy dat like Joe keeps hitting on this really important Point like verified trustworthy data if you can solve that problem and you then have a pipeline of information it it just makes the decomposition of large sets of information so much easier not to mention that AI agents

130. 1:01:40

     actually have Perfect Memory unlike us human beings unlike me I it might just be the time in life I'm at but I've never been more humbled than I have been in the last year from AI like any sort of every once in a while I just does something and I'm like whoa that is very impressive it's like magic it's Alchemy According to some people you know like it's pretty cool Joe what else do we want to kind of get at like I know we're gonna get into the technical stuff tomorrow but just like any couple things and you know any last sort of things as we kind

131. 1:02:12

     of no I I I I hope that people walk away from this and uh have questions uh and you know hopefully you know tomorrow they can bring some more questions too but I really this is one of the products and I'm not just saying because you guys are sitting here but I really that like it really opens the door to being creative as a software developer where you know it's allowing you know it's allowing me to to

132. 1:02:41

     just worry about building something cool instead of worrying about all this other infrastructure and everything else and so as soon as I start to pull that stuff away and I don't have to worry about all the nuances of like how am I going to get this data and how am I going to pull pull that API and how much you're going to pay for that it changes your perspective of like what I can actually start to build and any tool that's like

133. 1:03:04

     this that can do that I think is a net win for developers so I hope everyone even if you don't actually do something for the hackathon at least check it out and and see what it does for yourself I feel like every project on near should sign up for the hackathon just to use this and start building with the Masa AI agent like really like at what other time and then just like you can there's no downside I mean there's no downside right I'm not I'm not going to lose

134. 1:03:28

     moneyless number one you said there's a bount can make money there's a bounty I can make money and on top of all of it again it just sometimes you don't really realize like what's capable until you actually start to experience it and so this is why also think hackathons are great and you know where you can just come in I can mess around a bit if it fails it fails but at least I can just

135. 1:03:51

     now try I I have no pressure on it um you know I'm I'm certainly going to look into it for jiz project so I mean yeah it's uh Shard dog AI agent you know that's happening Joe I don't know what it's gonna be but it's gonna be wild but we're gonna make it we're gonna make it so easy for you to get up and running you know I love how Joe is really focusing on the creating space for creativity uh as an engineer and a developer um we'll show you tomorrow

136. 1:04:23

     just how easy that is um lots of example I think the biggest takeway I want everyone to leave tomorrow with is zero to something is not hard and the ability to go from zero to one today is so easy it's hours not days right it used to be months and this leverage that you now have by using tools in the AI space is just going to accelerate you so fast to the point of like validation and proof that you know shipping real product to users you can get there so quickly and

137. 1:04:55

     and that's one we really focus on tomorrow is like making you realize like how high the velocity can be today with some of these new tools that are in your kind of toolkit yeah awesome um anything El Clint you want to say anything that as we're wrapping up here no I think everyone should tune in tomorrow and to see how the M of magic Works in person how to set up Masa how to build an AI agent within five minutes see this is I I'm going to and then

138. 1:05:27

     we're going to clip that into a single video and then I'm going to make sure everybody watches that video and everybody makes an AI agent because it just I get so hyped on this it makes no there's no reason you shouldn't make an AI agent in the next month exactly in my mind exactly it wild well I just want to thank you guys I'm gonna do a little bit of like just so everyone listening the hackathon it's the redacted online hackathon and then I think we even have

139. 1:05:52

     please tell me I we've got the oh boom that's where you go sign up at the hackathon at the redacted devpost do.com over a thousand people already signed up but there's two verticals chain abstraction and also Ai and I think the AI stuff is a lot of agents and then Masa is doing some bounties as well so it's gonna be really cool and then also uh so everybody knows there's also the actual event where we're going to give out the awards and

140. 1:06:18

     have a bonus hacking on site cool stuff like that and that's November 9th through the 11th in Bangkok so that'll be really cool here's a little fun little thing there we go and then uh I just want to thank you all so much for uh being on here look forward to tomorrow and um until tomorrow have a great day awesome thanks for great s bye all