Block Zero Stage | Unstore Your Data for the AI Revolution With OnMachina

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   um with that please welcome to the stage Jonathan and Paulina will be telling you about un storing your data with AI and near thank you Clemens uh we are going to be doing demo so let's just see if we can get our machine hooked up here thank you okay all right great so that's all looking

2. 1:07

   good um okay so before we get to the demo we're going to talk a little bit about uh data and um oh uh oh y yes and what we're building at on Machina um so I'm Jonathan I'm yes we we are uh two of the founders of onina and uh we're building decentralized infrastructure starting with a storage Network so Paulina is going to talk a little bit about why we are starting with storage and what we think of the uh the storage landscape yeah so welcome to

3. 1:49

   the generation zetabyte uh as you can see by this graph the amount of data that people's produce is actually growing pretty rapidly starting from 2010 and 2025 can actually see from this graph that for the next roughly two years we'll grow about 30 to 25% and this is a very rapid transition from where we started as the phone get better you know the data images files became bigger and bigger and the thing about this is as a fact all of this data isn't struct well 90% of the data isn't

4. 2:23

   structured and uh how to work with that uh and why is this important or rather fact starting at 1970s all the data uh was rather queried or put in the database so you would know a question that you need to ask or statistics that you want to collect before you get to organize your data and that's why everything was put in the database correctly then once the Google started to emerge you had this mechanics of map produce and we shifted to a big data so

5. 2:54

   now you have profiles and all the other things and to collect statistics you can reduce and you have and you know the query later and then we have a data Lake as you can organize the data into the blobs and you have some agility in terms of how you can put a question and uh gather your statistics and now as we're heading into 2020s we are in 2020s now we got to intelligent data so all of our data can be unstructured and we still can work with that because of the AI we

6. 3:25

   no longer have to know the questions to structure our data correctly we just work with the data as it is and at the same time AI will require the data and the rocket ship of AI is fueled by the data and it REM it uh has to be trained on the huge amounts of data and refined of massive amounts of data and mention in the data leges and the hydrogen is the fuel for the rocket so this is a good analogy with the lake as we start to unstore the data and uh FedEd to the AI we have to bring the

7. 4:02

   data out of the lake into the rocket ship and more and more important it becomes to make the data actually being owned by the users because AI is not just power for for the human who use it like every one of us sooner or later goes to Chad to write some email to to the people you don't like for some reason being polite and yeah it becomes powerful for us but it also becames can be become powerful for all those corporations and for the Bad actors and

8. 4:33

   this is why it's very important to control your data as the huge evil corporations will try to vacuum clean it up and monetize your knowledge yeah so I I think that uh we're facing a situation now where as Pina said you know we have more data than ever before already and it's just going to continue to grow at incredible rates the big change is that we are now having uh the ability to

9. 5:00

   process and access and learn from unstructured data in ways that we never have before which is awesome but can also be scary um you know one of the things that uh that we believe is that as this uh data becomes accessible becomes usable becomes intelligent we don't want to see that just centralized into the hands of a few powerful institutions and we're coming out of an era where cloud computing has really been one of the greatest forces for centralization in the history of Technology We've Ended up with kind of

10. 5:30

    an oligopoly of just a few Cloud providers that provide most of of the Computing resources for the world uh but if Cloud you know was a big center of gravity you know big Mass AI is going to be even more that way like the the default that we're heading towards right now is everybody wants to give their stuff to open AI um everybody wants to feed their data into chat GPT and train it and make it better but I think it's

11. 5:54

    really important that we pause and we go wait a minute what is the impact to each of us us what is the impact to our businesses what is the impact to to uh the the ability of us as humans to learn and understand and interact with each other um you know one of the reasons why these big centers of gravity like like Cloud providers and and uh and AI providers uh succeed at sucking up so

12. 6:19

    much data is because when you have a centralized system it's easy to build a common API and when there's a common API that makes it easy for apps to uh live on top of of on top of those systems so with on Machina we are creating systems and structures where you can put a common API on top of an entire nebula of stars on top of a highly uh decentralized set of operators and data center managers and then you can

13. 6:50

    navigate around that and you can decide how much control you have over your data where it lives who uh you know who who has access to it so this is um you know this is I think a uh a really important thing to think about now as we're still in the early days of AI still in the early days of of unlocking all of this unstructured data um so that we can make good decisions and and potentially

14. 7:14

    different decisions than we've made recently so we started with a storage Network and we do have a common API that sits on top of a Federated set of of uh nodes across data centers and that API is very simple uh but you know simple can be really powerful too the our API is a rest API with five HTTP verbs get put post head and delete and uh you know that's a uh that that's good because it means it doesn't take a ton of time to understand how to interact with the storage system as I said we're going to

15. 7:45

    do a demo in just a couple of minutes um but to talk a little bit more about how it's built and why we're here at mircon and at the the hacker space um our identity management is totally integrated with near so if you have a near count then and you have access to onina now we're in a private beta right now so go to onina doio fill out the form or hit us up afterwards if you want to get Early Access and try it out but

16. 8:10

    near act as an O layer and then uh that connects to a proxy layer that sits on top of nodes um from different providers all over the world so right now we have we have nodes in uh in in three continents um in Europe Asia and North America and uh and those nodes live in data centers that are are on different networks and and from different providers so this is um you know a key

17. 8:35

    part of the decentralization here is that this is not all controlled and owned and operated by a single entity at the end of the day where where the data is living um and so you know as we as we unstore the data as we bring it out of these archives and out of these data lakes and feed it into AI systems it's really important to think about what this means and uh and um you know be very cognizant of also where we are in the AI Journey um we had one of our designer friends help us with

18. 9:08

    with some of these slides and he uh he decided to use AI to create some of this and he gave a prompt to uh to one of the AI image generators and said create a book in the style of a New York Times bestseller with the title unpredictable and so this was Result One unpredictable uh this was result too I don't even persable yeah uh unproductive you know this is I think another good thing to keep in mind about where we are

19. 9:38

    in this AI Revolution uh it doesn't always work yet it's a little bit dyslexic it's a little bit yeah it's a little bit uh um you know incomplete in its abilities so again you have to think about like wait a minute what am I sending my data into what are the risks what are the potential fallouts if you know uh the AI takes this kind of turn um and so we think that in a unpredictable World in a world where

20. 10:03

    things are rapidly changing decentralization is important because it helps us stay agile and properly built it gives us a great user experience and it gives us the ability to uh to own our data so all right we said that we would do a demo so let's go ahead and jump over to that um a console yeah I'm going to start in in a browser here so this is a uh this is an example web app that we built um it's just a simple JavaScript front-end app it's a handful of files put it together

21. 10:34

    in in uh basically a day and it gives a view I am logged in with my near testet account J bryce. testnet it gives a a sort of a file system View and uh you know I can create containers which are basically like folders so uh nearon 23 demo and we'll come back here and voila there it is you know I have some containers that uh that already exist um this is a picture of me with my first

22. 11:09

    computer y you know I was a lot cooler back then actually uh and we can you know upload files so go over here and let's pick uh you know maybe this and upload it so you know this is um pretty simple right we've got uh basic file browser functionality but again it yeah you know yep just add it so so this is all built in again a super simple stateless JavaScript front end app I could run this locally on my machine it's it's currently running on a on a netlify deployed website um and

23. 11:50

    it's using those five HTTP verbs get put post head delete to to create this interface what's cool about it and where it gets powerful is that because it's it's simple like that you can uh you can combine that into a lot of different commands now I'm going to use Curl here curl is a command line utility that exists on pretty much every uh Unix based system for many many decades um so if I uh if I go ahead and I I get my account status you'll see the

24. 12:21

    same list of containers including that one that I just created um you know I could go uh let's see let's do a a put here um so that will create a new container um test container zero y uh I can put a file into that container uh just do a a text file here and uh and now if we if we call a get on that container then we'll see you know that that I've now created a container I've a file so all of this is is pretty simple

25. 13:01

    stuff where it gets cool and Powerful is because it's these simple commands you can combine them together you can script them you can program them um to be much more much more powerful than that so uh if if we go back to to the top level of my account you'll see that I have I I like to script things I had uh created a thousand containers with a thousand objects in each of them so you know

26. 13:25

    basically throwing up millions of of things onto the system um and there are a few of these left but let's say I want to clean out all of the files in container 01 so um over here I have I have a file that just has um paths in it uh let's go ahead and update this so that instead of Z 0 it's 01 if I go back over here I can send uh send that file to the system um as a let me see there we go as a bolt command so this is going to be a bulk

27. 14:04

    delete command that I send that file and if we go back and we uh click in here you know it had a thousand objects in it you can see it has fewer they're getting deleted yeah it will take a little bit we refresh again fewer you know and and this is just going to keep running so you can you can combine these into really powerful Scripts and I'm just doing all of this directly from a a zsh command line here so this is you know nothing fancy this isn't a special client this

28. 14:36

    isn't anything like that just piecing these together the other the other side of all of this though is that um we have decentralized the way that nodes are added to the system so I uh I mentioned that we've got three continents I think that um we should bring Sal Paulo online into the network right now so I created a server just a little while ago if we look at this server you can see it's been on online for about two hours um

29. 15:02

    and it is uh S01 so we'll go back to uh my operator dashboard and uh this is DS in S01 now what this is going to do is I'm logged in again with my test net account it's going to create a sub account on near um we require staking some near as a little bit of security on the test net that's only one year um but basically to uh to to kind of get all of our operators into um a state where we're all helping to secure the network so we will create that

30. 15:43

    account all right that has now been created our stake has been accepted yes um it's created a credentials file uh I'll download that gives us some commands to run on our server to prep the file system so we'll go over here we'll uh run those um let me make sure that oh credentials file got downloaded it did okay so if we uh upload that to to our server all right that is on there now our file

31. 16:34

    system is prepped and uh and we can now run this command that will uh start up a a Docker image and connect to the network um yeah so what we just did we pretty much established a new nodes inside the storage Network so previously we had four as you saw on the slide we added the fifth one so what it's going to do right now is actually try to sync up all the file pieces from the other nodes and redistribute them to add for the new noes so let's create some new data and

32. 17:12

    get some activity yep happening here and uh as we put the new files they get redistributed in between the networks so every node has a piece of the file so as we span off the new node just now it will first sync the files from the previous nodes but as we add new files it would as well receive new file pieces so that's what we'll try to do right now Jonathan did did you just push a file I

33. 17:36

    missed that yes it takes a a few seconds to refresh and start pulling data but what we can we can come back and check it in just a minute but anyway the uh the thing that that is great about this is that the uh the operations and the uh the um the the front ends the back ends everything is decentralized in a way that uh that allows people to participate in this network to come en join the network add capacity to uh to

34. 18:09

    leave the network H you know we must have have uh have hit in the middle of a refresh rebuild cycle I'll keep an eye on it let's finish out with our last couple of slides Pina and then we can oh yeah quick summary actually we're running out of time as well yeah so can you press yes as a summary uh the data growing massively right now and it will continue on growing rapidly then uh we would need the data to become intelligent and uh AI will feed on it

35. 18:43

    will feed from this data as well it is important to uh have control of your data as it could it becomes more and more centralized as we use the projects like Facebook or Google cloud or open IB not so open recently so yeah centralization ation becomes more and more risk and that's why it is very important to have your data decentralized and start it on makina yeah all right so again we're going to be around here for a little bit come find us if you want to uh to get into the the private beta um on Machina doio

36. 19:17

    is the the website scan the QR code and uh yeah thank you thank