Data Platform Community Meeting (May '22)

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:10

   welcome again to another edition of community meetings this week is the data platform meeting where we're going to be going over triggering and alerts um yeah again the same format as always please feel free to drop any questions that you have using the q a tab below in the webinar um yeah drop us any kind of message that you want and we'll answer it as soon as uh we find a moment to jump in but uh

2. 0:37

   but yeah i'm not sure who's gonna uh lead the the meeting so uh i'll turn it over over to y'all i'm gonna present our slide all right got it oh awesome no problem this time um hey guys uh the platform team here we're gonna go through a couple of uh updates uh for what we have been doing for the past month and having some discussion on our new focus on alerts as a product and get some

3. 1:10

   feedbacks from you guys on what we want to you know focus on and we'll want to prioritize and we'll have some more polls and some questions and answers that we can go into i'm going to pass the mic to fro for the updates hey folks yeah it's exciting and i'd like to share more knowledge with you and uh we during the course of uh our restructuring of the docs for near indexers we put together quite a few new

4. 1:45

   tutorials which we'll talk through today these three are the data flow type of tutorials you can find very interesting i recommend you start with near data flow tutorial which talks in detail about how the transactions blocks and chunks and all and receipts and all those pieces fit together uh from the high level point of view there are no uh complicated um code example there just fewer colorful blocks that you can find on the beautiful

5. 2:24

   uh video recording there um you can dive deeper with a very specific example of a transfer transaction uh which talks like applies the same pattern that covers in the first explanation and applies to a specific uh transfer transaction and then uh there is a quite an old tutorial that we had before it's an advanced one but i still want to highlight it because it's very important when you want to learn uh how every single the octaneer is being transferred

6. 3:00

   through the blockchain and uh definitely you want to learn it if you want to understand it like 100 correctly um so uh with that said i passed my word to logan to give us update on the near lake side of things yeah hey everyone excited to be here uh for those who didn't understand i'm representing here uh a ukraine a somewhere here there and of course the lake it's just like for clarification why am i

7. 3:38

   why am i dressed like that uh yeah a few updates uh so recently we had the upgrade of near core to version 1.26.0 it was released and like on the mainnet so um no maintenance was required for the near lake framework users uh during these updates uh we captured on the picture one of the uh near like indexer like indexers uh indexer developers during the update because uh they have migrated to a near lake framework beforehand uh

8. 4:19

   next slide please also uh we have asked some of the community members about like how things are going with like so far and how did they feel during the last year core upgrade you can read through this uh anonymous uh testimonial uh it was made by anonymous labrad ban from the project fire or fair i hope i'll pronounce it correctly and the shameless plug that fair or failure is an upcoming nft

9. 4:53

   platform that connects digital art and crypto to the world of impact and charity they are essentially a marketplace whereby users can purchase art in support of their favorite charity or causes they that they feel passionate about so please wait for the launch they have soft launches on mainnet and very soon they will be launching like pretty loudly i believe uh we're very excited about the project uh next slide please uh yeah and a few updates here

10. 5:25

    and there uh near lake framework rust version received some stability improvements and we actually like very grateful to our community members who were creating pull requests and actually made it uh disability improvements so kudos to everyone who participated uh i remind you that during the previous community call uh we got a question about like um how to empower our neural lake framework based indexers with the start options uh that we used to have on the indexer framework like uh

11. 6:00

    from interruption like uh from the latest blog uh or like from the genesis or something like that so we have uh we have promised you uh to create a tutorial and we did it uh as we know someone uh already used it and like benefit from this tutorial so we're really happy to help uh also we have released new lake framework javascript version uh currently it's a little bit in beta i would say because

12. 6:34

    we have like we're not running any production uh code but it is fully used like usable and seems to be a production ready solution uh so please we welcome you to better test it and share your thoughts also we are grateful to our very active community member blood grecina who is firing us with the pull requests on javascript version it's really helpful so kudos to blood and like we're thinking about a new near-lake framework uh like language

13. 7:07

    uh which we're going to implement and i don't know maybe it will be python i don't know who knows let's see uh next slide please yeah and uh last minute update update uh like during this week like bef like especially for this community meeting we have created uh another tutorial uh that is utilizing the near lake framework javascript version uh like it is an nft indexer uh now on the screen you can see an example of uh what it actually prints during the indexing process uh

14. 7:50

    we have been trying to make this example uh as useful as possible in terms of you can grab the code uh read the tutorial grab the code like adjust it for your needs and just use it so please you're welcome uh and uh i'm not sure if i have done everything i guess it was the last update from my side and i'm i believe i'm passing the ball to froll or to tiffany yeah it's tiffany yeah so thank you i'm

15. 8:21

    here uh yeah so the last major part is uh we're going to have uh finally alerts uh that is going to be supported via deaf console and we have a lot of idea for what alerts would be and what it would look like it will have a lot of progression later down the face um but high level we're focusing on a couple of things to get started and you know the ones that we identify as very important for all of you here um so the first one is trans it's around transaction so for

16. 8:54

    transactions we want to notify users on all the transactions where a certain account is touched so any accounts that you are interested in you are curious about you want to understand you want to keep track on we want to notify you the second one will be only on failed transactions that um that you know you mainly immediately concerns you so that you can act on it um or you want to just you know keep track or anything and the second portion will be around contracts

17. 9:19

    um we have identified two major areas one is notify me when a certain method has been called um so you know there are a lot of con there are a lot of methods in your contract so instead of only looking at transaction you can go one layer deep into um you know like understanding what exactly has been has been uh act on and you know and uh plan your actions around it or notify me when a certain events have

18. 9:44

    been fired by a contract and the last portion is around accounts so you definitely want to keep track on when your account balance has changed right if there's any any money going in coming in or any money going out uh large amounts or a smaller amount if you if you really want to keep track um yeah so we have a poll ready um which also have other parts of of the questions that i have on the second slide here about a lot of different things but i want to pause here and launch the poll

19. 10:14

    um so that we're able to interact with it so the first question off the pole is actually about the late migration that uh bockton has just shared a lot on the updates that we have made on the lake front and we really want to understand your experience for the lake you know have you started their integration migration um you're you know do you want to are you considering are you interested

20. 10:38

    and you know we want to help you along the way and see where we can help better help you to migrate to the this awesome product that we have um let me know if you cannot see the poll by the way uh i think it's working well working fine yeah i think it pulls up there is one question um someone has they're asking in which scenario should they not be moving on to the near lake and for indexing yeah sure uh like i missed how we ended up

21. 11:10

    in the q a session already yeah yeah in my in my opinion uh we have only one uh but very major uh scenario when you should not be using near lake framework uh it's whenever your like indexing stuff requires to uh perform some queries uh into the blockchain itself so um and like you are about fully decentralized decision of course so uh like it's a little bit undocumented um stuff but whenever you run indexer based on indexer framework you can access uh

22. 11:57

    the view client uh which is basically uh the client and then your course site like a crate which allows you to perform like view calls uh into the chain like the data you have without like going through the rpc somewhere around like it's entirely on the on the single node uh so like i believe this case might be some sort of crucial for some projects but yeah i don't see any other scenarios maybe false let me let me expand on it a bit

23. 12:37

    uh so first of all i would say the the the only reason you want to avoid using your lake is that uh all in all new year lake is a centralized solution we store the blocks on s3 and you then pull the blocks from s3 so you will rely on uh pagoda inc to put the blocks on there um for you whereas if you use indexer framework you run your own node which is part of the decentralized peer-to-peer network and you have direct access to all the communication

24. 13:15

    the node does with all the rest of the network so that's the only difference um and and some like if you run your own node your own um network i mean then you can not use like because we only index uh the rest net and mainnet so yeah but you can still run near lake implementation the indexer itself which is based on top of index of framework and index your own local net just as well so yeah we have another question uh about

25. 13:55

    like any plans on configuration options for storage for all amazon haters uh it's already there so you actually can run your own near lake instance which is storing the blocks in json format and you can choose any s3 compatible storage for that but like pagada is not covering you in this case but you can do it by your own uh but sorry uh but also follow up on that i do have a question that i want to ask also you know the community here is um are you are you guys interested in the

26. 14:33

    storage right so that is also going to be a feedback that helped us at pagoda as a whole to to see if we want to you know if we want to develop resource in something um in a certain thing like this if you guys are interested we are going to think about the ways that we can integrate with decentralized storage or other storage that we can provide as a whole um through various ways that's

27. 14:56

    pagoda so uh really welcome all the uh feedbacks i can get on that front too it's not in the poll but uh but welcome answers yeah another question uh is accessing through indexer faster than the lake uh it depends on what kind of accession uh the question is uh if we go like if we're talking about something we can request through the rpc uh in case of uh the lake solution you will be using the like

28. 15:32

    external rpc so it might be a little bit slower than if you're running your own node with the rpc itself but in like if we're talking about running and getting the data uh i believe with lake it's like at least it feels faster because it starts immediately uh you're like you you can like get something faster but you will start like forever yeah for all let me extend it as well from my side so we are talking about near lake as a solution that is based on top of indexer

29. 16:11

    framework so first indexer framework will pick up the so first the network will pick up the block then the indexer framework will pick up the block then we will put the block onto s3 and only then the near lake clients will pick it up from s3 so there is some delay between the indexer framework picking up the uh the most recent finalized block and then between the lake would see it but this delay is

30. 16:47

    quite small and uh my estimate was around like a few hundred milliseconds so it shouldn't be a problem we are talking about finalization taking three seconds to to arrive hopefully that covers the question feel free to uh create follow-up questions if you need more clarifications yep sounds good yeah um there's no more question yet um yeah follow back on launching the polls i already see most of the participants here have answered them so

31. 17:28

    really appreciate that um so yeah so the prioritization on what are the things that we are going to prioritize uh for supporting the alerts um appreciate your answer solid and we're gonna pass it to uh our development and you know prioritize accordingly we really care about your feedback and you know where you care and more polls are so you already saw a question around alerts channels we are

32. 17:53

    thinking about supporting multiple channels as flexible as it can be um but we also hope to uh with your guys's help on prioritizing what we're going to support so i also saw a lot of um a lot of feedbacks around web hooks so um awesome that's that's actually we were thinking about supporting first so uh looks like we matched on on that one and um next question is what do you want to see in this community call so yeah we are doing it for the second time and we're

33. 18:23

    you know trying to improve on what we're going to do um and so yeah so really appreciate the answer for that one and the last one is we are potentially going to recruit more lab raps no just kidding but um but yeah we want to have more interaction with you guys and uh have you know direct user interviews conversations built long-term relationships and and yes potentially have more beta testing with you to uh see your

34. 18:49

    experience and you know improve the product around it so also really appreciate all the answers i got from that one um we actually don't have other set agendas uh for for this one but yeah so the fastest connection to our team i just saw the answer um yeah the fastest connection to our team we are uh all really active on all of our actually this is a great place to show that we uh we are actively active on all

35. 19:19

    of our repos we're active on github uh issues and we really welcome questions in sec overflow in the link here um and my personal information on like my countly to schedule any call um or discussion is all available from our weekly updates that uh boston here also amazingly prepared and send it out every week if you subscribe to the community thread on on the near github repo you will be able to see our updates and there are all the links places my discord telegram

36. 19:51

    anything feel free to reach out and yeah so that's i think that's one of the ways to reach me at least anyone wants to add on on that there's also um developer office hours that we have every day for devrel um every single day we have two different office hours near.org forward slash office dash hours you can come in and ask us any question and the deborah team will try to help um and uh yeah if we can't help we'll

37. 20:24

    forward you to one of these teams that can um so yeah yeah yeah i really appreciate the death row guys uh also just was also having this uh question passed by uh just yesterday from dorian so they've definitely been doing an awesome job and uh we're able to you know really connect and uh follow up on creating solutions bringing up you know any way we can help so uh yeah really really great channels everywhere and we really want to connect with you guys

38. 20:56

    but the course we have on youtube do you want to answer a few questions uh yeah there's a couple in here yeah um do you guys have any course tutorials on your university i'm not sure if you guys do yet um but maybe if there are any plans to collaborate with education team to support some of your tutorials or courses over there yeah i think i think that's a great idea so let's jump in yeah i think we are already doing quite a good job with the tutorials

39. 21:29

    which you can find in our indexer near indexers.your website as to specific tutorials on the near university we will definitely reach out to them and see if they will pick up our materials from from our tutorials but we are not in uh um in a great connection like we haven't set up the connection yet uh to make it a like a full course uh because today near lake is quite a young project out there

40. 22:08

    cool yeah it looks like you got a follow-up uh it'd be cool to have like a certificate covering indexing and you're like a general you could have like a near certified data engineer maybe um so yeah definitely take that into consideration cool man i would like to got this certificate as well cool well yeah keep uh keep posting your questions we have quite a bit of time to uh to answer them if you have any um feel

41. 22:40

    free to keep posting them otherwise was there were there any other agenda items that you guys had for for today um i don't think i have anything implant um this is a this is a shorter one but uh we are going to have more and more coming starting from next one because we'll have more of the levels and more you know detailed things that we're really going to share and you know interact and get more feedbacks on on every

42. 23:12

    on our stack and our deliveries on products um so yeah so be prepared for the for the longer longer ones more and more cool well yeah we can give it uh maybe one or two more minutes see if anybody has any more questions oh another one came in uh what's the state of pagoda dev consul and having near lake being a part of it a awesome question uh we have actually just have been talking about this this week and i just reached out to the console

43. 23:42

    team and um we have this great product and we really want to put on console it's not showing up yet um so a brief sharing on that product roadmap is uh just in the short term um immediate short term we are going to have a section on top of console that is going to be linking to an introduction to the lake and you know some tutorials some great tutorials that locked in here has been

44. 24:06

    creating and uh guiding you through the process of connecting skew lake via aws but um as the next step preview uh we potentially we're working on a solution to integrate with console in a more smoother way and we're still figuring out what that look like but hopefully you can you're able to directly manage your lake access via console with some other potential you know analytics on your usage and uh monitoring or your on your index or status more things to come um

45. 24:36

    but yeah short term we are working on a a quick introduction embed into console um so yeah good to know that you're you know you are also looking forward to that then cool i have another suggestion here yeah so from the poll i see that people are asking for technical walkthrough of uh during our community updates so i think we can actually run our in nft indexer tutorial another tutorial but the result

46. 25:15

    of the tutorial live so at least we can see it how how it's expected to work and i think it's a good use of our time and maybe you will have some other questions meanwhile or maybe you will ask what specific questions regarding the code yeah cool yeah go for it that'd be awesome yeah i hope you can see the my black screen we can maybe make the text a little bit bigger yep game a second like it's it's small for you but it's quite big

47. 25:50

    for me on a 27 inch screen you know okay is this fine yeah that's better yeah maybe open the website first uh and like uh just show the motivation part of it and cover it just okay like i don't want to share an entire screen just because it's too big and i will we're doing verbally uh-huh yeah give me a second ah yeah this one can you see it yep okay um yeah uh this is like a billion energy uh yeah and like meanwhile let me know please anyone who

48. 26:47

    is following tutorials i'm trying to write here uh i decided to go with the end section in the beginning uh to let you know what you got if you like walk through this tutorial uh also i'm putting a motivation section uh explaining why this tutorial might be useful so please let me know if uh it is fine for you like maybe you want some improvements on that and like i'll take it into account uh so as for motivation like uh like i

49. 27:20

    don't know if it's good to read it out loud while you see it it looks more like a lesson uh so i would skip it uh but like yeah we got it it was one of the biggest tutorial i believe so far so uh explaining everything beginning from the uh creating a directory but like uh in a few like myself words about the motivation uh a lot of projects around the nfd's are coming around at near uh so a lot of questions

50. 28:01

    regarding the indexer is something like uh oh i i need to get all the nfts uh like of the specific account or something like that and uh the answer is always like just index the entire blockchain and find every single nfd out there for this account and store it somewhere but how to do that like i believe this tutorial will help you but uh the application like the the code that

51. 28:31

    we have done is more like for the nft hunters i would say like to know every new single nfp out there immediately so i will demonstrate it like uh in in real life explore give me a second i'm typing very slowly uh for us to see it actually live we will copy the recent block and put it uh sorry i'm switching the screen to the terminal that's much more interesting yeah it's here oh

52. 29:19

    that was that was wrong region sorry i need to shop windows yeah i'm putting the freshest block height i i could find into the lake config and then just starting the stuff this inductor is made such way that we won't understand if it is working until any new nft will be minted out there so we can chill out and just wait for the results this is a debug errors from the near lake that is trying to get the newer information before it appeared actually

53. 30:14

    on the s3 so please don't be afraid i mean while we have a question near lake have out of the box support for building dashboards data visualization using leads like d3js i know this might be out of scope but it will certainly be helpful for integrators uh for do you want to answer that yeah i will take it so near lake is the foundation that allows you to collect the data and rear and build your own handlers to the data you receive from the chain

54. 30:51

    uh so i we believe the database visualization and dashboards are the next step and also we have a few projects in the community that we know are working on that problem space so you may either join them in the effort or build your own based on the near lake as your data source basically so um that would be great uh project to be built but it's it falls behind uh like outside of the scope of data platform team at this point in time yeah but still great great sounds a

55. 31:32

    great question and uh definitely highlights the needs of the community thanks yeah uh sorry i'll adjust a little bit but by hiding better or so it won't bother us during during demo and yeah it wasn't planned so please don't be mad at us or so a little bit shitty demo meanwhile i can teach you how to exit whim

56. 32:51

    now we see only the code nft means this uh datetime stuff is made out of the blocks daytime so it is real but it's the current time i believe but for what what time zone i don't know sorry it's definitely not oh yeah it's udc yeah oh great so it is udc thanks olga it's not so exciting demo as you might have expected but like imagine you have written all this stuff walking through

57. 33:42

    tutorial and then you got this result like it's more excited at that moment yeah i see that a few folks in the in the chat said that they will mint uh the entities now i i wonder if yeah any of those can you share which one of them is yours are you only listening for peras no we're listening for every uh one who is following the nap 171 uh but we're building links for for us and for mint base we want to say hi to rikki and nate and

58. 34:24

    their teams hello guys yeah when i see a comment that didn't pick up your uh nft somehow um was it i wonder if it was well you said this meeting i hope the contract was um yes it needs event standard yeah it's kind of weird demo i believe like uh we have created tutorial like uh one day ago no one has have been seen it and following and now like we got a lot of questions about the content of the tutorial which we're not showing like we

59. 35:17

    like maybe we shouldn't we should stop preparing tutorials and just like uh present them only here like during the community meeting so like wanna learn how to build indexer okay join our community meeting yeah we'll we'll have you cover it and always present you with the already built results here cool i don't see any other questions though um yeah okay i think ben ben wants to mint his mint his nfc and then if they're then if there's no more questions after

60. 35:59

    that we can uh probably wrap it up a little bit early but uh yeah hurry up ben we're waiting live it's the anticipation of this mint not too much pressure all right it's going oh rpc timeout

61. 37:00

    next time we will invite sre engineers on the meeting well that's all that's all the time we have for today folks um it's just not now we actually are watching demand now oh we're on mainnet he's minting on chestnut and we're watching for mainnet yeah this is a great point actually uh i can show how fastly we can switch the network without any problem like we just need to change the bucket name we're reading rom from the mainnet to test net and change the block height

62. 37:44

    let's get something like 100 blocks behind maybe 500 block behind yeah give me a second i'm learning how to use the explorer i hope i copied something nope what it was okay 100 blocks i'm ch i'm trying to copy the recent vlog you know okay yeah yeah thank you and let's go all right ben minter testnet nft oh hey we caught a freshly minted nft look at that look at that benji you are a star

63. 38:48

    yeah but like uh anything minted like you can notice this unknown marketplace we marked every others except those were covered during the tutorials like mint base and for us we mark them as unknown and this uh answer that like this response this output doesn't have like don't don't have links uh for the uh nfts because we don't know where it is and you know like another hi to ricky and uh to nate with their teams it was really

64. 39:27

    hard to find out how to build a url for your uh websites like i i spent like half an hour at least that's the crime it's definitely crime especially with the pieces when i need to split this by the column and like put it after the contract name and then after the slash i need to put all together this like without splitting this was amazing yeah thanks for the journey all right thanks everybody it was great to have you

65. 40:06

    and it was great to hear the comments and participate see your feedback on the polls yeah definitely and i think it would actually be kind of cool to create um those polls um uh like on a google sheet or something like a google poll so that way we could post it on the youtube when this gets uh uploaded and then yeah users can participate uh as well um but uh well prepare for that and uh share it all together cool yeah and i believe people voted for

66. 40:38

    technical walk-throughs and demos so we will be trying to prepare like um it was this like that kind of situation when we are like currently building something and just like in that phase that it's not working yet so uh i believe next time we will like we'll bring something to show you like hardcore like from the terminal of course yeah last time we were very excited for the lake and um you know now we're really wanting to understand what you care most about and we're going to

67. 41:12

    tailor everything based on what you need if if the technical walk through this then we'll have uh the live near certified data engineer program and the the poll will be a quiz at the end and you will get your certificate after so um so yeah thank you for all the answers awesome thank you so much everyone we appreciate it see you next time have a have a great rest of your day yeah see ya bye