NEAR Engineering Weekly

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:03

   we are live on youtube do you want to lead the meeting do you want me to well i think either way it's fine i mean i can just start with uh okay from let's start okay uh yeah i guess i'll start with some updates uh from the chain team and the sr team so the chanting on the for the last week uh i think we're still working on this uh charlie network investigation uh definitely made some progress there i

2. 0:42

   can now uh sustain the tvs up to uh 1000 uh for for eight shots uh which is still relatively low uh we tried the 2000 tps and that doesn't quite work yet so still some others get kicked out uh after running for a while so we're still investigating what's going on it seems that there's some uh strange uh network issue that caused this some messages to be delivered quite slowly that caused some

3. 1:18

   chunk producers to miss chunks we are still investigating why that is the case um yeah so uh on the kind of memory and the performance investigation side pyotr finished the the pr to add a performance metrics uh basically this measures the cpu usage of each thread and also added some logging if there's a significant delay in the in the processing of some some messages for for the actors uh and uh also pewter

4. 1:56

   did it in a way uh that is very non-intrusive it is hidden behind a feature flag so you can turn it on and off and also it is down through a procedural macro so this uh so essentially there's no not much visible change to the to the main body of the code it just there is a macro you can mark the uh the the actor uh handle functions wish and uh uh it's it's absolutely great and uh uh thanks a lot for putter uh thanks for watching for doing that work

5. 2:33

   um and and also uh uh i think assume filter also uh is finishing on the uh refactoring the uh the change he did with the memory uh the memory leak investigation and he's adding the uh the memory logging to uh to near core and refactoring a lot of the debugging code he had which is also great so misha is working on uh fixing the uh the cache contention that we found which is a regression uh that caused a node to fault of syncs

6. 3:15

   on testnet so that part is fixed uh but we also realized that uh uh the uh there's some issue with the cache itself and uh in inside view client and this could cause uh for example the curry to timeout and that sync meter is working on optimizing that as well uh basically we don't need to use the cache when we're serving the state part request which is quite expensive and don't

7. 3:46

   actually utilize the cash that much um so also i think i also made some progress in terms of the investigation related to memory spike that can sometimes just cause node to round off memory we're seeing a relatively short amount of time uh it is mostly related to state so i think i'll today discuss with mr on how we want the best address us i i hope uh yeah i think we can make a

8. 4:17

   uh effects in the in the short term very very soon hopefully uh and that should address the uh this issue is uh this memory and uh yeah that's pretty much uh the update from chain team and uh i think i think a lot of people are application this week so progress would be slower so for the i started team sandy was mostly working on improving uncle handbook and also figuring out the

9. 4:49

   issues with profiler automation and he also set up help set up a node for debugging rpc timeouts uh which is great and uh mario was mostly working on the with the bridge team on adding the necessary monitoring and systems they need uh he also finished setting up the issue notes for the 47 nodes for the british team to use and i believe they have already switched to using that and thanks mario for doing that and um

10. 5:27

    yeah that is that is pretty much all the update i have and uh yeah cool thanks bowen does anyone have any questions to born okay sounds like no let's go not a question but like about the checking essentially mentioned that this memory profiling code it's it's under the feature flag and we recently encountered uh performance regression in in a contract uh random code which is related to the fact that uh feature was unintentionally turned on and as a result it was very very slow

11. 6:15

    so so did you add the check that this uh this feature is not enabled on like uh on on a production notes for example in nerdy or something like that if this is a performance sensitive feature yeah i think that's a good point although it's i mean not clear what you mean by production no notes right like uh you do actually want to run some nodes with those features enabled so it's yeah you cannot just do a static

12. 6:47

    check by saying like you know you if you run your d you should not have this enabled well yeah but for example what we could have we could have like uh in enable like i don't know enable and optimize something like that and if this runtime flag is not present then then we can see you want some like yeah yeah i think i think that's probably yeah i think we can do something like that um but for now i think that um you know like since

13. 7:18

    so basically the way it is done is that the the feature is only introduced on the very top level so it's only in the near decrease it's not in any of the sub crates uh that's including near these so i i don't think there's a a possibility of of leaking accidentally leaking features and if you if you do not explicitly enable them they're not really enabled so um i mean sure i think we could add some uh

14. 7:47

    you know uh feature to like uh card like a bunch of features but for now we will i think we only have two uh i guess yeah we still can do that uh think maybe we can discuss this software okay thank you let's discuss it offline um and uh nikolai since you're talking maybe you can give an update on the contract time oh yes probably not not too much of an update but but still um so uh

15. 8:18

    so we uh oh we got mostly running this this stable rust rust stories so just just a few future few changes not yet fully merged but but we close very close here we also had the ocar planning session so so where we updated our website the ocrs in in air table and they kind of reflects of what uh what was done and what we what we're planning planning to do oh also uh we related uh possibility of using uh uh board civilization in uh in a compound compiled cache

16. 9:13

    and uh it turned out that um we couldn't do that at the moment because because we miss certain functionality uh more exactly uh generic bounce in in porsche which which wasn't there and uh and and so we we just we just need uh uh need the pull request washer merged and new version of wash published for for us to use it uh as uh civilization mechanism for contracts also also this long pending story with the with the profile data

17. 10:02

    seems to be a little bit stuck so um so let me remind the the story is that we want to put ability to to set the the profile data by profile we mean uh how how gas is being been spent on uh on web assembly code execution on host function execution and such and so uh we try to make it to make it uh usable in every view and in the black holes and it seems there is no like a clear

18. 10:45

    clear agreement so maybe um um bowen in particular so maybe you can uh speak up on what uh just just in a pull request what's what's your opinion how how would you like to uh us to do that because because i think it's uh it's an important uh general general purpose feature and also willem suggested to eventually provide provider functionality to that even in rpc calls so at least on a test net

19. 11:16

    uh people people can can actually extract from from the rpc how uh how how gas is being is being spent when uh uh when executing the contracts so what do those things does it make sense yeah it seems that you talked about several scenes right so uh uh i think oh i maybe i missed the like the first part you're talking about but are you asking like how i think about uh how we should add the profiler to the

20. 11:48

    into near core okay uh i think i commented on the pr uh but maybe there's some more discussion there i can take a look later okay okay yeah so i'm i'm talking about uh 37 30 in particular but uh that's uh i but like is there something specific that you you want me to to take a look or is it like more general yeah yeah it was just just a just a discussion uh uh like you suggested to to hide it um behind behind the fishing flag yeah yeah well i'm asking your opinion and i think

21. 12:29

    that's kind of stuck on it just working here as a kind of uh push try to push it forward and actually i just like like to get it uh agreed upon and merged so that's that's okay okay um okay so uh what else um uh also also on our okay our session we have uh we have a new uh new person who will join us uh uh after after after the christmas break uh alex cardiff so he he was very uh interested and uh

22. 13:09

    and even gave us uh a few practical recommendations about about using uh using rust the rustol chains and situation so i hope it will be it will be a good addition uh to the team um looking forward for everyone um the deadline for the lkrs is would be the second week of january actually i think january 15 to be more specific so you you can start working on drafts of the cars now but you have more time

23. 13:43

    and it's probably better if you do it in the beginning of january once we have organization wide occurs uh the nikolai's team they are in a more special position because their work is is very they have planned their work for for many months ahead of time so it's not going to be affected that much for the organization might be adjusted a little bit um so yes if you if you want to start doing like yours now you can just keep in mind that once

24. 14:15

    come organization you might need to do some minor adjustments to it um right how about alex rochanka can you talk about the status of the evm and surely the bridge too yeah i can i can do both hello everybody so the bridge update let's start with the bridge uh uh probably the most uh major update on the following week was that the rinky by testnet uh near testnet bridge uh is deployed uh by the hands of marcella marcel thanks

25. 14:48

    very much for this and uh and the robston uh bridge is also on the way is going to be there soon in general uh the team was prepared for the holiday season we are continuing our work on the bridge 2.0 um uh evgeny capune is focusing on this we are continuing work on the ui of the bridge and here is uh chad mainly is doing his work uh and uh we figured out with corbin that we already

26. 15:19

    have all of the needed designs for this so so designs are completed now i need to develop finalize the ui uh we're continuing our work on the monitoring and there are still some things uh that needs to be done there and with uh with regards to the testing the testing is also continued with the bridge uh good news that the previous problems uh inside of the bridge were caused by wrong scripts or like problems in scripts uh specifically problems with nonsense of the ethereum transactions

27. 15:56

    and it is not dependent on the smart contracts so hopefully the smart contracts are okay um also rap near was reviewed by marcella and i encouraging everyone who is in the core contracts repo the code owner of the core contracts repo to take a look at the pull request number 120 uh to be able to to say whether it is okay or not the more people uh will look uh at the pull request uh the better it will be in the end so that's it about the bridge um

28. 16:34

    with regards to the evm so late late motif of the week was phase 2 development and handover due to the changes in the team uh so bo is going to he is going to quit working with uh within the evm project um and uh so and his major part of his work were he was better transactions so we were doing lots of work on meta transactions with rethinking of of their logic and the way how how they are working in general and uh this information was handed over from bow to arto and

29. 17:14

    probably it is going to be continued within this short week um and besides that uh we are continuing to work on the anemone test suite so to test the evm implementation uh mike developed an excellent example of how to use pet shop deployed in the beta net so how to use it in 42 lines of code it is available in the your protocol slash examples uh uh rebel and we also have rescheduled the call even twice right now so the work group of the evm

30. 17:55

    if some of you were visiting it or wanted to do this uh it is now happening on fridays uh but in in the different time so it starts at seven morning san francisco 4 pm europe 11 pm china and it would last for one hour so we are extending this call we would like to bring more discussions there and make it like a call during which we are going to show to our community what was done within the week uh and maybe just a little bit more deeper and more

31. 18:28

    long discussions uh with regards to some major points that needs to be done within the next week so that's it from my site um any questions or what questions do you have oh yeah actually i have a question uh i think i mentioned it in uh in one of the strats on slack maybe uh so there's a bunch of uh transaction failures on uh on testnet and uh uh i think marcelo said this because we're

32. 19:07

    concurrently submitting a lot of headers that seems to me a not the best strategy in a way that we are like burning tokens unnecessarily on the transaction fees if we know that those transactions are going to fail yeah i'm looking i'm looking into that right now actually sorry i will fix it okay sounds good in general our strategy is to submit transactions in a batch asynchronously

33. 19:37

    when bridge is catching up with the theorem because that's that's how we actually are able to catch up really fast with ethereum is by submitting optimistically like 10 transactions in a time and hope that they all end up in the same block and they get rearranged according to the nuns and yeah sometimes you have some of them failing i think what you're referring to is something that causes more failures than that but in general it's okay for the bridge transactions to fail

34. 20:09

    um every now and then yeah i think i observed every morning i think at least for a while i was seeing like looking at all failures and there's no successes oh okay right okay so let's put it in the list uh marcella maybe you would be able to create an issue on the github so we can track this sure thanks thanks uh kendall can you give us please an update on wallet your helicopter maybe also to everyone else and uh let's give you time to fix it

35. 20:54

    and meanwhile uh bloodforlof can you please give update on node interfaces yep sure uh on the non interfaces side of things we were mostly focused last week on accurate computation of circulating supply and we identified quite a few issues with our lookup account uh implementation in javascript and we are still confirming that uh our result is expected because we have some discrepancies

36. 21:27

    and even when we dis extracted the get locked amount right from the contract itself adjusting it to the fact that we are already in phase 2 stage we still have the uh unexpected result from what we received from uh eric so me and bogdan are looking into that and currently we are blocked on yes and i guess uh to provide us with the like where exactly uh we lose the um count uh also i was working

37. 22:16

    on the uh explorer site the indexer side to collect various stats about our network for town halls and also for future charts we want to display on explorer site uh those are really uh and but haven't been merged yet i'm also finishing borsch project rework uh which basically means that our near borsch repo will only host the readme file and the static website and all all the rest implementations

38. 23:02

    will live in their respective repos so currently uh jazz implementation lives in its own repo and maintained by uh sergey i guess and worsh is maintained by me and bo uh we'll see how it goes uh further who is going to code on the rust part uh also bo helped us to implement json rpc and point to query contract code binary through our pc but this while this is in master i cannot provide any ata on when it will be available for test net in minute maybe bowen can update us

39. 23:55

    on this like because i'm not following the trade when we uh how to track trace down when and what includes releases um yeah on the explorer side of things we are exposing more data uh indexer version of explorer is currently deployed and under dedicated domains and we identified a few minor bugs there once those are fixed i guess we are ready to release it for testnet and mainnet

40. 24:38

    under main domains and we'll keep the old implementation for betternet and guildnet and this unblocked us on many items like like i closed at least a dozen but bug reports on explorer just with this uh indexer release so that's a great improvement there thanks for this work to bogdan and iphone uh working hard on backhand and front end to be future complete and i guess that's all from my side at this point of time

41. 25:17

    cool thanks lad um what questions does anyone have no questions all right great kendall are you back i have a question what are you getting yes oh yeah about the state of the received by by i also i also started implementing it so it will be ready in the coming days okay so uh vlad i also have a question not sure if it's a lot or to anyone else so do we have any definitive plan on uh on

42. 26:06

    update of the deployment of a new new version of of mainnet so that we need to organize and with which moment we need to have ready uh new costs ready yeah i think we can that can wait for the next but not the emit not the next one but the one afterwards so or like well depending on when when you guys feel comfortable uh of course first need to be released and tested and i don't think that's that is merged

43. 26:38

    because i think that is not merged because we're waiting for the uh the uh the the change to stay sync to actually uh recon to compile the the contracts uh just and store them in cash before we can change the fees okay all right cool kendall are you back yeah is that better yes much better okay great all right on the wallet side uh while testing last monday's release we uncovered a few bugs that we fixed over over the week so that release will likely be going out today prepares what's in that release check it

44. 27:24

    out here it's mainly a new send ui um a few bug fixes and then an adjustment basically making some of the balances a bit more accurate we found there so we'll have one more release before the holidays uh we're targeting to have in that release the account creation flow improvements just trying to make it much clearer to users what's happening with their accounts after they fund them to clear potentially clear of some issues there there'll also be improved staking

45. 27:55

    rewards calculations particularly when users have 2fa enabled and then hopefully uh also disable 2fa which is is working just doing some final testing on it so if all goes well we'll be putting that cutting the release today and then likely releasing it sometime between now and thursday and if you want to see what what's sort of left to do for the year here you can check out this here uh everything that's in progress that's in in progress and beyond will be released uh the four items that are

46. 28:26

    still in the backlog will be the first items targeted early next year uh what what questions can i answer for anyone kendall i recently figured out but maybe this was not a recent change i'm just not a big user of the explorer so recently figured out that uh the layout have changed and the information about the is taken it moved it it should probably move to the to the uh to the wallet itself are there any uh any in time frame when

47. 29:01

    uh when it is going to be displayed there i mean the the all like all of the big amount of different balances that we have locked ones staked ones uh stuff like that um are you asking when it will be displayed in the explorer or when it will be displayed in wallet because uh as far as i get it's from from blood it's never going to be displayed in explorer so there's a yeah we stinking rewards

48. 29:33

    are already displayed or rather sticky words are mostly displayed in the wallet uh everybody can see their balance accumulating especially if they're viewing the staking page but there were some issues with the stake in rewards calculation for 2fa for various reasons which the indexers helped us clear up um so that should be in you know by the holidays uh we are making we are going to majorly

49. 29:57

    refactor the way balances are displayed and we make some calculation adjustments to make things much clearer um if you follow the improved balances issue and that now one that i just sound out and i can also add something here you can kind of see what the plan is i think there are a few designs in there that are roughly the latest uh so if you're interested what it's going to look like

50. 30:21

    yeah yeah i was seeing this yeah thanks a lot uh maybe this would be a little bit so is this information uh would be available only after the login in the wallet no the profile page is publicly available so you can search any address and view it there that's already true yeah i'd like to also clarify the situation on explorer side and the profile page as well so on explorer we started initially with displaying on

51. 30:52

    the native balances uh meaning they like non-staking and staking balance that is native to near protocol we also have the notion of balances that are delegated or moved to lockup which are different accounts in terms of near protocol but they are like needs to be tied together for user uh representation and originally we tried to solve this problem explorer because it was a landing page

52. 31:31

    but quickly we realized that uh this is not a pass forward because there's balances could be various and there are tons of them so discussing internally we decided to scale back and provide the link in explorer to the profile page we can which you can find right now in the ui so probably if you haven't noticed the link it's a it's a ux bug and we need to address it definitely and make it more explorable and wallet is from what i can see with

53. 32:11

    this latest release should address the issue with the rewards that you refer to and once it blends you will see the proper uh latest balance with all the rewards you have yeah yeah well i i've seen i've seen that link i was trying to to access it but when i went there it was it was showing only the the current uh account balance without any uh information about this taking so that that was the problem yeah so the

54. 32:40

    link is big i mean anyone will be able to see it cool all right um you're getting kozakov mike and chad they all on the vacation today so they won't be able to give um summary of what the group was working on i'm gonna give like a very short in information here and then we can wrap up this meeting so we are actually going to sunset the engineering sync starting the new year for several reasons the primary reason said that we want our things that are public to the community to be engaged into the

55. 33:25

    community and uh since we want team things to be also engaging we're probably going to focus our our attention on these things and try to make them as engaging as possible so there's going to be a lot of work for the tls to figure out uh how to make them engaging and we're going to work on it all together and there is also duplication information happening during the the main engineering sync on monday with engineering of individual teams

56. 33:56

    so uh we gonna be sunsetting this engineering sync starting the new year that means uh the next week engineering sync is probably going to be the last one what we're going to replace it with is we're going to replace it with better agendas for each individual teams things so each team will start their things with what is the what is the goal uh what have been what has been done last week what are we going to do next week and this information is going to be directly taken into the

57. 34:27

    newsletter and all kind of marketing material that we're going to be distributing through the community and that's going to be basically our placement for the engineering sync the newsletter and that will also encourage uh the people who would normally ask the questions during this meeting like like we just did right now to actually ask these questions directly to the team members of this other team and you reach out to them through whatever tools they choose

58. 34:56

    to use whether this github projects i know github discussions so discord or whatever and actually get involved into the in these communication channels instead of just centralizing all these questions in in demand engineering things so that's the plan um and hope uh teals will start preparing by improving uh the agenda of the individual team engineering syncs and uh yeah that's it so many of you might not be

59. 35:28

    in the office next week so for those of you who we're not gonna see next week i wish you happy holidays and uh upcoming happy upcoming new year and thanks everyone for joining us today goodbye