Protocol Work Group Call #6 – December 15, 2023

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:04

   so hello and welcome to the sixth uh near protocol workg group call my name is Sarah I'm a technical program manager from pagoto working with devhub I've worked with several of you before and it's always nice to see you again um I'm excited today for us to look at a couple of net proposals and make a decision about them so this is our agenda first we'll discuss the objective for the call and provide some context about our near

2. 0:29

   developer EOS system as well as the NP process next we'll introduce all of the participants that are involved uh then we'll get into the main discussion of NEP 491 and net 508 you can find the link in the invite if you want to follow more closely or click into the GitHub issues um after that we'll open it up to Q&A and talk about next steps by the way it would be awesome to get to know anyone who's on the call today so please feel free to introduce yourself in the comments you can also drop any question or feedback that you

3. 1:01

   have in the chat or raise your hand uh so to get started we'll discuss the purpose of today's call we're here to publicly vote and arrive at a final decision on NEP 491 non-refundable storage and 508 restarting version two we're also here to celebrate the effort and time that everyone invested into these two NPS um it takes a lot of people to move this process forward and um you've all been wonderfully engaged and thorough as I went through a rigorous review by the community and many subject matter e

4. 1:34

   experts who care about making our ecosystem better before we jump in I want to give some background to help contextualize the call so um before we jump in let's talk about devhub the home base for nearb Builders we bring together individuals and organizations to help build a decentralized near developer ecosystem we provide social structure and tools to empower anyone to contribute to the near ecosystem and beyond our main tool for inter

5. 1:59

   interfacing with devhub is through Nev hub.org it's the Gateway for developers to make meaningful impact on near and Beyond you can share ideas proposals get support and sponsorship and find communities working on a variety of areas from protocol development to zero knowledge and you can also create your own communities in addition to the informal Community groups we also have formal

6. 2:22

   work groups like this one who help make decisions in various ecosystem areas such as protocol and contract standards these groups have selected recognized experts who engage with the community and help review proposals in a timely and uh Fair manner manner uh for example they help make decisions on technical proposals that you may want to get integrated across the ecosystem such as new contract

7. 2:46

   standards or protocol Improvement you can submit an idea for a technical proposal using a net a near enhancement proposal it allows anyone to propose an idea to the ecosystem and when a new when a new NEP is submitted the relevant workg group will help review and move it through our formal NP process in terms of the participants there are four key roles in the net process author or champion who is the person who writes the proposal and presents it to the workg group the moderator the person who facilitates the

8. 3:18

   process and helps ensure the net progresses the reviewer orme who is assigned an assigned subject matter expert who reviews the technical feasibility of the proposal and and an approver who was a work group member um and the group who makes the decisions just to visualize the process you can see here there are three stages draft review and voting and two possible outcomes approved or rejected all of the roles mentioned before help move the net through these stages and toward the

9. 3:50

   outcome most of the activity happens asynchronously on the nip itself within GitHub but when we do get to the voting stage the work group has an open public discussion to arrive at the best decision in a transparent way which is what we're doing here today we are executing this process and looking at two proposals that are in the final voting stage before we jump into the actual NP

10. 4:13

    reviews I wanted to highlight the participants who were heavily involved in these two NPS uh first we have Jacob or Kobe who's the author of NP 491 thank you for submitting thisn and for being here today I'm playing the Moder moderator role today in the sense of facilitating this call um we also had Michael Aki Alex and Robin as reviewers who did the technical assessment and we also have the work group members who

11. 4:39

    include Marcelo Michael Marson and Bowen in the interest of time we'll skip formal introductions a lot of you probably know each other anyway but please say hi in the chat or introduce yourself when you first speak and with that I'd love to move on to the actual NEP discussion so we'll first have the author of The NP present present um a summary of the NP and why they believe it's important and then from there we'll

12. 5:04

    have the work group discuss their findings and ultimately arrive at a final decision um let's begin with NEP 491 I'd like to invite Jacob or Kobe the author of The SNP to share a summary summary please take it away awesome thank you Sarah um hello everyone uh my name is Jacob I believe most people in here know me some people maybe don't um yes so I've authored this nap and I will talk about this for like

13. 5:35

    five minutes there is a long discussion around it I'm not able to go into technical details I'm not able to talk about all the Alternatives but here's the summary so at the core this naap is a protocol change which tries to make it easier for businesses to create an application on top of near specifically we're trying to help out on the registration process or new users so when you want to use a decentralized application you need an account on on the blockchain in this

14. 6:10

    case on your protocol and to create this account there is some cost involved now one view is to say well users have to bring their accounts and that's their problem however if you want to onboard millions of users that's really going to happen um we're currently in this phase where you have to help people as much as possible to get their account on the blockchain in the first place and we as the

15. 6:34

    community should make it as easy as possible for businesses to create an experience that will convince users to actually create this account without too much hassle and without people um being stopped in this onboarding process and there's one problem in particular that we're trying to address here which is the refund training atack so instead of saying a user has to bring their account what happens today

16. 7:00

    is often that the developer of the of the dep so basically the business just pays for the user to create the initial account it's not too much money and you can kind of assume that um over time the revenue produced by that user will overcome the initial cost to acquire the user but the problem is as you see here kind of in this simplified diagram um the smart contract has some funding pays for the creation of the

17. 7:30

    account and the user has full control over the account which is kind of what you want from webstream uh you give them full control over the account that means they can also delete the account and while deleting they get some money back because we have refund or the state um uh we have a refund for the money that you put down to reserve the state within the blockchain and that's the problem you

18. 7:56

    can just create accounts by the millions delete all of them and then you make a million times a sent and that's kind of enough money that people will actually want to do it I mean it's actually less than a scent but you know the scale is still the same problem so our solution here in this naap is rather simple we just cross out this receive refund Arrow we just make that impossible so we keep all the current situation everything stays the same you can do everything you did but you can now also create an account

19. 8:29

    and instead of paying for the account flat you can only reserve for storage with this new action that we want to add Reserve storage and then the account can be deleted that's fine but there is nothing to be gained so it's just without that second arrow and this allows businesses to actually create accounts for users in a very simple way it's kind of hard to mess this up like there's just because it's basically like you burn the money uh like you per tokens reserve for

20. 9:01

    storage you just avoid the attack and you don't have to do anything complicated to avoid it you still have to pay for it but that's kind of what people are willing to do um of course people don't really like this because we add a new action and that has lots of consequences throughout the stack all the wallets have to support this all the client implementations and just in general there's like lots of tooling

21. 9:25

    involved um so that's kind of the the con to it but the pro is it makes it much easier and it crucially allows it to be to implement this registration flow in a trustless environment so nether side has to trust the other uh today basically as a as a business I have to trust that my user doesn't make the refund attack or you can flip it the other way around and then the user has to trust the business that they will not delete their account if you let's say you don't give them a full access key

22. 10:01

    and they are just not allowed to delete the account but then the business can delete it or they can do some other stuff um there's also some solution that avoids this with smart contracts but that is much easier to mess up it's a lot more work for businesses so we can't really expect them to implement that on their own um we should probably help them out with that as well and create some

23. 10:24

    standards that would then be a contract standard NEP and yeah there's just much more risk involved with things being hacked um bugs in smart contract code and whatnot so to wrap this up in my own opinion as a former protocol developer I don't really like the change it's a weird action to add um but on the other hand from a developer experience which I think is the defining feature or one of the defining features of NE protocol I think we we have to do something like

24. 10:57

    this and because you all of the discussion we didn't come up with a simple solution that provides the same developer experience I still believe we have to go through with this and it will help the community and the overall growth of neor protocol that's it from my side great thank you so much for that awesome summary um and for submitting the proposal and for your ongoing work throughout all of this process um the community has reviewed and discussed this NEP on GitHub asynchronously along

25. 11:28

    with the smmes and the work group members um so next I'd like to invite Michael to summarize some of the benefits and concerns that we saw in the comments so go ahead Michael thanks Sarah and thanks Jacob for the great summary there um so on the on the concern side it's it's basically what Jacob said right the right right at the very end the solution proposed here is from a protocol perspective a little ugly you might say like it's it's this very specific action that is solving this very specific

26. 12:00

    problem and ideally in the future we actually won't need this because we'll come up with with with better Solutions so one thing that we've talked about is you can imagine that sort of the core problem here is users having to pay for accounts at all we all we we already have zero balance accounts for up to a certain amount of storage and if you could make those zero balance accounts

27. 12:24

    hold smart contract code then you would also solve this problem in in way so we've talked before there's sort of a now mostly defunct NEP about this idea of code sharing where you sort of deploy it once and then share it among all sorts of different accounts by referencing it say for example by the hash um and this would be another way of solving this problem and a solution that has more benefits Beyond

28. 12:48

    just this problem right code sharing is just generally useful besides this this application to zero balance accounts um but the issue there is is that that is a pretty large pretty complicated protocol change and doing it in in a timely fashion is at this moment probably not feasible so again like Jacob said a benefit of this approach is that it's simple it's something that we can Implement more or

29. 13:12

    less today and can release relatively quickly um so we can we can unblock these businesses that are trying to build on near and and again as Jacob said we do think that it is important to give businesses a good experience when they want to develop on so resolving this in a timely man is important and then on the on the contract side again just as Jacob said um you can you can do this right you can

30. 13:36

    do this with the so-called proxy contracts approach where you basically give a function call access key but you define functions in the smart contract that are able to do the actions that normally only a full access key can do and you sort of um uh prevent deleting while giving the the user full control over for the account in this way um but it's complex not just in terms of the development of the contract but also in terms of tooling for interacting with

31. 14:06

    that contract if a Dap wants to create its own function call access key then it actually needs to use a function call Action and you have to make sure that it's not actually gaining access to any of the sort of admin actions and so on and so forth there's there's a lot of complexity there like like Jacob said so our overall sort of benefits side of this is that this is a simple solution that we can Implement relatively quickly and release relatively quickly and it

32. 14:36

    should give developers The Experience they are expecting to get you know without this this possibility for the uh for the refund attack excellent thank you so much okay now that we've heard about the benefits and the concerns we'd like to give an opport opportunity for each working group member to share their opinion and how they're leaning which brings us to the voting stage um based on the net process we need we need a two-thirds majority to move forward with this NP and we've

33. 15:07

    already had each work gr member provide their voting indication on the NP itself in the GitHub discussion thread which you can refer to if you like but here's the summary um so we had um two working group members who initially indicated a vote to approve and then we had two vote two working group members to initi voted to reject however Bowen acted as a tiebreaker after further discussion um and recommended approving is there any further discussion we'd like to

34. 15:41

    have at this point yes Marson yes is the person voting against that um re very quick comments is yes I understand the Need for Speed and I'm really happy that you know we we're making sure that developer life is better at the same time IIT I'm a little bit disappointed because yes the code sharing is an absolutely crucial feature I think that Michael said that once available this will make developer lives a lot easier what we're talking here about is developers having to pay let's

35. 16:13

    say a dollar to deploy a contract and we're protecting from users stealing this dollar but it still CA developers a dollar right with global with code sharing or Global code storage they would have don't have to pay a dollar they have to pay five cents so this is a little bit of a disappointment that I think we are putting our efforts in a little bit wrer like we're fixing the shortterm things rather than investing

36. 16:37

    this time in the long ter but is like a metac comment uh two concrete problems I have here is first the reserve storage name is very misleading because it seems that you reserve and you can get it back but you cannot I know there's this whole discussion about how it should be called I personally think that we should add a word burn in there because these people in crypto understand that burning means

37. 17:00

    it's gone reserving has this feeling that you'll be able to get this back and the thing I'm worried about here is this opens up a very interesting reviance attack where let's say I can Target Marcel and be like hey Marcel sign this contract sign this transaction please it's doesn't do anything bad look it just creates account for you and reserves some storage like a thousand year worth of storage and Marcel will not know that by

38. 17:26

    doing it the Thousand Year is basically gone and this is important because this is a new action type this means all wallets have to be aware of that and have to show it very clearly to users as a warning right so we should really make sure that you know both wallet Implement people who are implementing wallets and the names are clearly warning people that this money will be gone from the

39. 17:49

    from this perspective thank you are there any other objections to moving forward and approving the snap yeah just one extra comment something that was discussed but I think it's not in the concerns was that we are locking some storage that won't be that we won't be able to remove error from the state uh that is some concern to think about uh we came to the conclusion that this is a no concern to some extent because that's happened today even with

40. 18:26

    the state that gets locked and people can refund back it's not happening because the amount they will get back is so so small so that's like we conclude that this is not really important but it's something to take into account let introduce it here yes y I also want to add one more comment like as Marchin said I agree that uh it's using a clear wording is definitely helpful um having said that I know that we also discussed an option of like refunding money back to the uh the smart

41. 19:05

    contract owners so if we use the word word burning then I maybe we're literally uh effectively removing that option from the table unless we you know come up with some other way so I just want to throw that out if that's a way I should think about it well the way it works right now is definitely burning maybe if we do that in the future it will be another protocal change probably there there was some discussion with stakeholders about the refund to the original sponsor

42. 19:45

    option and the stakeholders who responded to those comments basically indicated that this was a nice to have not a have to have it's relatively common for businesses to have a user acquisition cost even in traditional web two spaces so like like for example Spotify right what is the cost for Spotify to acquire a user through say advertising or something like this um uh so since businesses are are sort of okay with with burning the tokens and it's

43. 20:15

    simpler than than allowing a refund um that's why we went with with with it in its current form as Marcelo said you could potentially make another protocol change in the future to enable um refunding to sponsor but I as far as I can tell right now there's no demand for that so I don't think we should do it and again as marison said um probably the better place to put our efforts is into sort of

44. 20:40

    the generalized code sharing or you know Global Storage kind of kind of solutions so my my expectation is that it would be safe to call this a burn and put a whole bunch of warnings around it and in in the future hopefully this will be a sort of obsolete action because we'll have we'll have better better ideas Jacob yeah I think that makes a lot of sense what Michael just said there um

45. 21:06

    just to add a bit more flavor here if we actually want to allow this uh either refund to the original sponsor or somehow like trade whatever storage reservation you have there U that would be a major change internally like the way it's stored within the tribe we would have to add some information of who actually owns that and concept it's completely different because now we're

46. 21:30

    burning it and then it would be like it's still around so that there's still this potential of somehow an attack through couple of corners of getting the value anyway so I think at this point where we decide we want this feature we should say we want it to be burned and call it that and if we in the future want something else and rename it that would be a big change again and should have the same sort of um discussion that

47. 21:55

    we have now so if we decide that this propos is accepted we should call it burn that that's kind of my take any other comments okay so are we comfortable moving forward with an approval on NEP 491 I on this current discussion okay here we go all right so let's talk about some of the next steps to move this NP forward who will be implementing what is the current implementation status and some of the next

48. 22:56

    steps Jacob do you want to propose just making that language change like could you take that on as a Next Step uh we I mean if if you if you have a it the like changing the name as a Next Step at it uh I'll work with the our team member to make that happen I think that's one action item yeah we should do okay yeah I should say I authored that originally but uh lately Ur has been working on it so that's someone from the

49. 23:37

    current protocol team and Y is the engineering manager of that team so I'm happy to help out in some places here but I will not have the capacity to actually push this through with with all the work that needs to be done through the stack so that will have to be with the protocol team understood thank you anything else or are we ready to move on to the next one okay great thank you so much um

50. 24:10

    we'll continue to with the discussion of NP 508 I'd like to invite VAV one of the co co-authors of this snap um so please take it away hi hi everybody thank you uh so yeah my name is vatz uh I've been in pagola for just over a year and this is uh the naap that I will be proposing it's about resharding uh so first let me just quickly comment on what's the difference between sharding and resharding sharding which is one of the defining features of the near blockchain is the ability for

51. 24:41

    nodes to not track all of the shards or all of the state of the blockchain and this is the principle on which we're basing our scalability so ability to serve more and more transactions in traffic over time um so yeah near blockchain is divided into shards and rearing is the process in which the number of shards or more generally The Shard layout uh is changed uh in the past we've had one uh re sharting that's already happened

52. 25:15

    where we went from one Shard to for four shards that we have today and in this naap we are proposing two things uh first we're proposing a new implementation of rear ing that improves the performance and addresses a number of issues in the existing implementation and just to be clear those I'm not saying that the old one was bad it was fit for its purpose of going from one to four shorts it's just it wasn't suitable for uh going from four to another number

53. 25:48

    um so this is one of the proposals second one is we're proposing a new Shard layout uh that is to be deployed to the production networks namely test net First and M net later uh in the new Shard layout we will have five shards and we're going to get there by splitting one of the largest orry the largest Shard that we have today which is Shard number three into two smaller uh sharts and yeah uh talking just about rearing we need resharding um in order

54. 26:22

    to keep the size of the shorts small enough uh so that in the future when we have Charing fully implemented we can uh we can have nodes with some set minimal Hardware requirements to be able to track a predefined number of charts and we don't want to uh for those charts to get out of balance and if we can move on to the next slide um yes so in this implementation we we are only F focusing on splitting shorts uh in the future we may also want

55. 26:58

    to be moving The Shard boundary around or merging shards together uh in this first well second iteration of rearing uh we decided to only implement the the simplest operation that is currently needed um the new implementation is faster it's uh it is robust and should be flexible enough to support Future rearings IT fixes a number of issues and add support for some features that were not present during the time of the first rearing and we also made sure that it's

56. 27:33

    future compatible there is a number of other large projects that are upcoming in the near protocol such as status validation State sync or in memory try so we currently are not integrated fully like rearing would not work with those but we made sure that we can make it happen in the future so that it's not going to block any of the other projects um yeah and that is roughly it about the about re

57. 28:06

    Harding uh concerns is it also on me all right uh so there is a few concerns uh that should be discussed uh first increasing the number of charts means that we will have a permanent increase in the maximum capacity and this may require some extra or additional Hardware from validators on a permanent basis so this is one concerned we did discuss it with the note team who's most familiar with the topic and they said that we should be fine with it this is also one of the reasons why we're uh only going from four to five

58. 28:45

    shorts to make that uh increase smaller rather than for instance going from four to eight shorts where it would be um doubling the the total capacity uh under typical circumstances that maximum capacity is not fully utilized so we're not particularly worried uh but the recent events show that it could be fully utilized and that we don't always handle it as gracefully as we would want so we may want to revisit that actually

59. 29:18

    um the second concern is the temporary increase in resource usage while rearing is happening so rearing itself is actually quite a heavy process uh we made sure uh that it wouldn't impact the node operation again under under typical load uh so we may again want to revisit that and make sure it works under congestion uh there is we we did make it configurable to slow down resharding to allow for the regular note operation to have more priority over resharding but

60. 29:57

    there is a limit to how far we can go with it because we also need to finish rearing uh within a certain timeline um then we have the third concern which is a high storage overhead especially for archival nodes uh so while in the process of rearing uh we not only we will need some extra processing power but also storage and for archival notes the overhead is quite sign ific again we discussed this with the not team's most familiar with

61. 30:31

    it and they're expecting it to be fine but this is something that uh needs to be communicated um with the node operator so that they can prepare and have sufficient uh margin in their uh dis space um then the fourth concern is uh just inherent risk of deploying a large feature and any unknown bus in which case you know the the blockchain May stall which is actually all right if it

62. 31:03

    stalls for a short while and then recovers and continues or if say it only fails for a subset of nodes um we uh yeah it's hard to say what what could go wrong but it won't be nice uh so the resolution here is exhaustive testing we have a whole set of unit test integration tests we tested it in a small uh Network that was deployed to the cloud and then uh I also tested it on test net and Main net by hardcoding uh rearing to happen regardless of the rest of the network and all of this was

63. 31:44

    quite positive we also have uh good monitoring in place so that we can detect issues early and maybe even address it on the Fly we should have enough time for that uh as long as validators are paying attention uh and then the fifth concern is that um the current Shard layout uh has been around for quite a long while and the there may be a number of tools infrastructure or any other programs that have the current Shard layout

64. 32:18

    hardcoded and once The Shard layout changes during rearing those tools may break and we're trying to address that by announcing this change uh in advance across all possible channels giving the uh developers enough time to prepare for the change um that is all the concerns that I collected from the feedback but if I'm missing anything please shout yes anyone else um have any concerns they'd like to go ahead

65. 32:52

    Marson so for starters finally super excited to see the the number of shards increasing we should be doing it every week hopefully H but having said that as as you mentioned vas this is the first time it's happening I think since two years so the two main problems I see there is let's make sure that especially indexers are aware of this because this is the thing that has higher highest odds of breaking and highest odds of doing things like just tracking one

66. 33:22

    Shard and second is because I I see that you responded to my comments on GitHub and I can add exact pointers some of the rpcs will not work if you have machines that are not tracking all the shs so if you live with the assumption that everybody's tracking everything then yes things will work but if you have anybody who is not tracking all the shards at least for one Epoch they will be in a bad State potentially all right uh yeah we're

67. 33:53

    consciously making this rearing being being released before we allow notes to track uh not all shs and oh notes can not notes can already track not all shorts okay but do they that's that's a question from me to you okay fair enough Co we'll look into the rpcs a bit more then yeah and again I also think that most people are tracking older shards so like 90 but it's more about clearly saying that hey probably if you're not all said at least for like one to to two epochs you might be in a bad in a bad

68. 34:31

    shape okay fair enough any other concerns okay great thank you um uh based on the net process we need a two-thirds majority of the working group to move forward with this net n we've already had each work group member provide their voting indication on the NP itself um and in the GitHub discussion thread so here is the summary of that um based on the voting indications it seems like all work group

69. 35:11

    members were leaning to approve this NP so it seems like we're ready to approve this one are there any objections excellent thank you all right so next let's Identify some of the next steps to move this snap forward um such as who's going to be the implementer what's the implementation status and what are some next steps um sure uh I can help with that there is two implementers primary implementors that is myself and Shan from the protocol core team the

70. 35:46

    implementation status is that it's finished and uh the next steps is to get rearing included in the next release uh that is the 13 7 radies okay I've made a note of that and I'll update these slides after the meeting thank you so much for that sure thank you okay um any questions remaining after the discussion for either 491 or 508 all right very thorough um so I want to give a big thank you to the authors all the subject matter experts Who provided the technical

71. 36:33

    review all of the work group members and also the community for participating in this process and attending the call um we're very excited to work together to decentralize the development of near um just to close off the meeting I want to quickly highlight some of the ways you can get involved in the near developer ecosystem if you have a specific idea in mind you can submit it using devhub or you can find other

72. 36:57

    communities that are already existing to collaborate with you can also submit your own NP and join existing discussions about NPS in progress and you can go view all of The NPS and their various statuses on GitHub and join the discussions there you can also join the community telegram group it's a great place to connect with others and also a place to see announcements about meetings like

73. 37:21

    this you can learn more about devhub on our website and stay up to date on all of the social channels um we'll post the recording and summary from today as well as upcoming calls in all of those places um thank you so much everyone and take care enjoy the rest of your day