NEAR Core | Episode 01: Runtime Overview

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   thanks for joining this is the series on the internals of the nearer protocol and we're going to be focused mostly on the runtime this time uh runtime in near protocol is quite complex and to understand why certain things are done this way in the runtime we need to understand the motivation about it motivation behind the near runtime is that we want to have a very good developer experience for

2. 0:25

   people writing contracts in near time and uh what it means is that we don't want them to think that they're running on the blockchain too much or that our blockchain is sharded we don't want to expose the the internals of our of our blockchain we want them to just to write code uh with rust or assembly scrip or whatever you and using The Primitives that are familiar with like promises

3. 0:48

   from JavaScript or futures from rust and don't think exactly how it's going to be executed what it means for us as a developers of the blockchain is that we want to hide the complexity that we would otherwise EXP to developers inside the blockchain protocol and the place that hides all this complexity is called the runtime or most specifically transaction runtime so transaction runtime is this black box that have has lots of logic

4. 1:14

   about how to process transactions and how to execute the contracts in such a way that it looks very friendly to the contract developers this is going to be a series of videos about the transaction run time because this topic is very complex it's going to take quite some time to explore it the video today is going to be mostly about the overview of the transaction runtime and uh

5. 1:39

   um yeah so so we're not going to dive into the details of exactly how the assembly is executed or anything like that uh so let me start first uh by sharing with you like a little bit of drawing that I'm going to make here we go so uh near blockchain is shed which means we have a bunch of sharts executing stuff in parallel in the near blockchain both execution and state is shed some blockchains they don't shart state but in near we sh both execution

6. 2:16

   and state which allows uh to not have huge discs uh for the NOS if they choose to follow only one chart on your blockchain each of the charts has its own runtime I'm going to put a rectangle like this and each of these run times executes in parallel so that's the the thing that is common to many blockchains that are sharted is that there is a lot of parallel execution happening and because

7. 2:47

   execution is parallel um that means that it's going to the substraction is going to spill over into the contract execution so the contracts will be doing a synchronous Contra cross contract Contra calls in some cases as opposed to synchronous Contra calls that are typical to ethereum blockchain because ethereum blockchain is not sharted but we're not going to dive into the details of why uh

8. 3:11

   what is a synchronous uh contract call is from the perspective of the contract developer what you need to take from here is that there are multiple run times running on near blockchain one runtime per sh and this run times execute and parallel and you can think of which run time as the state machine and now I'm going to show you more or less what is the input and output of the state machine going to use cine which is our

9. 3:42

   typical development tool for the uh for the code of our blockchain so the the code that you want to to start looking into if you want to familiarize with our runtime leaves in the runtime folder under run types subfolder and uh the most of the high level things that you need to to know of they live in lip.

10. 4:05

    RS so in lip. RS we have this class called struct called runtime uh which in this case you see it's stateless and it's kind of hey Max can you please increase the font size a little bit thanks oops how do I increase the font size maybe I can decrease this size like this is it better it seems like yeah cool right decrease the size of the window uh so uh this is a bit janky here so don't put too much attention into it what we actually should be should have

11. 4:45

    had here is we shouldn't have had struck runtime because you know it's stateless there's nothing inside and we should have removed all this um reference self from this methods because self is not actually used here I mean there is no state in self um so this is just FYI this is the code that needs to be refactored but things that you're looking at the run time are basically two functions apply and apply Genesis can you see them in this popup

12. 5:14

    window that I'm presenting to you cool so apply function here it is and navigated to it that's basically where you're going to get like 60% of your knowledge about the runtime just by looking at the signature of this function like you can meditate on it take a nap and think about it like tomorrow what the signature of this function is but this is where like 60% of the knowledge of what runtime is doing is actually concentrated in the signature uh the signature represents

13. 5:44

    the state machine of the runtime What It Takes as an input and what it spits out as the output and um Additionally you can think of it as there is input there is output but you can think that there is a state and so you know when when the runtime executes something it also updates the state so as I said runtime is a state machine in one step of the state machine of the runtime is is execution of the one block or because our blockchain is

14. 6:16

    shed execution upon chunk so if you don't know uh chunk is basically a u a stru that contains the transactions and a state on a specific chart and chunks are aggregated together into the block across all sharts so one step of the of the of the of the near run time executes one Chunk on that chart and the two major two main inputs into the state machine are the receipts and transactions and what it spits out in the apply result if we go there are more

15. 6:56

    receipts and some other data that I will also go over basically more receipts and some information of what is the execution result of this thing you see here also there's some uh there are some Fields about the state we're going to touch it later and some stats some proofs this is all useful but uh not as critical for the runtime execution most of the important stuff is in outgoing

16. 7:21

    receipts and outcomes right now that's what you need to focus on so what are the receipts um let me go back to the near blockchain you have transactions and receipts as two main Primitives that uh runtime operates with receipts is basically a message that's been sent between sharts right we don't want sharts to work in completely isolation because if they work in in complete isolation that there wouldn't be any use of having a Shar blockchain that would be just like a bunch of

17. 7:58

    separate blockchains so block the sh shards need to communicate with each other and the way they communicate with each other is using the receipts so receipt is basically a message that shards exchange with each other or they exchange with themselves by just like you know saving it somewhere and then processing it later so think of a receipt as having exactly the same body as a transaction minus signature so receipt is basically a transaction that passed the validation that yes this is the account that issued

18. 8:28

    this transaction stripped of the signature some of the fees were paid almost exactly the same information of the transaction so what flying in between the sharts and the receipts it so when they execute when uh when um sorry when uh one shart executes one step of the of the runtime it consumes transactions a batch of transactions and receipts and it spits out it spits out a bunch of

19. 9:08

    receipts so that's what comes in that what comes out is a bunch of receipts some of these receipts they now fly to another shart on the next step some of them they Circle back and being reprocessed by the same shart on the next step and that's essentially the high level uh step of the of the state machine of the runtime it takes transactions and receipts and spits out the receipts so the question is where transactions are coming from

20. 9:38

    transactions are coming from the RPC so that's the RPC server uh of the of the blockchain node accepting the transactions from its users well most specifically first it's RPC then it gets into the transaction pool but you got the idea basically the users provide the transaction once they go for the transaction pool put into the chunk um they end up with with a runtime and run time consumes transaction receipts and spits out the more of the

21. 10:12

    receipts all right um now let's go back to the code and look at the at the body of the receipts and transactions second so here as I said this is where most of your focus should be if you're trying to understand the runtime the apply method of the runtime structure and uh two important Fields here are incoming receipts and transactions so incoming receipts are what I just said those are the receipts incoming from the other sharts some other shart wants us to process this on

22. 10:51

    some shart a wants to process on Shard B that's how they Rel the receipt and that's where it gets into the run time and uh here's the sign transaction so let's look at the sign transactions sign transaction has as I said the signature the hash the hash is computed on fly it's not actually serialized and passed through the network it's it's computed upon the distalization of the

23. 11:13

    transaction and then there is the transaction body itself the transaction body has some information on like who signed it what is the public key what is the NS um like who is receiving this transaction and a bunch of actions so if we look uh for instance in in the receipt let me me get receipt oops sorry the receipt receip here if you go all the way deep into the action receipt and I'm going to talk later what action receipt versus data receipt you also see that there's a bunch of actions in there so

24. 11:54

    this is basically the body of the body of the receipt the the list of the actions and the body of the transaction also the list of the actions all other information is kind of auxiliary um the sign ID again output data receivers we're going to go through all that but most of the stuff is inside the actions both for the receipt and the transaction and what is the action it's actually

25. 12:18

    what we want to do we want to do we want to create an account with this transaction or this receipt do we want to make a function call transfer some uh some nearer tokens um stake we have approve of stake blockchain delete a key add a key and delete an account so these are the actual things that we're going to execute our blockchain there stored inside the action and the reason why we

26. 12:41

    have a list of actions both in the transaction and the receipt is because unlike many other blockchains our transactions are batched meaning if you want to do multiple things in near you don't need to execute multiple transactions if I want to create an account deploy a contract add access key call some function I don't need to ex to execute a b to submit a bunch of signed transactions in a row and somehow make sure that they all execute in the right order and here you can I have a question

27. 13:14

    is the execution of a single action receipt or transactions Atomic so istion of entire receipt Atomic or only individual actions yeah the execution of the entire C is atomic so if one of the action fails uh uh then the whole receipt is unrolled or roll back sorry and this is useful when you want to issue a very typical badge transaction which is create account deploy a contract uh call initialize method on this contract add

28. 13:46

    an access key um this is like you know four very typical actions that you frequently want to bundle together and what we don't want to happen is that we we create an account deployed a contract but our initial initialization function failed and so we have this contract hanging in the air uh without even access key being added to it so no one can do anything with it and and to avoid

29. 14:10

    this the entire set of actions is getting rolled back so the entire receipt is automic does it mean that that the action uh actions they are performed in the same order that they are in this Vector yes it's important for them to performed in the same order and uh there also even some crazy things that you can do with this actions for instance you can create an account deploy a contract call a meth of method

30. 14:35

    on it delete this account because you don't need it anymore maybe I don't know and then create another account no I mean you cannot create another account currently but you got the idea you can do this kind of crazy stuff right now um and that's um uh that's that's the point there are uh some restricted sequences that you cannot do with this uh for certain reasons because we either didn't implement it yet because it would be

31. 15:00

    hard to implement for instance creating two different two different accounts in the same receipt you cannot do it right now you can only create one account uh you cannot uh do multiple times create delete back and forth because that would make us grindable because Del of the contract is very expensive just like on ethereum ethereum also has issue with this and it's actually one of the economic vulnerabilities of ethereum deleting a very large contract with a

32. 15:25

    very large State um so in our case for instance that would be a prohibited sequence if you try to create delete create delete multiple times in account um in in the same receip right um any other questions has a single destination account I don't see in this structure the information I I think it should have destination account and also predecessor like sender account different possibly

33. 15:57

    different from signer yeah so um destination and sender um are here so again I said that the auxiliary information of the receipt and transaction differs slightly right so in case of the receipt you have a fixed receiver ID you see it even leaves several level levels above the the actions right it doesn't leave even a receipt up it doesn't leave in action receipt it leaves all the way up here's the receiver ID who who is receiving this receipt and receiver ID also

34. 16:32

    identifies what would be the name of the of the of the account that you're creating so when you creating an account uh your action is going to be create account and your receiver ID is going to determine what is the name of the account that you're creating that's why when you go all the way into the actions and you look at what is the create account action there's nothing in here like it's empty it doesn't say what

35. 16:56

    account you want to create because the account name has been determined from the receiver ID and uh in terms of the signer and and sender so for the receipts we we're still going to dive into detail later what what what it means what predecessor ID is um in case of the transaction let's go into the transactions same transaction can I ask a short question about the receipts uh so about the actions um so so max you

36. 17:34

    said that there is no synchronous calls in inside of near blockchain so there are no compossibility of smart contracts like in ethereum but can can not can it be changed with the fact that all the actions are going to be implemented atomically here so like you're in for example if you applied this to defi you're buying here some tokens then you're putting this to the uh to some kind of pool and then you're like like calling some lendin

37. 18:09

    uh contracts and stuff like that can you do all of this in this set of actions yeah so the reason why the classic composability uh breaks is because the the classic ethereum composability allows you to do atomic multiple on multiple contracts at the same time the thing about is that these contracts can live on different shards and so you cannot create Atomic operation yet not going to spoil anything not not yet

38. 18:39

    maybe in the future who knows uh but right now you cannot have an atomic operation that expans two contracts that live on different shs because they live on different shs it's a synchronous and and batched transaction batched receipt that we just looked into is being executed as a batch on a single chart so this batch is executed only on one contract on a single shart you cannot

39. 19:01

    have a batch across two different shards but can you apply this to different contracts on A Single Shard technically we could have actually allowed it but the way we designed the runtime and this is like a really great segue question to how the reasons why our runtime is designed like that is we we didn't want to create any incentives for the people to try to Cate their contracts on the same Shard with someone else because when you have this incentives it's going to create very perverse system people will try to Cate

40. 19:34

    with exchanges they will try to certain shards are going to bloat for some reason and the the situation right now is that there is like literally no reason for someone to try to host their their contract with some other contract on on the same shart because um uh and uh because the way it works and let's go back to this diagram I showed right okay present in the correct window this time is that for instance if I'm on this chart a right and I'm

41. 20:12

    calling uh from contract LSM contract contract Bob if contract Bob leaves let's say actually in here what is going to happen is that Ellis contract will emit the receipt that was going to fly to the shart with Bob contract and that's what it's going to be executed so it's going to be executed One Step later right we still have blocks um now blocks chain so in in in Block X Lis is going to be executed in Block X+ one B is going to be executed and the way it's going to be executed is

42. 20:43

    sh with with with Bob contract is going to receive the receipt that says please call Bob um if Bob was leaving actually um not on this chart but on the same chart with Alice what would happen is that the receipt will still fly but to it to the same Shard and so it would be executed exactly the same way as if Bob was living on a different Shard and that creates like a fair system where there is no reason for you to try to put Bob on the same shart with Lis I don't know

43. 21:16

    cut the latency or somehow create some kind of atomic um execution or anything like that and this is kind of important for us because we don't want to think about this additional adversarial Vector where people will are trying to gem things on the same Shard this is like this makes this creates a new level of complexity if you try to think of right now we have to protect from people trying to put things on the same Shard you have a new game thetic model in there you need like you have

44. 21:45

    different attack vectors that you even think before so we simplified this thing and said okay there is no reason for people to ccate on the same Shard and also we were thinking and we're still thinking about Dynamic re sharting right we're thinking that our shards are going to be dynamically partitioned in the future so if one Shard accidentally blows we need to cut it into pieces you know and and repartition it and uh if people try to game this system try to

45. 22:09

    work against the dynamic or Shing when they've been you know put on a separate shart and then they try to go back to the original chart again then it's going to be just a n nasty game with our protocol again we don't want to think about it so this this whole this whole concept of removing incentives for people to Cate simplifies the prod IAL design like drastically right is sh a directly

46. 22:35

    sending it to sh b or is there some sort of coordinator um the coordination is is kind of implied so there's it's a complet different topic of how actually receipts are relate between the sharts there is no additional actor existing here all all of the actors in our blockchain um a validator notes that you know accept the chunks again the chunk is is the piece of the block specific to The Shard and uh they are

47. 23:07

    emitting those receipts they're exchanging them with each other in certain way that allows to ensure that those receipts are actually receipt but there is no like additional actor in the play who is you know takes the receipts and puts it into the other Shard there is there there is some mechanism of broadcasting that allows to do it pretty securely um with certain guarantees yeah but that's like it's a different topic which goes into the direction of our consensus and how blocks are produced and how synchronization between

48. 23:38

    the uh between different charts is happening all right so uh going back to the run time so with the transaction going back to the question of like where is the originator with in case of transaction the originator is the signer so whoever signs the transaction is considered to be the originator of the of this transaction because we use the assets on the account of the signer to pay for the initial fees every operation on a blockchain should cost something in in tokens

49. 24:14

    because it incurs certain computational cost to the validators uh that process uh this these transactions and so every time you convert transaction to the to the receipt you actually causing validator some CPU cycles and we have a fee for this have a fee that says what is the fee of converting transaction to the receipt actually and sending transaction uh and sending the produce the receipt across the network or like and processing it on

50. 24:44

    the other end once you receive it and therefore when transaction is being received the first time the first thing we do is we deduct the assets uh from we deduct the assets uh from the account that signed it and we convert this transaction to the receipt that's like the very short life cycle of the transaction as soon as it enters the runtime it becomes a receipt let's look at it let's look at this Supply function that I said is like the the core of the

51. 25:15

    of the entire runtime and just Trace what happens to the transactions here transactions so you see we Traen all the transactions in this in this method we take these transactions and um what we're doing we're producing receipts out of them so you see there's a method called process transactions but transaction but what it spits out it spits out to receipt so this method what basically does it takes transaction gives you receipt exactly the same set of actions stripped out of the signature

52. 25:48

    because the receipt is basically a trusted transaction right uh we we we when we exchange the receipts between shs we know that we can trust its receipts because originally the receipts were transactions so we already verified the signatures and we also verify that the amount of of gas that was attached to this receipt is also something that um you know the the transaction signer can pay because we actually have deducted it from the account um so yeah that's a very short

53. 26:21

    cycle life cycle of the transaction inside the run time it becomes the receipt quite quickly um and it can fail to become a receipt if for instance the person who signed the transaction the account did not exist or they didn't have enough assets actually when if the account did not exist it might actually be filtered out even before it enters the run time so one thing to know about the

54. 26:46

    runtime code in general is that the specifics of the runtime code that is makes it very different from other code in our near core is that it's pretty much Max can can can I ask a question so at the moment when when the transaction is converted into the receipt uh you need to know uh the uh the amount of money or the amount of tokens stored on the on the on the Creator uh on the

55. 27:10

    sender of the transaction so does this information needs to be uh synchronous in between all of the shards because the state is sharted yeah the state is sharted yeah so that's so if if they count if they count live on The Shard a and and the account Creator is send in a transaction to The Shard B how The Shard B would understand whether it is enough tokens uh on the account great question shark B doesn't need to know so what happens let's walk from the

56. 27:43

    scenario Alice Alis signs that Ellis signs a transaction and sends it to Shard a where a is like Ellis that's where your contract lives or like your account lives when the the transaction is signed Ellis needs to specify how much gas needs to be attached for all kind of operations in this transaction right it's very similar to what ethereum has but in ethereum you have Max gas limit

57. 28:07

    so you you're kind of specifying the maximum amount of gas that you're going to use in our case you're just saying that it's not the max gas limit it's actually what I going to attach to my transaction so when the transaction here is being converted to the receipt what we literally going to do I think somewhere here is actually we're going to deduct from the balance of Ellis the necessary amount of tokens to pay for

58. 28:33

    the gas that is going to be attached to this transaction let's look at it somewhere here verifying charge transaction it's pretty straightforward it's a it's the method tells you what it's actually doing it verifies transaction and charges for it it charges but who's doing this who is doing this the the Char that that hosts the Ellis's account executes the runtime this is the runtime that we're looking into this run

59. 29:00

    but this means but this means that this short yeah okay okay yeah I I got it but what about the transactions that Addis sent to the Shar B yes because she wants to F to to work with the uh with the contract that was deployed by Bob so when yeah so when Alice let's say wants to call I mean that's a great question right it's a bit of a different scenario but it's actually pretty similar so what the previous scenario

60. 29:32

    That We examined was Alice has a contract on Shard a and Ellis's contract wants to call Bob's contract on Shard on Shard B assume that Ellis doesn't have a contract and Ellis just wants to call Contract Bob here for that Ellis needs to have an account because you need to in order to to execute anything on the blockchain you need to have an account so suppose Ellis's account lives in

61. 30:03

    here Ellis uh creates a transaction uh that calls contract Bob and uses the signature from the from her own account and she knows that they're in assets on the LSS account she sends this transactions to the RPC node receives it and node knows that Ellis Lees Ellis's account lives on Shard a so this transaction is going to be routed to the account to the to the shart that hosts the account that signed the

62. 30:34

    transaction even though the contract that we are calling first is is on on Shard B so uh Shard a that holds ls's account will do all this deduction and that's all it's going to do it's going to it's going to convert transaction to the receipt and spit out the receipt and the receipt is going to fly to Bob's to Bob's to Bob's chart here and with some gas attached so on Shard a we have deducted

63. 31:07

    from LSS tokens certain amount of tokens to pay for the gas that we're going to attach and now this gas is attached with the receipt and it's being and it's flying to Bob's uh to to Shard B with Bob's contract and and Shard B doesn't need to verify that yes Alis indeed has this amount of tokens because receipt says say so receip says that don't worry we have already deducted all this gas from somewhere and this is the amount of gas that you can use for your own execution

64. 31:39

    and when Bob's uh when Bob's contract is done executing it it's going to send a refund back to LSUS uh to the LSS chart it happens automatically the user doesn't need to do it so some some amount of gas is not going to be used and the remaining gas that was used it's going to fly back to alysis uh to to Shard a with alysis account and going to be attached back to ls's account balance convert it back from gas to near tokens using the same price that was

65. 32:12

    initially used to convert tokens to gas but does it mean this receipt with gas attached is signed by validator a so that that's how validator be knows that it's valid or yeah so receipts receipts are uh produced receipts are artifacts of executing transactions right just like the state for instance in ethereum like when you execute transaction the input are the the batch of transaction the artifact is the new state route or the new state try right one of the artifacts

66. 32:46

    in our um chunk execution is uh a batch of receipts not only we give you new state on this chart but we also say here's the set of the receipts that you need need to execute another shirs and just like with the state route you trust it because validator signed this block of this chunk Val said this is the chunk I executed this is the new State Route this is the new set of the receipts that you need to execute another shs so it has the same kind of level of security as the new state rout and therefore the

67. 33:18

    receipts as I said the beginning unlike transactions they don't have signature the stript of the signature you just trust them because the only thing that you can produce the receipt is the validator when they process an a block user cannot submit receipt from the outside user can only submit transaction so you trust if if the receipt says certain gas was attached you trust it because some

68. 33:41

    validator you know computed it and deducted this amount of gas from someone's balance so this money coming from somewhere okay cool all right so this whole Dynamics so like how refunds work and like the receipts it's it's quite complex and we can like go for multiple scenarios um maybe in the future I think like the best way to understand runtime once you understand the signature of the

69. 34:11

    supply method is by reading the apply method so the specifics of the runtime code oops so the specific code of the runtime code is that that is different from the rest of the code in the newor core is that it's spec for the implementation which is also kind of embarrassing that we have spec defined by the implementation so if you if we were to write a spec of the runtime which we're trying to do for quite a long time right

70. 34:43

    now you would actually go over our code and reverse engineer what is the spec like what what are the rules like how what are the rules like do you execute transactions first before the receipts you first execute the receipts that are coming from the other chart so all the rules are basically defined by how code is written here which means the the cost of accidentally switching to lines of code that are

71. 35:09

    actually two lines of code that correspond to the spec rather than to the implementation is quite high right here is a more specific example like when you read the apply code uh the code of the apply method and you know let's let's keep all these things you know we like yes something is happening to the state some ization we converted the transactions to the receipts uh we processed something like

72. 35:34

    called delayed receipts I think local receipts so we processing local receipts then we processing uh incoming receipts and then I'm checking the balance the order of operation is extremely important here and I'm processing the proposals here this is basically the spec if we were right to write the spec of the runtime in pseud quot we would have said it rate over the transactions convert them to the receipts using the following algorithm then um take something that is called

73. 36:07

    local receipts that I haven't explained yet and do something with them then uh processing common receipts then check the balance at the end the the the the order of operation is extremely important here and if you accidentally working with the runtime code and you switch something that is actually correspond to this spec rather than implementation then you might actually break the protocol for instance I can switch this two this lines together and nothing actually is going to happen it's quite obvious in initialization

74. 36:38

    functions they don't do anything right it's not a spec but if I for instance replace the sign transaction and with local receipts first then I might break this spe so this is the big Nuance of working with the runtime code is that you had you got to thre carefully something can be the spec something can be implementation and if you mod Define it you need to know which one is which and the only way to know which one is which

75. 37:01

    is to get this kind of knowledge transfer that we are doing right now and and to actually learn what is spec and what is implementation all right um so let's just walk through this apply apply function step by step and um and I'm going to explain to you on High level what is happening here so yes go ahead so in ethereum there is a step when trans some validation is done for transaction if and if it's failed that this transaction cannot be included in a block at all

76. 37:41

    like for example if the account doesn't exist or it doesn't have enough balance to prepay for gas then this transaction cannot be included in a block and if someone includes it in a block that this block is completely rejected so what do we have in near like which transaction cannot be included in a block at all yeah so we have several steps of filtering transactions and balancing

77. 38:05

    them early on uh some of them happen before the transaction enters the M poool some of them happen when transaction exits the M Pool um I have a question regarding blocks so if some validators possibly running changed code which disables some filter is a block which includes an invalid transaction like a transaction which for which signer account doesn't exist or signature not valid or no like zero balance at all or low balance what

78. 38:38

    happens so this is something that I don't know from the top of my head right I don't know whether this is something that we determine as as a as a as as an error or like as a malicious behavior like this shouldn't be happening like protocol violation is this the protocol violation or is it just a implementation detail of the of the of the of the validator node and the way I can answer to your question right now is that I'm going to look into this runtime error so

79. 39:05

    as I said this apply uh method and it's um and its signature is like extremely important I know that all of the errors that can happen when runtime processes the block or the chunk are going to be in here in this runtime error so I'm going to look into it and I'm going to try to find something that going to tell me um that that something is that some error that says that the block

80. 39:30

    was incorrectly composed and uh what I'm looking into the storage here I don't think that it'ss here balance mismatch this is going to explain you later what it corresponds toat error um epoc manager this is not it so this is about uh how we distribute the rewards between the validators and receed validation error um and here you can see certain things is that we going to spit out an error if you don't have this sign

81. 40:02

    already like for instance someone tried to create a chunk with transactions that just don't have CER like that um then we're going to spit out it there so at least partially the answer to your question is yes we're going to have uh runtime barfing out an error if for instance someone try to give it a chunk with the transactions signed by non-existent account um even if it after if even if it happens after the B

82. 40:34

    pool performed like for example I know that if a block that a chunk it doesn't in that the state that is included in the chunk is before applying any transactions in that chunk so it might be possible that transactions are not valid like for for example transaction specify signer account ID and then the account with that account ID doesn't exist at all so will when will it will be

83. 41:11

    detected that this transaction cannot be valid for that state yeah so as I said like it definitely is going to be detected when runtime will try to process this but I think there are checks um like I know there are checks before the trans will will the blog be rejected that includes this trans transaction I think it will be rejected because the the you see this error this this error is global for the block right as I said the apply method apply method

84. 41:44

    X on the entire block right an entire chunk and it returns you only it doesn't return you Vector of Errors it returns you one error so this error that says that signer ID is invalid it corresponds to the entire chunk so it basically says in a chunk there is a receipt with invalid signer ID so that means from that I deduce you know by by reading this code is that the block is going to be rejected because otherwise why would you return an error like this for the entire

85. 42:14

    chunk um yeah so that's like the value of reading this error messages in the signature of the apply method that it kind of provides you with this information uh yeah so going back to the apply uh apply um method body so what you see here something about the state state is probably going to be a separate video of how exactly we modify the state when we process the when we process the blocks so I'm going

86. 42:48

    to skip it just ignore this nothing is happening here move along uh then validator account updates so what are the validator accounts uh these are basically the the the entities that produce the chunks in our system um and they need to be rewarded for doing that it's a proof of stake system the rewards needs to flow to them proportionally to how much they participated in it uh so there is something is happening on each Chunk in terms of uh in terms of like how we

87. 43:19

    maintain the solidor accounts and we don't need to go right now deep into this it's going to be a separate probably video about how we actually manage validator accounts how we distribute the rewards uh what is the Cadence what are the rules of updating the Distributing the rewards and upda the valid accounts now there's a bunch of initializations uh let's not go through them let's actually go through the through the code here so this is

88. 43:44

    something I have explained to you already the first thing we do we take the transactions that are meant to be produced processed in this block and we convert all of them into the receipts each one transaction means one receipt and here you see a very interesting thing happening here uh so this was actually added at some point later this is not necessary this is kind of an optimization of usability of our

89. 44:07

    runtime depending on where the transaction is being is has the same signer as the receiver we do something different in here so uh what what exactly do we do different uh we do a small optimization which says if L is for for instance calls let's say its own contract living on its own account right Alis call calls itself then we don't need to send produc a receipt that would fly and and uh and let's say to a different shart or be be processed on the next step we Tred to

90. 44:44

    actually process it right away and that's what is is called the local receipt local receipt is a receipt that's been tried to processed right away and uh given this whole you know conversation that I gave you about um how we don't want to incentivize people to Cate on the same shart this actually doesn't create any kind of perverse incentives this is like the only exception when we're going to do something specific when stuff is being executed on the same shart um it only

91. 45:15

    applies to the situation when the transaction is for the function call it calls itself that just allows people to um to wait in most of the cases 50% shorter when they do the contract call on the near blockchain because most of the contract calls on the near blockchain is just a single contract call there's not no cross invocation and instead of waiting for uh like two blocks they're going to wait for one block here to be finalized so it's just an optimization here so again

92. 45:46

    local receipt is the result of transaction calling an account uh which is the same as the account that signed its transaction and it's just an optimization of usability everything else is in outgoing receipts also you can see we keeping account of how much gas was burned and this is going to be important later and we are like recording the outcomes so outcome it's basically our way to record the result of of performance or action

93. 46:17

    in this case converion transaction into the receipt so converting transaction through the receipt might produce certain errors and people who do this on near blockchain they want to see them in RPC they want to see the for instance that they tried to sign with inv valid public key or you know stuff like that so that would produce an outcome that will be displayed uh eventually in one

94. 46:40

    of our interfaces um then let's keep this this is also like implementation details here uh this is Lambda also we don't need to look into what is inside the Lambda let's look at this large Cycles so basically the second point of of focus that you want to have while reading the apply method code is those large Cycles like when you trating over the transactions or you itating over the local receipts or you itating over the you know this is like

95. 47:10

    not for cycle but it's also important or you trating over the incoming receipts so let's let's just focus on these Cycles right now because we have only a few minutes left so the next thing as I said is after right after you converted transactions to the receipts you're going to try to process the receipts that are local receipts that correspond to the transactions that call the contract or the account on the same

96. 47:34

    shart so that's what you do you trate over the local receipts and you call the method process the receipt we can revisit it next time uh and at the same time what you're doing one second at the same time what you're doing is while you're doing all that you keeping track of am I outside the gas limit because we have a limit for the for the for the gas in our chunks so while you're doing all that you're keeping track of the total gas limit and if you if you did exceed the gas limit then instead of processing this receipt

97. 48:07

    and executing the function call or creating an account what you're doing is now you make it a delayed receip so the delayed receipt is a very interesting concept which is uh allows us to not drop the receipts uh so in our in our blockchain it's very very important to not drop the receipts and always execute them once receipt is is produced by uh converting transaction to the receipt or by doing a cross contract call it should be Pro it should be processed because uh this is the guarantee that you need to have if

98. 48:40

    you want to build any kind of defi on on near blockchain if you don't have guarantee that the receipt is going to be eventually executed you cannot have any kind of you know like locking unlocking scenario you cannot um you cannot even you know like exchange tokens on exchanges so this is like a fundamental guarantee of near blockchain is that the receipts are going to be eventually

99. 49:04

    executed and and that's why when we out of the gas for this block instead of just dropping this receipt we're putting it into delayed q and this delayed Q it Lees inside our state so the try the mer Patric TR that in ethereum just contains the state in our case also saves what is the current Q of the receipts that we haven't processed because we run out of gas limit on the previous for the previous block um this is the difference between

100. 49:36

     let's say the Minas that we have delete receipts q that leaves in a single in each single shart and so uh what we're doing here with the theay receipts uh we're trying to process them so uh once we went through the local receipts and now uh we're going to we're going to process them so somewhere here we have process receip things I forgot where is it Max does it mean that uh the fresh transactions they end up in the uh local receipts and only

101. 50:15

     then we are processing the delayed receipts so this is like uh last come first serves yeah so the local receipts have a uh priority over of the delayed receipts you try to process things that are local then you try to process delayed receipts um I think then uh you try to process incoming receipts so somewhere here you process delete receipts I don't remember where wait delete receipts so uh Delete receipt indexes

102. 50:47

     next available index so yeah you trat an over the so basically this function what it does it pops the delete receipts from this QA that lives in the state and processor yeah here's processor SE so you're extracting it here from from the state you literally here you can kind of see that you you go into the state and you say okay what else haven't we processed yet what is the delayed you

103. 51:10

     extract the receipt from the state from the miracle patri TR and then you increment this index that says the Q has moved and U then you process this delayed receipt so um then after you're done with the delayed receipts you process incoming receipts so you first take care of the receipts that of your own receipts you have a sh you first you care above all about your own stuff you care about your own local receipts you care about your own delayed receipts

104. 51:42

     only after you're done with all that then you start processing the incoming receipts from other sharts so incoming receipts are the receipts that you received from other sharts on the previous step and you also process them and and that's it that's B basically what the supply method does so again just to go over this major steps it converts transactions to the receipts it processes local receipts it processes delayed receipts it uh processes incoming receipts like four

105. 52:16

     things four major things that it does in addition of course to doing some work for upd and validator information and um finally finally the uh the last important thing that you see here is this check balance so this check balance is an extremely important method that we have that prevented a whole bunch of bugs in our system and what it does it makes sure that uh money are not coming from

106. 52:42

     nowhere and they're not disappearing into nowhere when the chunk is produced so it's basically an assertion it's just like this is basically an assertion living here it's a very very large and heavy an assertion that condition invariant yeah it's invariant make sure that variant holds it goes over this different manipulations with the balances that we had all this gas that was you know converted from tokens token

107. 53:06

     G tokens converted from gas uh even validat was distributed um gas burnt sometimes it can be accidentally burnt um and it makes sure it just adds up like it it just makes sure that um you know these two variables what should be and what shouldn't you know that this invariant actually holds otherwise you're going to have balance mismatch error and your not is going to panic because it's a broken invariant when did you last see a balance mismatch error what uh how long ago did you see a

108. 53:40

     balance mismatch error the previous time I mean we only saw it in the CI I think uh because we have a pretty good nightly with this s duck and all that so it only happened I think when we just forgot something like some specific Corner case and it was caught so yeah it only happened when we're actually merging things not in production I don't think it happened in production um yeah so that's it I think that's a good high level overview of the

109. 54:09

     apply method and uh if you focus on this on the signature again of the apply method on the signature try to understand what gets in what gets out on this four major cycles that I gave you overview of you will already have like 60% of understanding of the runtime that you might need if you're working with a runtime code like for instance if you're working with evm or um with RPC or like a building breach things like that so

110. 54:37

     that's it um does anyone have any questions um I have a question so as far as I see from the code uh we are not counting for moving transactions into receipts uh for for the gas so it cost nothing and uh so is this correct so as I said it actually costs almost anything we we were super diligent and actually putting fees everywhere so it actually costs things so when you do process transaction it

111. 55:11

     actually going to deduct a fee um for converting transaction to the receipt okay why then we don't have in the in the apply method counting for the the gas on on on the uh moving transaction into the receipt so in processing of the receipts we have that if if if we exceeded the amount then we need to move the resist oh yeah yeah yeah good point good point so um the limit for the gas p on

112. 55:45

     transaction and on everything else is different so you have a you have a basic a 50/50 split uh of the gas limit separately for processing converting transactions into receipts and and for processing receipts and uh this is also something that needs to be Revisited because it's a it's a it's a questionable game theoretic solution here we might want to revisit it later but because it's two separate

113. 56:09

     thresholds um they just the code is in different places all right I have to jump to different meeting thanks guys um we're gonna have more videos like this in case you are interested there's a lot of stuff to go over and yep have a good day byebye thank you bye