{"id":"YGUeTu4Cco4","title":"Whiteboard Series with NEAR | Ep: 55 NEAR on NEAR with Illia Polosukhin and Bowen Wang","description":"Illia Polosukhin (Co-Founder of NEAR Protocol) and Bowen Wang (Director of Protocol) discuss the consensus mechanism and the Nightshade design of the NEAR blockchain. \n\nTakeaways:\n• NEAR uses a hybrid consensus mechanism called Doomslug, which allows for faster block production and finalization.\n• The Nightshade design of NEAR eliminates the need for separate shard chains and uses one chain.\n• Preventing shard corruption and handling fraud proofs are complex challenges in sharded blockchains.\n• Stateless validation is a key concept in NEAR that addresses the issue of state growth in sharded blockchains. \n• Sharding is a core technology that enables scalability and decentralization in blockchain networks.\n• State validation improves fraud resistance and allows for lightweight validators.\n• Sharding limits the size of each shard and enables faster state access by storing the entire state in memory.\n• The user experience is simplified through the account model, access keys, and meta transactions.\n• WebAssembly is a versatile virtual machine that enhances developer tools and allows for easy deployment of smart contracts.\n• Chain signatures enable accounts to transact across multiple chains and enhance interoperability.\n\n00:00 Hybrid Consensus: Doomslug\n08:30 Nightshade Design: One Chain to Rule Them All\n27:55 Stateless Validation: Addressing State Growth\n40:36 User Experience and Developer Tools: Simplifying Account Management and Deployment\n49:09 WebAssembly: Enhancing Developer Tools and Smart Contract Deployment\n\nJoin NEAR's community:\nWebsite: https://near.org/\nTwitter:   / nearprotocol  \nBlog: https://near.org/blog/\nMedium:   / nearprotocol  \nReddit:   / nearprotocol  \nDiscord: https://near.chat/\nGitHub: https://github.com/near  https://github.com/nearprotocol\nDev Docs: https://docs.near.org/\nCreate a wallet: https://wallet.near.org/create\nLearn to Build On NEAR: https://www.near.university/\n#NEAR #nearprotocol #chainabstraction","channel_name":"NEAR Protocol","published_at":"2024-04-25T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/YGUeTu4Cco4/hqdefault.jpg","transcript_text":"Hello everyone, I'm Illia co -founder of NEAR and with me today, Bowen, who leads NEAR's protocol team and work group. And we have a very special episode because today compared to other episodes, we're actually going to talk about NEAR. So we wanted to actually go and dive in from the bottom up through all of the NEAR technology and... kind of explain piece by piece, as well as some of the changes that happened since a lot of the original papers introduced in 2019. So Bowen, let's dig in. I think, yeah, let's start with consensus. Yeah, sure. Yeah, I think this is part where it was not explained a lot in previous videos or other material about NEAR But today we use a... hybrid consensus called Doomslug So in the URA consensus, there are two types of them. One is BFT consensus, where everyone, there is a majority of the people need to actually reach agreement on what the block is. And basically what is produced is considered finalized. And then there's other type of consensus where there is the requirement for producing blocks is more relaxed, but then you can have forks. And the blockchain somehow needs something on top of that. to finalize blocks that have been produced. So this is the case with Ethereum, with Polkadot, where there is this block production logic with folk choice rule, and then there is a finality gadget on top of that that actually finalizes blocks. And near uses this approach as well. So the advantage here is that you don't actually have to wait a lot. So you're really in BFT consensus, at least there's two rounds of communication, and it adds a lot of latency. And here, by not actually requiring BFT consensus at the time of block production, we can actually pipeline block production with processing of the blocks. And as they get processed and more blocks get built on top, the previous ones actually get finalized in this way, like maximizing this throughput. So how this roughly works is that after this block is produced, Then it gets sent to obviously all the validators. And then they receive the process block validator that is indeed valid. And then they would start sending approvals to the next block producer. And then if they have sufficient approvals, they would produce the next block and so on. So I'll just write what? I'll just write push two. So there are actually two type of approvals. So it has endorsements. and skip. So this happens, endorsement happens when there's no block height being skipped. Basically, you process a previous block and you're sending approval for the next block. And then skip happens when some block is missed. Let's say the block producer is offline or there's a network delay, the block is skipped. Then essentially, we're saying it's like, skip basically saying like the previous block height is something and the next order is skipping to what is next height. And roughly the idea is that when you have this three blocks in a row where there's like a block h block, h plus one, block h plus two, then it actually finalizes h. And there is obviously the consensus needs to have like life provable, uh, safety and lifeness. And, um, uh, you know, we can, it, the, the proof is actually much more involved, but basically roughly the idea is that, um, uh, after a sufficient time past, uh, there will be a case where those, uh, three consecutive blocks will get produced and then the first one will get finalized. Yeah. So maybe when we're saying, so when the approvals are being sent to block, uh, T plus two, they in a way are finalizing T. Because in a simplistic term, there's 2F plus 1 approvals here, 2F plus 1 approvals here, and so 2 thirds plus 1 saw 2 thirds plus 1 saw some block. So this is kind of the, in a way, like an on -chain version of some BFT consensus, kind of like tandem maintain. The approvals are part of the block, so it's actually like, which block is being finalized is calculated on -chain by looking at the other blocks. So the benefit is, compared to traditional BFT, a block is produced, it's sent out to everyone, validators then send approvals back to the same validator. This validator aggregates all of these approvals and then sends out a block to everybody else. And so this usually actually requires more steps, especially because he need to... account that not everybody saw it. And so this usually requires multiple steps of communication and takes two, three or longer seconds. And this approach allows to pipeline in kind of the block speed and improve block speed. Yeah. Yeah, so basically, as these blocks are produced, while B is still waiting to be finalized, people already apply the blocks and apply the next blocks and so on. And the Demosrack also has this property that as long as the next block is produced, there's a concept called Demosrack finality, meaning that if both B and B plus 1 is produced, then at least there needs to be some malicious behavior in order for B to not get finalized. Yeah, because if this produces two F plus one signatures, then if there is a fork indeed, then somebody has signed maliciously and so you can slash them. OK. Yeah, so I mean, the big benefits been improving the kind of overall latency of the system, as well as one of the kind of not usually talked about in blockchain parts is that although there is a consensus layer, there's actually a lot of other nodes in the system, like exchanges, indexers that actually provide run -ins, RPC nodes. And. they have the same latency of whatever consensus is plus either optimistic or final, like if they're waiting for optimistic or final block. And so by speeding this up, at this point, this is already sent to the indexer who can actually process it and prepare this kind of the output. And so if you are hitting your RPC, you can actually have a optimistic. pretty much results, which in many cases is enough to start to visualize it in the front end. And then if there's a rework or something happens, especially after one block, you already have enough guarantees for pretty sizable amounts of value moved. All right. Well, let's go to the meaty part. So, NEAR has started as a chartered blockchain and has been kind of introduced in Nightshade design back in 2019. And so I think it's worth both diving into Nightshade, but also Bowen kind of led the work last year to introduce kind of Nightshade v2, which introduced the concept of stateless validation. I think... Let's dive into that. So I think one of the core idea of Nightshade is that in previous designs of sharded blockchains, usually there is some kind of main chain or people call it like relay chain or like the hub or like whatever names people use for it. It's basically like this chain that people use to orchestrate validators. People you can run like staking and then validator gets selected, assigned to shards and so on and so forth. And then separately for each shard, the names can be different, but conceptually there will be like a shard chain, meaning that for each shard it actually builds its own blockchain. So this is fine, but one of the problems is mainly that it gets very complex. So let's say you have a cross shard transaction. Now the problem is on both shard chains you can have forks. And say forks. And depending on how your transaction land, it could be that on the source shard, the transaction is executed on the canonical chain, meaning that the fork that becomes eventually final. But then when you send it to the destination shard to execute, it could be that on that part it landed in the non -canon. canonical fork, meaning that later there's a reorg that actually went to the other fork. That's a big problem because you have this atomicity failure. Let's say you're sending some tokens, then it will get debited on the sender account, meaning that they actually get the token deducted. And then on the receiver end, if there is this reorg, then suddenly they didn't actually get the token. So that's the problem. And then there are ways to solve this. But then it gets complex, you need to do this cross linking to like beacon chain and all the nones. Yeah, I mean, usually the approach is you add another chain, which is a whatever beacon chain, which now you communicate to it. And then from there you communicate to like it finalizes blocks and then you can communicate kind of needs to be included. And so when this gets finalized the next block, you guarantee inclusion or some kind of processing. and ordering and everything. And that creates a massive bottleneck because now this needs to actually receive all the messages from everywhere and be able to like route them. And yeah, kind of creates additional latency for the cross -short communication because now you need to have block time here, block time here, and potentially one more block time here, if not more, because you still need to wait for finalization. So it can be taking like four block times. at least if not longer. Yeah, I think that's one thing. Another thing is that from an implementation point of view, this is actually way more complex. You need to handle a lot more kernel cases than if you just only have one chain that you need to deal with, like forks and switch between forks and so on. So let's not do that. Yeah, so that's where the nice idea came in. So the idea is that instead of having like, having shard chains, we just have like one chain to rule them all. And so inside this chain, it has blocks that... Sorry. So it has a chunk for each shard. So chunk is basically a shard block. I'm sorry. So... Kind of like implicitly, you can think of there's still like this short chain idea where if like you actually having the next chunk point to the previous chunk and so on. But on the level of like how they're being orchestrated, they're being orchestrated as part of the block. And there is only one blockchain that contains the chunks. So the blocks don't actually contain the physical chunks and the way... is handled is that it actually only contains the chunk header. And this is to make sure that blocks can be propagated very fast, but still provides enough metadata about each shard chunks so that people can request the corresponding chunks if they need to. So obviously not every node in the network cares about every shard, but for the shard they do care about it. And once you see the header, they can... verify it's rigid and then can actually request what the chunk is according. So yeah, that's kind of the, I guess the core idea of, of Nation. And then I think another very important aspect is always read the data variability, how we call it guarantee this in a shard network. This unlike a non -sharded blockchain. The one problem is that we need to handle inherently here is that from every shard from the view, the other shard it's running. kind of light crime for other shards. So it needs to be convinced that the other shards data actually are available. Otherwise, there could be attacks of withholding data and then actually generate receives and create this double spending attack where they do the reorg and then the data on one of the forks is actually not available. So the way we do this is that for every chunk we produce, it actually first get erasure coded into pieces and they get distributed to all the validators in the system. And there's a system parameter on like how many validators can collectively reconstruct to collectively reconstruct the chunk using the shares you get. So I think the in production threshold is set to one third, meaning that as long as a third of the other is honest, we can actually reconstruct the chunk for every single. Yeah, so maybe to kind of from a specific chunk perspective, or as it was put for data -available sharding, as it gets produced, it gets erasure -coded into parts. So a number of parts, which as Bowen said, depends on the parameter. And then these parts are then sent to other. validators in other shards or like across the whole network. And so, so you have like we, you know, we want to go. And so ACE, the kind of the benefit of this, that the subset of this validators can reconstruct this chunk if needed. So even if, you know, all of the eventually validators in the chart go offline or they're malicious, if this procedure happened, it's possible to reconstruct it from the. validators and other shards. So this provides a very beneficial guarantee because this validator is actually not going to send approvals for the next block if they haven't seen the chunks from every other shard. So if they haven't received their part of a chunk. So we actually weave in the data availability directly into consensus by doing this. And I think one other optimization we did is that after we produce every chunk, it's actually not executed first. It's first sent out to all the validators, sorry, all the validators of the track list chart. So that then basically when they receive it, they start executing. And then the whoever produces chunk also start executing the chunk. This is to avoid this 2X latency you would introduce otherwise. which is like you first execute the chunk and then you distribute to all the people out there and then they all execute it. So, yeah. So this is again to compare this, for example, Tendermint where you order transactions first, then you execute, which for example, if you know the execution can take like another second, then you send for approvals, then you... wait for two thirds of approvals. And so like, and you probably wait for another round generally. And so that's kind of why, you know, you have extra latency on most of the block production. And so this pretty much simplifies this by always kind of, you know, as soon as you have data, you send it out. As soon as you have kind of executed and validated, uh, invariance, you approve it, and as soon as approval is collected, you kind of repeat that. And all of the kind of all the invariants have been calculated kind of retroactively. All right, so maybe it's good to then cover the stateless validation. Yeah. So in the original natural design, there's one very complex piece, which is how do we prevent entire shard from getting corrupted? So even though there is this like, you know, every epoch, the viaduct shuffles and the assignment to shards is random, but the epoch is still relatively long and there is this... assumption that it's possible for the shard to get adaptively corrupted after the assignment of either to shard is known to everyone who can just see the assignment of either to sharding the entire blockchain. So the idea is that, well, if a shard is corrupted, meaning let's say it has a majority of over two thirds of malicious viadators, then it can potentially actually do a invalid state transition and let's say mean some. a lot of tokens. And then the part that's very annoying to deal with here in this sharded blockchain setting is that they actually, let's say you do something to identify the malicious behavior, it's not sufficient to just like roll back the state of this shard because they actually can, after mean that a lot of tokens in an invalid state transition, they can actually distribute that to or send that to many different accounts on many different shards. So that even though, all the other shards are still doing various state transitions. And because just like one shard getting corrupted essentially can corrupt the entire system, like all the shards becoming valid in a way. So that's the difficult thing to deal with. And in the original nightshade design, the proposal is to use fraud proofs. So the idea is that there's this kind of people called fishermen who so responsibilities to watch the blockchain to make sure that they know in last stage transition actually happens. And essentially the idea is that as long as there's one honest fisherman or validator in a shard, then if an in last stage transition happens, they can submit what is called a fraud proof. So what this does is that it actually has the partial state. of the state that's touched during the execution of the chunk that's questionable. Obviously, it has the pref state root and obviously the post state root. So basically saying that here is the state that's touched during the execution of this chunk and here is a pref state root. anyone else in the system, any validators who's not tracking the shard, can do the same execution and arrive at some post state route by doing themselves. And then they can see whether that state route is indeed invalid, meaning that whoever is doing the invalid state transition, they necessarily would have a different state route because they're not following the rules. So then this can prove to everyone that their state transition is invalid. and then they get slashed accordingly. So this is a fraud proof part. But another difficult part actually happens after the fraud proof is verified. So let's say you actually verify the fraud proof. Now, you're also in a difficult spot because, as I said before, it's not sufficient to just reset the state of a specific shard because it could have already polluted other shards. So the idea here is that you actually roll back state of all shards. So just as an example, right, if, for example, something got corrupted here in this in this chart to the fishermen would send a fraud proof here, everybody would validate it. And now they need to for this block kind of almost roll back to the state that was before across all of this. Yeah. Yeah. So. So this is obviously a problem and a challenge that comes with that is that we need to make sure that also works well with consensus because Dynastrack has its own rules on how approvals should work. And for example, you cannot skip over an endorsement. But then if you actually roll back the state, if you completely disregard some part of the blockchain, then you actually run into this problem that because some approvals actually already existed, has been created. If you just disregard and rebuild from, let's say, this block, disregard this part and build on top of this block, then it will actually violate the rules that Doomsdrock has because those blocks already have their approvals. So the idea is that you don't actually get rid of all the blocks that come after the last valid one. What you do is that you still keep them in the history, but then, you're just building new blocks which the state root set to the state root as of before the invariant state transition so that it doesn't actually mess up the consensus. That's actually not the end of the complexity here. There's actually many more. For example, what is the challenge window? This is the part that gets very tricky because you want the window to be long enough so that it's difficult for people to let's say DDoS the network to prevent some valid challenge from being submitted into the network. But you also don't want to wait for too long because there's still some notion of finality that people wants. So this is similar to the or like almost exactly the same as how optimistic roll up works. But as we all know today that people don't like the fact that there is a someday like window where they need to wait before they can do like withdraw and. And here, definitely, people don't want to wait for seven days before they confirm their transactions are valid. And another part is that we also need to handle invalid challenges, because we cannot assume every fraud proof that's being submitted to the chain is actually valid. Then the problem is that we need to figure out how to slash people if they actually submit invalid fraud proofs. And... And then all of that makes things very complex. And I think the, I mean, ultimately the thing that makes us really reconsider whether this is the best thing to implement in practice is that even if we get all of those implemented, I mean, although they are complex, we can implement them. There is no really good way to test this thing. So basically this is an extremely complex mechanism that's or like the most complex part of the protocol, but there's almost never expected to get triggered in practice. That means like while you can test in like all the testing environment, it's like you still cannot be fully confident that it will work well in practice. And the problem with that is that if it doesn't work well, someone's going to lose money. And that's extremely dangerous situation to be in. And that's why we... thought about this whole thing again and decided to reconsider whether this is the approach to take. Yeah, so I think a lot of it is also similar to after the growlup. There's so many different economic attacks that actually opened this up where now you need to be very clear what your kind of different staking amounts and kind of money at stake depending on how much money are being transacted on the network. And that also... means you need to add a lot more kind of economics -based kind of re -rating on staking and potentially have higher requirements so that if something slashes. But at the same time, we haven't seen actually improved stake. We haven't seen anyone being actually explicitly malicious. So far, everybody just had bugs in the code. And given this is complex, actually, the bugs in the code here would be... Yeah, it would be... And that's not even necessarily like a... bugs in the code in terms of a deterministic sense. It's very problematic if there is, let's say, a non -determinism in the wrong time. It could even be related to hardware. So let's say you have that caused the state transition to diverge in the sense, let's say, 20 % of nodes does something, and the other 80 % does the other kind of state transition. That's completely screwed, because now the 80 % will try to slash the 20 % saying that they are malicious. And then the 20 % will think that, The other 80 % is malicious and then try to slash each other. Yeah, for context, those kind of things can be different in implementation on different clients that just have one of them doing fully implement the spec, right? And then everybody gets slashed. So because of this, you guys developed a new approach. So let's introduce it. And while including, yeah, so one of the kind of, I think, holy grails that been talked a lot about is stainless validation because there is kind of a number of things that it solves. But the biggest actually challenge around sharding as well is, well, around any blockchain is state growth. And this is challenge of as you get more transactions, as you get more. kind of smart contracts as you get more users, your state is growing. And in non -sharded blockchain, this is extremely challenging because you actually, every single computer in the network needs to validate all of the transactions, maintain all of the state to be able to access, right? And so Ethereum has this kind of buttons and topics around state growth, so that their state keeps growing. I mean, even Solana now has, you know, working around like to do. For example, compression, which is just a layer two on top to kind of move out the state out of the state space. And sharding actually addresses that because now you can keep adding shards to continue pretty much maintaining somewhat of state per shard. But the challenge is when new validators joining or validators get reassigned between shards, actually need to download the state of the shard. And so they need to catch up and sync into this. Now all of that is implemented right now in Nightshade and it's working, but it is a pretty kind of expensive procedure to do constantly. And that's why Validate is only rotate kind of once 12 hours and they need to, or like 12 to 14 hours. And then they need to catch up and kind of re -sync into the new shard they assigned. Yeah, so normally like in the sharded blockchain, like in a specific shard, everyone validates the state transition or validates a chunk by actually having the state locally and actually execute the chunk to see that the post state root actually agrees with what the chunk proposer proposes and then moves forward. But yeah, the problem is you actually need to have the state locally and that's potentially expensive and also... in a way, limits how many auditors you can have in the system. And the statorless validation idea is that, well, so basically there's two roles in the system. There's block producer. And then there's validator. So for simplicity, we can collapse block and chunk production into one row. So basically, they produce blocks and chunks. And in addition to that, this is a key difference, is that they also produce state witness. OK, so what is state witness? So we were talking about fraud proof before. We were saying that you need this partial state, and the proof that it actually belongs to the correct state is dictated by the state root. And precisely what a state witness is is basically the state that is touched during the execution of a chunk alongside with the proof that it actually belongs to the correct state as specified by some state root. So. Why is this useful? Well, then because other validators who don't actually maintain any state of the shard locally can actually execute the chunk with state witness. and then they can statelessly verify that this state transition in the chunk is actually valid. And the advantage here is that then the validators can actually rotate every single block into a different shard because they don't have to download states. So there's like zero cost of them rotating other than maybe need to establish some new network connection or downloading some like state witness from other people. Maybe just for example, so you want send, A send B 10 coins, that's a transaction. And so what you do, you include that A has 100, B has 10 right now as your state witness, which includes a proof to the state root. that this is indeed the previous state. And so then given this transaction and this state witness, you can receive that the new state is A90, B20, and that the new stage root is updated this way. So stage root plug T plus 1. So that's how you verify that state transition of a shard is actually valid. Now, like we said, validators can actually just rotate on every single block to another shard. This way, it is essentially, and also the shuffling is done through the randomness beacon that we have on chain that's unpredictable. So adversary cannot know a priori how the - not did their assignment to charge what was going to work. And because it's changing actually on every single block and also unpredictable because of the randomness beacon, then essentially the idea is that adaptive corruption actually won't work because you cannot even know two blocks in advance which validator is going to assign to which chart. And that's one of the core idea behind serious validation. So the stimulus validation has actually many, many different benefits. We'll talk about how it addresses the concern we had with snipe shade and fraud proofs, and also talk about the decentralization, let's say. For example, the validator needs to now can be more lightweight because it doesn't need to hold the state locally and only needs to kind of execute the chunk with the standard witness that it actually receives. So potentially you can have a lot more validators. In the system, we have more shards. Another thing about decentralization is that even though those block producers, they actually are heavier compared to the auditors. They actually not only produce the chunks, but also execute them and produce the witness. They themselves can't do much than what they can already do today. It's not because you're running heavier, more expensive hardware, they somehow have more control of the system. actually the control is the same as in the nightshade system we have today. Basically, it can choose what transactions to include and what not to include, but it cannot provide state witness because it will get actually verified first that it is indeed belongs to the correct state route. And because the state route is actually agreed on correctly by the validators, there is limited amount of things that broad producer can do even if it wants to act maliciously. And another great benefit of this design that plays nicely with sharding is that because now we have sharding, we can actually limit the size of each shard to some fixed number, let's say, like 50 gigabytes, right? So state size. Then what can happen is that those block producers, because they actually run more expensive hardware, they can actually hold the entire state in memory. So let's say you want to hold 50 gigabytes of state in memory, then potentially 64 gigabytes of RAM machine should actually be sufficient. And that's actually not even that expensive. And because validators actually execute the chunks using still witness, that's also in memory when the chunk's being executed. So now all the state access, including region rights, are down in memory. And that's going to be a huge improvement over the current design where I can actually go to disk. And that's a huge bottleneck. Yeah, I think maybe it's worth mentioning that any blockchain that uses Merkle -Patricia trees, for example, for storing state, ends up having a as a major bottleneck to pretty much throughput. Because you, like to actually access and change something, you need to have kind of a logarithm of reads from disk to actually find, I mean, either find or kind of find what you need to update in the state. And, you know, on top of it, you add the complexity of the data underlying database, which is another logarithm. And so, you know, if you have, kind of a billion records, this ends up being a few hundred disk reads per kind of one read from a specific item in the state. And so that's a pretty heavy lift, even with SSDs kind of flash memory. And so transition of full state of memory indeed speeds this up by... more than that factor because you don't need to rate the disk. And it allows to have this node. And this is only possible because through sharding, we can actually limit the size of each individual shard and able to grow the number of shards as well over time as more demand, as more applications are adding. So maybe, yeah, we're talking, I mean, you mentioned it, right, that validators is kind of a huge benefit. They now just need to receive a block, validate, right, and submit it. And in theory, they can actually join and leave even faster than ebook time. This is not a current implementation, but it's possible to have shorter lived validators because they have such a kind of transient job. At the same time, this means you can have so many of them and potentially even, you know, I mean, we discussed this like having in the browser, right? You open a browser tab, it kind of connects to the network, starts receiving blocks, signing them, and then you close it, it kind of disconnects. I mean, for that to happen, I mean, this is still like somewhat like expensive to do because you're still actually wrong the execution. But I always think like this architecture is more future facing in the sense that... or today you actually validate the state transition using execution against the witness. But let's say the zero -knowledge technology matures in like two to three years, where we can generate ZK proof pretty fast. And then, indeed, the writer only needs to validate one zero -knowledge proof and don't actually even need to do any execution. Now, it's not only that. The verification is extremely fast, but also that the data overhead is very low. You just receive like one ZK proof, depending on the scheme, a few hundred bytes or a few kilobytes. So that's way lower overhead than what we do today. Yeah, so this is if block producers are able to produce ZK proof pretty much in the block time. For example, then they can just send that instead of sending state witness and transactions. And so then all of this becomes even more lightweight, which is amazing as we've seen the K technology to continue speed up kind of with every year. Okay. So this is the core technology, I mean, the blockchain, the sharding, and the ability to continue scaling the transaction throughput. The other interesting part has been how do we make all of this easier to use and really enable experience? And while Bowen includes that, I think. One of the big improvements is actually the whole idea of the sharding being kind of packaged as a single blockchain. So if you go to Near, it's actually really hard to find shards. You need to dig pretty deeply because the idea was to actually abstract that out from the users and developers. And so we did have a few reshardings recently, which kind of brought it up more that it's happening. But generally speaking, you need to actually go to like... block level RPC to actually explore that. And that is part of the vision that as users you ideally should not think about the blockchain and especially should not think about how it's actually partitioned and scaled underneath because this allows a lot more flexibility to change some of this kind of design decisions without affecting users and developers at all. Now, A lot of things we've done on VM account level as well was in that focus, like how do we make it easier for developers to build this? Actually, I remember one thing we didn't talk about is cross -shard transactions. So let's maybe start with that. So maybe we can start with a simple example where let's say, perhaps, Alice .Near on shard 0 and Bob .Near on shard 1. And then they want to say the transaction. It's actually... I was just sending Bob some near tokens, one near token. So this is a very simple transaction, but the kind of the lifetime of the transaction from the kind of execution point of view is that the transaction first gets sent to shard 0. The validers of shard 0 executed there. And here it actually, in this step, it deducts. One year from now. And it has a receipt that carries this transfer of one near into Shard1. Obviously, we use Bob as the receiver. And on Shard1, once it's actually getting executed here, then basically, they can't get one more near. Now, that's kind of from the runtime point of view. But the complexity comes from the fact that, oh, it comes from the problem of like, how each shard knows about the receipts that's coming from another shard. So this goes back into the chain. So in every block, there is this chunk header. Because all the value is still tracking the chain itself, even though it may not track the actual transaction and the state of other shards, so it knows. what the chunk header from every other shard is. So specifically, so let's see this chunk header. It has this outgoing receipt route. that which is more colonization of all the outgoing receipt route from shard zero to other shards. And here, when actually needs to execute the receipt from shard one, it can receive the receipts from like some request receipt from people or like valid from shard zero and then actually verified against the outgoing receipt route that's actually agreed upon through consensus. And then it knows what are the receipts are going. into SHARP1 and actually can know whether it has received all the receipts. And then also make sure that the receipts that they actually receive is actually the right receipt from SHARP0. Yeah, so maybe from more, so there's like few levels of abstraction that happening here. From the developer perspective or user perspective, Alice, and Bob .Near, they actually kind of exist almost like on separate chains. And that's one of the interesting ways to think about it. Near kind of gives every account a chain and then just allows them to really effectively communicate with each other. And so as Alice wants to send a message, it gets sent to Bob in the next block time. usually unless there is any congestion in the network. And so and then Bob can you know send a message back or send a message to somebody else in this and so kind of what this allows to do interestingly is that we actually can allocate one account on near a full capacity of a blockchain right so you know you you know you have app chains or you have kind of roll -ups. And so you can imagine that any account on NEAR can have capacity up to a roll -up, up to a upchain, up to whatever maximum one shard can handle. And so we actually have, for a while, been running specifically Aurora contract, which is an EVM as a smart contract on NEAR, as a separate shard, pretty much being able to process its own transactions and not being affected by other shard throughput. And so that's a really kind of powerful tool because, as I said, from a user and developer perspective, you are not really kind of need to think about this. You just deploy a contract and it works. And then the network itself, the governance of it, can decide to group those things into one shard or separate shards depending on the load of the network. Now, right now, this is more of a kind of protocol team and validators deciding and executing this. But over time, this can become a lot more kind of programmatic based on the load. And we call it dynamic resharding. And so again, the kind of core idea that as the network scales, you want to continue changing where different contracts living and how they grouped based on what's the most optimal for using resources and kind of allowing everyone to have. enough throughput for their smart contracts and accounts. Now on the chain side, as you mentioned, there's a chunk header, but the way to think about it, each chunk, when they send parts with data availability, they also attach the outgoing receipts. And so as part, again, of consensus data availability, we also do message passing. And so in a way, with We tried to weave in all of the pieces of communication that shards need to have, and in a way, chains need to have into one system that is very much always passing information forward. And what you're trying to avoid is, as much as possible, requesting information from other chains and from other validators, because that always adds an extra lag, delay, wait, retry, try another validator. So that's kind of the core idea. From a user side, you have a single chain with accounts and you're kind of interacting with them. From a developer side, you have almost like your own blockchain where you can really easily communicate with other chains. And you don't need to have individual user accounts managed separately. And then from a kind of consensus side, it's all weaved in into one system. to ensure delivery, correctness, and low latency of communication. All right, so yeah, let's talk about VM. Yeah, so one of the major choices we made was to really go with the WebAssembly. And so for those who are not familiar, WebAssembly is a virtual machine developed by Microsoft, Google, Mozilla, and other companies. It is used in every browser. And it also runs on edge, on edge devices. It's a very versatile, generic virtual machine that has has a lot of non -web3 kind of push behind it. And the benefit is there's a number of compilers that already exist, and specifically Rust, C, and a few others who are kind of highly supported, allowing to build kind of performant and with Rust usually more correct programs. And being kind of a generic framework, it allows to have other... somewhat complex or more complex applications actually being able to run really easily. So I mentioned Aurora, which is an EVM. And so this is literally an EVM written for one of the EVM clients that was kind of pretty much imported as a smart contract compiled and run into WebAssembly. So that's kind of the power you get by using such a generic kind of virtual machine. Now, on top of this, we build kind of a different account model. to really simplify the access for users as well as create a versatility in places for creating more secure usage. So one of the, again, major decisions was instead of accounts being named because of the hash of the public key, it access it, it's actually named based on a string that follows somewhat of a domain name. rules, right? So there's a near account which can create everything .near. And there's other top level accounts like Aurora and now TG, which creates .tg accounts. And so this Alice .near account has kind of information stored, which is how much near it has, how much it staked or locked. and then it has a contract hash. There's also storage information and then there's additional information that we call access keys. And access keys is how you know which key can have access to this account and with what permissions. And so what this allows to do is you can create a house with the account. And now you can have, for example, we call them full access keys that are, for example, accessed from your phone, from your laptop, maybe from a bank. if you have custody. But also you can have access, we call them functional call keys. And so they allow, for example, a specific application to be able to submit transactions on your behalf from this frontend. So what you can do, you can then go and log in into this app and say, when you're approving adding this key, that it only can, for example, call one function on behalf of the account. And so then this front end, this application, can now send transactions of that sort without you needing to go to confirm them in the wallet every time. Then you can go and remove this key and manage these permissions. So this is very similar to using Twitter or Facebook or anything else. You have approved applications and devices, and you can pretty much manage that. There's also kind of amount of gas attached here to pay for the... The important thing is that the function call access key does not allow the owner to transfer money from that key. So it protects the user that they don't have to worry about that part. Exactly. And so it tells me the other thing is because you have a contract hash on the account... So any account, you can create a regular account and then turn it into a smart contract account, or turn it into a smart contract, or turn it back into your own account. And so the idea is you can deploy at any time a specific contract on this that has specific, I mean, this can be anything from just a regular whatever, DAX to a specialized contract for your account. For example, you can deploy a multi -sig. And now, all of this, you know, your removal of full access keys and you just have function call keys that can call your own account to approve some transactions. And so this is very powerful because now you can turn existing account into a multisig, right? So imagine you have a lot of, you know, you start a new account, it was empty, it didn't matter. And then you received, for example, locked tokens, or you have an NFT revenue stream. that is now targeted at your account, but you cannot change it anymore because that's baked into an NFT contract. So now you can turn your account into a multisig and have like some even external custodians, 2FA, whatever you want as a way to manage this. There's also a really cool other use case that keep on pioneering, which is called trial account. This is when somebody else creates account, funds it with money. and gives you a function called key to do something, but you only can limit, like you only limit it with what you can do. You cannot withdraw the money from this account. And so if you want to get a full access key, you need to repay back to whoever gave you this trial account. So it's a very flexible system that allows a lot of different use cases. And this, you know, we see kind of more and more folks experimenting with what they can do, including you can have one account. So for example, Near has a contract on it which allows to pretty much add a new key to this new function called Key to this account, which then if somebody has a private key for this public key, they can call on behalf of the account to create a new account. What this allows to do is I can now create a link that I give somebody else. And when they click on it, it creates them a new account and deposits some Near on it. So they can pretty much create new accounts. from a link non -custodial through this approach, right? So, there's ways to expand, kind of have a share, almost like a shared account, where different keys have different permissions for different users who hold the private key for that. Sidekings. And then, one of the use cases we've originally not envisioned that people will use through this contract, but it is, someone complex is actually paying on behalf of this user, right? Let's say Alice came in and she only has zero balance. How do we actually enable that? And the kind of one of the things that Bowen team introduced early 2023 has been to have a specialized meta transaction call. It allows to send a transaction on behalf of Alice. So Alice needs to sign with one of the keys that she has. And then this is sent, so send transaction, but then this is sent to Relayer. Relayer actually submits a transaction, pays the fee, and then it goes to Alice's account. and gets executed from there. So it actually wraps this transaction in another transaction. So when it signs it creates a new transaction that has the signed transaction as the payload. Yeah, so here there's a kind of transaction signed by Relair, which contains transaction signed, like a delegated action signed by Alice. And so Relair pays the fee to the network. and then this delegated action gets executed off Atlas's account. And so this allows to now do a lot of things. For example, a lot of the biggest applications on NEAR are using this approach now to send transactions on behalf of users, everything from free transfers for stable coins, being able to interact with games, being able to claim tokens and everything else. And so it's a very powerful primitive. that kind of works pretty well with this whole architecture and the access key system. Now, everything you see is like we're trying to kind of abstract and make it easy to use accounts and make it easy to use the blockchain, right? Again, on NEAR itself, we're trying to remove the need to think about shards. And with this, we're trying to remove the need to think about kind of transaction fees, at least initially. And so one of the things that we're introducing, the march is, or introduced in the march, is in 2024, is the idea of chain signatures. And so chain signatures is kind of expanding on all of this. But now you're able to pretty much call a method sign to kind of specific subset of validators who. and you want to tell them a payload. and so -called derivation. and what they will return you back actually a signed payload. signature, like a signature for this payload, but this signature is not from your key. This signature is actually now based on a new key that a set of validators now maintain that is owned by them. And we are setting them up such that it's subset of near validators as well as some eigen layer. kind of risk taker, we see you. Yeah, so the idea is that none of the validators here actually hold the full key for users, so none of them collectively, individually can actually do anything malicious, and there is actually a threshold upon which they need that many validators to actually sign to actually create the final signature. And this is essentially a threshold, a signature scheme that actually fixes the signature. but allow the validator to rotate. Yeah, so for example, there's specific popkey that all of this validators have. And then you imagine maybe one of them left, leaves, goes offline. You can still get the same popkey. You can still get signatures for the same popkey by subset. And then, so this was like v4, v4 left, and for example, v6 joint. And they pretty much constantly reshare. parts of the key in such a way that none of them contain, none of them have the full key, but they're all still able to sign if they come together for a deterministic public key. Now, there is an additional derivation used and your account, right? So it's Alice, for example, requested the signature that actually determines the exact key. And so the way to think about it, you have a near account. You have Alice .Near, which now has associated addresses on Ethereum and multiple of them, depending on derivation. It has a Bitcoin address. It has Solana address, I don't know, Taser address, et cetera. So you have addresses on every single chain that you're able to sign pretty much transactions. So this really allows to now have a new account that is able to transact across all the other chains where the custody of this is shared between the validators and your account. And this is very powerful because, remember, your accounts can be contracts, which means a contract can now transact on other chains. Because of the derivations, you can actually have a contract which has an independent addresses where you can deposit, I don't know, ordinals, which this contract can issue near NFTs for. You can have internal mapping that, oh, this address is owned by Illia, this address is owned by Bowen, let's swap this ownership. So you can program that. You can obviously have direct usage as if you go with your near account. You log in into a Magic Eden. You transact there. And depending if it's like smart contract on Solana or on Ethereum, it actually executes through the right signature. And you can receive assets on either chains. So you're allowed to really kind of lift up the experience from, OK, I need to choose a chain. I need to bridge there. I need to figure out what's a gas dock in there to kind of really start abstracting this out. And there's like a few other pieces like gas layers to enable that. Yep. So one chain to rule them all. And you can have all your transactions handled through like a smart contract on there. Yeah. So welcome for any developers interested in chain signatures. Go out, check out documentation. For learning more about the protocol, there's a Nomicon .io. which has a lot of the spec, as well as invite everyone to github .com, near .com, and EPS, near enhancement proposals. And this is where a lot of the discussions about the protocol are happening and the design updates for the specification are. Yeah, thanks everyone. All right, thank you.","transcript_synced_at":"2026-11-03T06:00:55.346Z","segments":[{"start_seconds":3,"text":"Hello everyone, I'm Illia co -founder of NEAR and with me today, Bowen, who leads NEAR's protocol team and work group.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3s"},{"start_seconds":12,"text":"And we have a very special episode because today compared to other episodes, we're actually going to talk about NEAR.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=12s"},{"start_seconds":18,"text":"So we wanted to actually go and dive in from the bottom up through all of the NEAR technology and...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=18s"},{"start_seconds":28,"text":"kind of explain piece by piece, as well as some of the changes that happened since a lot of the original papers introduced in 2019.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=28s"},{"start_seconds":38,"text":"So Bowen, let's dig in. I think, yeah, let's start with consensus. Yeah, sure.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=38s"},{"start_seconds":46,"text":"Yeah, I think this is part where it was not explained a lot in previous videos or other material about NEAR But today we use a...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=46s"},{"start_seconds":58,"text":"hybrid consensus called Doomslug So in the URA consensus, there are two types of them.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=58s"},{"start_seconds":66,"text":"One is BFT consensus, where everyone, there is a majority of the people need to actually reach agreement on what the block is.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=66s"},{"start_seconds":73,"text":"And basically what is produced is considered finalized.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=73s"},{"start_seconds":76,"text":"And then there's other type of consensus where there is the requirement for producing blocks is more relaxed, but then you can have forks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=76s"},{"start_seconds":86,"text":"And the blockchain somehow needs something on top of that. to finalize blocks that have been produced.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=86s"},{"start_seconds":93,"text":"So this is the case with Ethereum, with Polkadot, where there is this block","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=93s"},{"start_seconds":99,"text":"production logic with folk choice rule, and then there is a finality gadget on top of that that actually finalizes blocks. And near uses this approach as well.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=99s"},{"start_seconds":108,"text":"So the advantage here is that you don't actually have to wait a lot.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=108s"},{"start_seconds":113,"text":"So you're really in BFT consensus, at least there's two rounds of communication, and it adds a lot of latency.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=113s"},{"start_seconds":118,"text":"And here, by not actually requiring BFT consensus at the time of block production, we can actually pipeline block production with processing of the blocks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=118s"},{"start_seconds":130,"text":"And as they get processed and more blocks get built on top, the previous ones actually get finalized in this way, like maximizing this throughput.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=130s"},{"start_seconds":139,"text":"So how this roughly works is that after this block is produced, Then it gets sent to obviously all the validators.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=139s"},{"start_seconds":151,"text":"And then they receive the process block validator that is indeed valid. And then they would start sending approvals to the next block producer.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=151s"},{"start_seconds":163,"text":"And then if they have sufficient approvals, they would produce the next block and so on.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=163s"},{"start_seconds":171,"text":"So I'll just write what? I'll just write push two. So there are actually two type of approvals.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=171s"},{"start_seconds":182,"text":"So it has endorsements. and skip.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=182s"},{"start_seconds":192,"text":"So this happens, endorsement happens when there's no block height being skipped.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=192s"},{"start_seconds":199,"text":"Basically, you process a previous block and you're sending approval for the next block. And then skip happens when some block is missed.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=199s"},{"start_seconds":208,"text":"Let's say the block producer is offline or there's a network delay, the block is skipped.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=208s"},{"start_seconds":215,"text":"Then essentially, we're saying it's like, skip basically saying like the previous block height is something and the next order is skipping to what is next height.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=215s"},{"start_seconds":230,"text":"And roughly the idea is that when you have this three blocks in a row where there's like a block h block, h plus one, block h plus two, then it actually finalizes h.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=230s"},{"start_seconds":243,"text":"And there is obviously the consensus needs to have like life provable, uh, safety and lifeness.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=243s"},{"start_seconds":251,"text":"And, um, uh, you know, we can, it, the, the proof is actually much more involved, but basically roughly the idea is that, um, uh, after a sufficient time past, uh,","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=251s"},{"start_seconds":261,"text":"there will be a case where those, uh, three consecutive blocks will get produced and then the first one will get finalized. Yeah.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=261s"},{"start_seconds":268,"text":"So maybe when we're saying, so when the approvals are being sent to block, uh, T plus two, they in a way are finalizing T.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=268s"},{"start_seconds":280,"text":"Because in a simplistic term, there's 2F plus 1 approvals here, 2F plus 1 approvals here, and so 2 thirds plus 1 saw 2 thirds plus 1 saw some block.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=280s"},{"start_seconds":292,"text":"So this is kind of the, in a way, like an on -chain version of some BFT consensus, kind of like tandem maintain.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=292s"},{"start_seconds":301,"text":"The approvals are part of the block, so it's actually like, which block is being finalized is calculated on -chain by looking at the other blocks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=301s"},{"start_seconds":312,"text":"So the benefit is, compared to traditional BFT, a block is produced, it's sent out to everyone, validators then send approvals back to the same validator.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=312s"},{"start_seconds":325,"text":"This validator aggregates all of these approvals and then sends out a block to everybody else.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=325s"},{"start_seconds":330,"text":"And so this usually actually requires more steps, especially because he need to... account that not everybody saw it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=330s"},{"start_seconds":338,"text":"And so this usually requires multiple steps of communication and takes two, three or longer seconds.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=338s"},{"start_seconds":345,"text":"And this approach allows to pipeline in kind of the block speed and improve block speed. Yeah.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=345s"},{"start_seconds":354,"text":"Yeah, so basically, as these blocks are produced, while B is still waiting to be finalized, people already apply the blocks and apply the next blocks and so on.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=354s"},{"start_seconds":365,"text":"And the Demosrack also has this property that as long as the next block is produced, there's a concept called Demosrack finality, meaning that if both B","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=365s"},{"start_seconds":374,"text":"and B plus 1 is produced, then at least there needs to be some malicious behavior in order for B to not get finalized.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=374s"},{"start_seconds":383,"text":"Yeah, because if this produces two F plus one signatures, then if there is a fork indeed, then somebody has signed maliciously and so you can slash them.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=383s"},{"start_seconds":398,"text":"OK. Yeah, so I mean, the big benefits been improving the kind of overall latency of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=398s"},{"start_seconds":404,"text":"the system, as well as one of the kind of not usually talked about in blockchain","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=404s"},{"start_seconds":410,"text":"parts is that although there is a consensus layer, there's actually a lot of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=410s"},{"start_seconds":416,"text":"other nodes in the system, like exchanges, indexers that actually provide run -ins, RPC nodes. And.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=416s"},{"start_seconds":424,"text":"they have the same latency of whatever consensus is plus either optimistic or final, like if they're waiting for optimistic or final block.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=424s"},{"start_seconds":436,"text":"And so by speeding this up, at this point, this is already sent to the indexer who can actually process it and prepare this kind of the output.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=436s"},{"start_seconds":445,"text":"And so if you are hitting your RPC, you can actually have a optimistic.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=445s"},{"start_seconds":452,"text":"pretty much results, which in many cases is enough to start to visualize it in the front end.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=452s"},{"start_seconds":462,"text":"And then if there's a rework or something happens, especially after one block, you already have enough guarantees for pretty sizable amounts of value moved.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=462s"},{"start_seconds":480,"text":"All right. Well, let's go to the meaty part.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=480s"},{"start_seconds":486,"text":"So, NEAR has started as a chartered blockchain and has been kind of introduced in Nightshade design back in 2019.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=486s"},{"start_seconds":494,"text":"And so I think it's worth both diving into Nightshade, but also Bowen kind of led the","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=494s"},{"start_seconds":501,"text":"work last year to introduce kind of Nightshade v2, which introduced the concept of stateless validation.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=501s"},{"start_seconds":509,"text":"I think... Let's dive into that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=509s"},{"start_seconds":518,"text":"So I think one of the core idea of Nightshade is that in previous designs of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=518s"},{"start_seconds":525,"text":"sharded blockchains, usually there is some kind of main chain or people call it like relay chain or like the hub or like whatever names people use for it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=525s"},{"start_seconds":536,"text":"It's basically like this chain that people use to orchestrate validators.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=536s"},{"start_seconds":540,"text":"People you can run like staking and then validator gets selected, assigned to shards and so on and so forth.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=540s"},{"start_seconds":546,"text":"And then separately for each shard, the names can be different, but conceptually there will be like a shard chain, meaning that for each shard it actually builds its own blockchain.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=546s"},{"start_seconds":557,"text":"So this is fine, but one of the problems is mainly that it gets very complex.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=557s"},{"start_seconds":567,"text":"So let's say you have a cross shard transaction. Now the problem is on both shard chains you can have forks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=567s"},{"start_seconds":576,"text":"And say forks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=576s"},{"start_seconds":581,"text":"And depending on how your transaction land, it could be that on the source","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=581s"},{"start_seconds":587,"text":"shard, the transaction is executed on the canonical chain, meaning that the fork that becomes eventually final.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=587s"},{"start_seconds":594,"text":"But then when you send it to the destination shard to execute, it could be that on that part it landed in the non -canon.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=594s"},{"start_seconds":603,"text":"canonical fork, meaning that later there's a reorg that actually went to the other fork. That's a big problem because you have this atomicity failure.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=603s"},{"start_seconds":613,"text":"Let's say you're sending some tokens, then it will get debited on the sender account, meaning that they actually get the token deducted.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=613s"},{"start_seconds":623,"text":"And then on the receiver end, if there is this reorg, then suddenly they didn't actually get the token. So that's the problem.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=623s"},{"start_seconds":630,"text":"And then there are ways to solve this.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=630s"},{"start_seconds":633,"text":"But then it gets complex, you need to do this cross linking to like beacon chain and all the nones.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=633s"},{"start_seconds":638,"text":"Yeah, I mean, usually the approach is you add another chain, which is a whatever beacon chain, which now you communicate to it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=638s"},{"start_seconds":646,"text":"And then from there you communicate to like it finalizes blocks and then you can communicate kind of needs to be included.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=646s"},{"start_seconds":655,"text":"And so when this gets finalized the next block, you guarantee inclusion or some kind of processing.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=655s"},{"start_seconds":662,"text":"and ordering and everything.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=662s"},{"start_seconds":664,"text":"And that creates a massive bottleneck because now this needs to actually receive all the messages from everywhere and be able to like route them.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=664s"},{"start_seconds":675,"text":"And yeah, kind of creates additional latency for the cross -short communication because now you need to have block time here, block time here, and potentially one more block time here, if not more, because you still need to wait for finalization.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=675s"},{"start_seconds":688,"text":"So it can be taking like four block times. at least if not longer. Yeah, I think that's one thing.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=688s"},{"start_seconds":696,"text":"Another thing is that from an implementation point of view, this is actually way more complex.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=696s"},{"start_seconds":701,"text":"You need to handle a lot more kernel cases than if you just only have one chain that you need to deal with, like forks and switch between forks and so on.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=701s"},{"start_seconds":710,"text":"So let's not do that. Yeah, so that's where the nice idea came in.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=710s"},{"start_seconds":719,"text":"So the idea is that instead of having like, having shard chains, we just have like one chain to rule them all.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=719s"},{"start_seconds":728,"text":"And so inside this chain, it has blocks that... Sorry.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=728s"},{"start_seconds":735,"text":"So it has a chunk for each shard. So chunk is basically a shard block.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=735s"},{"start_seconds":745,"text":"I'm sorry. So...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=745s"},{"start_seconds":752,"text":"Kind of like implicitly, you can think of there's still like this short chain idea where if like you actually having the next chunk point to the previous chunk and so on.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=752s"},{"start_seconds":762,"text":"But on the level of like how they're being orchestrated, they're being orchestrated as part of the block. And there is only one blockchain that contains the chunks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=762s"},{"start_seconds":772,"text":"So the blocks don't actually contain the physical chunks and the way...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=772s"},{"start_seconds":779,"text":"is handled is that it actually only contains the chunk header.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=779s"},{"start_seconds":783,"text":"And this is to make sure that blocks can be propagated very fast, but still","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=783s"},{"start_seconds":789,"text":"provides enough metadata about each shard chunks so that people can request the corresponding chunks if they need to.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=789s"},{"start_seconds":800,"text":"So obviously not every node in the network cares about every shard, but for the shard they do care about it. And once you see the header, they can...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=800s"},{"start_seconds":807,"text":"verify it's rigid and then can actually request what the chunk is according.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=807s"},{"start_seconds":814,"text":"So yeah, that's kind of the, I guess the core idea of, of Nation.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=814s"},{"start_seconds":821,"text":"And then I think another very important aspect is always read the data variability, how we call it guarantee this in a shard network.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=821s"},{"start_seconds":830,"text":"This unlike a non -sharded blockchain.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=830s"},{"start_seconds":834,"text":"The one problem is that we need to handle inherently here is that from every shard from the view, the other shard it's running.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=834s"},{"start_seconds":844,"text":"kind of light crime for other shards. So it needs to be convinced that the other shards data actually are available.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=844s"},{"start_seconds":853,"text":"Otherwise, there could be attacks of withholding data and then actually generate receives and create this double spending attack where they do the reorg and then the data on one of the forks is actually not available.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=853s"},{"start_seconds":867,"text":"So the way we do this is that for every chunk we produce,","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=867s"},{"start_seconds":872,"text":"it actually first get erasure coded into pieces and they get distributed to all the validators in the system.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=872s"},{"start_seconds":879,"text":"And there's a system parameter on like how many validators can collectively reconstruct to collectively reconstruct the chunk using the shares you get.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=879s"},{"start_seconds":891,"text":"So I think the in production threshold is set to one third, meaning that as long as a third of the other is honest, we can actually reconstruct the chunk for every single.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=891s"},{"start_seconds":902,"text":"Yeah, so maybe to kind of from a specific chunk perspective, or as it was put for","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=902s"},{"start_seconds":909,"text":"data -available sharding, as it gets produced, it gets erasure -coded into parts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=909s"},{"start_seconds":918,"text":"So a number of parts, which as Bowen said, depends on the parameter.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=918s"},{"start_seconds":925,"text":"And then these parts are then sent to other. validators in other shards or like across the whole network.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=925s"},{"start_seconds":936,"text":"And so, so you have like we, you know, we want to go.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=936s"},{"start_seconds":941,"text":"And so ACE, the kind of the benefit of this, that the subset of this validators can reconstruct this chunk if needed.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=941s"},{"start_seconds":948,"text":"So even if, you know, all of the eventually validators in the chart go","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=948s"},{"start_seconds":953,"text":"offline or they're malicious, if this procedure happened, it's possible to reconstruct it from the.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=953s"},{"start_seconds":961,"text":"validators and other shards. So this provides a very beneficial guarantee because this validator is","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=961s"},{"start_seconds":969,"text":"actually not going to send approvals for the next block if they haven't seen the chunks from every other shard.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=969s"},{"start_seconds":976,"text":"So if they haven't received their part of a chunk.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=976s"},{"start_seconds":982,"text":"So we actually weave in the data availability directly into consensus by doing this.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=982s"},{"start_seconds":990,"text":"And I think one other optimization we did is that after we produce every chunk, it's actually not executed first.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=990s"},{"start_seconds":998,"text":"It's first sent out to all the validators, sorry, all the validators of the track list chart.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=998s"},{"start_seconds":1006,"text":"So that then basically when they receive it, they start executing. And then the whoever produces chunk also start executing the chunk.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1006s"},{"start_seconds":1015,"text":"This is to avoid this 2X latency you would introduce otherwise.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1015s"},{"start_seconds":1018,"text":"which is like you first execute the chunk and then you distribute to all the people out there and then they all execute it. So, yeah.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1018s"},{"start_seconds":1026,"text":"So this is again to compare this, for example, Tendermint where you order","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1026s"},{"start_seconds":1031,"text":"transactions first, then you execute, which for example, if you know the","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1031s"},{"start_seconds":1038,"text":"execution can take like another second, then you send for approvals, then you...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1038s"},{"start_seconds":1048,"text":"wait for two thirds of approvals. And so like, and you probably wait for another round generally.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1048s"},{"start_seconds":1059,"text":"And so that's kind of why, you know, you have extra latency on most of the block production.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1059s"},{"start_seconds":1068,"text":"And so this pretty much simplifies this by always kind of, you know, as soon as you have data, you send it out.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1068s"},{"start_seconds":1074,"text":"As soon as you have kind of executed and validated, uh, invariance, you approve it, and as soon as approval is collected, you kind of repeat that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1074s"},{"start_seconds":1084,"text":"And all of the kind of all the invariants have been calculated kind of retroactively.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1084s"},{"start_seconds":1093,"text":"All right, so maybe it's good to then cover the stateless validation. Yeah.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1093s"},{"start_seconds":1098,"text":"So in the original natural design, there's one very complex piece, which is how do we prevent entire shard from getting corrupted?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1098s"},{"start_seconds":1111,"text":"So even though there is this like, you know, every epoch, the viaduct shuffles and the assignment to shards is random, but the epoch is still relatively long and there is this...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1111s"},{"start_seconds":1122,"text":"assumption that it's possible for the shard to get adaptively corrupted after the assignment of either to shard is known to everyone who can just see the assignment of either to sharding the entire blockchain.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1122s"},{"start_seconds":1136,"text":"So the idea is that, well, if a shard is corrupted, meaning let's say it has a","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1136s"},{"start_seconds":1143,"text":"majority of over two thirds of malicious viadators, then it can potentially actually do a invalid state transition and let's say mean some.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1143s"},{"start_seconds":1151,"text":"a lot of tokens. And then the part that's very annoying to deal with here in this sharded blockchain","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1151s"},{"start_seconds":1158,"text":"setting is that they actually, let's say you do something to identify the malicious behavior, it's not sufficient to just like roll back the state of this shard because","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1158s"},{"start_seconds":1168,"text":"they actually can, after mean that a lot of tokens in an invalid state transition, they can actually distribute that to or send that to many different accounts on many different shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1168s"},{"start_seconds":1179,"text":"So that even though, all the other shards are still doing various state transitions.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1179s"},{"start_seconds":1184,"text":"And because just like one shard getting corrupted essentially can corrupt the entire system, like all the shards becoming valid in a way.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1184s"},{"start_seconds":1192,"text":"So that's the difficult thing to deal with. And in the original nightshade design, the proposal is to use fraud proofs.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1192s"},{"start_seconds":1202,"text":"So the idea is that there's this kind of people called fishermen who so","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1202s"},{"start_seconds":1209,"text":"responsibilities to watch the blockchain to make sure that they know in last stage transition actually happens.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1209s"},{"start_seconds":1217,"text":"And essentially the idea is that as long as there's one honest fisherman or","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1217s"},{"start_seconds":1222,"text":"validator in a shard, then if an in last stage transition happens, they can submit what is called a fraud proof.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1222s"},{"start_seconds":1233,"text":"So what this does is that it actually has the partial state.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1233s"},{"start_seconds":1240,"text":"of the state that's touched during the execution of the chunk that's questionable.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1240s"},{"start_seconds":1248,"text":"Obviously, it has the pref state root and obviously the post state root.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1248s"},{"start_seconds":1257,"text":"So basically saying that here is the state that's touched during the execution of this chunk and here is a pref state root.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1257s"},{"start_seconds":1269,"text":"anyone else in the system, any validators who's not tracking the shard, can do the same execution and arrive at some post state route by doing themselves.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1269s"},{"start_seconds":1281,"text":"And then they can see whether that state route is indeed invalid, meaning that whoever is doing the invalid state transition, they necessarily would have a different state route because they're not following the rules.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1281s"},{"start_seconds":1293,"text":"So then this can prove to everyone that their state transition is invalid. and then they get slashed accordingly.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1293s"},{"start_seconds":1302,"text":"So this is a fraud proof part. But another difficult part actually happens after the fraud proof is verified.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1302s"},{"start_seconds":1309,"text":"So let's say you actually verify the fraud proof.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1309s"},{"start_seconds":1315,"text":"Now, you're also in a difficult spot because, as I said before, it's not","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1315s"},{"start_seconds":1323,"text":"sufficient to just reset the state of a specific shard because it could have already polluted other shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1323s"},{"start_seconds":1329,"text":"So the idea here is that you actually roll back state of all shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1329s"},{"start_seconds":1338,"text":"So just as an example, right, if, for example, something got corrupted here in","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1338s"},{"start_seconds":1344,"text":"this in this chart to the fishermen would send a fraud proof here, everybody would validate it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1344s"},{"start_seconds":1351,"text":"And now they need to for this block kind of almost roll back to the state that was before across all of this. Yeah.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1351s"},{"start_seconds":1360,"text":"Yeah. So. So this is obviously a problem and a challenge that comes with that is that we","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1360s"},{"start_seconds":1366,"text":"need to make sure that also works well with consensus because Dynastrack has its own rules on how approvals should work.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1366s"},{"start_seconds":1375,"text":"And for example, you cannot skip over an endorsement.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1375s"},{"start_seconds":1378,"text":"But then if you actually roll back the state, if you completely disregard some","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1378s"},{"start_seconds":1384,"text":"part of the blockchain, then you actually run into this problem that because some approvals actually already existed, has been created.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1384s"},{"start_seconds":1390,"text":"If you just disregard and rebuild from, let's say, this block, disregard this part","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1390s"},{"start_seconds":1396,"text":"and build on top of this block, then it will actually violate the rules that Doomsdrock has because those blocks already have their approvals.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1396s"},{"start_seconds":1405,"text":"So the idea is that you don't actually get rid of all the blocks that come after the last valid one.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1405s"},{"start_seconds":1414,"text":"What you do is that you still keep them in the history, but then, you're just building new blocks which the state root set to the state root as of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1414s"},{"start_seconds":1423,"text":"before the invariant state transition so that it doesn't actually mess up the consensus.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1423s"},{"start_seconds":1430,"text":"That's actually not the end of the complexity here. There's actually many more. For example, what is the challenge window?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1430s"},{"start_seconds":1439,"text":"This is the part that gets very tricky because you want the window to be long enough so that it's","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1439s"},{"start_seconds":1447,"text":"difficult for people to let's say DDoS the network to prevent some valid challenge from being submitted into the network.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1447s"},{"start_seconds":1454,"text":"But you also don't want to wait for too long because there's still some notion of finality that people wants.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1454s"},{"start_seconds":1460,"text":"So this is similar to the or like almost exactly the same as how optimistic roll up works.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1460s"},{"start_seconds":1467,"text":"But as we all know today that people don't like the fact that there is a someday like window where they need to wait before they can do like withdraw and.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1467s"},{"start_seconds":1476,"text":"And here, definitely, people don't want to wait for seven days before they confirm their transactions are valid.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1476s"},{"start_seconds":1485,"text":"And another part is that we also need to handle invalid challenges, because we cannot assume every fraud proof that's being submitted to the chain is actually valid.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1485s"},{"start_seconds":1496,"text":"Then the problem is that we need to figure out how to slash people if they actually submit invalid fraud proofs.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1496s"},{"start_seconds":1504,"text":"And... And then all of that makes things very complex.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1504s"},{"start_seconds":1509,"text":"And I think the, I mean, ultimately the thing that makes us really reconsider","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1509s"},{"start_seconds":1515,"text":"whether this is the best thing to implement in practice is that even if we","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1515s"},{"start_seconds":1522,"text":"get all of those implemented, I mean, although they are complex, we can implement them. There is no really good way to test this thing.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1522s"},{"start_seconds":1530,"text":"So basically this is an extremely complex mechanism that's or like the most complex part of the protocol, but there's almost never expected to get triggered in practice.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1530s"},{"start_seconds":1540,"text":"That means like while you can test in like all the testing environment, it's like you still cannot be fully confident that it will work well in practice.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1540s"},{"start_seconds":1551,"text":"And the problem with that is that if it doesn't work well, someone's going to lose money. And that's extremely dangerous situation to be in.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1551s"},{"start_seconds":1560,"text":"And that's why we... thought about this whole thing again and decided to reconsider whether this is the approach to take.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1560s"},{"start_seconds":1570,"text":"Yeah, so I think a lot of it is also similar to after the growlup.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1570s"},{"start_seconds":1574,"text":"There's so many different economic attacks that actually opened this up where now you","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1574s"},{"start_seconds":1579,"text":"need to be very clear what your kind of different staking amounts and kind of money at stake depending on how much money are being transacted on the network.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1579s"},{"start_seconds":1590,"text":"And that also... means you need to add a lot more kind of economics -based kind of re -rating on staking and potentially have higher requirements so that if something slashes.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1590s"},{"start_seconds":1602,"text":"But at the same time, we haven't seen actually improved stake. We haven't seen anyone being actually explicitly malicious.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1602s"},{"start_seconds":1609,"text":"So far, everybody just had bugs in the code. And given this is complex, actually, the bugs in the code here would be... Yeah, it would be...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1609s"},{"start_seconds":1617,"text":"And that's not even necessarily like a... bugs in the code in terms of a deterministic sense.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1617s"},{"start_seconds":1622,"text":"It's very problematic if there is, let's say, a non -determinism in the wrong time. It could even be related to hardware.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1622s"},{"start_seconds":1630,"text":"So let's say you have that caused the state transition to diverge in the sense,","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1630s"},{"start_seconds":1635,"text":"let's say, 20 % of nodes does something, and the other 80 % does the other kind of state transition.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1635s"},{"start_seconds":1641,"text":"That's completely screwed, because now the 80 % will try to slash the 20 % saying that they are malicious.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1641s"},{"start_seconds":1646,"text":"And then the 20 % will think that, The other 80 % is malicious and then try to slash each other.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1646s"},{"start_seconds":1654,"text":"Yeah, for context, those kind of things can be different in implementation on","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1654s"},{"start_seconds":1658,"text":"different clients that just have one of them doing fully implement the spec, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1658s"},{"start_seconds":1667,"text":"And then everybody gets slashed. So because of this, you guys developed a new approach. So let's introduce it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1667s"},{"start_seconds":1675,"text":"And while including, yeah, so one of the kind of, I think, holy grails that been","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1675s"},{"start_seconds":1681,"text":"talked a lot about is stainless validation because there is kind of a number of things that it solves.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1681s"},{"start_seconds":1688,"text":"But the biggest actually challenge around sharding as well is, well, around any blockchain is state growth.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1688s"},{"start_seconds":1699,"text":"And this is challenge of as you get more transactions, as you get more. kind of smart contracts as you get more users, your state is growing.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1699s"},{"start_seconds":1708,"text":"And in non -sharded blockchain, this is extremely challenging because you actually, every single computer in the network needs to validate all of the transactions, maintain all of the state to be able to access, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1708s"},{"start_seconds":1720,"text":"And so Ethereum has this kind of buttons and topics around state growth, so that their state keeps growing.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1720s"},{"start_seconds":1728,"text":"I mean, even Solana now has, you know, working around like to do.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1728s"},{"start_seconds":1733,"text":"For example, compression, which is just a layer two on top to kind of move out the state out of the state space.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1733s"},{"start_seconds":1741,"text":"And sharding actually addresses that because now you can keep adding shards to continue pretty much maintaining somewhat of state per shard.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1741s"},{"start_seconds":1751,"text":"But the challenge is when new validators joining or validators get reassigned between shards, actually need to download the state of the shard.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1751s"},{"start_seconds":1761,"text":"And so they need to catch up and sync into this. Now all of that is implemented right now in Nightshade and it's working, but it is a pretty kind of expensive procedure to do constantly.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1761s"},{"start_seconds":1772,"text":"And that's why Validate is only rotate kind of once 12 hours and they need to, or like 12 to 14 hours.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1772s"},{"start_seconds":1780,"text":"And then they need to catch up and kind of re -sync into the new shard they assigned.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1780s"},{"start_seconds":1786,"text":"Yeah, so normally like in the sharded blockchain, like in a specific shard,","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1786s"},{"start_seconds":1792,"text":"everyone validates the state transition or validates a chunk by actually having the state locally and actually execute the chunk to see that the post state root","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1792s"},{"start_seconds":1802,"text":"actually agrees with what the chunk proposer proposes and then moves forward.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1802s"},{"start_seconds":1808,"text":"But yeah, the problem is you actually need to have the state locally and that's potentially expensive and also...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1808s"},{"start_seconds":1815,"text":"in a way, limits how many auditors you can have in the system.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1815s"},{"start_seconds":1820,"text":"And the statorless validation idea is that, well, so basically there's two roles in the system.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1820s"},{"start_seconds":1827,"text":"There's block producer. And then there's validator.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1827s"},{"start_seconds":1834,"text":"So for simplicity, we can collapse block and chunk production into one row.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1834s"},{"start_seconds":1841,"text":"So basically, they produce blocks and chunks.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1841s"},{"start_seconds":1849,"text":"And in addition to that, this is a key difference, is that they also produce state witness. OK, so what is state witness?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1849s"},{"start_seconds":1860,"text":"So we were talking about fraud proof before.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1860s"},{"start_seconds":1862,"text":"We were saying that you need this partial state, and the proof that it actually belongs to the correct state is dictated by the state root.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1862s"},{"start_seconds":1869,"text":"And precisely what a state witness is is basically the state that is touched during the execution of a chunk alongside with the proof that it actually belongs to the correct state as specified by some state root.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1869s"},{"start_seconds":1882,"text":"So. Why is this useful?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1882s"},{"start_seconds":1884,"text":"Well, then because other validators who don't actually maintain any state of the shard locally can actually execute the chunk with state witness.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1884s"},{"start_seconds":1899,"text":"and then they can statelessly verify that this state transition in the chunk is actually valid.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1899s"},{"start_seconds":1907,"text":"And the advantage here is that then the validators can actually rotate every single block into a different shard because they don't have to download states.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1907s"},{"start_seconds":1917,"text":"So there's like zero cost of them rotating other than maybe need to establish some new network connection or downloading some like state witness from other people.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1917s"},{"start_seconds":1927,"text":"Maybe just for example, so you want send, A send B 10 coins, that's a transaction.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1927s"},{"start_seconds":1937,"text":"And so what you do, you include that A has 100, B has 10 right now as your state witness, which includes a proof to the state root.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1937s"},{"start_seconds":1951,"text":"that this is indeed the previous state.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1951s"},{"start_seconds":1953,"text":"And so then given this transaction and this state witness, you can receive that the new state is A90, B20, and that the new stage root is updated this way.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1953s"},{"start_seconds":1967,"text":"So stage root plug T plus 1. So that's how you verify that state transition of a shard is actually valid.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1967s"},{"start_seconds":1979,"text":"Now, like we said, validators can actually just rotate on every single block to another shard.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1979s"},{"start_seconds":1985,"text":"This way, it is essentially, and also the shuffling is done through the randomness beacon that we have on chain that's unpredictable.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1985s"},{"start_seconds":1994,"text":"So adversary cannot know a priori how the - not did their assignment to charge what was going to work.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=1994s"},{"start_seconds":2001,"text":"And because it's changing actually on every single block and also unpredictable because of the randomness beacon, then essentially the idea is that adaptive","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2001s"},{"start_seconds":2011,"text":"corruption actually won't work because you cannot even know two blocks in advance which validator is going to assign to which chart.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2011s"},{"start_seconds":2020,"text":"And that's one of the core idea behind serious validation.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2020s"},{"start_seconds":2028,"text":"So the stimulus validation has actually many, many different benefits.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2028s"},{"start_seconds":2034,"text":"We'll talk about how it addresses the concern we had with snipe shade and fraud proofs, and also talk about the decentralization, let's say.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2034s"},{"start_seconds":2043,"text":"For example, the validator needs to now can be more lightweight because it doesn't need to hold the state locally and only needs to kind of execute the chunk with the standard witness that it actually receives.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2043s"},{"start_seconds":2055,"text":"So potentially you can have a lot more validators. In the system, we have more shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2055s"},{"start_seconds":2063,"text":"Another thing about decentralization is that even though those block producers, they actually are heavier compared to the auditors.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2063s"},{"start_seconds":2070,"text":"They actually not only produce the chunks, but also execute them and produce the witness. They themselves can't do much than what they can already do today.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2070s"},{"start_seconds":2080,"text":"It's not because you're running heavier, more expensive hardware, they somehow have more control of the system.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2080s"},{"start_seconds":2087,"text":"actually the control is the same as in the nightshade system we have today.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2087s"},{"start_seconds":2092,"text":"Basically, it can choose what transactions to include and what not to include, but it cannot provide state witness because it will get actually verified first that it is indeed belongs to the correct state route.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2092s"},{"start_seconds":2105,"text":"And because the state route is actually agreed on correctly by the validators, there is limited amount of things that broad producer can do even if it wants to act maliciously.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2105s"},{"start_seconds":2117,"text":"And another great benefit of this design that plays nicely with sharding is that","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2117s"},{"start_seconds":2124,"text":"because now we have sharding, we can actually limit the size of each shard to some fixed number, let's say, like 50 gigabytes, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2124s"},{"start_seconds":2133,"text":"So state size.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2133s"},{"start_seconds":2141,"text":"Then what can happen is that those block producers, because they actually run more expensive hardware, they can actually hold the entire state in memory.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2141s"},{"start_seconds":2149,"text":"So let's say you want to hold 50 gigabytes of state in memory, then potentially 64 gigabytes of RAM machine should actually be sufficient.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2149s"},{"start_seconds":2156,"text":"And that's actually not even that expensive.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2156s"},{"start_seconds":2159,"text":"And because validators actually execute the chunks using still witness, that's also in memory when the chunk's being executed.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2159s"},{"start_seconds":2166,"text":"So now all the state access, including region rights, are down in memory.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2166s"},{"start_seconds":2172,"text":"And that's going to be a huge improvement over the current design where I can actually go to disk.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2172s"},{"start_seconds":2179,"text":"And that's a huge bottleneck.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2179s"},{"start_seconds":2182,"text":"Yeah, I think maybe it's worth mentioning that any blockchain that uses Merkle -Patricia trees, for example, for storing state, ends up having a as a major bottleneck to pretty much throughput.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2182s"},{"start_seconds":2198,"text":"Because you, like to actually access and change something, you need to have kind of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2198s"},{"start_seconds":2205,"text":"a logarithm of reads from disk to actually find, I mean, either find or kind of find what you need to update in the state.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2205s"},{"start_seconds":2216,"text":"And, you know, on top of it, you add the complexity of the data underlying database, which is another logarithm.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2216s"},{"start_seconds":2221,"text":"And so, you know, if you have, kind of a billion records, this ends up being a few hundred disk reads per kind of one read from a specific item in the state.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2221s"},{"start_seconds":2238,"text":"And so that's a pretty heavy lift, even with SSDs kind of flash memory.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2238s"},{"start_seconds":2245,"text":"And so transition of full state of memory indeed speeds this up by...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2245s"},{"start_seconds":2252,"text":"more than that factor because you don't need to rate the disk. And it allows to have this node.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2252s"},{"start_seconds":2260,"text":"And this is only possible because through sharding, we can actually limit the size","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2260s"},{"start_seconds":2264,"text":"of each individual shard and able to grow the number of shards as well over time as more demand, as more applications are adding.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2264s"},{"start_seconds":2277,"text":"So maybe, yeah, we're talking, I mean, you mentioned it, right, that validators is kind of a huge benefit.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2277s"},{"start_seconds":2282,"text":"They now just need to receive a block, validate, right, and submit it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2282s"},{"start_seconds":2289,"text":"And in theory, they can actually join and leave even faster than ebook time.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2289s"},{"start_seconds":2295,"text":"This is not a current implementation, but it's possible to have shorter lived validators because they have such a kind of transient job.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2295s"},{"start_seconds":2305,"text":"At the same time, this means you can have so many of them and potentially even, you know, I mean, we discussed this like having in the browser, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2305s"},{"start_seconds":2312,"text":"You open a browser tab, it kind of connects to the network, starts receiving blocks, signing them, and then you close it, it kind of disconnects.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2312s"},{"start_seconds":2321,"text":"I mean, for that to happen, I mean, this is still like somewhat like expensive to do because you're still actually wrong the execution.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2321s"},{"start_seconds":2327,"text":"But I always think like this architecture is more future facing in the sense that...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2327s"},{"start_seconds":2332,"text":"or today you actually validate the state transition using execution against the witness.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2332s"},{"start_seconds":2339,"text":"But let's say the zero -knowledge technology matures in like two to three years, where we can generate ZK proof pretty fast.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2339s"},{"start_seconds":2348,"text":"And then, indeed, the writer only needs to validate one zero -knowledge proof and don't actually even need to do any execution. Now, it's not only that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2348s"},{"start_seconds":2357,"text":"The verification is extremely fast, but also that the data overhead is very low.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2357s"},{"start_seconds":2361,"text":"You just receive like one ZK proof, depending on the scheme, a few hundred bytes or a few kilobytes.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2361s"},{"start_seconds":2369,"text":"So that's way lower overhead than what we do today.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2369s"},{"start_seconds":2374,"text":"Yeah, so this is if block producers are able to produce ZK proof pretty much in the block time.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2374s"},{"start_seconds":2381,"text":"For example, then they can just send that instead of sending state witness and transactions.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2381s"},{"start_seconds":2387,"text":"And so then all of this becomes even more lightweight, which is amazing as we've seen the K technology to continue speed up kind of with every year.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2387s"},{"start_seconds":2404,"text":"Okay. So this is the core technology, I mean, the blockchain, the sharding, and the","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2404s"},{"start_seconds":2414,"text":"ability to continue scaling the transaction throughput.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2414s"},{"start_seconds":2420,"text":"The other interesting part has been how do we make all of this easier to use and really enable experience?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2420s"},{"start_seconds":2433,"text":"And while Bowen includes that, I think.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2433s"},{"start_seconds":2436,"text":"One of the big improvements is actually the whole idea of the sharding being kind of packaged as a single blockchain.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2436s"},{"start_seconds":2444,"text":"So if you go to Near, it's actually really hard to find shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2444s"},{"start_seconds":2447,"text":"You need to dig pretty deeply because the idea was to actually abstract that out from the users and developers.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2447s"},{"start_seconds":2454,"text":"And so we did have a few reshardings recently, which kind of brought it up more that it's happening.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2454s"},{"start_seconds":2462,"text":"But generally speaking, you need to actually go to like... block level RPC to actually explore that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2462s"},{"start_seconds":2470,"text":"And that is part of the vision that as users you ideally should not think about","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2470s"},{"start_seconds":2476,"text":"the blockchain and especially should not think about how it's actually partitioned and scaled underneath because this allows a lot more flexibility to change some of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2476s"},{"start_seconds":2485,"text":"this kind of design decisions without affecting users and developers at all.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2485s"},{"start_seconds":2491,"text":"Now, A lot of things we've done on VM account level as well was in that focus, like how do we make it easier for developers to build this?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2491s"},{"start_seconds":2503,"text":"Actually, I remember one thing we didn't talk about is cross -shard transactions. So let's maybe start with that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2503s"},{"start_seconds":2517,"text":"So maybe we can start with a simple example where let's say, perhaps, Alice .Near on shard 0 and Bob .Near on shard 1.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2517s"},{"start_seconds":2530,"text":"And then they want to say the transaction. It's actually...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2530s"},{"start_seconds":2539,"text":"I was just sending Bob some near tokens, one near token.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2539s"},{"start_seconds":2543,"text":"So this is a very simple transaction, but the kind of the lifetime of the transaction from the kind of execution point of view is that the transaction first gets sent to shard 0.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2543s"},{"start_seconds":2555,"text":"The validers of shard 0 executed there. And here it actually, in this step, it deducts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2555s"},{"start_seconds":2565,"text":"One year from now.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2565s"},{"start_seconds":2571,"text":"And it has a receipt that carries this transfer of one near into Shard1.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2571s"},{"start_seconds":2581,"text":"Obviously, we use Bob as the receiver.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2581s"},{"start_seconds":2584,"text":"And on Shard1, once it's actually getting executed here, then basically, they can't get one more near.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2584s"},{"start_seconds":2593,"text":"Now, that's kind of from the runtime point of view.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2593s"},{"start_seconds":2596,"text":"But the complexity comes from the fact that, oh, it comes from the problem of like, how each shard knows about the receipts that's coming from another shard.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2596s"},{"start_seconds":2606,"text":"So this goes back into the chain.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2606s"},{"start_seconds":2614,"text":"So in every block, there is this chunk header.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2614s"},{"start_seconds":2620,"text":"Because all the value is still tracking the chain itself, even though it may not track the actual transaction and the state of other shards, so it knows.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2620s"},{"start_seconds":2629,"text":"what the chunk header from every other shard is. So specifically, so let's see this chunk header.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2629s"},{"start_seconds":2639,"text":"It has this outgoing receipt route.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2639s"},{"start_seconds":2646,"text":"that which is more colonization of all the outgoing receipt route from shard zero to other shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2646s"},{"start_seconds":2655,"text":"And here, when actually needs to execute the receipt from shard one, it can receive","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2655s"},{"start_seconds":2661,"text":"the receipts from like some request receipt from people or like valid from shard zero and then actually verified against the outgoing receipt route that's actually agreed upon through consensus.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2661s"},{"start_seconds":2672,"text":"And then it knows what are the receipts are going. into SHARP1 and actually can know whether it has received all the receipts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2672s"},{"start_seconds":2680,"text":"And then also make sure that the receipts that they actually receive is actually the right receipt from SHARP0.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2680s"},{"start_seconds":2690,"text":"Yeah, so maybe from more, so there's like few levels of abstraction that happening here.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2690s"},{"start_seconds":2699,"text":"From the developer perspective or user perspective, Alice, and Bob .Near, they actually kind of exist almost like on separate chains.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2699s"},{"start_seconds":2711,"text":"And that's one of the interesting ways to think about it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2711s"},{"start_seconds":2715,"text":"Near kind of gives every account a chain and then just allows them to really effectively communicate with each other.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2715s"},{"start_seconds":2723,"text":"And so as Alice wants to send a message, it gets sent to Bob in the next block time.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2723s"},{"start_seconds":2732,"text":"usually unless there is any congestion in the network.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2732s"},{"start_seconds":2737,"text":"And so and then Bob can you know send a message back or send a message to somebody","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2737s"},{"start_seconds":2742,"text":"else in this and so kind of what this allows to do interestingly is that we","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2742s"},{"start_seconds":2748,"text":"actually can allocate one account on near a full capacity of a blockchain right so you know you you know you have app chains or you have kind of roll -ups.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2748s"},{"start_seconds":2759,"text":"And so you can imagine that any account on NEAR can have capacity up to a roll -up, up to a upchain, up to whatever maximum one shard can handle.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2759s"},{"start_seconds":2768,"text":"And so we actually have, for a while, been running specifically Aurora contract,","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2768s"},{"start_seconds":2773,"text":"which is an EVM as a smart contract on NEAR, as a separate shard, pretty much","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2773s"},{"start_seconds":2781,"text":"being able to process its own transactions and not being affected by other shard throughput.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2781s"},{"start_seconds":2788,"text":"And so that's a really kind of powerful tool because, as I said, from a user and developer perspective, you are not really kind of need to think about this.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2788s"},{"start_seconds":2798,"text":"You just deploy a contract and it works. And then the network itself, the governance of it, can decide to group","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2798s"},{"start_seconds":2805,"text":"those things into one shard or separate shards depending on the load of the network.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2805s"},{"start_seconds":2814,"text":"Now, right now, this is more of a kind of protocol team and validators deciding and executing this.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2814s"},{"start_seconds":2822,"text":"But over time, this can become a lot more kind of programmatic based on the load. And we call it dynamic resharding.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2822s"},{"start_seconds":2830,"text":"And so again, the kind of core idea that as the network scales, you want to continue changing where different contracts living and how they grouped","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2830s"},{"start_seconds":2839,"text":"based on what's the most optimal for using resources and kind of allowing everyone to have.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2839s"},{"start_seconds":2846,"text":"enough throughput for their smart contracts and accounts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2846s"},{"start_seconds":2852,"text":"Now on the chain side, as you mentioned, there's a chunk header, but the way to","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2852s"},{"start_seconds":2859,"text":"think about it, each chunk, when they send parts with data availability, they also attach the outgoing receipts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2859s"},{"start_seconds":2868,"text":"And so as part, again, of consensus data availability, we also do message passing.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2868s"},{"start_seconds":2873,"text":"And so in a way, with We tried to weave in all of the pieces of communication that shards need to have,","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2873s"},{"start_seconds":2881,"text":"and in a way, chains need to have into one system that is very much always passing information forward.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2881s"},{"start_seconds":2890,"text":"And what you're trying to avoid is, as much as possible, requesting information from other chains and from other validators, because that always adds an extra lag, delay, wait, retry, try another validator.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2890s"},{"start_seconds":2904,"text":"So that's kind of the core idea. From a user side, you have a single chain with accounts and you're kind of interacting with them.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2904s"},{"start_seconds":2911,"text":"From a developer side, you have almost like your own blockchain where you can really easily communicate with other chains.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2911s"},{"start_seconds":2921,"text":"And you don't need to have individual user accounts managed separately.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2921s"},{"start_seconds":2927,"text":"And then from a kind of consensus side, it's all weaved in into one system.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2927s"},{"start_seconds":2932,"text":"to ensure delivery, correctness, and low latency of communication.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2932s"},{"start_seconds":2943,"text":"All right, so yeah, let's talk about VM. Yeah, so one of the major choices we made was to really go with the WebAssembly.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2943s"},{"start_seconds":2956,"text":"And so for those who are not familiar, WebAssembly is a virtual machine developed by Microsoft, Google, Mozilla, and other companies.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2956s"},{"start_seconds":2968,"text":"It is used in every browser. And it also runs on edge, on edge devices.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2968s"},{"start_seconds":2975,"text":"It's a very versatile, generic virtual machine that has has a lot of non -web3 kind of push behind it.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2975s"},{"start_seconds":2983,"text":"And the benefit is there's a number of compilers that already exist, and specifically Rust, C, and a few others who are kind of highly supported, allowing to","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2983s"},{"start_seconds":2994,"text":"build kind of performant and with Rust usually more correct programs.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=2994s"},{"start_seconds":3001,"text":"And being kind of a generic framework, it allows to have other...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3001s"},{"start_seconds":3006,"text":"somewhat complex or more complex applications actually being able to run really easily. So I mentioned Aurora, which is an EVM.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3006s"},{"start_seconds":3014,"text":"And so this is literally an EVM written for one of the EVM clients that was kind of pretty much imported as a smart contract compiled and run into WebAssembly.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3014s"},{"start_seconds":3024,"text":"So that's kind of the power you get by using such a generic kind of virtual machine.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3024s"},{"start_seconds":3030,"text":"Now, on top of this, we build kind of a different account model.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3030s"},{"start_seconds":3036,"text":"to really simplify the access for users as well as create a versatility in places for creating more secure usage.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3036s"},{"start_seconds":3047,"text":"So one of the, again, major decisions was instead of accounts being named because of","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3047s"},{"start_seconds":3055,"text":"the hash of the public key, it access it, it's actually named based on a string that follows somewhat of a domain name.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3055s"},{"start_seconds":3065,"text":"rules, right? So there's a near account which can create everything .near.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3065s"},{"start_seconds":3073,"text":"And there's other top level accounts like Aurora and now TG, which creates .tg accounts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3073s"},{"start_seconds":3083,"text":"And so this Alice .near account has kind of information stored, which is how much near it has, how much it staked or locked.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3083s"},{"start_seconds":3095,"text":"and then it has a contract hash.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3095s"},{"start_seconds":3100,"text":"There's also storage information and then there's additional information that we call access keys.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3100s"},{"start_seconds":3109,"text":"And access keys is how you know which key can have access to this account and with what permissions.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3109s"},{"start_seconds":3117,"text":"And so what this allows to do is you can create a house with the account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3117s"},{"start_seconds":3123,"text":"And now you can have, for example, we call them full access keys that are, for","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3123s"},{"start_seconds":3130,"text":"example, accessed from your phone, from your laptop, maybe from a bank.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3130s"},{"start_seconds":3140,"text":"if you have custody. But also you can have access, we call them functional call keys.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3140s"},{"start_seconds":3149,"text":"And so they allow, for example, a specific application to be able to submit transactions on your behalf from this frontend.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3149s"},{"start_seconds":3160,"text":"So what you can do, you can then go and log in into this app and say, when you're approving adding this key, that it only can, for example, call one function on behalf of the account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3160s"},{"start_seconds":3173,"text":"And so then this front end, this application, can now send transactions of that sort without you needing to go to confirm them in the wallet every time.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3173s"},{"start_seconds":3181,"text":"Then you can go and remove this key and manage these permissions. So this is very similar to using Twitter or Facebook or anything else.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3181s"},{"start_seconds":3190,"text":"You have approved applications and devices, and you can pretty much manage that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3190s"},{"start_seconds":3196,"text":"There's also kind of amount of gas attached here to pay for the...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3196s"},{"start_seconds":3201,"text":"The important thing is that the function call access key does not allow the owner to transfer money from that key.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3201s"},{"start_seconds":3208,"text":"So it protects the user that they don't have to worry about that part. Exactly.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3208s"},{"start_seconds":3218,"text":"And so it tells me the other thing is because you have a contract hash on the account...","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3218s"},{"start_seconds":3224,"text":"So any account, you can create a regular account and then turn it into a smart contract account, or turn it into a smart contract, or turn it back into your own account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3224s"},{"start_seconds":3232,"text":"And so the idea is you can deploy at any time a specific contract on this that has","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3232s"},{"start_seconds":3239,"text":"specific, I mean, this can be anything from just a regular whatever, DAX to a specialized contract for your account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3239s"},{"start_seconds":3248,"text":"For example, you can deploy a multi -sig.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3248s"},{"start_seconds":3250,"text":"And now, all of this, you know, your removal of full access keys and you just have function call keys that can call your own account to approve some transactions.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3250s"},{"start_seconds":3262,"text":"And so this is very powerful because now you can turn existing account into a multisig, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3262s"},{"start_seconds":3268,"text":"So imagine you have a lot of, you know, you start a new account, it was empty, it didn't matter.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3268s"},{"start_seconds":3273,"text":"And then you received, for example, locked tokens, or you have an NFT revenue stream.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3273s"},{"start_seconds":3280,"text":"that is now targeted at your account, but you cannot change it anymore because that's baked into an NFT contract.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3280s"},{"start_seconds":3287,"text":"So now you can turn your account into a multisig and have like some even external custodians, 2FA, whatever you want as a way to manage this.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3287s"},{"start_seconds":3299,"text":"There's also a really cool other use case that keep on pioneering, which is called trial account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3299s"},{"start_seconds":3305,"text":"This is when somebody else creates account, funds it with money.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3305s"},{"start_seconds":3309,"text":"and gives you a function called key to do something, but you only can limit, like you only limit it with what you can do. You cannot withdraw the money from this account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3309s"},{"start_seconds":3317,"text":"And so if you want to get a full access key, you need to repay back to whoever gave you this trial account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3317s"},{"start_seconds":3322,"text":"So it's a very flexible system that allows a lot of different use cases.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3322s"},{"start_seconds":3328,"text":"And this, you know, we see kind of more and more folks experimenting with what they can do, including you can have one account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3328s"},{"start_seconds":3339,"text":"So for example, Near has a contract on it which allows to pretty much add a new key","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3339s"},{"start_seconds":3346,"text":"to this new function called Key to this account, which then if somebody has a private key for this public key, they can call on behalf of the account to create a new account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3346s"},{"start_seconds":3357,"text":"What this allows to do is I can now create a link that I give somebody else.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3357s"},{"start_seconds":3361,"text":"And when they click on it, it creates them a new account and deposits some Near on it. So they can pretty much create new accounts.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3361s"},{"start_seconds":3369,"text":"from a link non -custodial through this approach, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3369s"},{"start_seconds":3375,"text":"So, there's ways to expand, kind of have a share, almost like a shared account, where different keys have different permissions for different users who hold the private key for that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3375s"},{"start_seconds":3387,"text":"Sidekings.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3387s"},{"start_seconds":3390,"text":"And then, one of the use cases we've originally not envisioned that people will use through this contract, but it is, someone complex is actually paying on behalf of this user, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3390s"},{"start_seconds":3403,"text":"Let's say Alice came in and she only has zero balance. How do we actually enable that?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3403s"},{"start_seconds":3412,"text":"And the kind of one of the things that Bowen team introduced early 2023 has been","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3412s"},{"start_seconds":3423,"text":"to have a specialized meta transaction call. It allows to send a transaction on behalf of Alice.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3423s"},{"start_seconds":3433,"text":"So Alice needs to sign with one of the keys that she has.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3433s"},{"start_seconds":3438,"text":"And then this is sent, so send transaction, but then this is sent to Relayer.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3438s"},{"start_seconds":3447,"text":"Relayer actually submits a transaction, pays the fee, and then it goes to Alice's account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3447s"},{"start_seconds":3457,"text":"and gets executed from there. So it actually wraps this transaction in another transaction.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3457s"},{"start_seconds":3462,"text":"So when it signs it creates a new transaction that has the signed transaction as the payload.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3462s"},{"start_seconds":3468,"text":"Yeah, so here there's a kind of transaction signed by Relair, which contains transaction signed, like a delegated action signed by Alice.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3468s"},{"start_seconds":3482,"text":"And so Relair pays the fee to the network. and then this delegated action gets executed off Atlas's account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3482s"},{"start_seconds":3489,"text":"And so this allows to now do a lot of things.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3489s"},{"start_seconds":3492,"text":"For example, a lot of the biggest applications on NEAR are using this approach now to send transactions on behalf of users, everything from free","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3492s"},{"start_seconds":3502,"text":"transfers for stable coins, being able to interact with games, being able to claim tokens and everything else.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3502s"},{"start_seconds":3512,"text":"And so it's a very powerful primitive.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3512s"},{"start_seconds":3514,"text":"that kind of works pretty well with this whole architecture and the access key system.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3514s"},{"start_seconds":3522,"text":"Now, everything you see is like we're trying to kind of abstract and make it easy to use accounts and make it easy to use the blockchain, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3522s"},{"start_seconds":3530,"text":"Again, on NEAR itself, we're trying to remove the need to think about shards.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3530s"},{"start_seconds":3535,"text":"And with this, we're trying to remove the need to think about kind of transaction fees, at least initially.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3535s"},{"start_seconds":3541,"text":"And so one of the things that we're introducing, the march is, or introduced in the march, is in 2024, is the idea of chain signatures.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3541s"},{"start_seconds":3556,"text":"And so chain signatures is kind of expanding on all of this.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3556s"},{"start_seconds":3561,"text":"But now you're able to pretty much call a method sign to kind of specific subset of validators who.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3561s"},{"start_seconds":3572,"text":"and you want to tell them a payload. and so -called derivation.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3572s"},{"start_seconds":3583,"text":"and what they will return you back actually a signed payload.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3583s"},{"start_seconds":3588,"text":"signature, like a signature for this payload, but this signature is not from your key.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3588s"},{"start_seconds":3595,"text":"This signature is actually now based on a new key that a set of validators now","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3595s"},{"start_seconds":3605,"text":"maintain that is owned by them.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3605s"},{"start_seconds":3612,"text":"And we are setting them up such that it's subset of near validators as well as some eigen layer. kind of risk taker, we see you.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3612s"},{"start_seconds":3621,"text":"Yeah, so the idea is that none of the validators here actually hold the full key for users, so none of them collectively, individually can actually do anything","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3621s"},{"start_seconds":3631,"text":"malicious, and there is actually a threshold upon which they need that many validators to actually sign to actually create the final signature.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3631s"},{"start_seconds":3639,"text":"And this is essentially a threshold, a signature scheme that actually fixes the signature. but allow the validator to rotate.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3639s"},{"start_seconds":3649,"text":"Yeah, so for example, there's specific popkey that all of this validators have. And then you imagine maybe one of them left, leaves, goes offline.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3649s"},{"start_seconds":3659,"text":"You can still get the same popkey. You can still get signatures for the same popkey by subset.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3659s"},{"start_seconds":3665,"text":"And then, so this was like v4, v4 left, and for example, v6 joint.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3665s"},{"start_seconds":3672,"text":"And they pretty much constantly reshare.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3672s"},{"start_seconds":3675,"text":"parts of the key in such a way that none of them contain, none of them have the","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3675s"},{"start_seconds":3682,"text":"full key, but they're all still able to sign if they come together for a deterministic public key.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3682s"},{"start_seconds":3688,"text":"Now, there is an additional derivation used and your account, right?","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3688s"},{"start_seconds":3693,"text":"So it's Alice, for example, requested the signature that actually determines the exact key.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3693s"},{"start_seconds":3700,"text":"And so the way to think about it, you have a near account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3700s"},{"start_seconds":3706,"text":"You have Alice .Near, which now has associated addresses on Ethereum and multiple of them, depending on derivation.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3706s"},{"start_seconds":3717,"text":"It has a Bitcoin address. It has Solana address, I don't know, Taser address, et cetera.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3717s"},{"start_seconds":3728,"text":"So you have addresses on every single chain that you're able to sign pretty much transactions.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3728s"},{"start_seconds":3738,"text":"So this really allows to now have a new account that is able to transact across","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3738s"},{"start_seconds":3743,"text":"all the other chains where the custody of this is shared between the validators and your account.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3743s"},{"start_seconds":3752,"text":"And this is very powerful because, remember, your accounts can be contracts, which means a contract can now transact on other chains.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3752s"},{"start_seconds":3760,"text":"Because of the derivations, you can actually have a contract which has an independent addresses where you can deposit, I don't know, ordinals, which this contract can issue near NFTs for.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3760s"},{"start_seconds":3773,"text":"You can have internal mapping that, oh, this address is owned by Illia, this address is owned by Bowen, let's swap this ownership.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3773s"},{"start_seconds":3782,"text":"So you can program that. You can obviously have direct usage as if you go with your near account. You log in into a Magic Eden.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3782s"},{"start_seconds":3791,"text":"You transact there.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3791s"},{"start_seconds":3792,"text":"And depending if it's like smart contract on Solana or on Ethereum, it actually executes through the right signature.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3792s"},{"start_seconds":3800,"text":"And you can receive assets on either chains.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3800s"},{"start_seconds":3804,"text":"So you're allowed to really kind of lift up the experience from, OK, I need to choose a chain. I need to bridge there.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3804s"},{"start_seconds":3811,"text":"I need to figure out what's a gas dock in there to kind of really start abstracting this out. And there's like a few other pieces like gas layers to enable that.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3811s"},{"start_seconds":3821,"text":"Yep. So one chain to rule them all.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3821s"},{"start_seconds":3823,"text":"And you can have all your transactions handled through like a smart contract on there. Yeah. So welcome for any developers interested in chain signatures.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3823s"},{"start_seconds":3833,"text":"Go out, check out documentation. For learning more about the protocol, there's a Nomicon .io.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3833s"},{"start_seconds":3839,"text":"which has a lot of the spec, as well as invite everyone to github .com, near .com, and EPS, near enhancement proposals.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3839s"},{"start_seconds":3850,"text":"And this is where a lot of the discussions about the protocol are happening and the design updates for the specification are.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3850s"},{"start_seconds":3861,"text":"Yeah, thanks everyone. All right, thank you.","url":"https://www.youtube.com/watch?v=YGUeTu4Cco4&t=3861s"}],"youtube_url":"https://youtube.com/watch?v=YGUeTu4Cco4","url":"https://near-pedia.xyz/videos/YGUeTu4Cco4"}