{"id":"UTs9ZkX401M","title":"'Sharding Research' by Alex Skidanov | Graph Day | Blockchain","description":"Our Co-Founder Alex Skidanov (  / alexskidanov  ) spoke on the topic - 'The frontier of sharding research' at Graph Day on January 25th, 2019. \n\nIn this video, Alex covered Sharding, Validators, and how the protocols are looking at scaling Blockchains. He also spoke about how Near Protocol is building a public sharded blockchain with the focus on usability.\n\n\nGraph Day (https://thegraph.com/graphday) was an event hosted by Graph Protocol (https://thegraph.com/) to discuss the future of Web3 with the dedicated developers, investors, and community members. \n\nMore about NEAR: https://nearprotocol.com/\n\n\nFollow the latest from NEAR Protocol on, \n\nDiscord: https://near.ai/discord\nMedium: https://near.ai/medium\nTwitter: https://near.ai/twitter\nGitHub: https://near.ai/github","channel_name":"NEAR Protocol","published_at":"2019-02-07T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/UTs9ZkX401M/hqdefault.jpg","transcript_text":"hi everyone and let's talk about shorting before we start let me quickly introduce myself I was I'm a co-founder of near protocol which held briefly introduced in a second before near protocol I was working for a company called MEMC pole and mem sequel was a short database I was employee number one at home sequel and I built most of the early sharding and then I was driving charging when it became more mature and the for mom sequel i was at Microsoft that was a decade ago and we started near protocol around August last year it's one of the younger protocols and as a shorter blockchain we have a huge emphasis on usability we invest heavily into developer experience into user experience with the blockchain how they can onboard easier and near protocol is one of the few teams that has a very strong team with specifically with the industry backgrounds we have three people four we have four people from home Seco we have many eggs Googlers we have many people who have gold medals from ACM ICPs which is a competitive programming and one of the one of the key things about near protocol is that we are very open with the community and so we're not when we're not behind like a walled garden we work closely with other protocols we work closely with the theory on foundation with cosmos and others and so the everything that happens in the sharding research we share a lot of that bit between us and so in this presentation I will I will show you what charting is how charting works and what sort of problems still exist in charting so first of all what charting is right so let's say we have a blockchain so that could be a theorem or block chain or Bitcoin or or any blockchain and that and that blockchain has some number of transactions per second it can process so let's say it's 12 and let's say 12 is not enough let's say we want more maybe we want let's say 136 so one thing we can do is we can run three blockchains if each of them can process twelve transactions that gives us 36 if we want to have thousand transactions per second we can run hundred blockchains and if we want to have ten thousand transactions per second we can we can run 1,000 chip block chains and that's it we thought with all the scalability soft there's a small problem the slight problem which is when we have one blockchain and there's some population of people who who are willing to validate that blockchain and produced blocks if you want to compromise the blockchain you need to corrupt or control 51% of them once you have multiple block chains those people they were when this blockchain had 12 transactions per second there was some limitation there was some reason why the blockchain couldn't have processed more maybe maybe the network was at capacity maybe processing was of capacity maybe storage but there was some reason why the equipment process more so when we have you know ten ten ten block chains we cannot expect all validators validate all the block chains because they once they start validating one they are at capacity and that means that validators are split between them and now if you want to corrupt one blockchain you only need to corrupt sufficient number 51 percent of the people who validated which is but if you have ten block chains that becomes five percent of people if you have thousand block chains you need to corrupt point zero five percent of all the validators which is that doesn't sound very hard but that would only work if you can actually control so if you let's say let's say it's a half power that is that is deciding the transaction or at stake for you to be able to corrupt one one of the chains you either have to be able to control where your hash power goes to which shirt or where your state goes or you have to be able to to identify people in a particular shirt and some somehow corrupt them you'd either bribe them or hack them in some way and so the most common solution to this problem is to rotate the validators we will assign validators to shards so let's say every short has 100 validators they will produce some few blocks and then we'll completely you shuffle them and so now first of all I don't have control where my validation power goes but also I cannot really corrupt someone unless I can corrupt very fast right so unless I can corrupt them during during that time frame well the validating the shard by the time we corrupt them they already got you shuffled and at that point they belong to different shirts and so the idea here is that if we believe that the total population has less than 25% let's say malicious actors then once we sample let's say 100 of them we can we can do some mathematical modeling and show that the number of malicious actors the probability of having let's say I take a 3% of malicious actors in the chart is very low right so you know like let's say the full population is 10,000 and your sample 100 if you know there's 25% here at most then the probability that you have 33 malicious actors 33 percent militias like there's here is very low it can be considered to be zero and so once that is the case let's say this is you and you want to know what is the latest look in the blockchain you go to the current validators and you ask them what is the current what is the latest block and if you hear the response from two-thirds of them and you use some sort of DFT consensus and that response is the same you know that block is correct because you know that only that 2/3 of them are actually honest and when those validates were producing a block they asked for it from the previous validators and by similar reasoning the block they they got was correct block at that time and then there's sort of induction here and it goes all the way to the base of the interaction with the first validators they were building on top of Janice's block which is known to be correct and so this block chain the entire block chain is correct for as long as you make an assumption that all the validators along the way were correct which is not a very reasonable assumption and we'll get back to it in a second but there's one problem which is shooting those validators that is that should be secure computation that computation should happen on chain but we don't have a chain we have multiple chains which which chain is responsible for actually rotating the validators and the most common solution to that problem is to have the chain have one blockchain to rule them all and in the theorem it is called the beacon chain in polka dotted cultural a chain we will be referring to this main chain throughout the presentation and so their chain everybody validated right so the short chains the validators are split among them but they the mange and everybody validates it and the main chain primary responsibility is to rotate the validators assign them to shirts and do some other computations which are global to the entire system an example of such computation which will be very useful is like clearly the main chain has significantly higher security because everybody validates it and it would have been nice if we could somehow give that security from the main chain to all the short chains and one way to do that is for short chains to occasionally snapshot the half of the latest block to the beacon chain to the main chain if the short chain has natural to their hair to the main chain and the main chain finalize the block with that snapshot then on the third chain later you can in the fork choice rule you can say do not respect any chain that does not contain that block and now forking anywhere before that block occurred is impossible because unless you formed the main chain but the mention has significantly higher security so that is called cross-linking and that helps you to prevent forking but it's working the only malicious behavior that can be exercised as a matter of fact not if you corrupt a shark in some way you can fork but you can also do another attack called invalid state transition what invalid state transition is is let's say we have a transaction in which Alice sends Bob ten tokens and we have the state in which Alice has don't ten tokens and Bob has zero tokens if you get that state and apply this transaction the state you're supposed to get is the state in which Alice has zero tokens and walk has ten and so any block which apply state in this way is valid what can happen is there could be a block produced in which you know in initial state Alice had ten plugins the transaction was applied and now Bocas thousand that is an invalid state transition and you probably often heard about forking it is he recently had a fork but you're unlikely to have heard about invalid state transition attacks that is because in non Charlotte blockchains invalid state transition is impossible the reason for that is imagine that we actually have majority of health power or majority of stake belonging to malicious actors and at some point they produced an invalid block so this block a prime contains some transaction which was applied appropriately in any modern block chain like a theorem or a Bitcoin all the validators are validating all the blocks and all the users are validating on the block so if you're using Bitcoin you would be downloading the entire ledger and validating every single block and so the moment you see the a prime you will see it as invalid you will discard it and so you will not even consider B prime C prime or D prime as well that blocks because you know they built on top of invalid state and so even though the honest actors built a shorter chain every single client will know that that chain is correct however in a Charlotte block chain that problem does exist and to understand why we first need to talk a little bit about cross heart communication right in the in the Charlotte block chain if we have three different sharks three different block chains and let's just say for now that the only kind of transactions that exist is payment transactions the let's say I have an account in the first shark and the person I'm sending money to has account on the second shirt there is nobody who validates both chains and so the transaction cannot be in the first block because nobody because because then the validators can I change the state of the second of the destination account and the transactions can the transaction cannot really be on the second chain because then the validator cannot change the state of the of the sender and so what needs to happen is this there's got to be some way of change to communicate and the way it's usually done is through this concept of receipt where with the transaction that touches multiple shirts it originates in one of the shirts for example in a Chardon which let's say Elias is sending money to Bob the transaction will originate and the Alice's chart it will debit her account and then the validators of this chart will send some sort of receipt to the second shirt with signatures confirming that the transaction was correct and then in the second chart the validators will verify the signatures and apply the transaction to Bob now with crusher transactions in place what can happen is if we manage to corrupt short one I can produce completely invalid block block B in which we mean some tokens out of thin air and then later will produce a block C which otherwise loops correct in Block C no transition is applied improperly and from that block with sending those tokens which were minted to another shirt the second chart has no way to validate the entirety of the first shirt right they maybe are able to validate Block C but they cannot validate the entire history because you know if there's thousand shards and everybody is sending cross card transactions - sure - sure - cannot validate all the shirts and so - sure sure - has no way to to know that the that the block wasn't correct so one way to solve that was offered is let's have let's not allow cross-court communication between English arts let's say that only sound shirts can communicate between each other and the chard every shirt validates all the shirts that let this neighboring all its neighbors right this way if there's a cross heart communication and let's say Block B was invalid then chart two because third one is its neighbour validated all the blocks and the short one and it knows block D is invalid so this attack is no longer possible and if someone wants to send money from one star to another which is not a neighbor they will just send through multiple shirts and this is what the problem occurs again right because well like corrupting short one is not sufficient anymore but now corrupting just two shirts which is still significantly less than full system is sufficient who will corrupt corrupt child one produced invalid block sent across drug transaction - sure - which is also owned by us so we will accept the cross card transaction and we will send it to chart 3 chart 3 can only valid a chart - which are - is completely valid there is no there is no invalid state transition in the shark - so how it is solved today well one way or how it can be solved in the future one way is you can pretend the does not exist you can you can try to build your security in such a way that you believe you convinced that at no point the validator set in any shard will be corrupted alternatively you can you can try to have some cryptographic proof that the entire blockchain is correct when you're sending a crosshair transaction there is such cryptographic primitives they called snarks but today they not quite usable they were very slow right they're usable for private transactions so zeeky zeeky zeeky caches using them but to valid to prove that the entire blockchain is correct that is not that is not quite there yet the long term that will be the best solution possible the most promising solution today is called fisherman where the idea is that for as long as you have at least one honest validator that one is validator would be monitoring with what is happening in other blocks in other shirts and if they observe the distant cross card transaction happening from from the shirt that they validating from the state that they they have not observed before so actually the scenario will be the charges completely overtaken there's more than two-thirds of malicious actors they produce an invalid blog they never show this blog to the owners validator they produce a valid block on top of it otherwise valid besides the fact to disapproving an invalid block initiate crash or transaction the owners validator will notice that this is happening download the blocks identify that this block is invalid and create some proof some some small proof that that this is happening and send this proof to the short torso now at this point sure to will quickly verify the proof the proof is very lightweight and know that the cross card transaction that was initiated is incorrect the problem is that monitoring the shard identifying this this problem happened done loading the block validating it it takes a lot of time and so the challenge period the period during which the period that far to will be waiting after crucial transaction was initiated before they shorten that no challenge will arrive should be pretty long right so so let's say both shards of producing blocks every three seconds so cross our transaction can really without fishermen be finished in like six seconds with the fisherman it needs to be on the order of minutes so the system becomes significantly slower but the bigger problem is that this blog be the own it Valley that actually wouldn't load the blog be the problem is that the dishonest validator might have never even published it because in the short - they they cannot be done loading all the blog's from all the shirts if these thousand shirts and there's 1 megabyte blog produced every few seconds in every shirt you would have to download one gigabyte of data every few seconds that is infeasible so char - cannot you 11 10 loading this block to verify the transaction and so this block might not even exist anywhere nobody nobody nobody besides the dishonest validators might have seen that block and so in this case the onus validator cannot download it cannot validate it cannot craft the challenge that problem is called data availability problem very few protocols actually really work on this problem right the only protocols that that that publicly say that they work on this problem that they believe it exists is polka dot and the etherium most of the other protocols they there's nothing in their technology which actually tries to to go to work with data availability to work around the data availability problem and yeah and the problem is that it's not only cross hard communication that is broken is also snapshotting to the beacon chain you only snapchatting to the beacon chain the half of the latest block you never reveal the actual block right so the onus validators might not be able to download the block of the shard was corrupted and they cannot so the short can completely stalled and so there are some proposals the usual they mostly work around this idea of using erasure codes and providing some extra redundancy in the data that is available and then data is sample split most of those are the very early stages and so if you interested in this particular topic you can you can search for a paper by the italic called route proofs and let me finish my presentation with quick overview of how people actually build sharding today right so cosmos is one example of something which is very similar to Chardon right they have a cosmos hub which is which looks a lot like a beacon chain and they have zones which look a lot like shorts few things about coordinates is that first validators do not rotate around zones validators assigned to the zone and it reduces the security a little bit and they the way cause most positions it is that the users of the zone have for themselves their responsibility to identify how many validators exist there how much they trust them and how much they trust the zone and then cosmos the cross shard communication cross down communication is carried out through a protocol called IBC inter blockchain communication which allows you to move assets around between the designs but it doesn't really allow you to do any more complex code execution polka-dot is somewhat similar to to cosmos however the big difference is that a the actual runtime of every blockchain is stored on on the chain so in cosmos every blockchain is is completely different binary in polka-dot in principle it can be built in such a way that the para para chains of the same binaries they don't have to be but they could and the runtime of the para chain is stored in the own chain meaning that validators in principle could rotate however in what polka-dot is planning to ship this year they will not be rotating so polka dot will have the same security as cosmos when they ship and poke a lot a lot using fisherman for the securities of the you don't really have to trust that you don't need to trust that two-thirds of the validator disk is our honest it doesn't have to trust that ninety-nine per se it's enough to try that is atleast one honest validator / / shard I silica is you probably heard of silicon the concept of sharding silica only delivered state the only short processing meaning that every node to have to store the entire state which is a huge limitation and also they for cross card transactions they completely shut down the entire the completely shot shutdown sharding and process them serially and the theorem serenity near they very similar unfortunately running out of time but they effectively everything I I described today this is effectively how etherium serenity and near work and cool so let me finish here quick thing is that we have whiteboard series where we talk to two founders of different protocols we talk to cosmos to solana where we talk a lot about technology so check it out near yes this YouTube and also next Tuesday we're revealing our deaf net if you in the city google it it's or the link is near a guys there's definite unfortunate don't have it on the slide thank you","transcript_synced_at":"2026-11-04T04:32:54.273Z","segments":[{"start_seconds":1,"text":"hi everyone and let's talk about shorting before we","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1s"},{"start_seconds":8,"text":"start let me quickly introduce myself I was I'm a co-founder of near protocol which held briefly introduced in a","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=8s"},{"start_seconds":18,"text":"second before near protocol I was working for a company called MEMC pole and mem sequel was a short database I was employee number one at home sequel","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=18s"},{"start_seconds":25,"text":"and I built most of the early sharding and then I was driving charging when it became more mature and the for mom","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=25s"},{"start_seconds":33,"text":"sequel i was at Microsoft that was a decade ago and we started near protocol around August last year it's one of the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=33s"},{"start_seconds":42,"text":"younger protocols and as a shorter blockchain we have a huge emphasis on usability we invest heavily into","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=42s"},{"start_seconds":49,"text":"developer experience into user experience with the blockchain how they can onboard easier and near protocol is","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=49s"},{"start_seconds":56,"text":"one of the few teams that has a very strong team with specifically with the industry backgrounds we have three","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=56s"},{"start_seconds":64,"text":"people four we have four people from home Seco we have many eggs Googlers we have many people who have gold medals from ACM ICPs which is a competitive","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=64s"},{"start_seconds":73,"text":"programming and one of the one of the key things about near protocol is that","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=73s"},{"start_seconds":81,"text":"we are very open with the community and so we're not when we're not behind like a walled garden we work closely with other protocols we work closely with the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=81s"},{"start_seconds":88,"text":"theory on foundation with cosmos and others and so the everything that happens in the sharding research we share a lot of that bit between us and","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=88s"},{"start_seconds":97,"text":"so in this presentation I will I will show you what charting is how charting works and what sort of problems still","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=97s"},{"start_seconds":106,"text":"exist in charting so first of all what charting is right so let's say we have a blockchain so that could be a theorem or","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=106s"},{"start_seconds":113,"text":"block chain or Bitcoin or or any blockchain and that and that blockchain has some number of transactions per second it can process so let's say it's","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=113s"},{"start_seconds":122,"text":"12 and let's say 12 is not enough let's say we want more maybe we want let's say 136 so one thing","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=122s"},{"start_seconds":130,"text":"we can do is we can run three blockchains if each of them can process twelve transactions that gives us 36 if we want to have","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=130s"},{"start_seconds":140,"text":"thousand transactions per second we can run hundred blockchains and if we want to have ten thousand transactions per second we can we can","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=140s"},{"start_seconds":147,"text":"run 1,000 chip block chains and that's it we thought with all the scalability","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=147s"},{"start_seconds":153,"text":"soft there's a small problem the slight problem which is when we have one blockchain and there's some population","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=153s"},{"start_seconds":162,"text":"of people who who are willing to validate that blockchain and produced blocks if you want to compromise the blockchain you need to corrupt or","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=162s"},{"start_seconds":170,"text":"control 51% of them once you have multiple block chains those people they","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=170s"},{"start_seconds":178,"text":"were when this blockchain had 12 transactions per second there was some limitation there was some reason why the blockchain couldn't have processed more","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=178s"},{"start_seconds":185,"text":"maybe maybe the network was at capacity maybe processing was of capacity maybe storage but there was some reason why the equipment process more so when we","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=185s"},{"start_seconds":193,"text":"have you know ten ten ten block chains we cannot expect all validators validate all the block chains because they once","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=193s"},{"start_seconds":201,"text":"they start validating one they are at capacity and that means that validators are split between them and now if you want to corrupt one blockchain you only","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=201s"},{"start_seconds":208,"text":"need to corrupt sufficient number 51 percent of the people who validated which is but if you have ten block chains that becomes five percent of","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=208s"},{"start_seconds":216,"text":"people if you have thousand block chains you need to corrupt point zero five percent of all the validators which is","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=216s"},{"start_seconds":222,"text":"that doesn't sound very hard but that would only work if you can actually control so if you let's say let's say","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=222s"},{"start_seconds":231,"text":"it's a half power that is that is deciding the transaction or at stake for you to be able to corrupt one one of the chains you either have to be able to","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=231s"},{"start_seconds":239,"text":"control where your hash power goes to which shirt or where your state goes or you have to be able to to identify people in a particular shirt and some","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=239s"},{"start_seconds":247,"text":"somehow corrupt them you'd either bribe them or hack them in some way and so the most common solution to this problem is","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=247s"},{"start_seconds":255,"text":"to rotate the validators we will assign validators to shards so let's say every short has 100 validators they will produce some few blocks and then","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=255s"},{"start_seconds":264,"text":"we'll completely you shuffle them and so now first of all I don't have control where my validation power goes but also I cannot really corrupt someone unless I","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=264s"},{"start_seconds":272,"text":"can corrupt very fast right so unless I can corrupt them during during that time frame well the validating the shard by the time we corrupt them they already got you shuffled and at that point they","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=272s"},{"start_seconds":282,"text":"belong to different shirts and so the idea here is that if we believe that the total population has less than 25% let's","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=282s"},{"start_seconds":290,"text":"say malicious actors then once we sample let's say 100 of them we can we can do","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=290s"},{"start_seconds":297,"text":"some mathematical modeling and show that the number of malicious actors the probability of having let's say I take a","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=297s"},{"start_seconds":304,"text":"3% of malicious actors in the chart is very low right so you know like let's say the full population is 10,000 and","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=304s"},{"start_seconds":311,"text":"your sample 100 if you know there's 25% here at most then the probability that you have 33 malicious actors 33 percent","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=311s"},{"start_seconds":319,"text":"militias like there's here is very low it can be considered to be zero and so once that is the case let's say this is","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=319s"},{"start_seconds":326,"text":"you and you want to know what is the latest look in the blockchain you go to the current validators and you ask them what is the current what is the latest block and if you hear the response from","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=326s"},{"start_seconds":334,"text":"two-thirds of them and you use some sort of DFT consensus and that response is the same you know that block is correct because you know that only that 2/3 of","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=334s"},{"start_seconds":343,"text":"them are actually honest and when those validates were producing a block they asked for it from the previous validators and by similar reasoning the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=343s"},{"start_seconds":352,"text":"block they they got was correct block at that time and then there's sort of induction here and it goes all the way","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=352s"},{"start_seconds":361,"text":"to the base of the interaction with the first validators they were building on top of Janice's block which is known to be correct and so this block chain the entire block chain is correct for as","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=361s"},{"start_seconds":369,"text":"long as you make an assumption that all the validators along the way were correct which is not a very reasonable assumption and we'll get back to it in a second but there's one problem which is","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=369s"},{"start_seconds":377,"text":"shooting those validators that is that should be secure computation that computation should happen on chain but","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=377s"},{"start_seconds":386,"text":"we don't have a chain we have multiple chains which which chain is responsible for actually rotating the validators and the most common solution to that problem","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=386s"},{"start_seconds":394,"text":"is to have the chain have one blockchain to rule them all and in the theorem it is called the beacon","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=394s"},{"start_seconds":402,"text":"chain in polka dotted cultural a chain we will be referring to this main chain throughout the presentation and so their","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=402s"},{"start_seconds":409,"text":"chain everybody validated right so the short chains the validators are split among them but they the mange and everybody validates it and the main","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=409s"},{"start_seconds":418,"text":"chain primary responsibility is to rotate the validators assign them to shirts and do some other computations","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=418s"},{"start_seconds":425,"text":"which are global to the entire system an example of such computation which will be very useful is like clearly the main","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=425s"},{"start_seconds":433,"text":"chain has significantly higher security because everybody validates it and it would have been nice if we could somehow","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=433s"},{"start_seconds":440,"text":"give that security from the main chain to all the short chains and one way to do that is for short chains to","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=440s"},{"start_seconds":447,"text":"occasionally snapshot the half of the latest block to the beacon chain to the main chain if the short chain has natural to their hair to the main chain","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=447s"},{"start_seconds":455,"text":"and the main chain finalize the block with that snapshot then on the third chain later you can in the fork choice rule you can say do not respect any","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=455s"},{"start_seconds":463,"text":"chain that does not contain that block and now forking anywhere before that block occurred is impossible because","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=463s"},{"start_seconds":472,"text":"unless you formed the main chain but the mention has significantly higher security so that is called cross-linking","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=472s"},{"start_seconds":479,"text":"and that helps you to prevent forking but it's working the only malicious behavior that can be exercised as a","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=479s"},{"start_seconds":486,"text":"matter of fact not if you corrupt a shark in some way you can fork but you can also do another attack called","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=486s"},{"start_seconds":494,"text":"invalid state transition what invalid state transition is is let's say we have a transaction in which Alice sends Bob ten tokens and we have the state in","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=494s"},{"start_seconds":503,"text":"which Alice has don't ten tokens and Bob has zero tokens if you get that state and apply this transaction the state you're supposed to get is the state in","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=503s"},{"start_seconds":512,"text":"which Alice has zero tokens and walk has ten and so any block which apply state in this way is valid what can happen is","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=512s"},{"start_seconds":520,"text":"there could be a block produced in which you know in initial state Alice had ten plugins the transaction was applied and now Bocas thousand that is an invalid","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=520s"},{"start_seconds":529,"text":"state transition and you probably often heard about forking it is he recently had a fork but","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=529s"},{"start_seconds":537,"text":"you're unlikely to have heard about invalid state transition attacks that is because in non Charlotte blockchains invalid state transition is impossible","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=537s"},{"start_seconds":545,"text":"the reason for that is imagine that we actually have majority of health power or majority of stake belonging to","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=545s"},{"start_seconds":553,"text":"malicious actors and at some point they produced an invalid block so this block a prime contains some transaction which was applied appropriately in any modern","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=553s"},{"start_seconds":562,"text":"block chain like a theorem or a Bitcoin all the validators are validating all the blocks and all the users are validating on the block so if you're","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=562s"},{"start_seconds":569,"text":"using Bitcoin you would be downloading the entire ledger and validating every single block and so the moment you see the a prime you will see it as invalid you will discard it and so you will not","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=569s"},{"start_seconds":578,"text":"even consider B prime C prime or D prime as well that blocks because you know they built on top of invalid state and so even though the honest actors built a","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=578s"},{"start_seconds":587,"text":"shorter chain every single client will know that that chain is correct however in a Charlotte block chain that problem","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=587s"},{"start_seconds":596,"text":"does exist and to understand why we first need to talk a little bit about cross heart communication right in the in the Charlotte block chain","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=596s"},{"start_seconds":604,"text":"if we have three different sharks three different block chains and let's just say for now that the only kind of transactions that exist is payment","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=604s"},{"start_seconds":612,"text":"transactions the let's say I have an account in the first shark and the person I'm sending money to has account","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=612s"},{"start_seconds":619,"text":"on the second shirt there is nobody who validates both chains and so the transaction cannot be in the first block because nobody because because then the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=619s"},{"start_seconds":627,"text":"validators can I change the state of the second of the destination account and the transactions can the transaction cannot really be on the second chain because then the validator cannot change","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=627s"},{"start_seconds":636,"text":"the state of the of the sender and so what needs to happen is this there's got to be some way of change to communicate","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=636s"},{"start_seconds":643,"text":"and the way it's usually done is through this concept of receipt where with the transaction that touches multiple shirts","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=643s"},{"start_seconds":650,"text":"it originates in one of the shirts for example in a Chardon which let's say Elias is sending money to Bob the transaction will originate and the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=650s"},{"start_seconds":659,"text":"Alice's chart it will debit her account and then the validators of this chart will send some sort of receipt to the second shirt with signatures confirming that the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=659s"},{"start_seconds":668,"text":"transaction was correct and then in the second chart the validators will verify the signatures and apply the transaction","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=668s"},{"start_seconds":675,"text":"to Bob now with crusher transactions in place what can happen is if we manage to corrupt short one I can produce","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=675s"},{"start_seconds":683,"text":"completely invalid block block B in which we mean some tokens out of thin air and then later will produce a block","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=683s"},{"start_seconds":691,"text":"C which otherwise loops correct in Block C no transition is applied improperly and from that block with sending those tokens which were minted","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=691s"},{"start_seconds":698,"text":"to another shirt the second chart has no way to validate the entirety of the first shirt right they maybe are able to","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=698s"},{"start_seconds":706,"text":"validate Block C but they cannot validate the entire history because you know if there's thousand shards and everybody is sending cross card transactions - sure - sure - cannot","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=706s"},{"start_seconds":714,"text":"validate all the shirts and so - sure sure - has no way to to know that the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=714s"},{"start_seconds":721,"text":"that the block wasn't correct so one way to solve that was offered is let's have let's not allow cross-court","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=721s"},{"start_seconds":729,"text":"communication between English arts let's say that only sound shirts can communicate between each other and the chard every shirt validates all the shirts that let this neighboring all its","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=729s"},{"start_seconds":738,"text":"neighbors right this way if there's a cross heart communication and let's say Block B was invalid then","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=738s"},{"start_seconds":747,"text":"chart two because third one is its neighbour validated all the blocks and the short one and it knows block D is invalid so this attack is no longer possible and if someone wants to send","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=747s"},{"start_seconds":756,"text":"money from one star to another which is not a neighbor they will just send through multiple shirts and this is what the problem occurs again right because","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=756s"},{"start_seconds":764,"text":"well like corrupting short one is not sufficient anymore but now corrupting just two shirts which is still significantly less than full system is sufficient who will corrupt corrupt","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=764s"},{"start_seconds":772,"text":"child one produced invalid block sent across drug transaction - sure - which is also owned by us so we will accept the cross card transaction and we will","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=772s"},{"start_seconds":780,"text":"send it to chart 3 chart 3 can only valid a chart - which are - is completely valid there is no there is no","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=780s"},{"start_seconds":787,"text":"invalid state transition in the shark - so how it is solved today well one way or how it can be solved in the future","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=787s"},{"start_seconds":795,"text":"one way is you can pretend the does not exist you can you can try to build your security in such a way that you believe you convinced that at no","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=795s"},{"start_seconds":804,"text":"point the validator set in any shard will be corrupted alternatively you can you can try to have some cryptographic","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=804s"},{"start_seconds":811,"text":"proof that the entire blockchain is correct when you're sending a crosshair transaction there is such cryptographic primitives they called snarks but today","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=811s"},{"start_seconds":820,"text":"they not quite usable they were very slow right they're usable for private transactions so zeeky zeeky zeeky caches using them but to valid to prove that","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=820s"},{"start_seconds":828,"text":"the entire blockchain is correct that is not that is not quite there yet the long term that will be the best solution possible the most promising solution","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=828s"},{"start_seconds":838,"text":"today is called fisherman where the idea is that for as long as you have at least one honest validator that one is validator would be monitoring with what","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=838s"},{"start_seconds":846,"text":"is happening in other blocks in other shirts and if they observe the distant cross card transaction happening from from the shirt that they validating from","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=846s"},{"start_seconds":854,"text":"the state that they they have not observed before so actually the scenario will be the charges completely overtaken","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=854s"},{"start_seconds":862,"text":"there's more than two-thirds of malicious actors they produce an invalid blog they never show this blog to the owners validator they produce a valid block on top of it otherwise valid","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=862s"},{"start_seconds":870,"text":"besides the fact to disapproving an invalid block initiate crash or transaction the owners validator will notice that this is happening download","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=870s"},{"start_seconds":879,"text":"the blocks identify that this block is invalid and create some proof some some small proof that that this is happening","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=879s"},{"start_seconds":886,"text":"and send this proof to the short torso now at this point sure to will quickly verify the proof the proof is very lightweight and know that the cross card","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=886s"},{"start_seconds":894,"text":"transaction that was initiated is incorrect the problem is that monitoring the shard identifying this this problem happened","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=894s"},{"start_seconds":902,"text":"done loading the block validating it it takes a lot of time and so the challenge period the period during which the period that far to will be waiting after","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=902s"},{"start_seconds":911,"text":"crucial transaction was initiated before they shorten that no challenge will arrive should be pretty long right so so let's say both shards of producing","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=911s"},{"start_seconds":919,"text":"blocks every three seconds so cross our transaction can really without fishermen be finished in like six seconds with the fisherman it needs to be on the order of","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=919s"},{"start_seconds":928,"text":"minutes so the system becomes significantly slower but the bigger problem is that this blog","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=928s"},{"start_seconds":934,"text":"be the own it Valley that actually wouldn't load the blog be the problem is that the dishonest validator might have","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=934s"},{"start_seconds":941,"text":"never even published it because in the short - they they cannot be done loading all the blog's from all the shirts if these thousand shirts and there's 1","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=941s"},{"start_seconds":950,"text":"megabyte blog produced every few seconds in every shirt you would have to download one gigabyte of data every few seconds that is infeasible so char -","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=950s"},{"start_seconds":957,"text":"cannot you 11 10 loading this block to verify the transaction and so this block might not even exist anywhere nobody","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=957s"},{"start_seconds":964,"text":"nobody nobody besides the dishonest validators might have seen that block and so in this case the onus validator cannot download it cannot validate it","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=964s"},{"start_seconds":972,"text":"cannot craft the challenge that problem is called data availability problem very few protocols actually really work on","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=972s"},{"start_seconds":982,"text":"this problem right the only protocols that that that publicly say that they work on this problem that they believe","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=982s"},{"start_seconds":990,"text":"it exists is polka dot and the etherium most of the other protocols they there's nothing in their technology which","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=990s"},{"start_seconds":998,"text":"actually tries to to go to work with data availability to work around the data availability problem and yeah and","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=998s"},{"start_seconds":1006,"text":"the problem is that it's not only cross hard communication that is broken is also snapshotting to the beacon chain you only snapchatting to the beacon chain the half of the latest block you","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1006s"},{"start_seconds":1014,"text":"never reveal the actual block right so the onus validators might not be able to download the block of the shard was corrupted and they cannot","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1014s"},{"start_seconds":1021,"text":"so the short can completely stalled and so there are some proposals the usual they mostly work around this idea of","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1021s"},{"start_seconds":1028,"text":"using erasure codes and providing some extra redundancy in the data that is available and then data is sample split","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1028s"},{"start_seconds":1037,"text":"most of those are the very early stages and so if you interested in this particular topic you can you can search for a paper by the italic called","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1037s"},{"start_seconds":1045,"text":"route proofs and let me finish my presentation with quick overview of how","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1045s"},{"start_seconds":1054,"text":"people actually build sharding today right so cosmos is one example of something which is very similar to Chardon right they have a cosmos hub","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1054s"},{"start_seconds":1061,"text":"which is which looks a lot like a beacon chain and they have zones which look a lot like shorts few things about coordinates","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1061s"},{"start_seconds":1070,"text":"is that first validators do not rotate around zones validators assigned to the zone and it reduces the security a","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1070s"},{"start_seconds":1078,"text":"little bit and they the way cause most positions it is that the users of the zone have for themselves their responsibility to identify how many","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1078s"},{"start_seconds":1086,"text":"validators exist there how much they trust them and how much they trust the zone and then cosmos the cross shard communication cross down communication","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1086s"},{"start_seconds":1094,"text":"is carried out through a protocol called IBC inter blockchain communication which allows you to move assets around between","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1094s"},{"start_seconds":1102,"text":"the designs but it doesn't really allow you to do any more complex code execution polka-dot is somewhat similar","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1102s"},{"start_seconds":1110,"text":"to to cosmos however the big difference is that a the actual runtime of every","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1110s"},{"start_seconds":1117,"text":"blockchain is stored on on the chain so in cosmos every blockchain is is completely different binary in polka-dot in principle it can be built in such a","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1117s"},{"start_seconds":1125,"text":"way that the para para chains of the same binaries they don't have to be but they could and the runtime of the para chain is stored in the own chain meaning","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1125s"},{"start_seconds":1134,"text":"that validators in principle could rotate however in what polka-dot is planning to ship this year they will not be rotating so polka dot will have the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1134s"},{"start_seconds":1142,"text":"same security as cosmos when they ship and poke a lot a lot using fisherman for the securities of the you don't really","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1142s"},{"start_seconds":1149,"text":"have to trust that you don't need to trust that two-thirds of the validator disk is our honest it doesn't have to","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1149s"},{"start_seconds":1158,"text":"trust that ninety-nine per se it's enough to try that is atleast one honest validator / / shard I silica is you probably heard of silicon the concept of","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1158s"},{"start_seconds":1166,"text":"sharding silica only delivered state the only short processing meaning that every node to have to store the entire state which","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1166s"},{"start_seconds":1173,"text":"is a huge limitation and also they for cross card transactions they completely shut down the entire the completely shot shutdown sharding and process them","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1173s"},{"start_seconds":1182,"text":"serially and the theorem serenity near they very similar unfortunately running out of time but they effectively","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1182s"},{"start_seconds":1192,"text":"everything I I described today this is effectively how etherium serenity and near work","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1192s"},{"start_seconds":1197,"text":"and cool so let me finish here quick thing is that we have whiteboard series","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1197s"},{"start_seconds":1206,"text":"where we talk to two founders of different protocols we talk to cosmos to solana where we talk a lot about technology so check it out","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1206s"},{"start_seconds":1213,"text":"near yes this YouTube and also next Tuesday we're revealing our deaf net if you in the city google it it's or the","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1213s"},{"start_seconds":1221,"text":"link is near a guys there's definite unfortunate don't have it on the slide thank you","url":"https://www.youtube.com/watch?v=UTs9ZkX401M&t=1221s"}],"youtube_url":"https://youtube.com/watch?v=UTs9ZkX401M","url":"https://near-pedia.xyz/videos/UTs9ZkX401M"}