Block and chunk producer selection algorithm in simple nightshade

This post primarily focuses on how block and chunk producers will be selected once sharding is enabled.

Background

Due to the complex nature of challenges, we decide to enable sharding first with all block producers tracking all shards and we call this version The Simple Nightshade. Naturally it implies that block producers will have to run more expensive hardware, which harms the decentralization of the network. Fortunately, not everyone who stakes will have to track all shards — chunk producers, who are solely responsible for producing chunks in some shard, will only need to track one shard and therefore can run with the same hardware requirement as what we require today. Furthermore, we can have a large set of chunk producers to increase the number of validators, thereby improving the decentralization of the network.

Proposal

To ground the proposal in concrete numbers, let’s say we have up to 100 block producers and up to 300 chunk producers (these numbers are subject to change). When validators stake, they specify whether they would like to become a block producer or a chunk producer. At the end of each epoch, the proposals are collected and we calculate the new block and chunk producer set by

• If there are N proposals for block producers, the top min(100, N) proposals are selected.
• If there are M proposals for chunk producers, then we pool the proposals for block producers and chunk producers together, order them by their stake in decreasing order, and take the first min(100 + 300, N + M) proposals to be chunk producers.

We will not discuss in this post how block and chunk producers map to each height. There is already an existing proposal on this topic and we can mostly follow the same proposal. Some important things to note here:

• Each shard should have at least a few different chunk producers so that even if a chunk producer is offline, the shard can still keep operating.
• As we can see from the selection process mentioned above, block producers can and in most cases will be chunk producers as well.
• Those who stake to become chunk producers should only be responsible for one shard so that there is no unpredicted increase in hardware requirement. On the other hand, block producers are free to be chunk producers in multiple shards since they already track every shard and it takes very little extra effort to produce chunks.

The total reward will be split equally between block producers and chunk producers, i.e, block producers will together earn 50% of the reward and the same with chunk producers. For each individual block producer, their reward will be calculated as

total_block_producer_reward * adjusted_online_ratio * validator_stake / total_block_producer_stake

Similarly, for each individual chunk producer, their reward will be calculated as

total_chunk_producer_reward * adjusted_online_ratio * validator_stake / total_chunk_producer_stake

where adjusted_online_ratio is computed as

min((online_ratio - 0.9) / (0.99 - 0.9), 1)

Notice that in the majority of cases total_chunk_producer_stake will be the same as the sum of all stakes between block producers and chunk producers. Therefore for someone with a large amount of stake, per capita return is strictly better if they stake for block producer.

Drawbacks

• Implementation requires adding a new action StakeForChunkProducer. Because of this, the current staking pool implementation will not work directly for chunk producers and there needs to be a separate staking pool implementation. However, in the longer term this may not be a huge concern since chunk producers will likely not require a lot of tokens and they may be able to stake their own tokens without needing to pool tokens from other token holders.

Perhaps this should be cross-posted in http://gov.near.org/c/gov/6

Discussed with @birchmd and Misha yesterday again about this. We realized that

• The scheme that I proposed above will burn some reward due to the way total chunk producer stake is computed, which is not ideal.
• We want to incentive people with enough stake to become block producers to ensure the security of the entire system. This means that if the per capita return is the same for both chunk producers and block producers, since chunk producers run cheaper hardware, they would be more profitable.
• In the proposal above chunk producers are chunk producers for only one shard. This is to reduce the complexity of the overall design. However, Misha pointed out that it may be desirable to incentivize chunk producers to validate some number of shards that is roughly proportional to their stake.

@eatmore I remember when we discussed this during one of the chain syncs, you had some ideas as well. Do you mind sharing your thoughts here?

Bowen:

The scheme that I proposed above will burn some reward due to the way total chunk producer stake is computed, which is not ideal.

Can you explain why the reward gets burnt, and how much approximately?

Bowen:

We want to incentive people with enough stake to become block producers to ensure the security of the entire system. This means that if the per capita return is the same for both chunk producers and block producers, since chunk producers run cheaper hardware, they would be more profitable.

The initial proposal explicitly says that it is more beneficial to be a block producer, and here you assume that it is not. I find the argument in the original proposal sound, why here you assume it might not be more beneficial to be a block producer if one has sufficient stake to be one?

Bowen:

In the proposal above chunk producers are chunk producers for only one shard. This is to reduce the complexity of the overall design. However, Misha pointed out that it may be desirable to incentivize chunk producers to validate some number of shards that is roughly proportional to their stake

What is the advantage of making chunk producers validate proportional to their stake?

First some nomenclature:

• block producers (BPs) produce blocks
• chunk producers (CPs) produce chunks (for some shard)

I will assume all BPs are also CPs for all shards (I think that makes the most sense). There also exist chunk-only producers (COPs) (that is the whole point of this proposal). I think it is important to distinguish BPs and COPs, and note that CPs = BPs \union COPs.

alexatnear:

Can you explain why the reward gets burnt, and how much approximately?

The reason some is burned is because the “chunk producer” reward is really the reward for COPs, but the total stake used in the denominator is the total CP stake (i.e. including both COPs and BPs). This was done on purpose to ensure that BPs always earn more reward per unit stake.

alexatnear:

why here you assume it might not be more beneficial to be a block producer if one has sufficient stake to be one?

We are not assuming Bowen’s proposal does not make it correct to always be a BP if possible, I think that it does. The only point is that whatever scheme we choose must have this property, otherwise we will not have enough block producers.

alexatnear:

What is the advantage of making chunk producers validate proportional to their stake?

I think having a correlation between hardware and stake is a nice property. Already we want BPs to have more hardware than COPs, and also more stake too. Now, if the minimum stake to become a BP is T, and I have T-1 staked as a COP, then I am in an awkward position where I could probably become a BP if I wanted to (getting a little bit more stake is likely possible), and it should improve my net return (assuming the rewards scheme is working properly). But I must upgrade my hardware first from something that can do one shard all of a sudden to something that can do all shards. That feels like a discontinuity which could be smoothed out if we had COPs with very high stake validating for multiple shards.

Another nice feature is being able to unify BP and COP proposals. All proposals would be to be a CP for some number of shards. BPs are those that are CPs for all shards. It is not clear exactly how this would work in terms of how proposals are accepted/prioritized, but I think could be a nice foundation for a clean solution.

birchmd:

The reason some is burned is because the “chunk producer” reward is really the reward for COPs, but the total stake used in the denominator is the total CP stake (i.e. including both COPs and BPs). This was done on purpose to ensure that BPs always earn more reward per unit stake

If total_chunk_producer_stake includes the stake of block producers, then block producers should also receive chunk producer reward, in addition to block producer reward, and then no stake is burnt.

birchmd:

I think having a correlation between hardware and stake is a nice property. Already we want BPs to have more hardware than COPs, and also more stake too. Now, if the minimum stake to become a BP is T, and I have T-1 staked as a COP, then I am in an awkward position where I could probably become a BP if I wanted to (getting a little bit more stake is likely possible), and it should improve my net return (assuming the rewards scheme is working properly). But I must upgrade my hardware first from something that can do one shard all of a sudden to something that can do all shards. That feels like a discontinuity which could be smoothed out if we had COPs with very high stake validating for multiple shards.

What’s wrong with this discontinuity, though?

With the model in which I am explicitly stating whether I want to validate all shards, or just one, I know exactly what hardware I need as a pool. If the shards are assigned to me proportional to the stake, then I need to be constantly upsizing / downsizing my hardware with delegations coming and going.

The way I understand Bowen’s proposal is the following, in pseudocode:

struct Proposal {
   account_id: AccountId,
   want_to_be_bp: bool,
   stake: Balance,
}

fn get_block_producers(proposals: Vec<Proposal>) {
  proposals.filter(|x| x.wants_to_be_bp).sort(stake).reverse().take(100)
}

fn get_chunk_producers(proposals: Vec<Proposal>) {
  proposals.sort(stake).reverse().take(400)
}

// here `block_producers` is the top 100 based on `get_block_producers`
fn compute_block_producer_reward(block_producers: Vec<Proposal>, total_reward: Balance) {
  let total_stake = block_producers.map(|x| x.stake).sum();
  let total_bp_reward = total_reward / 2;
  block_producers.map(|x| x.stake * total_bp_reward / total_stake)
}

// here `chunk_producers` is the top 400 based on `get_chunk_producers`
fn compute_chunk_producer_reward(chunk_producers: Vec<Proposal>, total_reward: Balance) {
  let total_stake = chunk_producers.map(|x| x.stake).sum();
  let total_cp_reward = total_reward - total_reward / 2;
  chunk_producers.map(|x| x.stake * total_cp_reward / total_stake)
}

It doesn’t appear that any stake gets burnt, the hardware requirements are known in advance, and block producers get significantly higher reward that chunk producers.

alexatnear:

If total_chunk_producer_stake includes the stake of block producers, then block producers should also receive chunk producer reward, in addition to block producer reward, and then no stake is burnt.

The concern with allowing BPs to “double-dip” by also getting reward from the chunk producer side is that there will be no incentive for anyone to be chunk producers. In Bowen’s proposal the fraction of reward going to BPs and COPs is capped so that there is an incentive for both roles to be filled. If there are all BPs and no COPs then half the reward is unclaimed, creating an incentive to have COPs. On the other hand, if there are many COPs, and not very many BPs then upgrading a large-stake COPs to a BP could increase its reward per unit staked.

alexatnear:

I need to be constantly upsizing / downsizing my hardware with delegations coming and going

Yes, I agree this is an unfortunate feature of stake and hardware having no intrinsic relationship. It could be that there is not an elegant way to have it work well.

there will be no incentive for anyone to be chunk producers.

Technically, we don’t need an incentive to be a chunk producer: someone becomes a chunk producer if they don’t have enough stake to be a block producer, so they don’t really have a choice.

But it does matter when it comes to delegating: delegating to those who can’t be a block producer will be most certainly less profitable than to someone who can be a block producer.

Why not have total_chunk_producer_stake only include those who doesn’t want to be a block producer? It appears to create a natural market in which approximately the same stake will end up in both buckets, with a smaller number of larger pools as block producers, and a larger number of smaller pools as chunk producers?

alexatnear:

Technically, we don’t need an incentive to be a chunk producer: someone becomes a chunk producer if they don’t have enough stake to be a block producer, so they don’t really have a choice.

But it does matter when it comes to delegating: delegating to those who can’t be a block producer will be most certainly less profitable than to someone who can be a block producer.

Yes, the delegation is an important piece. If there is no incentive to be a COP, then rather than several token holders each create a COP, they could pool their resources to create one BP (e.g. via out-of-band cooperation or via delegation), and this could be a more profitable configuration, which would be bad for decentalization.

alexatnear:

Why not have total_chunk_producer_stake only include those who doesn’t want to be a block producer?

If the COP reward were only divided among COPs (and none of it burned), and there was equal stake in both BP and COP roles, then the reward per unit staked would be equal on both sides. However, there is greater overhead on the BP side because they need to run more expensive hardware to validate all shards, thus the profit on the BP side is lower. Therefore, it may be more profitable for some BPs to become COPs.

I suppose it’s not clear how many BPs could switch to COPs and be more profitable. It could be that it is acceptable to have there be more stake on the COP side as long as sufficient stake remains on the BP side. Maybe I should try to estimate what the equilibrium stake distribution looks like under some assumptions of cost overhead.

@alexatnear

Edit: I realized I made a mistake in my approximate calculation yesterday. The simple result (still using unrealistic assumptions like zero variance in stake distribution) is that the stake fraction is equal to the cost fraction (i.e. 10% higher cost for BPs means 10% more stake on the COP side). I’ll still do a more detailed calculation today.

Here is an approximate calculation of the equilibrium stake fraction (COP over BP) as a function of the increased hardware cost (BP over COP), assuming that COPs get all the total_chunk_producer_stake, as you suggested. It is only approximate because it makes a variety of bad assumptions, such as the variance of the stake distribution being zero. I’ll do a more precise calculation tomorrow, but I thought I would share this preliminary result.

When the hardware costs are the same the stake ratio is 1 (i.e. equal total stake for both COP and BP), as expected. But this changes very rapidly. Even with a 10%-20% increase in hardware cost, there is ~100x more stake on the COP side. This seems unacceptable to me because the stake on the BP side is what provides security to the network.

It will be interesting to see if a more precise calculation changes these results significantly.

In your model what percentage of the pool profits go towards offsetting hardware costs? That sounds like an important parameter (e.g. if it’s a negligible amount, naturally the ratio will remain 1).

Also note that block producers can set a higher fee (for example it’s not infeasible to expect that BPs will average 10-15%, and CPs will average 1-5% fees).

Sorry, I updated the post to fix a mistake I made yesterday. The stake fraction is much less extreme now.

alexatnear:

In your model what percentage of the pool profits go towards offsetting hardware costs?

I’m currently assuming a profit of 0 (i.e. return = cost). But we could sub in something closer to reality here.

alexatnear:

Also note that block producers can set a higher fee (for example it’s not infeasible to expect that BPs will average 10-15%, and CPs will average 1-5% fees).

This is a good point. The model currently does not include delegation (i.e. I am assuming BPs and COPs get full return from their share of the stake).

Another way we can tune things to make the stake fraction equal is to not split the rewards evenly. If f is the fraction of the reward allocated to BPs and x is the multiplicative factor increasing the BP hardware cost (i.e. COPs pay C and BPs pay xC), then the simple model predicts there is the same amount of BP and COP stake when f = x / (x + 1) or equivalently x = f / (1 - f). E.g. when f = 0.5 then x = 1, which makes sense since this corresponds to equal sharing of rewards, so they must have equal costs. E.g. If x = 1.2 (BPs pay 20% more in hardware) then we set f = 6 / 11 (i.e. BPs get ~55% of the reward instead of 50%) to obtain an equal amount of stake in both BP and COP roles.

alexatnear:

Also note that block producers can set a higher fee (for example it’s not infeasible to expect that BPs will average 10-15%, and CPs will average 1-5% fees).

Are you assuming that the per capita return is the same here? If so, why would delegators choose block producers with a much higher fee? Is it because they think block producers are more reliable?

Without doing any models or math, my gut feeling is that if you separate BP and CP rewards completely, it will settle on something along the lines of there being 10% more stake staked to Chunk Producers, but Block Producers having ~15% fees compared to Chunk Producers having ~5% fees, thus (a) making it approximately the same yield for the delegators, and (b) Block Producers having the extra fees offsetting higher hardware costs.

I don’t think delegation is very important when it comes to computing how much stake ends up in each role. Delegators have the same choice as anyone running a node (do I put my stake behind a block producer or a chunk-only producer?), and if BPs do have higher fees then the setup is even similar where the overhead for backing a BP is higher than backing a COP (the difference of course is the overhead is a fraction of the earnings rather than a constant amount). So then a similar model would apply and we would end up with a similar distribution of stakes. So for now I am going to ignore delegation, but if you feel strongly about it then I can try including it.

I’ll write up and post the model + calculations once I am finished with them. Then others can double check my work, point out any too outlandish assumptions, etc.

I’ve completed all the analysis I think can be done with a simple model. The full write up (including the assumptions and calculations) is here.

The main result is the following:
Let f be the fraction of the total reward given to block producers and x be the multiplicative factor on the block producers’ hardware cost over chunk-only producers (i.e. if COPs pay C then BPs pay xC). The entirety of the BP reward fraction is split among BPs (as in @Bowen 's proposal) and the entirety of the COP reward fraction (1 - f) is split among COPs (with no burn, as in @alexatnear 's modification of the proposal). Then the ratio of total COP stake to total BP stake is bounded as follows

(1-f)/f <= (total COP stake) / (total BP stake) <= x (1-f)/f

From this I conclude:

• f = 0.5 (as in the original proposal) is not an acceptable value since this will skew the stake away from BPs
• f will need to be tuned depending on the value of x
• f should be a protocol parameter so that it can be tuned even after we ship this feature

Overall, though, I think @alexatnear 's modification of the proposal is a good one so long as we tune f appropriately. I like that we do not need to burn reward as in Bowen’s original proposal and I like that the parameter f gives us explicit control over the security (total BP stake) vs decentralization (total CP stake) trade off.

Sounds good. I believe x also depends on the number of shards we have and maybe there is a way to dynamically adjust the value of x based on overall system load, but we can go with a protocol level constant for now.

To summarize the discussion that just happened on the Chain sync:

Michael’s model concludes that, assuming that CPs and BPs stake their own assets, the ration of |CPs| to |BPs| will be between (1-f)/f and x(1-f)f where f is the percentage of stake that goes to BPs, and x is the price difference in hardware. Thus, with f= 0.5, the ratio will be between 1 and x. Assuming x is somewhat high, the ratio might get pretty high.

I claim that if we incorporate the delegation fees, the ratio will get very close to (f-1)/f. Specifically, see my comment above:

alexatnear:

Without doing any models or math, my gut feeling is that if you separate BP and CP rewards completely, it will settle on something along the lines of there being 10% more stake staked to Chunk Producers, but Block Producers having ~15% fees compared to Chunk Producers having ~5% fees, thus (a) making it approximately the same yield for the delegators, and (b) Block Producers having the extra fees offsetting higher hardware costs.

In this model say f is 50%, so the same reward is paid to the block producers and the chunk producers. Say the total stake of BPs is bs and the total stake of CPs is cs, and their fees are correspondingly bf and cf. Say my stake, as a delegator, is negligibly small compared to b and c. Then if I delegate to one of the BPs, I will receive half_reward * (1 - bf) / bs, and if I delegate to one of the CPs, I will receive half_reward * (1 - cf) / cs. Correspondingly, the stakes will converge to values such that bs / cs = (1 - bf) / (1 - cf), and with my assumed rates of 15% and 5% it converges to the ratio of 0.89, which is acceptable, while block producers receive almost 3x more reward (0.89 x 3 specifically).

@alexatnear @Bowen

I’ve updated my model to include delegation. The full write up is here.

The final result is:

bp_cop_stake_ratio_bound835×116 5.21 KB

where f and x are as before, S_COP is the total chunk-only producer stake (including delgation), S_BP is the total block producer stake (including delegation), D_COP is the total stake delegated to chunk-only producers (note: D_COP <= S_COP), and \phi_COP is the delegation fee charged by chunk-only producers.

Notice the upper bound is now a linear combination of two terms. In the limit of no delegation (D_COP = 0) we recover the result of the original model (a good sanity check). We also recover the original result in the limit \phi_COP -> 1, which is also reasonable (if the fee is 100% then there is no reason to delegate so we should recover the original result).

In the limit where the total stake is dominated by delegation (S_COP ~= D_COP), then we have an upper bound given by ((1-f)/f)(1 + (x-1)\phi_COP). I haven’t fully thought through the implications of this bound yet, but I don’t like the idea that the stake ratio can be constrained based on the whims of the chunk-only producers.

I’ll think about the implications of this result some more. But let me know if you have any thoughts on it as well.

Could you explain why the upper bound is asymmetric, i.e, why does it only rely on chunk only producers?

This is just the way the algebra shakes out. The upper bound comes from the following condition: in equilibrium, it will not be more profitable for any chunk-only producer to switch to being a block producer. The lower bound comes from the symmetric condition: in equilibrium, it will not be more profitable for any block producer to switch to being a chunk-only producer.

In the lower bound case, the terms involving BP quantities (like D_BP / S_BP) cancel out which results in a cleaner expression. In the upper bound case they do not cancel out, so there are chunk-only producer quantities which remain.

The reason things cancel out in one case but not the other is because there is an inherent asymmetry in the setup in the cost term. In both cases the cost term takes the form (x-1)C, but it appears on different sides of the inequality in the different cases. It is clear that x - 1 >= 0 and C >= 0, so the lower bound on (x-1)C is also 0. When this cost term is on the smaller side of the inequality we can simplify the expression by replacing it with its lower bound (ie 0). When the cost term is on the larger side of the inequality we replace it with its upper bound, which comes from the condition that chunk-only producers are profitable (we could instead use the condition that block producers are profitable, but this does not help in simplifying the expression). When replacing the cost term with its upper bound we pick up the extra terms ultimately stick around until the final result.

There isn’t much here at the “whim” of the chunk producers. Both D_{cop} and S_{cop} have some other equilibrium constraints not captures in this formula. E.g. people will not choose not to delegate to chunk producers at all (and make the coefficient on the right skyrocket to x) just to “mess with the system”, because not delegating to chunk producers at all is bad for them.

Also, S_{cop} is present on both sides of the inequality, so we are expressing the upper bound for a ratio in terms of its nominator, which is not ideal.

We also fix φ, which in itself is a subject of reaching an equilibrium (e.g. block producers might want to lower their φ to attract more delegated stake to their side?).

The real givens we have are: f, C, x, and also the total stake of the pools, and the total delegated stake.

Separately, while fixing some precise upper and lower bound is helpful, are there objections to the model in which the reward is split between block producers and chunk producers in some proportion (e.g. equally) with the information we have so far?

I suggest we proceed with building it, because intuitively, assuming high percentage of delegations, it converges to a split of stake close to the ratio of how the reward is split between the BPs and CPs, and the noise in how people actually make decisions will outweigh the all the fine-tuning we can make.

alexatnear:

Also, S_{cop} is present on both sides of the inequality, so we are expressing the upper bound for a ratio in terms of its nominator, which is not ideal.

I agree it’s not ideal, but I do like that the upper bound is expressed in terms of quantities which have some easy interpretation, so we can apply any intuition we have about them. E.g. your intuition is that D_COP / S_COP will be close to 1. I could work harder to try and untalgle everything until it is all in terms of external variables.

alexatnear:

I suggest we proceed with building it

I agree that based on the information we have so far it looks like this incentive mechanism should be fine. I’m writing a simulation of it now. This will give empirical data in addition to the analytical work I’ve already done. I think this is an important step before we move forward with a production implementation because getting the incentive mechanism wrong could be very bad for the system, so it is worth the additional time investment to have confidence it will work as intended.

I agree with Alex that we should not spend too much time optimizing the model based on unverified (and maybe unverifiable) assumptions. For example, it is hard to estimate the percentage of delegations chunk producers will get. It could be that most of their stake come from delegations, or the other way around — most people use their own stake and do not accept delegations.

Sure, my goal is to not optimize the any model parameters (e.g. f). I just want to make sure the incentive mechanism is not exploitable or creates an undesirable equilibrium. And we can test what happens (or try to analytically calculate bounds) for a wide range of possibilities, including when there is more or less delegation happening.

I’ve done some work with my simulation of the incentive mechanism and found some interesting results. The code for the simulation is on my GitHub: GitHub - birchmd/near-bp-sim

The key results are:

1. I still think the reward fraction needs to be in favour of the block producers (in the plots shown below I have f = 0.55. And it should still be a protocol parameter so that it can be adjusted in the future.
2. There is a sort of phase-transition that occurs as the chunk producers increase their delegation fee. Counter-intuitively, chunk producers pull the equilibrium stake fraction towards their side by increasing their delegation fee to a relatively high number (>50%). This is illustrated in the plots below. I think the reason this happens is as follows: there will always be some delegators on both BP and COP sides, even when COP fees are higher because the reward is finite, and if all delegated stake is on the side with lower fees it is split among many participants. Whereas, if some switch to the higher fee side then they earn more despite paying a higher fee because the reward on that side is split among fewer participants. Moreover, because the delegation fee is higher on the COP side, and the hardware costs are lower, it attracts more node operators as well. The net effect is more total stake on the COP side. It is also true that the BPs cannot re-balance the stake fraction by increasing their own delegation fee. This is probably a good thing since it means it does not create a situation where it is a race to the top.
3. In the parameter ranges we expect in reality (2 <= x <= 4, \phi_BP = 0.15, \phi_COP <= 0.4), there are more block producers than chunk producers. And even outside that range there is never more than twice as much stake on the chunk-only producer side. I.e. we can establish practical bounds 0.5 <= S_COP/S_BP <= 2, and this seems like an acceptable range.

My conclusion is that this incentive mechanism seems like it will work well for what we want to accomplish.

equilibrium_stake_ratio_11018×449 27.1 KB

equilibrium_stake_ratio_21005×454 28.7 KB

I will be the first to admit that I don’t understand everything above, but I do have a couple thoughts.

1. I saw a few comments about the cost of hardware for Block Producers vs Chunk Producers. This seems irrelevant when considering the current seat cost and the rewards lost when a validator is kicked. Even if this was reduced by 10x it still doesn’t look like a material cost.
2. If Simple Nightshade is an intermediate solution, doesn’t that inevitably hinder a final Nightshade solution?

I listened to Alex’s latest chain link video and this whole thing appears to be redesigning the consensuses mechanism on the fly. I’m not wanting to rail on anyone but this is all completely different than what was sold in the white paper and absolutely is attempting to swap out a nuclear reactor in a submarine in the middle of the pacific. You are dealing with game theory consensus mechanism changes on the fly.

The change to nightshade lite has been pretty opaque and I would have missed it if I hadn’t dropped in for latest videos.

I’m a little confused about the status of sharding with the guildnet and if that’s the version going to the main net first, then implementing the chunk producers, then implementing whatever Alex is discussing which I’ve heard designs similar to DOT or Cosmos etc

Also will re-iterate that hardware being 2% as expensive as the amount of money staked is completely appropriate. Developer time to set up an integrate a validator is going to cost far more than hardware cost.

Simple nightshade does not prevent us from going to full nightshade. Once everything is ready, we can remove the restriction that block producers need to track all shards

Georgehosterguy:

I’m not wanting to rail on anyone but this is all completely different than what was sold in the white paper and absolutely is attempting to swap out a nuclear reactor in a submarine in the middle of the pacific

I don’t think that is an accurate analogy. For large protocol changes it takes a very long time from ideation to implementation and will not be done in a hurry precisely to make sure that we don’t miss anything in the process. The discussions we had are about new ideas and it does not imply that we will necessarily implement them. As for the current status, what is deployed on guildnet is usable sharding, but it is flawed from a security point of view – slashing is not built and that is why we decided to go with simple nightshade first.

Does “tracking all shards” and “validating on all shards” mean the same thing?

Bowen:

As for the current status, what is deployed on guildnet is usable sharding, but it is flawed from a security point of view – slashing is not built and that is why we decided to go with simple nightshade first.

Could you explain how guildnet is “flawed from a security point of view”?
Are there two options: Build/Enable slashing or Simple Nightshade?

sdmurphy:

Does “tracking all shards” and “validating on all shards” mean the same thing?

Yes.

sdmurphy:

Could you explain how guildnet is “flawed from a security point of view”?
Are there two options: Build/Enable slashing or Simple Nightshade?

Yes. There are other alternatives (sync sharding for example) that may require larger changes.