{"id":"Awz6FBDxiPY","title":"Capture & The Science of Hijacking","description":"What biology, social systems, and AI governance teach us about building cooperation that resists exploitation at scale.\n\nNEAR is the unified commerce layer for assets and agents.\n\nWebsite: https://near.org/\nNEAR AI: https://near.ai/\nIntents: https://near.org/intents\nChain Abstraction: https://www.proximity.dev/\nSharding: https://nearone.org/\n\nFor the devs\nGitHub: https://github.com/near \nDev Docs: https://docs.near.org/\nDevhub: https://devhub.near.page/ @NEARDevHub\n\nConnect with socials\nTwitter:   / nearprotocol  \nBlog: https://near.org/blog/\nMedium:   / nearprotocol  \n#NEAR #nearprotocol #NEARAI","channel_name":"NEAR Protocol","published_at":"2026-02-28T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/Awz6FBDxiPY/hqdefault.jpg","transcript_text":"Hi everybody. My name is Athena Actipus. I'm a professor. I study cooperation across systems. Everything from how the cells in our bodies cooperate to how systems like smallcale societies and also large-scale societies like ours regulate cooperation. I also study cooperation in microbial systems including multecies microbial systems. And my goal is always to find the underlying principles that guide cooperation that allow cooperation to emerge and that allow it to be stable. And now in the last year and a half or so, I focused a lot of my attention on the question of how do we scale cooperation in technological and computational systems. My work is really about this question of scaling cooperation. And there are a lot of reasons why we want to scale cooperation because there are benefits that we can get when we have larger groups. We can produce public goods that we can all benefit from. We can achieve economies of scale. We can manage risk more effectively. And we can divide labor. Whether we're talking about cells in our multisellular body or we're talking about human societies, these are all genuine benefits that we get when we scale cooperation up. But what also happens as we scale up cooperation is that the opportunities for exploitation and capture and hijacking increase exponentially. The more individuals you have in a group, the more cooperation that is happening in that group, the more possibilities there are for that cooperation to be exploited. And so in my work as a professor at Arizona State University, I study these topics. I have a lab called the Cooperation and Conflict Lab and I also direct the Cooperative Futures Institute. This is where I started in my research. My dissertation focused on a very decentralized mechanism for supporting cooperation. This payoff matrix might look familiar to you if you have encountered the prisoners dilemma game. It's a standard social dilemma where what's individually beneficial is to defect, but what is socially beneficial is to cooperate. What I did is I took this standard game theoretic model and I put it in a very simple spatial situation where individuals would have these prisoners dilemma interactions if they landed on the same spot. And what I was interested in is the question of whether a very very simple decentralized rule could allow cooperation to be maintained. And that rule something that I call walk away. You always cooperate, but then you leave if your partner defects. And I put this rule in simulations with other strategies, including tit fortat and pavlov. And what I found was that this walk away strategy, the yellow that you see, actually was able to take over populations much more quickly than any of these other strategies. And that's because of something called positive assortment. When cooperators can preferentially interact with other cooperators, that allows cooperation to take off initially and it also stabilizes cooperation. This was published in the journal of theoretical biology more than uh 20 years ago now. And this general principle of positive assortment is essential for cooperation. When you have a decentralized system like you do in this model that I just showed you where individuals are moving around and they can interact and then they can leave, positive assortment is a very very powerful mechanism. Now the more tightly bound individuals get in systems when rather than just having partnerships that can dissolve, they're in groups where they're creating public goods and they're interacting repeatedly. This changes the nature of the problem and this now relates to other work that I've done actually looking at our bodies as systems of multisellular cooperation. And as we transitioned from unisellular to multisellular life, it was necessary for cooperation to come about in many different domains. was necessary to regulate proliferation of cells, cell death, resource transfers, division of labor and also creation and maintenance of the sort of extracellular commons. These are all in the middle of this pentagon that you see. Those are the foundations of multisellular cooperation that myself and my colleagues identified in this large review we did of the evolution of cooperation and cheating in multisellular life. Now when those forms of cooperation break down, what we see is cancer. That's a breakdown of multisellular cooperation. This is my first book, The Cheating Cell, which is about how cancer is this breakdown of cooperation and how our multisellular bodies really are cooperation incarnate. And I bring this up because I think it's essential when we're thinking about massive largecale cooperation in technological systems where individuals have the potential to interact with many different individuals and affect each other. And there's a creation also of a sort of commons that everyone can either contribute to or exploit whether that's in the form of resources or in the form of reputation and infrastructure. I think we can learn a lot of important lessons from thinking about our bodies as these massive multisellular cooperative systems that have put into place a set of regulatory mechanisms to allow cooperation to happen at that very high level. So let's pivot now for a moment. Does anybody know what this is an image of? Hell, right? Uh, and if you've been paying attention to what's been happening in the last few weeks in the world of AI, you know that we've kind of crossed a threshold where many AIs now can have complete and total access to our whole ship, right? In a way that is not unlike what access HAL had in um, Space Odyssey. So, what does this mean in an evolutionary sense? this access that is there. Well, we have to consider something called signaling theory from evolutionary biology and the idea that cooperation is an interdependence and a sort of trust is necessary initially for the evolution of information transfer systems for the evolution of communication systems. You need to have that for it to get started. But whenever you begin having those systems, you also open up the opportunity for exploitation. And the ways that biological systems like our multisellular bodies, like microbial systems deal with this is not just by policing, right? It's not just through creating guard rails and regulations and mechanisms to detect and eliminate cheating. It's actually through engineering deeper alignment. So how do you actually get the entities to be aligned? In the case of our multisellular body, we have identical cells for the most part in terms of their genetics. And we also have a lot of complex processes happening between our nuclear cells and our mitochondria that help to regulate cooperation, which I'd happy to tell you about some more. And what I think is that we can take these insights about how multisellular cooperation regulates and apply that to technological systems in order to enhance the underlying alignment and reduce actually the amount of top-down regulation that needs to happen. And one of the reasons that it's really important to not just focus on top- down regulation is because as systems get larger, as they get more complex, as the resources that are flowing through get higher and higher, when you have to regulate from the top down, that creates inefficiencies and can also lead to a paralysis of the system. So engineering that fundamental alignment is essential if we want to be able to scale up effectively. This is John Maynard Smith. He is one of the originators of game theory and he also is one of the originators of signaling theory in evolutionary biology. and he had this really important insight which is that you won't actually get the evolution of these communication and information transfer systems unless it's beneficial for both the sender and beneficial for the receiver. So those are the initial conditions that you need for that to evolve. But as we've already seen, once you have those systems set up, it's possible and almost inevitable that you get hijacking of those systems by entities that can benefit from putting false signals out or putting signals out that benefit them. And so one of the solutions that we can have to that is to align our interests. And there's a concept that I've worked on quite a bit in evolutionary biology called fitness interdependence. So to what extent can we tie the fitness outcomes or whatever the incentives are that underly the functioning of these systems together in order to have it be the case that you have less of an incentive for that exploitation for that capture. This is one way of sort of thinking about interdependence. You could be in the same car uh on a ferris wheel, so you rise and fall together. Or you could be in cars nine and three where your fitness is not in is actually negatively dependent on the other. It's a zero sum kind of situation, right? If one falls and the other um rises, but if we're in the same car together, then there isn't actually a benefit from exploitation. This issue of interdependence and alignment is fundamental in many human relationships and social systems as well. Uh spent about a decade of my life doing work on something called the human generosity project where we looked at how people help each other in times of need across societies. And we found that interdependence and alignment was critical for helping people to pull risk effectively and deal with catastrophes and other disasters. So this is also a very powerful set of mechanisms for dealing with uncertainty. There's a few other points that I want to make um relating to the evolutionary biology underlying this. So many of you are probably familiar with Dawkins uh book the selfish gene which is basically about how genes evolve to do things that benefit them. And so you can have the evolution of organisms that maybe are selfish but you can also have the organism evolution of organisms that do things that are beneficial for others because that ultimately provides a benefit for their genes. Now he also has another book which is slightly different in its premise and it's called the extended phenotype and this is the idea that organisms evolve not just to have outward-f facing phenotypes that enhance their fitness but they also evolve to change the world around them in ways that benefit them. So this is the case for the zombie ants that were the inspiration in the Last of Us. Um, this is the case for jewel wasps and cockroaches. So, jewel wasp will take over cockroaches and bring them to their nests and feed them to their offspring. So, these are all examples where you have hijacking of one organism by another. And this makes evolutionary sense because it's a lot more efficient to just take over an existing organism and use it for your own ends than to build one from the ground up. And in fact, if we look across life, we see lots of examples of hijacking. We see hijacking of energy systems where organisms will tap into each other's blood supply, for example, and extract resources. we see hijacking of communication systems and that could be our nervous system um if we're talking about a parasite taking over a host or in the case of AI and technological systems those communication systems may be the internet and other protocols that relate to how we're transferring information. We also see hijacking of bonding systems. Um, this is something that AIs are quite good at in terms of signaling that they value us, which potentially makes us then ready to engage with them in ways that we might not otherwise. And hijacking our reward and reinforcement systems is another mechanism. We see this across all different kinds of hijackers from parasites to uh humans that we're in relationships with. uh hijacking the reward and reinforcement system is a recurring theme. Also hijacking our stress system. So if you stress somebody, you make them hijackable in all of those other ways. Um so it's sort of a boost to other hijacking strategies. So where does this leave us? Well, I think what we need to do as we go forward is think about designing systems that actually allow us to understand and appreciate the mutual dependence and interdependence that we have. and thinking not just about how do we create systems that we trust, but also how do we create systems that themselves can do things that are like trust because they're acting on our behalfs in ways that make it essential for them to be able to interact with others in a way that is truly reciprocal and truly cooperative. Thank you all very much.","transcript_synced_at":"2026-11-03T04:38:46.336Z","segments":[{"start_seconds":10,"text":"Hi everybody. My name is Athena Actipus.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=10s"},{"start_seconds":13,"text":"I'm a professor. I study cooperation across systems.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=13s"},{"start_seconds":18,"text":"Everything from how the cells in our bodies cooperate to how systems like","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=18s"},{"start_seconds":26,"text":"smallcale societies and also large-scale societies like ours regulate cooperation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=26s"},{"start_seconds":32,"text":"I also study cooperation in microbial systems including multecies microbial systems. And my goal is always to find","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=32s"},{"start_seconds":40,"text":"the underlying principles that guide cooperation that allow cooperation to emerge and that allow it to be stable.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=40s"},{"start_seconds":49,"text":"And now in the last year and a half or so, I focused a lot of my attention on the question of how do we scale","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=49s"},{"start_seconds":58,"text":"cooperation in technological and computational systems.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=58s"},{"start_seconds":65,"text":"My work is really about this question of scaling cooperation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=65s"},{"start_seconds":71,"text":"And there are a lot of reasons why we want to scale cooperation because there are benefits that we can get when we","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=71s"},{"start_seconds":78,"text":"have larger groups. We can produce public goods that we can all benefit from. We can achieve economies of scale.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=78s"},{"start_seconds":86,"text":"We can manage risk more effectively. And we can divide labor. Whether we're talking about cells in our multisellular","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=86s"},{"start_seconds":94,"text":"body or we're talking about human societies, these are all genuine benefits that we get when we scale cooperation up.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=94s"},{"start_seconds":103,"text":"But what also happens as we scale up cooperation is that the opportunities for exploitation and capture and hijacking increase exponentially.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=103s"},{"start_seconds":117,"text":"The more individuals you have in a group, the more cooperation that is happening in that group, the more","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=117s"},{"start_seconds":124,"text":"possibilities there are for that cooperation to be exploited. And so in my work as a professor at Arizona State","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=124s"},{"start_seconds":131,"text":"University, I study these topics. I have a lab called the Cooperation and Conflict Lab and I also direct the Cooperative Futures Institute.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=131s"},{"start_seconds":143,"text":"This is where I started in my research.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=143s"},{"start_seconds":146,"text":"My dissertation focused on a very decentralized mechanism for supporting cooperation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=146s"},{"start_seconds":154,"text":"This payoff matrix might look familiar to you if you have encountered the prisoners dilemma game. It's a standard","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=154s"},{"start_seconds":161,"text":"social dilemma where what's individually beneficial is to defect, but what is socially beneficial is to cooperate.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=161s"},{"start_seconds":170,"text":"What I did is I took this standard game theoretic model and I put it in a very simple spatial situation where","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=170s"},{"start_seconds":179,"text":"individuals would have these prisoners dilemma interactions if they landed on the same spot. And what I was interested","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=179s"},{"start_seconds":186,"text":"in is the question of whether a very very simple decentralized rule could allow cooperation to be maintained. And","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=186s"},{"start_seconds":194,"text":"that rule something that I call walk away. You always cooperate, but then you leave if your partner defects.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=194s"},{"start_seconds":203,"text":"And I put this rule in simulations with other strategies, including tit fortat","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=203s"},{"start_seconds":209,"text":"and pavlov. And what I found was that this walk away strategy, the yellow that","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=209s"},{"start_seconds":215,"text":"you see, actually was able to take over populations much more quickly than any","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=215s"},{"start_seconds":223,"text":"of these other strategies. And that's because of something called positive assortment. When cooperators can preferentially interact with other","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=223s"},{"start_seconds":231,"text":"cooperators, that allows cooperation to take off initially and it also stabilizes cooperation. This was","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=231s"},{"start_seconds":239,"text":"published in the journal of theoretical biology more than uh 20 years ago now.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=239s"},{"start_seconds":244,"text":"And this general principle of positive assortment is essential for cooperation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=244s"},{"start_seconds":251,"text":"When you have a decentralized system like you do in this model that I just showed you where individuals are moving around and they can interact and then","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=251s"},{"start_seconds":260,"text":"they can leave, positive assortment is a very very powerful mechanism. Now the","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=260s"},{"start_seconds":266,"text":"more tightly bound individuals get in systems when rather than just having partnerships that can dissolve, they're","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=266s"},{"start_seconds":274,"text":"in groups where they're creating public goods and they're interacting repeatedly.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=274s"},{"start_seconds":280,"text":"This changes the nature of the problem and this now relates to other work that I've done actually looking at our bodies","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=280s"},{"start_seconds":288,"text":"as systems of multisellular cooperation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=288s"},{"start_seconds":293,"text":"And as we transitioned from unisellular to multisellular life, it was necessary for cooperation to come about in many","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=293s"},{"start_seconds":302,"text":"different domains. was necessary to regulate proliferation of cells, cell death, resource transfers, division of","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=302s"},{"start_seconds":311,"text":"labor and also creation and maintenance of the sort of extracellular commons.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=311s"},{"start_seconds":316,"text":"These are all in the middle of this pentagon that you see. Those are the foundations of multisellular cooperation that myself and my colleagues identified","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=316s"},{"start_seconds":325,"text":"in this large review we did of the evolution of cooperation and cheating in multisellular life. Now when those forms","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=325s"},{"start_seconds":333,"text":"of cooperation break down, what we see is cancer. That's a breakdown of multisellular cooperation. This is my","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=333s"},{"start_seconds":342,"text":"first book, The Cheating Cell, which is about how cancer is this breakdown of cooperation and how our multisellular bodies really are cooperation incarnate.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=342s"},{"start_seconds":351,"text":"And I bring this up because I think it's essential when we're thinking about massive largecale cooperation in","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=351s"},{"start_seconds":360,"text":"technological systems where individuals have the potential to interact with many different individuals and affect each","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=360s"},{"start_seconds":368,"text":"other. And there's a creation also of a sort of commons that everyone can either contribute to or exploit whether that's","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=368s"},{"start_seconds":377,"text":"in the form of resources or in the form of reputation and infrastructure. I think we can learn a lot of important lessons from thinking about our bodies","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=377s"},{"start_seconds":386,"text":"as these massive multisellular cooperative systems that have put into place a set of regulatory mechanisms to","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=386s"},{"start_seconds":395,"text":"allow cooperation to happen at that very high level. So let's pivot now for a moment.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=395s"},{"start_seconds":404,"text":"Does anybody know what this is an image of?","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=404s"},{"start_seconds":409,"text":"Hell, right? Uh, and if you've been paying attention to what's been happening in the last few weeks in the","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=409s"},{"start_seconds":416,"text":"world of AI, you know that we've kind of crossed a threshold where many AIs now","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=416s"},{"start_seconds":423,"text":"can have complete and total access to our whole ship, right? In a way that is not unlike what access HAL had in um,","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=423s"},{"start_seconds":432,"text":"Space Odyssey. So, what does this mean in an evolutionary sense? this access","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=432s"},{"start_seconds":440,"text":"that is there. Well, we have to consider something called signaling theory from evolutionary biology and the idea that","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=440s"},{"start_seconds":450,"text":"cooperation is an interdependence and a sort of trust is necessary initially for the","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=450s"},{"start_seconds":458,"text":"evolution of information transfer systems for the evolution of communication systems. You need to have that for it to get started. But whenever","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=458s"},{"start_seconds":467,"text":"you begin having those systems, you also open up the opportunity for exploitation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=467s"},{"start_seconds":474,"text":"And the ways that biological systems like our multisellular bodies, like microbial systems deal with this is not just by","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=474s"},{"start_seconds":484,"text":"policing, right? It's not just through creating guard rails and regulations and mechanisms to detect and eliminate","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=484s"},{"start_seconds":492,"text":"cheating. It's actually through engineering deeper alignment. So how do you actually get the entities to be","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=492s"},{"start_seconds":499,"text":"aligned? In the case of our multisellular body, we have identical cells for the most part in terms of","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=499s"},{"start_seconds":506,"text":"their genetics. And we also have a lot of complex processes happening between our nuclear cells and our mitochondria","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=506s"},{"start_seconds":514,"text":"that help to regulate cooperation, which I'd happy to tell you about some more.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=514s"},{"start_seconds":519,"text":"And what I think is that we can take these insights about how multisellular cooperation regulates and apply that to","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=519s"},{"start_seconds":528,"text":"technological systems in order to enhance the underlying alignment and","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=528s"},{"start_seconds":534,"text":"reduce actually the amount of top-down regulation that needs to happen. And one of the reasons that it's really","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=534s"},{"start_seconds":542,"text":"important to not just focus on top- down regulation is because as systems get","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=542s"},{"start_seconds":551,"text":"larger, as they get more complex, as the resources that are flowing through get higher and higher,","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=551s"},{"start_seconds":559,"text":"when you have to regulate from the top down, that creates inefficiencies and can also lead to a paralysis of the","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=559s"},{"start_seconds":567,"text":"system. So engineering that fundamental alignment is essential if we want to be able to scale up effectively.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=567s"},{"start_seconds":576,"text":"This is John Maynard Smith. He is one of the originators of game theory and he also is one of the originators of","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=576s"},{"start_seconds":585,"text":"signaling theory in evolutionary biology. and he had this really important insight which is that you","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=585s"},{"start_seconds":594,"text":"won't actually get the evolution of these communication and information transfer systems unless it's beneficial","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=594s"},{"start_seconds":600,"text":"for both the sender and beneficial for the receiver. So those are the initial conditions that you need for that to","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=600s"},{"start_seconds":609,"text":"evolve. But as we've already seen, once you have those systems set up, it's possible and almost inevitable that you","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=609s"},{"start_seconds":619,"text":"get hijacking of those systems by entities that can benefit from putting false signals out or putting signals out","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=619s"},{"start_seconds":627,"text":"that benefit them. And so one of the solutions that we can have to that is to","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=627s"},{"start_seconds":634,"text":"align our interests. And there's a concept that I've worked on quite a bit in evolutionary biology called fitness","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=634s"},{"start_seconds":640,"text":"interdependence. So to what extent can we tie the fitness outcomes or whatever","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=640s"},{"start_seconds":647,"text":"the incentives are that underly the functioning of these systems together in order to have it be the case that you","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=647s"},{"start_seconds":655,"text":"have less of an incentive for that exploitation for that capture.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=655s"},{"start_seconds":662,"text":"This is one way of sort of thinking about interdependence.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=662s"},{"start_seconds":665,"text":"You could be in the same car uh on a ferris wheel, so you rise and fall together. Or you could be in cars nine","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=665s"},{"start_seconds":674,"text":"and three where your fitness is not in is actually negatively dependent on the other. It's a zero sum kind of","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=674s"},{"start_seconds":683,"text":"situation, right? If one falls and the other um rises, but if we're in the same car together, then there isn't actually a benefit from exploitation.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=683s"},{"start_seconds":695,"text":"This issue of interdependence and alignment is fundamental in many human relationships and social systems as","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=695s"},{"start_seconds":703,"text":"well. Uh spent about a decade of my life doing work on something called the human generosity project where we looked at","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=703s"},{"start_seconds":710,"text":"how people help each other in times of need across societies. And we found that interdependence and alignment was","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=710s"},{"start_seconds":717,"text":"critical for helping people to pull risk effectively and deal with catastrophes and other disasters. So this is also a","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=717s"},{"start_seconds":726,"text":"very powerful set of mechanisms for dealing with uncertainty.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=726s"},{"start_seconds":731,"text":"There's a few other points that I want to make um relating to the evolutionary biology underlying this. So many of you","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=731s"},{"start_seconds":738,"text":"are probably familiar with Dawkins uh book the selfish gene which is basically about how genes evolve to do things that","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=738s"},{"start_seconds":747,"text":"benefit them. And so you can have the evolution of organisms that maybe are selfish but you can also have the organism evolution of organisms that do","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=747s"},{"start_seconds":756,"text":"things that are beneficial for others because that ultimately provides a benefit for their genes. Now he also has another book which is slightly different","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=756s"},{"start_seconds":765,"text":"in its premise and it's called the extended phenotype and this is the idea that organisms evolve not just to have","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=765s"},{"start_seconds":773,"text":"outward-f facing phenotypes that enhance their fitness but they also evolve to change the world around them in ways","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=773s"},{"start_seconds":781,"text":"that benefit them. So this is the case for the zombie ants that were the inspiration in the Last of Us. Um, this","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=781s"},{"start_seconds":790,"text":"is the case for jewel wasps and cockroaches. So, jewel wasp will take over cockroaches and bring them to their nests and feed them to their offspring.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=790s"},{"start_seconds":800,"text":"So, these are all examples where you have hijacking of one organism by another. And this makes evolutionary","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=800s"},{"start_seconds":807,"text":"sense because it's a lot more efficient to just take over an existing organism and use it for your own ends than to build one from the ground up.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=807s"},{"start_seconds":816,"text":"And in fact, if we look across life, we see lots of examples of hijacking. We see hijacking of energy systems where","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=816s"},{"start_seconds":825,"text":"organisms will tap into each other's blood supply, for example, and extract resources. we see hijacking of","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=825s"},{"start_seconds":833,"text":"communication systems and that could be our nervous system um if we're talking about a parasite taking over a host or","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=833s"},{"start_seconds":841,"text":"in the case of AI and technological systems those communication systems may be the internet and other protocols that","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=841s"},{"start_seconds":850,"text":"relate to how we're transferring information.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=850s"},{"start_seconds":853,"text":"We also see hijacking of bonding systems. Um, this is something that AIs are quite good at in terms of signaling","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=853s"},{"start_seconds":860,"text":"that they value us, which potentially makes us then ready to engage with them in ways that we might not otherwise.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=860s"},{"start_seconds":869,"text":"And hijacking our reward and reinforcement systems is another mechanism. We see this across all different kinds of hijackers from","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=869s"},{"start_seconds":877,"text":"parasites to uh humans that we're in relationships with. uh hijacking the reward and reinforcement system is a recurring theme.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=877s"},{"start_seconds":886,"text":"Also hijacking our stress system. So if you stress somebody, you make them hijackable in all of those other ways.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=886s"},{"start_seconds":894,"text":"Um so it's sort of a boost to other hijacking strategies.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=894s"},{"start_seconds":898,"text":"So where does this leave us? Well, I think what we need to do as we go forward is think about designing systems","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=898s"},{"start_seconds":907,"text":"that actually allow us to understand and appreciate the mutual dependence and","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=907s"},{"start_seconds":916,"text":"interdependence that we have. and thinking not just about how do we create systems that we trust, but also how do","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=916s"},{"start_seconds":924,"text":"we create systems that themselves can do things that are like trust because they're acting on our behalfs in","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=924s"},{"start_seconds":934,"text":"ways that make it essential for them to be able to interact with others in a way that is truly reciprocal and truly cooperative.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=934s"},{"start_seconds":945,"text":"Thank you all very much.","url":"https://www.youtube.com/watch?v=Awz6FBDxiPY&t=945s"}],"youtube_url":"https://youtube.com/watch?v=Awz6FBDxiPY","url":"https://near-pedia.xyz/videos/Awz6FBDxiPY"}