[REDACTED] 2024 | Perspectives on AI and Consciousness

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:05

   I'm mie I'm a writer at near and I've been lucky to know Alex and Ilia for a few years now I hope you don't need to be introduced to them after two days at redacted but they're the co-founders of near protocol and near Ai and since we're here at the end of our two days I thought it would be fun to zoom out and talk about the goals for user owned AI for GI um and kind of what the point of all of this work is pursuing um

2. 0:36

   conscious machines so just going to ask them some interesting questions and let them ask each other interesting questions and we'll see where we go um so to kick off does one of you want to try to Define AGI okay so I have my own definition I Define AGI as a machine that is better than I at everything I can do okay that's a high standard higher than most I well I I I would my definition is everything that any best person at that

3. 1:09

   skill yeah that's that's Open the Eyes definition as well yeah and Alex just assumes he's the best at everything so he might be so yeah AI that matches or surpasses human cognitive abilities so if we are pursuing AGI then I would argue we're also pursuing conscious AI so that is the premise of this fireside chat I'll also give a disclaimer that Alex asked me to give which is none of us claims to be an expert on Consciousness but I would posit these

4. 1:35

   two are experts on AI so we'll land somewhere in the middle of AI and Consciousness I guess um so people love the origin story of near and um I I guess I'm just curious for you to retell the story of why you named near AI what you did yeah so so I kind of liked I liked Singularity since the moment when I had to spend uh 16 hours with a friend of mine on the way from M to Moscow in a train and he just read it was 2005 so the singularity

5. 2:08

   near just came out he read it and so I was tortured for 16 hours where he was telling me how the all-encompassing is uh artificial intelligence as he would call it will um is everything we need uh and he was very convincing so at the end of the trip I was I was thinking a lot about it uh and this concept then then in 2012 I finally decided to read the book seven years later and I liked it a lot and then uh like my license plate has been near uh since way longer than near existed and

6. 2:39

   the the near domain I also bought way before I even contemplated the idea of starting a company right and so yeah and then when IL and I started a company they I don't know I had the domain I was like hey what do you think of near he's like yeah near sounds like a cool name you know rolls the tongue so let's go with it so it wasn't about pursuing The Singularity you just like to the name well it was okay so yeah I mean the the plan was I mean which is still the plan AI developer a researcher

7. 3:09

   AGI yeah and uh yeah effectively at the time we were thinking we're going to be the first trillion dollar company uh that was a plan in 17 no recollection now there's few other companies so maybe you know aiming 10 uh but uh yeah the idea that effectively like through software engineering we were able to actually Advance AI that was kind of the premise for near AI it was conceived in the train out of tulon to Paris uh after we got um very jet legged

8. 3:45

   very jet lag very uh ton also is an interesting place uh and so we were we were coming from the iclr conference where there was a lot of program synthesis papers and we're like this is happening it's it's clearly here and so we need to we need to capitalize on this and actually uh uh kind of front run it by getting way better data and get a way better uh effectively uh AI developer so here we are six years later but uh the mission is still there full circle

9. 4:21

   that was 2017 7 years later it's now seven years wow yeah IIA you've you obviously were in AI researcher before starting near and I know you developed an interest in an early age um what made you first get interested in Ai and and in studying it well yeah I mentioned this quite a few times but uh the movie artificial intelligence that I think came out in 200000 or 2001 so I was like 10 years old I think we had a class trip to watch

10. 4:52

    it um but uh yeah somehow it left an impression on me and then I was reading a lot of Science Fiction um and so that always kind of um seem to me clear that we need to have an AI that will be our companion it will be our partner it will be um effectively removing all the mundane tasks uh I may have thought also it will you know go trade on Market make a lot of money and then uh help to improve the life everywhere um but to me

11. 5:21

    it's always yeah I was imagining it would have some form of Consciousness um and at the same time kind of omnipresent to be able to execute any actions anywhere and so uh now as I was you know growing up teenager trying to do stuff with machine learning back then the the reality was not there but uh yeah um continued working on this kind of across working in uh machine learning field and

12. 5:56

    then joining Google focusing more on text knowledge understanding and uh pushing the envelope Transformers and then herei Alex what about you when did you decide to start working on AI you were doing something else before yeah so around 2000 14 I want to say I was I was playing an online game uh and there was a particular mechanic that required solving a simple capture uh and it was uh it was automatable even without AI

13. 6:29

    because there are websites where people can solve captures for you but they really didn't like paying for it and so yeah I then loaded there was a framework back in the day called lasagna you know you have layers in a Transformer so it's lasagna yeah and uh yeah and I remember I I I I really did not expect it to work because I think in 2014 it was not quite clear like 2012 was When de leing

14. 6:53

    started and I think two years later majority of people I definitely uh didn't quite catch up yet to what was possible and I remember I trained this neural network and I ran it and it had like 99% accuracy and I was like holy cow you know like how how could it possibly be because you know like I think as an engineer before machine learning you you sort of think of machines as rigid right and the capture it's like a it's not rigid yeah and so

15. 7:17

    and so from there I was doing a lot of AI and I was working for this company called single store and I was like okay guys I'm I'm leaving by and they were they were trying to you know uh we we have the saying in Russian which is like to stretch an owl on the globe uh so they tried to introduce a into into the work of single store but they failed so so yeah I left and they started um you know what would become near eventually um so IIA you said your

16. 7:48

    motivation is basically augmenting human intelligence and making making us more efficient is that fair do you still consider that your motivator for working on this yeah I think I mean there is a kind of multiple Dimensions obviously to where I can lead but I think the most obvious one which already has been happening for over a decade right has or 20 years now has been augmenting our intelligence right Google being the

17. 8:18

    first effectively machine Learning System that became globally available for everyone to augment intelligence to find any information you want and so I mean the current Reincarnation of that is extend of that and as this machines are becoming int more intelligent themselves they you know able to execute more complex actions uh make decisions now the where we go from here is indeed

18. 8:44

    a branch of either kind of effectively merging a human machine that's one direction or creating a new um beings that are kind of coexisting with us and I think probably given how World structured Pro all of these directions will be explored and so um but at the same time I think the our our kind of humanity is driven by selfishness and so augmenting ourselves is a way likelier and more

19. 9:18

    resource allocated path than creating another species which you know generally humanity is very good at destroying species not creating but at the same time given we have kind of you know we created digital environment for digital species to to exist there's Al an interesting opportunity for that we created cores that's why you're here GES um yeah Alex you've said that you're motivation is perhaps more defensive so

20. 9:47

    what motivates you to pursue building AGI or super intelligent AI uh I don't if it's more defensive but generally I I see what you you mean defensive I see sorry leading question you can tell your version I takes me time to catch up yeah so so generally uh I I really don't like the span of life that is uh uh allocated by default I think it's pretty short uh and with every passing day it feels shorter uh and I think AI is the uh is the more when I

21. 10:18

    was um like when I was in the train with that person I think one of the things that uh particularly was inspiring for me that that he the way he was looking at it this the to immortality right because if we can trans transfer our brains into a machine you can leave for as long as you can maintain the machine right and so you know if if if if someone wants to live much longer than

22. 10:40

    uh you know the pre-allocated 100 years that's a a more reliable path and even you know so longevity would be the second path but longevity gives you longer time span but not redundancy while if you move yourself to a machine not only you can live much longer you also have redundancy so if something happens you can uh you can move yourself restore from backup restore from a backup yeah or or like make a couple of

23. 11:04

    copies yeah so it's uh so that's what's been driving me for quite a while and still drives me so yeah ideally so yeah IIA wants to augment a human I want to uh yeah I want to fully move into the into the virtual you're ready to be a machine well I mean that that is effective like as as as we integrating closer and closer the final step of that integration you're full like if that is

24. 11:26

    indeed the the most conspiracy we good um yeah as as we become more integrated right the our intelligence may move into the digital realm uh if that's a more uh effective path to do that or we'll have a more uh conscious beings that exist independently and then uh we'll be figuring out how to negotiate with that Alex you've also said before that you are also motivated by uh beating someone potentially bad to building egi

25. 12:09

    uh can you talk a bit about that right about what like the idea that that someone if someone else builds it first they might have the wrong motivation so it's your obligation to build it so I think generally and I think it's very interesting From perspective of uh current alignment efforts if you talk to people from antropia from uh there are some other entities that pursue alignment many of them look at alignment From perspective

26. 12:35

    of how do we make models that we build like right so an Tropic has clae right and so they they say how do we make CLA to be safe uh which is not quite uh the risk I don't think the risk From perspective of entropic should be hey the CLA that clae will escape and do something harmful the actual threat we have is that some malicious actors right and some um you know there are some some

27. 12:58

    countries which we believe might be implicated in the future in doing that that some malicious actors train a malicious Ai and and use it for for for malicious purposes and naturally none of what anthropic is doing in any way is preventing that and uh mostly I think if if people actually build Ai and use it for bad not much we can do it's an entity that is smarter than us so no matter what we do uh it's

28. 13:24

    a game that we will lose but we can we can defend against certain things that people can do before before AI is reached right so a good example I like to give is that I'm pretty sure we're very close to the moment where AI will be much better than us at reading and analyzing code and because most of the like you know Linux operating system is millions of lines of code of course it has bugs but security model is effectively such that a very smart human

29. 13:48

    by looking at that code will not find them because many many smart humans looked at it and didn't find it but something smarter than human could right and so if someone has access earlier than others to to utility that allows you to do that right they could be a good guy who will just open the bug reports and Patch all of those or they could be a bad guy who will bring down the whole internet uh right and that will be a massive U you know bad event

30. 14:13

    but we can prepare to that right like we can for example invest heavily into formal verification and you write majority of the core infrastructure in a way that is formally verified so that you know that no matter how smart your opponent is uh they would not be able to break it right so so like that would be a more I think reasonable approach to safety where you try to predict uh the particular you know attack attack vectors and start patching them ahead of time AI for safety on the

31. 14:38

    way to conscious AI yeah how will we know when we have achieved conscious AI or that AGI is conscious well I think we kind of bypass the definition of Consciousness in general oh yeah I guess I didn't come back to asking that okay what's Consciousness yeah I think that's that's a question everybody is uh struggling with and I think that's why that's why things have been very hard to Define um and kind of AI space is yeah like

32. 15:09

    there's every everything from people who already thought that who thought llms even like two three years ago already conscious right and kind of hey we you know effectively killing a conscious being by like stopping the program mhm uh two people who like hey this is just a you know token parrot and just predicts the next token and it's just a uh and so I think the same question is like you

33. 15:34

    know is dogs conscious um like kind of unclear but the I think the best definition I've um kind of saw has been effectively understanding of a presence of self so this is when whatever the system is it's able to understand that it is exists within some world and it can execute actions and it can effectively uh like reflect on its own presence in the system MH uh now how to quantify that and like what's a benchmark for that yeah is a little bit

34. 16:14

    hard because the problem is if you assume um like if you test for that and it's like a talken parot it will effectively tell you everything that you think will make them conscious that's why what conscious people sound like and yeah yeah uh so and I think the the the experiment you were mentioning that if you remove everything about Consciousness from the training data and then train on it yeah it's from from Ilia yeah from from the other Ilia yeah

35. 16:45

    so if you remove all the Consciousness data and then you ask effectively are you conscious if it's able to infert that but I think all of this is yeah pretty limited um and given like we don't have introspection even in how our head works it's uh yeah it's pretty challenging but uh yeah so that that is a very interesting question where where that boundary lies yeah our Consciousness doesn't emerge in isolation either so uh an AI

36. 17:16

    that doesn't know the words for Consciousness or feelings isn't isn't necessarily emerging the same way ours does but at the same time we can think like well when you and this is to the question is like it's predicting next token everything you need like you can you know hypothesize that this is effectively what we're doing we're just predicting the next sensory input and the most effective way to do that if

37. 17:41

    understanding kind of our presence in the world and being to reflect on that and effectively Consciousness emerges as a way to the most effectively compress information and predict next toing and predict you know not just shortterm but also long term and effectively optimize all the chemicals that are driving our uh kind of body and so the the question there is like what is the effective

38. 18:05

    alternative of that in the digital system and I mean people have been trying a lot of invi man and kind of you know Video Games Etc as a way to kind of create the environment where there is uh such thing and that goes into like well let's say we create a simulation right we have a lot of AI running around and kind of um figuring outs and and yeah like you know if you control C that simulation like how do we know where

39. 18:35

    where we are on the on the spectrum of I stopped the program to I kill a bunch of conscious beings but if the person sees us to exist is he killed if he yeah he just sees us to exist in the moment yeah right it's so so we were discussing this because the the the experiment is like if you you know restart the world effectively from a snapshot and do two thing like do some you know harm to a person twice is this like did the person experience

40. 19:06

    this twice or once or uh zero time as well as if you if that person never existed have the like did that harm was done or not right yeah yeah um or like here's another thought experiment imagine that imagine that it turns out that uh people never figured out how to do anesthetics instead they figured out how how to do a drug which raises the last hour of memories so you always experience what happens but you just never remember is it actually is it actually a good pain killer or

41. 19:38

    not what about you Alex do you think we'll know when we when AGI has been achieved I I think um I I don't have a very good uh sort of opinion here but I suspect that we will we will see it from from like a from second order second order facts yeah just by interacting with it at some point you will be like well I have no doubt that I'm interacting with the conscious being there are some people who already feel that way about AI systems they're interacting with so

42. 20:09

    maybe we're already there and we don't know well I mean effectively that was a touring test right like can if you talk sufficient amount of time with this can you guess if it's a person or not that that was in a way a uh and I think like you know there's still whole like there's definitely T Test when you don't ask it hard questions is effectively past yeah so now it's all about like

43. 20:34

    when you interact with it can it uh can it effectively like the way we test that this is llm right now is like can it solve hard problems and harder problems problems that even people cannot solve anymore uh or some people cannot solve anymore and so I think the reality is like we're approaching that it's just it's still a program right so like it it totally like I actually think it's tottally possible

44. 21:01

    we have a a a machine system that is smarter than any of us individually has more advanced skills can act in this digital world right given what we're building yeah and do anything and is not conscious because it's still effectively just a machine that you know acts on the input now if you put that machine into a world World effectively give it some feedback loop and give it some value function that it it is trying to optimize that's when we

45. 21:39

    can start actually assume that some form of Consciousness may arrive and this is where like an autonomous agents is actually an interesting thought experiment because uh you you're giving them a mission right this can be anything from you know make more money on Exchange or uh you know let's save the planet and not let it um not not let Humanity destroy itself and uh you know give it some resources like money you know crypto assets and then you Loop it effectively to you know

46. 22:11

    keep thinking about what is possible and how I should do it and you let it act by deploying capital and the question there is like yeah at what stage we can consider this a Consciousness conscious uh versus just carrying out yeah versus carrying out yeah yeah yeah I mean if that's if if AGI is just like a super powerful calculator then why do you think people are so afraid of it also are we actually just go ahead well to that question I think people are afraid of people and so people project

47. 22:50

    the people's fear to AI yeah people kill people machines don't kill people so far yeah yeah there's computers that you know on purpose killed people it's mostly yeah but but I think there's a couple things one is that we gen first of all people are generally afraid of competition because you know someone you know you know some someone who competes with you can can can can make you IR relevant uh and I think historically people with power quite frequently were

48. 23:22

    malicious right so if we're thinking of AI as a you know extremely powerful entity it makes sense that we would have some in inherent inherent fear of it MH um yeah I mean Jeffrey Hinton basically says that if um he's scared of super powerful AI because it can start manipulating people but it's like if it's just carrying out the actions of its it will be a person manipulating people using I mean that's already happening it's not this is not a yeah like you can already do that there's Twitter Bots

49. 23:56

    already trying to do this this this has been a thing before even before generative AI so like that's not a not a new problem yeah yeah and like it will get more and more sophisticated and you know create like you can you know imagine it creating more advanced strategies but like somebody gave the it this Mission effectively at this point mhm do you think we'll see is it likelier to emerge as one very powerful centralized system or more like a kind of emergent

50. 24:27

    swarm of many small agents or many small models um do you have an idea of what what those systems will look like as they grow yeah it's a bit of a false dichotomy in the sense that it could be what I think it would be is it would be a very large model a single model great but that model will be decentralized I guess we have heard that from you before so what about youa I I think it's the same as um um how we are effectively you

51. 25:01

    know a big model and like individually we are I assuming we're conscious and uh you know capable but the reality is the if you on the metal level there's a society being that is a collective of all of the people and actually Society being is way more resilient it have been surviving you know for probably you know tens of of thousands of years now it's able to main you know kind of uh in different ways Advance itself and and upgrade itself

52. 25:37

    and make each cell of itself live longer through technology and Innovation and so like from a evolutionary biology right you can think we're cells and then Society is the body so probably similar thing will happen here again given what we're building there will be large out bit decentralized models but they themselves can be interconnected into a network uh that effectively exchanges

53. 26:07

    information and uh is able to kind of provide a bigger context and maybe like some of them are more specialized than others uh maybe not from a even information perspective but from just access to resources right like a model like uh or from uh kind of ability to interact with some specific subset of other kind of people and uh yeah so like different different types of specialization that can occur there What

54. 26:41

    You Remind Me Do you think the society is conscious so this is a great example yeah the great question uh it's probably it it's a subconscious sub subconscious I think I think there there's a sub I mean effectively a subconscious you know design to for sur for society to survive right which propagates into evolutionary need to replicate for ourselves so yeah I don't think there's a yeah I mean given what

55. 27:11

    especially us killing the planet right now there's like very given how hard it is to coordinate people there's like a very conscious level but there's definitely some subconscious organizational things right that you know uh lead us to work together in different ways and and rely on each other so so one interesting way to look at it is uh is to try to understand why why Consciousness emerged to begin with what is the evolutionary

56. 27:37

    advantage and I think uh I didn't put too much thought to it but I think one advantage of Consciousness is that it's a it improves planning it's much easier to plan when you when you have Consciousness right so Society is not particularly good at planning right so from this perspective it could be argued that maybe it doesn't have Consciousness but neither uh neither are um model is today that good at planning right so so

57. 28:01

    they're relatively good at U answering in in one shot but if you I mean they can do Chain of Thought but relatively poorly right so from this perspective and we know that when you when we train a regular large legging model uh we don't tell her to be intelligent we just tell her predict the next token right but as it tries to compress all the information given to her at some point there's uh you know like gradually it realizes that by being intelligent about

58. 28:26

    what it says it can compress it much better right it doesn't have to memorize every possible situation it just needs to understand it conceptually and then it can compress that understanding so from this perspective once we get better and better at planning like you know GPT 401 is presumably open as for step towards that uh that's where Consciousness is more likely to emerge MH right so that's uh so so yeah once

59. 28:51

    once we start investing in planning we need to pay more attention but but I think so I agree with that but I think the Consciousness in planning is required because you're part of the plan like you individually as a as a being is part of the plan that's why you need to understand how like you will act and react in the future like that's the explainability part uh that you need for

60. 29:15

    planning if if you're just like planning for somebody else right that it can be like I mean you can just use the smt search right and like create a plan for when when the Searcher itself is not involved in the process so I think it's it will require some embodiment what whatever that I mean it can be very digital you know navigating internet and this is like it's always funny now to reread some of the science fiction

61. 29:41

    because clearly you know physical robotics is not where AI evolution is happening because AI look by definition is digital being it lives in digital world bringing it back to physical world is actually a hard hard work and letting it Rome in in the in the digital world is where uh kind of it's the freest but I think some form of embodiment is required uh for it to like or like some form of identifier is to use the

62. 30:13

    diaspora anal is required to effectively understand that you're present and how your actions will impact yourself well we're already five minutes over I hadn't any more questions for you but speaking of planning I guess we're out of time so we'll solve Consciousness another day um thank you guys that was fun thanks everybody for coming