{"id":"meaoCJs5F2U","title":"NEAR AI office hours #19 with Hero - smarter, faster, AI-powered crypto decisions.","description":"Illia & Alex discuss the latest AI research topics and unpack some NEAR.AI focus areas.\r\n\r\nGuest: Hero\r\nhttps://hero.io/\r\nhttps://x.com/HeroAisearch\r\n\r\nYour go-to for smarter, faster, AI-powered crypto decisions.\r\n\r\nNEAR AI: https://near.ai/\r\nIllia Polosukhin https://x.com/ilblackdragon\r\nAlex Skidanov https://x.com/AlexSkidanov\n\nAlex interviews Vyas the Chief Science Officer of Apta AI, an AI spin-out from the University of Cambridge. They dive into Apta’s work in conversational AI, the limitations of language models like GPT-4, and the integration of AI with Web3 through their partnership with Hero. Key topics include the Task Switch paper, agentic frameworks, and the future of search in Web3.\n\nTakeaways\n• Apta, a spin-out from the University of Cambridge, develops AI solutions for Web3, with a focus on conversational AI.\n• Apta's collaboration with Hero aims to create the \"Google of Web3\" by providing users with advanced search capabilities for Web3 insights.\n• Apta's approach addresses the limitations of large language models by using specialized agents for deeper, faster insights.\n• The \"task switch\" problem in LLMs is identified as a key challenge, and Apta is developing solutions to improve context switching without performance degradation.\n• Apta is working toward launching Hero AI Search in late 2024, integrating Web3-specific insights for users.\n\n\nTimestamps:\n0:00 - Introduction and guest introduction\n0:15 - Vas from Apta AI explains the company's background\n1:02 - Task Switch paper origins and collaboration\n1:46 - Introduction to Hero and Apta partnership\n2:30 - Understanding Hero’s vision for Web3 search\n3:49 - Limitations of current LLMs like GPT-4\n5:00 - How large language models work\n6:10 - AI's limitations with logical and complex tasks\n9:00 - Real-world examples and problem-solving with LLMs\n13:19 - Limitations of agentic frameworks and how APTA solves them\n18:39 - Task Switch paper explained\n23:16 - Task switching challenges for businesses\n33:05 - Conversation on solutions to task switching issues\n36:58 - Apta’s agent-based solution and specialized agents\n38:47 - Hero AI search and pre-computed data\n42:50 - Applications of AI in Web3\n44:00 - Integrating Web3 tools and the future of conversational interfaces\n48:05 - How Apta collaborates with Hero for specialized AI development\n53:01 - Future of AI interfaces and final thoughts","channel_name":"NEAR Protocol","published_at":"2024-09-24T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/meaoCJs5F2U/hqdefault.jpg","transcript_text":"okay hi everyone this is uh episode number I don't remember of office hours uh today we have vs with us from apto Ai and hero um yeah do do you want to tell a couple words about yourself and what appon her is yeah thanks Alex thanks for having me on the show um yeah so I'm vas I am the chief science officer at APTA we are a AI spin out from the University of Cambridge so we're basically PhD researchers in AI in natural language processing we saw a problem we felt in the space of natural language and buildt some technology to try and solve that problem and we're applying that to the space of web three and that's our partnership with here in a nutshell I see so so the paper about the tax task switch that is a uh that paper is L from within aftera that's not good question so the task switch paper is a collaboration between University of Cambridge which is where I was doing a PhD and with sispa which is a it's an institution in Germany and the technology we developed there or the ideas we proposed there we took them forwards in apar and deployed them as part of our solution because effectively we're trying to build conversational AI which can access deep insights so that used some of the insights we developed in that paper I see sense I see and uh and and here is something completely separate right or or are yeah maybe I can give a brief idea behind how this is all structured so everyone's on the same page so hero is looking to develop basically the as one of the tools they want to offer an ecosystem they want to develop the Google of web 3 so they want to provide a search engine where people can access information about web 3 they can ask questions such as where can I buy Pepe and beyond that they also want to allow people to ask slightly more deeper questions or more sophisticated insights so this could be something like give me the top three meme coins that have gone down the most in the last 19 days so it's slightly complex questions but the bit where APTA comes in is allowing these insights through a conversational interface so I think everyone is kind of coming to terms with the fact that the future of search is no longer us coming up with key words in Google but more of the perplexity style a natural conversation with the data you have or the directories under the hood so at apter we basically develop the conversational interface but with a bit of a Twist I think we fundamentally believe that the probabilistic nature or the auto regressive nature of llms is not sufficient to answer some some of the deep questions people have I know there's a lot of like buzz around GPD 401 with this Chain of Thought but um even with that there's a fundamental limitation maybe we can go on to that later in the conversation with how these conversational llms are used to access the these kind of deep insights so the structure basically is the paper is discussing one particular limitation of llms specifically something called task switch and that kind of feeds into one of the overall framework of how APTA deals with with explicit challenges in llms and comes up with the structure we think solves all these challenges right and so so what is the so so you mentioned the fundamental limitation of the auto regressive shall we just jump in I think that's a very interesting yeah yeah good yeah that's my favorite topic nowadays um I think so I me I'm sure you know Alex and a lot of the listeners here already know that um the modern day large language models typically refer to these the decoder architecture in the Transformer architecture which was proposed in 2017 2018 but the decoder architecture has this particular operation where when it sees a sequence of words or tokens as we say natural language processing tasks the way it generates its output is by predicting the next most probable word or token and I guess to give a bit insight into why it does this I guess there's only three main stages of training these large language models there's a pre-training stage where these decodes are literally exposed to lots and lots of sentences and trained to completely sentences or predict the next most probable word given the sentence shown then there's a second stage which I would say makes them very smart and that's the instruction tuning stage so over here you just expose this pre-trained large language model to lots and lots of supervis tasks so this can range from translation to summarization question answering and then you ask the model you train the model to be able to solve these tasks with the reference correct outputs and the third stage usually less talked about is the alignment stage that's just to ensure the model is kind of tailored to the or aligned with how humans expect answers to be and I think old ways were using rlf um reinforcement learning with human feedback but more modern approaches use like DPO poo kind of approaches from the reinforcement learning literature but that's less relevant to the constraint I would say the constraint is that these large language models are at the end of the day however clever an answer they're generating it's based on a probabilistic output and I think as a community people came to this conclusion that this really gives very powerful models in terms of understanding language so I think me and you I think I I love using chat GPT for example to summarize emails or come up with nice poems but if I want to do a complex task or logical task using a probabilistic sequence just based on how much data it's seen or having some good grasp of language is not sufficient I think a typical example I always give is if I ask chbt can you find me the 11th Fibonacci number it can probably answer that correctly so don't get me wrong with that that sense but this is not how I would say is the correct way to answer the question and to prove my point I would say I can make this question increasingly complex and the model will have a decreasing level of accuracy in answering the question so right now the way the model tried to answer the question at least gb41 it would do some kind of chain of thought it would say from its parametric memory it knows the formula for the nth Fibonacci number and then was substitute the formula the 11 and come up with the 11th Fibonacci number for me but if you come up with a if you ask a humanist kind of question or if you have a deterministic program what I would normally say is a human we understand the question as a natural language input calculate the 11 Fibonacci number but we don't go and then come up with the most probable sequence of words to try and answer the question we in a way say okay we're going to activate our tool it could be a computer program or it could be our something we've learned in school where we know the process of working out the think and this is almost a deterministic process you've activated after understanding a language I think as a community in the whole NLP space I think when 202 before came everyone kind of started moving towards what everyone call the agentic framework so this is function calling you use the large language model what it's very good at which is understanding language and synthesizing language but to do actual solving of problems you feed it to specialized modules and functions so I think that's like a solution to one of the main problems with a probabalistic framework but I can also going to some challenges around the agentic framework why it's also not sufficient to the tasks Enterprises business care about nowadays and any thoughts on that Alex so so you think because like like when I tried to to use your example right to me uh looking for the 11 Fibonacci number is closer to what GPT for o is doing right because in my head if I wasn't like obviously the most likely you know if you ask me for the 100 fibon number I would just go and write a script right so that is much closer to function falling but if you ask me 11 I would try to compute it on my head right which is which is very similar to GPT 401 because first of all with a very high Chance by the 11th number I would have made at least one mistake right which is which is similar to probabilistic I think it's a good point to raise I think the example I've given here there's actually a way through a chain of language to try and come to the answer especially if it's superficial like number 11 but if I for example said the 11th prime number there's actually no Pro I haven't tried it actually in chpd I've be impressed actually not L press 11 is still quite small so it's probably seen it sufficient times to give you the right answer but um I think it's interesting what you're saying if it's something quite common we do use language to try and answer it if it's something which is clearly algorithmic in trying to answer the question and you haven't explicitly been exposed to that um input we are kind of triggering a tool as you said writing a python script interesting Insight actually so finding that balance between at what point do you move from r language to moving to those functions guess in the business application I think usually it's very more very complex tasks it's not um language based task I don't know what you would say to that yeah and I I haven't used gp4 in a while right now because it got loged out and I never bother to log in again I just gr with not not not not the El gr but yeah yeah yeah gr it's uh it's fast fast lpu based right yes yeah yeah internally at near we using uh we have an inference router which uses hyperbolic and the fireworks uh but uh but I myself use Gro primarily because I have it logged in and you know it's it's it's quickly accessible yeah but but GPT 4 right back in the day at least I'm pretty sure that still exists right uh for some questions it would spin up an interpreter right and actually write a python script yeah right so so that so that would be interesting to compare if you ask it various Fibonacci numbers if you ask the o1 I I don't know if o1 has access to the interpreter but ask questions at which point go ahead so from my understanding so a lot of these popular platform based large language models have access to some I would call gener General functions so these are an interpreter a python interpreter they have a web search agent they have a file analysis or file upload agent so these are always generic agents and I think that's um I think the question you're asking is at what point does it trigger them and what point is the Reliance language right but especially in case of for of o1 it's interesting to see in which case it would use you know its Chain of Thought and in which case it would switch to interpreter specifically in the context of Fibonacci numbers and prime numbers right because some of them it could be able to find yeah try it out I suspect because it's been trained explicitly I mean they haven't said so by everyone saying in different sources has been explicitly trained in a Chain of Thought manner it's going to lean towards always trying to do Chain of Thought if it can right but but so so what you mentioned is that so so APTA is is is using agents right that's that's the implication from from what you after we try to go a level Beyond this so we argue that this agentic framework is really cool and solves a lot of problems in how llms are deployed but there's three still fundamental problems with these generalist agentic Frameworks at least with llms and I think the first Main and obvious problem is for a business these generalist llms or agents don't have access to bespoke agents which the business cares about there's a specific business function which the business has spent 20 years developing the llm such as gp4 doesn't have access to interfacing that function so no access to bespoke agents is one problem and I guess related to that then there's no access to the bespoke data streams the business May care about but um we can always imagine a Utopia where people can link up their data streams and their own bisoke functions with generalist llms there's still like a third problem which I think I've observed in dealing with lot of companies as part of an AI consultancy we used to run last year and the problem was it's too slow to run these business processes on the Fly I think a classic example I gave is for example if you ask um chat GPT I'm come for the random example what's been the correlation between the price of apples and oranges in the last seven months in Ghana it's a very specific very precise question expressed through natural language but requires a series of mathematical modelings and solutions as well as access to bespoke data streams so in the Utopia let's imagine the llm has access to the data let's say has access to a database with all the stuff and it has access to the correct agents to do the processing what it has to do on the Fly then is develop some kind of machine learning model which will extract the correlations or do whatever is required mathematically to answer the question and to do all of this on the Fly we could easily take an hour and I think in no real use experience is a user willing to wait an hour to answer have their questions answered in the conversation so I was say a limitation of the agentic framework in this current setup and how it's deployed in most examples I've seen is that you can't extract deep insights on the Fly even though they're welld designed to extract those insights you can't do it on the Fly because it's too slow there's a bottleneck in the actual processing and I think that's what we're trying to solve with apter I think to some extent we've solved it quite well with application to Hero where they're trying to do for the web three space I think it's uh something interesting uh an interesting observation which is like somewhat relevant is that um we were playing with some uh you know developer agents and we notice that if you want to to do a front end an agent can you know like it can do it on the flight right you can design pretty complex front ends on the fight but if you want it to be you know like Implement some business logic it requires quite a bit of iterating and it it's taking time and hypothesis here is that that's because this so much frent code online right for for any particular business case you know the probability that you it's seen a lot of it is is low so it means like you know Chain of Thought and iterating but there's so much front and code that it can just right yeah I like in this case um so so let's do the same example right you you want the correlation between the the the price of apples and oranges yeah and don't remember the rest of the of the in the last seven months so very specific kind of right so so so like in case in case of uh of APTA how do you how do you approach it how do how do you make it fast yeah great question so I guess the ID APPA is the second you go to a specific business use case or specific domain let's say for example the web 3 domain or it could be the legal domain it could be the healthcare domain you've already in a way defined the scope of complex questions people care about in that space and when You' defined the scope and you can you can imagine some distribution of questions there will always exist a tail you can never handle people come with really weird questions and you will know how to handle for the majority of questions even though the details of the question are different you can set up your framework to store intermediaries to allow deep computation or complex analysis to be done on a regular basis in the background but the to answer the specific question you only by light computation on top of the intermediary outputs so for example in the case of apples and oranges correlations you may have in advance if you know people care about correlations between different objects such as fruits for every pair of top 200 fruits people care about you may have already pre-computed their correlations and to answer the specific question you may have to adjust the computation to do it for the last seven months or the last X for the range you care about which kind of shows you've done the complex analysis for the case you care about beforehand and to actually answer the question on the Fly you have those intermediary outputs and process them bit further and F tune them for the specific sorry fune is a bad word to use but adjust them for the specific question being asked so fact use the main knowledge to to sort of try to predict what is the yeah I mean I would say it's two ways you can be domain knowledge is how to start off in determining which kind of background age you're going to develop to do the intermediary calculations but you can also be a bit blind in that you can just release the tool and so that's going be the next stage of what app and hero are going to do and just see what people ask and you immediately can come up with clusters of kind of questions people ask and cover the space of what's of Interest automatically so you can either go bottom up or top down and think in trying to come up with a scope for the specific domain like in this case if someone adds for rep oranges you go through the slope process initially yeah exactly I mean best you can only go through SL it's I guess a parallel on software engineering is is caching in this case you're not caching the final output because there too many different outputs you're kind of caching the process which should be required yeah Best Way describe it okay so um uh the initial context the the way I learned about this whole effort right was from this task switch paper right so I guess Yeahs it would be interesting to to to to explain what a task switch is right was the problem there yeah of course so I think task switch in a way before you describe it to give a bit of motivation is illustrating a limitation in how we perceive what these generalist almost AGI kind of models should be used as as opposed to what's fundamentally Limited in how they're trained and developed so what Tas switch is saying is intuitively I use chbd this way let's say I'm writing a email to a colleague I'll use chat GPD to help synthesize my email now I want to use chat GPT to help me correct some code I don't continue in the same chat session I tend to create a new chat session to answer that code or answer my code related query and the question which rose to me when we started this project was why do we intuitively change our chat session when we change the task we're asking the llm to solve and I think this intuitively comes from people having perhaps tried it in the same session and have maybe visibly or intuitively experienced some kind of performance degradation so what Tas switch papers effectively looking at is one formalizing and two examining the extent to which models such as jbt or llama or mistro have a performance degradation when the conversation history includes a task which is different to the final task being asked so for example I could be asking about email synthesis summarization in my history of the conversation and my new question my latest question could be help me solve this coding problem so there's a task switch from the history to the final question and the task switch paper is basically examining to what extent are models affected by this kind of task switch and I guess maybe feeding on into the overall framework or limit ation of llms is more looking at when you want to deploy a genuinely AGI kind of system a generalist like model which can kind of handle anything you want to behave a bit more like a human a human doesn't reinitialize thems when they move to a different task they have some could be implicit or some explicit way of ignoring the history of the conversation so far if there's a task switch and the way these llms are trained or designed right now there's no hard rule telling them to ignore or not ignore the history is an implicit attention weight attached to the history whilst the inference happens and because of that there's always a probabilistic chance there's too much weight attached to the history which has an adversarial impact on the answer to the question for the latest question so I guess that's a not so brief summary of T switch Yeah so so I yes I have two questions uh immediate followup I guess the first one is uh so my understanding is that that is part of the uh sort of logic that you are building into into APTA right so so so I guess the question would be why is like given I can start a new chat right uh why is Task switch sort of important short term why why should we you know Implement uh protections against it into the systems yeah I think I think I would have twofold answer to this so in the short term it's okay I mean not to be a bit cheeky with this it's almost like saying a quote if you ask the people so the creator of Ford said this quote if you ask people what they wanted they would said I want a faster horse so it's like almost this people are happy changing conversations now because they're used to it in a way but you can have a bit of a paradigm shift if you didn't have to have a new conversation you literally had a model which is your expert co-pilot or your virtual assistant which can handle everything one go in a way it's a more elegant solution and more useful in the long term but a more practical perspective in the short term let's say people don't care about the effort of changing conversation or reinitializing the Practical the Practical concern is as a human user who hasn't trained the model or is not exposed to what the model is good at or what is bad at they don't necessarily know when they are switching tasks for example even when I move from asking about python to asking about HTML code I could for a specific model be switching tasks but I don't know as a user if this is a sufficient task switch for me to Mo go and create a new conversation I may naively continue in the same session and be getting poor results for no reason if I had instead a model which could handle that kind of switch explicitly I don't know if that's like a fair answer in your opinion I find it very interesting right that people are getting people are learning very fast to use systems like for example you know Google uh like you can get way better well I guess now Google is uh now it's very hard to get anything useful from Google but back in the day it was it was much it was uh like if you knew how to use Google you would find results much faster right than if you just like naively write a query especially in like earlier days but but I think the the other uh question is so so the PDF like I read through the through the PDF and it's it's doing a very good job of uh sort of investigating the problem from a like there's a very principled approach right to to measuring it uh on a variety of problems but that PDF at least the PDF I have doesn't doesn't look at all to the solutions right but my understanding is that you guys already have some progress towards the mitigating the tax task switch Yeah it's a good point so this paper just to put everyone on the same page Tas switch paper actually proba worth mentioning Beyond since you were provided the paper it's also been accepted to em L as well so if anyone happens to be in Miami in November please come check it out there as well so I guess the aim of the paper beyond that Shameless plug was that to um formalize and be the one of the first introductions to a problem I think people already knew but hadn't formalized in their heads and as you said there's only I in a way identify the problem and formalized in the principled way but hasn't sort to come to any solutions to it I guess that was me be future work but as you said since then we've been got a few ideas and how to solve it and so I think as AP we have a solution which works in an agenic framework but there are many players who'd also be interested in using this in a conversational interface so there's still a massive use case for llms beyond the agentic framework you actually just want to use them for normal conversation where you can still come across T switches so how do you resolve that kind of situation and to me to solve this problem it actually goes back to a fundamental machine learning problem the reent task switch is a problem is because the input and when I say input I'm referring to both the conversation history and the latest question this entire input is in a way out of distribution for the model so what I mean by out of distribution is during training the models will actually never come across conversations where there exists a task switch if you think about the different training steps we have the pre-training Stage IT effectively just is is exposed to piing next most probable word in unsupervised way then we have the instruction tuning stage instruction tuning is a supervised task spe task specific training setup but you do include conversational data sets in this instruction tuning the conversational data sets included are for specific tasks it's only question answering or it's only entertainment conversation there's no stage at which during this training you're telling the model how to deal with the fact you have a task switch from the history to the latest question so to me this almost comes out exactly the same problem as having an input which lies out of distribution for the model so the model is not too confident and not too sure how to handle it during inference it'll do its best it's a strong grasp of language so we'll try to answer this question the best it can but it hasn't been exposed sufficiently to assign the right way to the history as opposed to the latest question so so in an ideal setting if it had been trained in this way it in a way would have automatically implicitly learned to assign attention weights of zero maybe to the entire history but right now because it hasn't seen it there's nothing telling it to and there's no there's no implicit bias on the weights of the attention weights on the history so then coming back to the question if we've established this is a problem of being out of distribution the solutions we can kind of Leverage come from the whole area of outer distribution detection or resolution and to me this compos down to two things it's either you can develop a detector this detector is built on existing LMS so you try to detect out of distribution samples and you flag them so you explicitly tell the llms out of distribution and to deal with it in some way you can have some clever principled approach to that which kind of goes back to being a bit like an agentic framework because you now have not just the llm running naively on the input you have external systems working with the input to help inform the lm's decision but beyond the detector and I think this is will be a neat solution it's going to what like we do in the aders robustness space so my PhD is in the area of adversar robustness which I would argue is exploring one very specific kind of a distribution setting basically where there's a malicious perturbation of the input to deceive a model so going back to answering the question to make a model robust to out of distribution samples such as task switches we can just effectively do something called adversarial training we retrain the large language model but just give it examples of what it fails at and it would learn perhaps implicitly to now handle task switching I think it could be a very simple solution you just update the supervisor fine tuning stage or the instruction tuning stage to include task switches and the conversational tasks and then suddenly I think we we'd have a dramatic Improvement but that comes of course at a greater cost so it's the compromise between do we want to retrain these models versus building external detectors on top the ones more elegant but also more expensive uh so so I remember um in the uh when you do pre-training right when you do pre-training you have a lot of documents but the documents naturally are not like you know if your context size is 16,000 your documents are not necessarily you know a multiples of 16,000 tokens and so they already sort of package multiple of them into the context right so so the so the model already has seen many times uh you know drastic test switch right because when you do pre-training like one of them could be uh right and there's like a u there's usually end of sequence token right and so the model learned that if there is an end of sequence token it should ignore everything to the left uh and moreover something interesting is that of course on the architecture side uh you can if you want to just math it out right uh because you know you already have this uh attention attention mask right you can as well say hey I know what the boundaries are let me just zero out the tension the question is how would you know to do that you need external system yeah yeah that's that's a test time right but what I'm saying is that uh they found that the training time it doesn't matter right so they they did experiments and they found that whether or not you mask out the previous context does not affect at all the model performance right and so they don't bother right because obviously easier not to affect the mask right so the model is already seeing a lot of task Swit just it has this token which obviously doesn't have inference time right that inference time if you do a task switch the token is not there the mod is confused right but but you but something you mentioned is that now if we sort of extrapolate this idea to the fine tuning instruction tuning phase right it doesn't have to be more expensive I think I think it's sufficient uh to just like you know if you have multiple examples annotated like you have 50,000 sequences instead of training on each sequence separately you just concate them right them together yeah right right and then you naturally with Without End of sequence talking right you just say hey this is a continuation of the the same conversation interesting point you actually mentioned at the end of sequence token I think what this what it suggests to me is the fact it made no difference there include to not actually mask out the contact and just leave it there and have that token it shows the model is very heavily relying on that token to help informance decision and secondly remove that is kind of moving it off the manifold of data it's ever seen yes in a way actually I would even argue you can take you could improve the whole process by removing the Reliance on the end of sequence token as well maybe during the pre-training as well to just get the model used to their existing a sudden change in task change the flow of the conversation so again as you said the cheapest solution is L stacking them during instruction following but I think one add there is to added your instruction following You' have to retain the prompts or the instructions with previous task it's not just about shoving the context in yeah if you remove the the end of sequence token during pre-training uh your model will learn that the moment it finish the sentence uh it can start you know hallucinating absolutely arbitrary stuff right which is not because during inference what they do they they mask out like the end of sequence token cannot be produced right but but if you but but if you train without having end of sequence token then the model you will have no way of preventing model from switching to a completely different task when you don't want it to happen yeah agre I didn't mean remove the end of sentence sequence all together I meant um what if you sampled it from a range of different tokens so you don't rely heavily on a single token so the model learns to have a method of not being dependent on that token existing during his training but even if it's arbitrary it can add noise during the process which you hope could be compensated by the instruction following stage but the sampling could help maybe override its dependence but I guess um it's too expensive a research research approach perhaps to come with a robust solution in trying assing this approach uh so so the solutions you already mentioned right you mentioned uh pre-training uh on some examples of the task switching did you guys try any of them or is it all right now it's I think it's usually easiest simply to go with open source models such as llama 3.1 so it's actually better to build in detected kind of approaches but for our application apter the agentic framework solves it anyway so the third solution I guess I didn't even discuss is and I guess this is where the agentic framework Falls in even Chain of Thought in a way solves this problem the second you explicitly instruct the model to evaluate the answer it's coming to it is able through language maybe to discard its emphasis on history so even though this not a hard rule it is in a way a solution as well so this is an autocorrection kind of solution and you can use Chain of Thought to do that but you could also use a agentic framework to do that the second you pass bits to agents the models been explicitly asked to work out what inputs it needs to put into each agent to answer the question and the second is giving the inputs required to answer the question for the different agents it knows those inputs don't exist in the history so it won't even pass them to the model example would be in the history tell me the top synthesiz this emo for me and then the next question you ask llm is give me the correlation between apples and oranges it the inputs it passes to the agents are apples and oranges there's no reason it would try to pass anything about the synthesized email in the past so that kind of dependence is automatically through uh physical consideration and L they would discard yeah also one interesting thing to to measure here would be uh when you uh when you look at the cost of the task switch of the you know the performance drop would be interesting to see what happens if right before you're doing a Tas task switch you just add one sentence saying let's switch to a different task right so so if you have a natural language q that there's a different task do we still see the performance degradation I wouldn't be surprised if do right because it still has you know yeah that's question so this I would say so if I was to summarize the solutions we're having so far is we had one which is retraining or ad training people say we have explicit external detectors and then you have a auto correction solution and this fourth solution is now requiring human collaboration this is the user knowing they're about to change a task sufficiently and informing the model model that be ready and I think that would be very effective solution but to me perhaps falls in the same category as the human changing the session I don't know if you would argue that's the same thing no I think I think it's uh it's quite different um it makes more you think it's more natural yeah exactly yeah so I mean it becomes more like a human expert in that sense but you could say under the hood okay let's say if you're training users to use your system you say oh if you're changing tasks just let the model know and under the hood all the model does is actually reinitialize it itself but but if we want user to to inter intervene right the user might as yeah yeah it's uh we still require user to have the the main expertise of the system they have to have when to intervene yeah yeah iation having the automatic resolution and not the human collaboration yeah so so so you keep on mentioning agents right so under the hood APTA is still reliant on the on like interaction between agents right it's uh like architecture it is a uh multi-agent system maybe I can Define the agent better so agent can be anything from a very specific F un llm so a very complex thing to even just being a calculator so agent is literally just some specialized function and the llm is just something which is able to identify the relevant expert modules or agents we can call them to be activated what inputs to pass to them how to chain them together and once they've chained the outputs together how to feed it back to the llm so the answer can be synthesized for the user yeah so so is it like under the hood yeah but but under the hood it is still like multiple stru multiple specialized actors effectively and an orchestrator right yeah nice putting it yeah yeah uh so uh you mentioned uh go going back to the to the earlier conversation you mentioned that uh like one of the problems is that if you don't have um like if you if you don't have a way to precompute some of the information that might be useful it is it it is somewhat slow to get the results right so in the context of web three so so you're applying up today to the hero uh uh offerings uh what are some examples of something that is you know that uh that exists as that information that is computed and available to to the I think a really good example is so give the general brief I would say we want to give insights which are in a way usually ring fenced for experts people who know how to do mathematical modeling people know how to do programming they know how to get these answers and even then it might take them a half an hour and hour so an example of a question someone might say is I am following influencer I would name someone to shame that for no reason influencer X okay on and on YouTube they've told me to go and buy Solana it's going to go up apparently now should I should I trust this influencer how often have they been correct in the past now on the Fly you could go and extract all the videos of this influencer transcribe them correlate them with what happened with the price and then go and answer the user oh this person's been correct 33% of the time that's way better than all the other influencers we've seen who are correct 1% of the time so go ahead and maybe trust them more than you would trust anyone else now a very simple expert agent you can imagine in the past in the background is for the top thousand influencers or top 10,000 influencers we could have pre-transcriptional in this case the example of the computation would be transcribing a lot of videos right ahead of time even retrieving I would say so preparing the correct data streams to retrieve it transcribe it and Beyond transcribe it you have to use some clever natural language processing technology to identify what the content of what they're saying is which tokens do they refer to which direction are they saying those tokens are moving in so there some level of named entity recognition and then some kind of intent around those entities has to be extracted right so yeah that's an interesting U that's an interesting application one thing that is interesting is that if you if you like ask Gemini to look at some video uh it can immediately give you answers about that video which is very that's like a mindblowing experience it almost looks like it already watched it right because for Gemini it can attemp to hold video at at once right so you can give it like a half an hour video and it will tell you oh yeah yeah that's what happens in it you know several seconds later which is fascinating but of course it doesn't scale like if you want to to watch the whole you know history of someone's videos that's I think Beyond even the whole history of videos for a particular influencer what if you want to correlate it with 100 influencers or yes you want to do something which actually very cheap if You' already done it but very very expensive if you have to do it every time on the fly in a way MH I think another classic example is like anything which requires computation in terms of you to build an entire model around it or this long processing so the example I gave is long processing you get information you have to then go and break AP information prepare a database but there's also many examples where you have to train models so for example hero we're training a model to try and detect rug pools automatically you don't want to train this model on the fly but you can get this insights to inference on the Fly very easily I think that's like another classic example I would promote as an example of why this intermediary output structure is use when you extract deep insights in application yeah and so how far all of that so so is it in production or like is it in research stage where how far is it yeah good question so right now we're planning to launch the hero AI search in Q4 so end November December is the plan um we've in the phase of collecting feedback from internal testers and few select testers as well so as as you can imagine we came up with ideas of we thought was interesting but the idea of having a few testers also comes up with identifying what they think is interesting as well as questions to ask also a big part of this is when you just release a product to someone they don't know what to ask so it's actually all about educating the users and how to exploit the platform best for the use case for example we could always have this influencer validator tool right if no one knows about it they would never even maybe ask the question which they could really benefit from but they don't know about it so that's a big part of how would you streamline the suggested queries or the interface of the system to allow users to exploit the most of the system right and so so here already has some of the like it has a wallet browser uh a payment platform right is it uh is it is it like part of the bigger picture do they naturally feed to the search or was it more of uh uh creating early products the market yeah I think it's a very pertinent question so the idea here is that we want a single point of entry for all your web Three Needs or crypto needs in the space so even though the initial hero AI search is in a way allowing insights and information about cryptocoins or exchanges the next layer is looking to make it more executional so there are a lot of like individual companies out there which have developed this kind of Technology anyway the advantage of having a single ecosystem with all the different tools it means your AI has ability to interact with all these tools as well because they're within the same ecosystem and so the idea would be then you could have your conditional execution such as can you always can you automatically go and buy me Bitcoin if it crosses 50k and sell it if it goes above 60k that's my strategy but I don't want to monitor it myself but you tell this execution agent not the kind of Agents we referring to right now to go and do this process for you and the whole point llm there is literally just an interface to what is very in a way very simple function under the hood and the advantage of having everything in one place is it makes it very easy to interface everything yeah so so like part of the idea there would be having access to the like effectively if the user is doing it from within the browser that is integrated then there is natural access to the to the state uh to the current state of the user right yeah so in a way you can personalize it to the extent that user wants personalization if they want to share their wallet address then they can go and do the transactions for them if they want to share other information and through conversations as well if they want to allow some level personalization they would have the ability to do so yeah and so given you investing in SE that heavily is is it your is it your vision that in the future people will be primarily interacting with the with the web through a conversation or like how do you envision you know like two years from now yeah I mean interfacing with so I I wouldn't say I would take credit for the vision of um there being a natural language interaction with surge I think that seems to be migration in the community anyways I think it was first really I guess um coined or really shown demonstrated through perplexity Ai and then I think everyone jumped on that bandwagon like Gemini came out and you have their summary at the beginning every time you do Google search so I don't think it's necessarily A Vision but more of a need for how people will interact anything else will seem clunky and archaic you want to be able to interact with your tools naturally your language because we have now tools such as LMS which can understand language very well there's no need for us to inate code how we communicate even when you Google in know historically you're kind of coding your question this I would say um question top of my head okay say I want to know where is Prada in my location I say near me Prada why am I writing like that you know there's no need to exclude a filler words if I'm having a normal conversation and even though people got used to communicating that way in the past I think people are very very quickly moving to a future where they want to communicate naturally with all the tools they have so again I would say it's not a vision it's a it's a basic need to have that interface yeah but I would emphasize this point it's very hard to make that expectation people have very robust natural language is so varied and so easy for machines to misinterpret what you're saying robustness is the key and that's why I really believe robustness cannot be achieved through a simple probabilistic framework you need deterministic functions under the hood or very specialized modules which do that thing extremely well and you only rely on the LM to do thing it does extremely well which is understand the language it shouldn't be doing much more than that in my opinion y uh and so so out of curiosity why is a um so so it seems like for hero the web three search is like one of the core um offerings right uh is it like why why is it that you know like this core uh feature is developed in an external entity is it like have it happened historically or there is some motivation behind it sure you mean like why are they partnering with APTA to get this developed question yeah yeah it's a good question I mean I think the simplest answer to that is it's extremely difficult I think today to make this work right now with the technology in place and as a team we at APTA so our research has been linked to this and previously to apter we ran an AI consultancy uh where we de develop products for lots of different companies around the world or the consulting or even due diligence for a lot of the VCS and based on that we already came across this narrative where we wanted to build something which allows deep insights through natural language without that latency and we had we didn't have a partner to go and deploy that for we had the AI knowledge we didn't have the domain expertise to apply it so it just seemed like a very good fit for APPA to work with hero they didn't necessarily have their tools or the expertise to build it or after had the tool and expertise but didn't have the domain knowledge to apply it so it just seem like a very synergistic setup in my opinion okay Al so after and here are completely unrelated I I thought it was the same team running feels like the same team nowadays but um two separate companies so Apon is actually developing this for the web 3 but in parallel we're also going to looking to apply this technology to multiple different Industries because as you can imagine it's very useful for any industry where you want a natural language access to deep information web the just seems like a really good an apt setting because it's so Rife with scams and confusion and noise it's a space where I think everyday users really benefit from having some transparency and some more informed decisions around the information out there and through natural language because they don't necessarily have the coding skills or programming skills to go and access that information otherwise and to to better understand UPA so so so the product of APTA is both the uh the infrastructure and all the agents on top of it right so all the agents are built-in house it's it's not like a API or a framework that others can can use to BU their right yeah so to build the actual specialized experts and functions under the hood we work very closely with the domain we're working in so in this case hero because we don't come from a native um crypto background to me I never knew people followed influences in the crypto space but as I've entered this space everyone follows influences a lot and seems to be a big driver of traffic and also a Big Driver of price based on what different influences say so this kind of domain expertise is proved by Hero so we developed the technology once we have provided the domain expertise in house and go and deploy it using our Air Technology but the source of what we build comes from the team we're working with Y and uh I guess besides the task switch uh paper are there any other interesting uh like research papers or any anything else like like maybe something about agents that is a something people can look read and to get to better understand technology behind APTA yeah to understand technology behind APTA I think the AP end day because it's using so many different tools from large language models it almost comes down to coming summarizing the biggest limitations LMS and if you want to deploy them how do people solve those limitations so classic problems are like hallucination That's a classic problem everyone discusss and there's lots of solutions out there everything from rag to selfcheck GPT which is actually published by one of our co-founders then there's even agentic framework help solve that problem then there's a problem of bias then there's a problem of adversar robustness there's a problem twitch as we give an example um these are things I'm going top of my head so if I you address this question you're saying I basically come say to yourself what problem am I having with llm and that kind of drives the research paper which is worth investigating because it's such a broad area each of these topics I would struggle to say this is the one paper to go and say it solves everything it's more about incremental updates in each of these areas in sumary and and and as of today is it uh what is what is powering up is it like is it llama is it chat GPT what is the what are the underlying yeah so as of now um we aren't llm loyal as you can imagine as as no one else is right now is the second we can get the fastest cheapest LM we swap do that so right now we're finding as most useful to use llama 3.1 because that gives the flexibility to find unit a bit more but as I said this is not what I would say is the focus we're very agnostic to the llm um right now we're also actually investigating how we can get on to Gro to improve our speed as well under the hood M yeah but I mean in the future could like Sova they have also very cool chips being relaunched and released so as I would generally say this to everyone in the field there's no need for loyalty as a customer go for the best it's so easy to switch between them and everyone's providing access to very similar interfaces there's very little engineering work required to do the actual switch Yeah so uh uh speaking of loyalty I think some people have an opinion that the models are sufficiently different that you know switching the model is not just changing the API end point so so it's interesting I mean I think depends if you're talking about a coding model or you're talking about a race specialized function I would agree but the second you're using the models as we are as just an interface to understand language I think they all I as good as it need to be actually to be honest yeah so like I don't see for example I would never use a Chain of Thought model and that's heading there's no reason for us to do that we're already solving that problem through different functions or agents it's now literally only what inference uh speed and cost yeah the motivator in yeah so so you mentioned switching to Gro as of today are are you guys hosting them yourself or you using some other inflence provider we're using different Cloud providers right now I see yeah I think it's um comes down to hosting your own machines is very very expensive right now and doesn't seem like the most useful use of resources under you deployed and it becomes like a clear narrative you have also because we want to explore faster machines so if you host your own gpus right now and you find the status quo is lpus through easy access in the future then there's no it's not worthwhile investing in that resource in my opinion right now yeah yeah and if there's someone else like hero who has a like a similar use case they want to build a conversation agent they want to have a more robust uh specialized uh is like after already at a stage where people can reach out and and integrate or or like is it very much so but more in they have to they have to guide the expert agents under the hood of course y because I guess the idea being we don't want to be the generalist the generalist is always kind of being done by the big companies out there we want to be robust for the specific business so the structure and framework is very much ready to be ported to different Industries but as you can imagine the teething process can be slightly complex sometimes you can have very finicky agents or very specialized functions which the LM do necessarily understand very well so you have to work with making sure it's robust as I say in 2 hours anyone can build agentic framework it can take two months to two years to make it robust MH that's where all the clever AI tricks and research comes in yeah so so what is the interface like let's say I'm a company similar to Hero right uh will it be my Engineers building agents on top of uh opta with like help or or is it or is it more of like me me sharing my domain knowledge but someone on the UPA side is ilding the agent I look forward to a Utopia where we can almost provide like an eight endpoint to the framework and you can Define your own functions and your agents and they just feed into this API in a clever way I think that's a lot of like even like open AI a lot API are trying to do that right now with generalist llms or I've seen some Lang chain offering function calling that's a Utopia but from my experience this is not robust there are a lot of things you have to do to actually make it work well um I mean as a classic examples in context learning the llm has to be given a lot of examples of what kind of output from different inputs to give that specific agent it's a very small example that is and you don't know what unless you are building that agent yourself in a very close quarter to the llm you can't do a lot of these tricks which are required to make it robust so the current setup for us is at least they provide their domain expertise they may have even developed code or modules to do specific things we don't want to reinvent the wheel in any way we just want to interface it in many cases I think usually people have an idea what's important and they haven't developed it so then we work with them or if it's not necessary if we understand the problem well enough we can go and develop it ourselves yeah my uh my good friend IIA who's usually here with me but not today he I he had a child so congratulations to him yeah uh his uh his Utopia is that there's an agent that builds agents so you go to you know to an agent and you explain your use case and and the agent is sufficiently proficient in building agents um yeah we're gonna have a very very productive economy yep y okay almost out of time so uh um I would love to hear any closing thoughts about you know like how you envision the future what's uh what's next and uh yeah I think for my closing thought in one line summary would be let's use tools for what they're good at not overhype them and you can take that from it yeah but it's been really fun chatting thanks so much yeah okay thank you thank you for coming and I think thanks everybody for watching","transcript_synced_at":"2026-11-03T05:32:27.924Z","segments":[{"start_seconds":3,"text":"okay hi everyone this is uh episode number I don't remember of office hours","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3s"},{"start_seconds":9,"text":"uh today we have vs with us from apto Ai and hero um yeah do do you want to tell a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=9s"},{"start_seconds":18,"text":"couple words about yourself and what appon her is yeah thanks Alex thanks for having me on the show um yeah so I'm vas","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=18s"},{"start_seconds":26,"text":"I am the chief science officer at APTA we are a AI spin out from the University of Cambridge so we're basically PhD","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=26s"},{"start_seconds":34,"text":"researchers in AI in natural language processing we saw a problem we felt in the space of natural language and buildt","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=34s"},{"start_seconds":41,"text":"some technology to try and solve that problem and we're applying that to the space of web three and that's our partnership with here in a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=41s"},{"start_seconds":51,"text":"nutshell I see so so the paper about the tax task switch that is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=51s"},{"start_seconds":57,"text":"a uh that paper is L from within aftera that's not good question so the task switch","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=57s"},{"start_seconds":66,"text":"paper is a collaboration between University of Cambridge which is where I was doing a PhD and with sispa which is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=66s"},{"start_seconds":73,"text":"a it's an institution in Germany and the technology we developed there or the ideas we proposed there we took them","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=73s"},{"start_seconds":81,"text":"forwards in apar and deployed them as part of our solution because effectively we're trying to build conversational AI","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=81s"},{"start_seconds":88,"text":"which can access deep insights so that used some of the insights we developed in that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=88s"},{"start_seconds":93,"text":"paper I see sense I see and uh and and here is something completely separate","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=93s"},{"start_seconds":101,"text":"right or or are yeah maybe I can give a brief idea behind how this is all structured so everyone's on the same","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=101s"},{"start_seconds":108,"text":"page so hero is looking to develop basically the as one of the tools they want to offer an ecosystem they want to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=108s"},{"start_seconds":117,"text":"develop the Google of web 3 so they want to provide a search engine where people can access information about web 3 they","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=117s"},{"start_seconds":124,"text":"can ask questions such as where can I buy Pepe and beyond that they also want to allow people to ask slightly more","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=124s"},{"start_seconds":132,"text":"deeper questions or more sophisticated insights so this could be something like give me the top three meme coins that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=132s"},{"start_seconds":139,"text":"have gone down the most in the last 19 days so it's slightly complex questions but the bit where APTA comes in is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=139s"},{"start_seconds":146,"text":"allowing these insights through a conversational interface so I think everyone is kind of coming to terms with the fact that the future of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=146s"},{"start_seconds":154,"text":"search is no longer us coming up with key words in Google but more of the perplexity style a natural conversation","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=154s"},{"start_seconds":162,"text":"with the data you have or the directories under the hood so at apter we basically develop the conversational interface but with a bit of a Twist I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=162s"},{"start_seconds":171,"text":"think we fundamentally believe that the probabilistic nature or the auto regressive nature of llms is not sufficient to answer some some of the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=171s"},{"start_seconds":180,"text":"deep questions people have I know there's a lot of like buzz around GPD 401 with this Chain of Thought but um even with that there's a fundamental","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=180s"},{"start_seconds":188,"text":"limitation maybe we can go on to that later in the conversation with how these conversational llms are used to access the these kind of deep","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=188s"},{"start_seconds":196,"text":"insights so the structure basically is the paper is discussing one particular limitation of llms specifically something called task switch and that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=196s"},{"start_seconds":205,"text":"kind of feeds into one of the overall framework of how APTA deals with with explicit challenges in llms and comes up","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=205s"},{"start_seconds":213,"text":"with the structure we think solves all these challenges right and so so what is the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=213s"},{"start_seconds":221,"text":"so so you mentioned the fundamental limitation of the auto regressive shall we just jump in I think that's a very interesting yeah yeah good yeah that's","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=221s"},{"start_seconds":229,"text":"my favorite topic nowadays um I think so I me I'm sure you know Alex and a lot of the listeners here already know that um","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=229s"},{"start_seconds":237,"text":"the modern day large language models typically refer to these the decoder architecture in the Transformer architecture which was proposed in 2017","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=237s"},{"start_seconds":246,"text":"2018 but the decoder architecture has this particular operation where when it sees a sequence of words or tokens as we","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=246s"},{"start_seconds":254,"text":"say natural language processing tasks the way it generates its output is by predicting the next most probable word","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=254s"},{"start_seconds":261,"text":"or token and I guess to give a bit insight into why it does this I guess there's only three main stages of training these large language models","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=261s"},{"start_seconds":270,"text":"there's a pre-training stage where these decodes are literally exposed to lots and lots of sentences and trained to completely sentences or predict the next","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=270s"},{"start_seconds":279,"text":"most probable word given the sentence shown then there's a second stage which I would say makes them very smart and that's the instruction tuning stage so","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=279s"},{"start_seconds":287,"text":"over here you just expose this pre-trained large language model to lots and lots of supervis tasks so this can","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=287s"},{"start_seconds":295,"text":"range from translation to summarization question answering and then you ask the model you train the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=295s"},{"start_seconds":302,"text":"model to be able to solve these tasks with the reference correct outputs and the third stage usually less talked about is the alignment stage","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=302s"},{"start_seconds":310,"text":"that's just to ensure the model is kind of tailored to the or aligned with how humans expect answers to be and I think","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=310s"},{"start_seconds":318,"text":"old ways were using rlf um reinforcement learning with human feedback but more modern approaches use like DPO poo kind of approaches from the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=318s"},{"start_seconds":327,"text":"reinforcement learning literature but that's less relevant to the constraint I would say the constraint is that these large language models are at the end of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=327s"},{"start_seconds":335,"text":"the day however clever an answer they're generating it's based on a probabilistic output and I think as a community people","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=335s"},{"start_seconds":342,"text":"came to this conclusion that this really gives very powerful models in terms of understanding language so I think me and you I think I I love using chat GPT for","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=342s"},{"start_seconds":351,"text":"example to summarize emails or come up with nice poems but if I want to do a complex task or logical task using a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=351s"},{"start_seconds":360,"text":"probabilistic sequence just based on how much data it's seen or having some good grasp of language is not sufficient I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=360s"},{"start_seconds":367,"text":"think a typical example I always give is if I ask chbt can you find me the 11th","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=367s"},{"start_seconds":375,"text":"Fibonacci number it can probably answer that correctly so don't get me wrong with that that sense but this is not how I would say is the correct way to answer","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=375s"},{"start_seconds":383,"text":"the question and to prove my point I would say I can make this question increasingly complex and the model will have a decreasing level of accuracy in","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=383s"},{"start_seconds":391,"text":"answering the question so right now the way the model tried to answer the question at least gb41 it would do some kind of chain of thought it would say","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=391s"},{"start_seconds":399,"text":"from its parametric memory it knows the formula for the nth Fibonacci number and","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=399s"},{"start_seconds":405,"text":"then was substitute the formula the 11 and come up with the 11th Fibonacci number for me but if you come up with a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=405s"},{"start_seconds":413,"text":"if you ask a humanist kind of question or if you have a deterministic program what I would normally say is a human we understand the question as a natural","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=413s"},{"start_seconds":421,"text":"language input calculate the 11 Fibonacci number but we don't go and then come up with the most probable sequence of words to try and answer the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=421s"},{"start_seconds":429,"text":"question we in a way say okay we're going to activate our tool it could be a computer program or it could be our","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=429s"},{"start_seconds":437,"text":"something we've learned in school where we know the process of working out the think and this is almost a deterministic process you've activated after","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=437s"},{"start_seconds":445,"text":"understanding a language I think as a community in the whole NLP space I think when 202 before came everyone kind of started moving towards what everyone","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=445s"},{"start_seconds":453,"text":"call the agentic framework so this is function calling you use the large language model what it's very good at which is understanding","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=453s"},{"start_seconds":460,"text":"language and synthesizing language but to do actual solving of problems you feed it to specialized modules and functions so I think that's like a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=460s"},{"start_seconds":469,"text":"solution to one of the main problems with a probabalistic framework but I can also going to some challenges around the agentic framework why it's also not","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=469s"},{"start_seconds":476,"text":"sufficient to the tasks Enterprises business care about nowadays and any thoughts on that Alex","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=476s"},{"start_seconds":484,"text":"so so you think because like like when I tried to to use your example right to me uh looking for the 11 Fibonacci number","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=484s"},{"start_seconds":491,"text":"is closer to what GPT for o is doing right because in my head if I wasn't like obviously the most likely you know","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=491s"},{"start_seconds":500,"text":"if you ask me for the 100 fibon number I would just go and write a script right so that is much closer to function falling but if you ask me 11 I would try","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=500s"},{"start_seconds":508,"text":"to compute it on my head right which is which is very similar to GPT 401 because first of all with a very high Chance by the 11th number I would have made at","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=508s"},{"start_seconds":516,"text":"least one mistake right which is which is similar to probabilistic I think it's a good point to raise I think the example I've given","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=516s"},{"start_seconds":525,"text":"here there's actually a way through a chain of language to try and come to the answer especially if it's superficial like number 11 but if I for example said","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=525s"},{"start_seconds":533,"text":"the 11th prime number there's actually no Pro I haven't tried it actually in chpd I've be impressed actually not L press 11 is still quite","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=533s"},{"start_seconds":541,"text":"small so it's probably seen it sufficient times to give you the right answer but um I think it's interesting what you're saying if it's something","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=541s"},{"start_seconds":549,"text":"quite common we do use language to try and answer it if it's something which is clearly algorithmic in trying to answer the question and you haven't explicitly","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=549s"},{"start_seconds":557,"text":"been exposed to that um input we are kind of triggering a tool as you said writing a python script interesting Insight actually so","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=557s"},{"start_seconds":566,"text":"finding that balance between at what point do you move from r language to moving to those functions guess in the business","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=566s"},{"start_seconds":574,"text":"application I think usually it's very more very complex tasks it's not um language based task I don't know what you would say to that yeah and I I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=574s"},{"start_seconds":582,"text":"haven't used gp4 in a while right now because it got loged out and I never bother to log in again I just gr with","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=582s"},{"start_seconds":590,"text":"not not not not the El gr","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=590s"},{"start_seconds":597,"text":"but yeah yeah yeah gr it's uh it's fast fast lpu based right yes yeah yeah","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=597s"},{"start_seconds":607,"text":"internally at near we using uh we have an inference router which uses hyperbolic and the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=607s"},{"start_seconds":614,"text":"fireworks uh but uh but I myself use Gro primarily because I have it logged in","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=614s"},{"start_seconds":621,"text":"and you know it's it's it's quickly accessible yeah but but GPT 4 right back in the day at least I'm pretty sure that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=621s"},{"start_seconds":628,"text":"still exists right uh for some questions it would spin up an interpreter right and actually write a python script yeah right so so that so","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=628s"},{"start_seconds":636,"text":"that would be interesting to compare if you ask it various Fibonacci numbers if you ask the o1 I I don't know if o1 has access to the interpreter but","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=636s"},{"start_seconds":645,"text":"ask questions at which point go ahead so from my understanding so a lot of these popular platform based large language","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=645s"},{"start_seconds":653,"text":"models have access to some I would call gener General functions so these are an interpreter a python interpreter they","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=653s"},{"start_seconds":661,"text":"have a web search agent they have a file analysis or file upload agent so these are always generic agents and I think","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=661s"},{"start_seconds":669,"text":"that's um I think the question you're asking is at what point does it trigger them and what point is the Reliance language right but especially in case of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=669s"},{"start_seconds":677,"text":"for of o1 it's interesting to see in which case it would use you know its Chain of Thought and in which case it would switch to interpreter specifically","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=677s"},{"start_seconds":684,"text":"in the context of Fibonacci numbers and prime numbers right because some of them it could be able to find yeah try it out I suspect because it's been trained","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=684s"},{"start_seconds":693,"text":"explicitly I mean they haven't said so by everyone saying in different sources has been explicitly trained in a Chain of Thought manner it's going to lean towards always trying to do Chain of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=693s"},{"start_seconds":701,"text":"Thought if it can right but but so so what you mentioned is that so so APTA is is is using agents","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=701s"},{"start_seconds":710,"text":"right that's that's the implication from from what you after we try to go a level Beyond this so we argue that this agentic","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=710s"},{"start_seconds":719,"text":"framework is really cool and solves a lot of problems in how llms are deployed but there's three still fundamental problems with these generalist agentic","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=719s"},{"start_seconds":728,"text":"Frameworks at least with llms and I think the first Main and obvious problem is for a business these generalist llms","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=728s"},{"start_seconds":735,"text":"or agents don't have access to bespoke agents which the business cares about there's a specific business function","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=735s"},{"start_seconds":743,"text":"which the business has spent 20 years developing the llm such as gp4 doesn't have access to interfacing that function","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=743s"},{"start_seconds":751,"text":"so no access to bespoke agents is one problem and I guess related to that then there's no access to the bespoke data streams the business May care about but","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=751s"},{"start_seconds":760,"text":"um we can always imagine a Utopia where people can link up their data streams and their own bisoke functions with generalist llms there's still like a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=760s"},{"start_seconds":768,"text":"third problem which I think I've observed in dealing with lot of companies as part of an AI consultancy we used to run last year and the problem","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=768s"},{"start_seconds":777,"text":"was it's too slow to run these business processes on the Fly I think a classic example I gave is for example if you ask","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=777s"},{"start_seconds":783,"text":"um chat GPT I'm come for the random example what's been the correlation between the price of apples and oranges","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=783s"},{"start_seconds":793,"text":"in the last seven months in Ghana it's a very specific very precise question expressed through natural language but requires a series of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=793s"},{"start_seconds":802,"text":"mathematical modelings and solutions as well as access to bespoke data streams so in the Utopia let's imagine the llm has access to the data let's say has","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=802s"},{"start_seconds":810,"text":"access to a database with all the stuff and it has access to the correct agents to do the processing what it has to do","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=810s"},{"start_seconds":818,"text":"on the Fly then is develop some kind of machine learning model which will extract the correlations or do whatever is required mathematically to answer the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=818s"},{"start_seconds":827,"text":"question and to do all of this on the Fly we could easily take an hour and I think in no real use experience is a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=827s"},{"start_seconds":834,"text":"user willing to wait an hour to answer have their questions answered in the conversation so I was say a limitation of the agentic framework in this current","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=834s"},{"start_seconds":841,"text":"setup and how it's deployed in most examples I've seen is that you can't extract deep insights on the Fly even though they're welld designed to extract","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=841s"},{"start_seconds":849,"text":"those insights you can't do it on the Fly because it's too slow there's a bottleneck in the actual processing and I think that's what we're trying to solve with apter I think to some extent","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=849s"},{"start_seconds":858,"text":"we've solved it quite well with application to Hero where they're trying to do for the web three","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=858s"},{"start_seconds":865,"text":"space I think it's uh something interesting uh an interesting observation which is like somewhat","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=865s"},{"start_seconds":871,"text":"relevant is that um we were playing with some uh you know developer agents and we","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=871s"},{"start_seconds":878,"text":"notice that if you want to to do a front end an agent can you know like it can do it on the flight right you can design","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=878s"},{"start_seconds":886,"text":"pretty complex front ends on the fight but if you want it to be you know like Implement some business logic it requires quite a bit of iterating and it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=886s"},{"start_seconds":893,"text":"it's taking time and hypothesis here is that that's because this so much frent","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=893s"},{"start_seconds":901,"text":"code online right for for any particular business case you know the probability that you it's seen a lot of it is is low so it means like you know Chain of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=901s"},{"start_seconds":908,"text":"Thought and iterating but there's so much front and code that it can just right","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=908s"},{"start_seconds":917,"text":"yeah I like in this case um so so let's do the same example right you you want","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=917s"},{"start_seconds":924,"text":"the correlation between the the the price of apples and oranges yeah and don't remember the rest of the of the in the last seven months","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=924s"},{"start_seconds":932,"text":"so very specific kind of right so so so like in case in case of uh of APTA how do you how do you approach it how do how","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=932s"},{"start_seconds":941,"text":"do you make it fast yeah great question so I guess the ID APPA is the second you go to a specific business use case or","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=941s"},{"start_seconds":949,"text":"specific domain let's say for example the web 3 domain or it could be the legal domain it could be the healthcare domain you've already in a way defined","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=949s"},{"start_seconds":957,"text":"the scope of complex questions people care about in that space and when You' defined the scope and you can you can imagine some","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=957s"},{"start_seconds":965,"text":"distribution of questions there will always exist a tail you can never handle people come with really weird questions and you will know how to handle for the majority of questions even though the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=965s"},{"start_seconds":974,"text":"details of the question are different you can set up your framework to store intermediaries to allow deep computation","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=974s"},{"start_seconds":981,"text":"or complex analysis to be done on a regular basis in the background but the to answer the specific question you only","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=981s"},{"start_seconds":988,"text":"by light computation on top of the intermediary outputs so for example in the case of apples and oranges correlations you may have in advance if","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=988s"},{"start_seconds":997,"text":"you know people care about correlations between different objects such as fruits for every pair of top 200 fruits people care","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=997s"},{"start_seconds":1006,"text":"about you may have already pre-computed their correlations and to answer the specific question you may have to adjust the computation to do it for the last seven","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1006s"},{"start_seconds":1015,"text":"months or the last X for the range you care about which kind of shows you've done the complex analysis for the case you care about beforehand and to actually answer","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1015s"},{"start_seconds":1024,"text":"the question on the Fly you have those intermediary outputs and process them bit further and F tune them for the specific sorry fune is a bad word to use","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1024s"},{"start_seconds":1031,"text":"but adjust them for the specific question being asked so fact use the main knowledge to to sort of try to predict what is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1031s"},{"start_seconds":1040,"text":"the yeah I mean I would say it's two ways you can be domain knowledge is how to start off in determining which kind of background age you're going to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1040s"},{"start_seconds":1049,"text":"develop to do the intermediary calculations but you can also be a bit blind in that you can just release the tool and so that's going be the next stage of what app and hero are going to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1049s"},{"start_seconds":1057,"text":"do and just see what people ask and you immediately can come up with clusters of kind of questions people ask and cover the space of what's of Interest","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1057s"},{"start_seconds":1066,"text":"automatically so you can either go bottom up or top down and think in trying to come up with a scope for the specific","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1066s"},{"start_seconds":1074,"text":"domain like in this case if someone adds for rep oranges you go through the slope process initially yeah exactly I mean best you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1074s"},{"start_seconds":1082,"text":"can only go through SL it's I guess a parallel on software engineering is is caching in this case you're not caching the final output because there too many","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1082s"},{"start_seconds":1091,"text":"different outputs you're kind of caching the process which should be required yeah Best Way describe","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1091s"},{"start_seconds":1099,"text":"it okay so um uh the initial context the the way I learned about this whole effort right was from this task switch","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1099s"},{"start_seconds":1108,"text":"paper right so I guess Yeahs it would be interesting to to to to explain what a task switch is right was the problem","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1108s"},{"start_seconds":1117,"text":"there yeah of course so I think task switch in a way before you describe it to give a bit of motivation is illustrating a limitation in how we","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1117s"},{"start_seconds":1127,"text":"perceive what these generalist almost AGI kind of models should be used as as opposed to what's fundamentally Limited","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1127s"},{"start_seconds":1134,"text":"in how they're trained and developed so what Tas switch is saying is intuitively I use chbd this way let's say I'm","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1134s"},{"start_seconds":1142,"text":"writing a email to a colleague I'll use chat GPD to help synthesize my email now I want","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1142s"},{"start_seconds":1150,"text":"to use chat GPT to help me correct some code I don't continue in the same chat session I tend to create a new chat session to answer that code or answer my","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1150s"},{"start_seconds":1159,"text":"code related query and the question which rose to me when we started this project was why do we intuitively change","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1159s"},{"start_seconds":1167,"text":"our chat session when we change the task we're asking the llm to solve and I think this intuitively comes from people","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1167s"},{"start_seconds":1174,"text":"having perhaps tried it in the same session and have maybe visibly or intuitively experienced some kind of performance degradation so what Tas","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1174s"},{"start_seconds":1182,"text":"switch papers effectively looking at is one formalizing and two examining the extent to which models such as jbt or","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1182s"},{"start_seconds":1192,"text":"llama or mistro have a performance degradation when the conversation history includes a task which is different to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1192s"},{"start_seconds":1201,"text":"the final task being asked so for example I could be asking about email synthesis summarization in my history of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1201s"},{"start_seconds":1209,"text":"the conversation and my new question my latest question could be help me solve this coding problem so there's a task switch from the history to the final","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1209s"},{"start_seconds":1217,"text":"question and the task switch paper is basically examining to what extent are models affected by this kind of task switch and I guess maybe feeding on into","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1217s"},{"start_seconds":1226,"text":"the overall framework or limit ation of llms is more looking at when you want to deploy a genuinely AGI kind of system a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1226s"},{"start_seconds":1235,"text":"generalist like model which can kind of handle anything you want to behave a bit more like a human a human doesn't reinitialize thems when they move to a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1235s"},{"start_seconds":1244,"text":"different task they have some could be implicit or some explicit way of ignoring the history of the conversation so far if there's a task switch and the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1244s"},{"start_seconds":1253,"text":"way these llms are trained or designed right now there's no hard rule telling them to ignore or not ignore the history","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1253s"},{"start_seconds":1263,"text":"is an implicit attention weight attached to the history whilst the inference happens and because of that there's always a probabilistic chance there's","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1263s"},{"start_seconds":1271,"text":"too much weight attached to the history which has an adversarial impact on the answer to the question for the latest","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1271s"},{"start_seconds":1280,"text":"question so I guess that's a not so brief summary of T switch Yeah so so I yes I have two questions uh immediate followup","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1280s"},{"start_seconds":1289,"text":"I guess the first one is uh so my understanding is that that is part of the uh sort of logic that you are","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1289s"},{"start_seconds":1297,"text":"building into into APTA right so so so I guess the question would be why is like given I can start a new chat right uh","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1297s"},{"start_seconds":1306,"text":"why is Task switch sort of important short term why why should we you know Implement uh protections against it into","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1306s"},{"start_seconds":1314,"text":"the systems yeah I think I think I would have twofold answer to this so in the short term it's okay I mean not to be a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1314s"},{"start_seconds":1323,"text":"bit cheeky with this it's almost like saying a quote if you ask the people so the creator of Ford said this quote if you ask people what they wanted they","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1323s"},{"start_seconds":1331,"text":"would said I want a faster horse so it's like almost this people are happy changing conversations now because","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1331s"},{"start_seconds":1339,"text":"they're used to it in a way but you can have a bit of a paradigm shift if you didn't have to have a new conversation you literally had a model which is your","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1339s"},{"start_seconds":1346,"text":"expert co-pilot or your virtual assistant which can handle everything one go in a way it's a more elegant solution and more useful in the long","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1346s"},{"start_seconds":1354,"text":"term but a more practical perspective in the short term let's say people don't care about the effort of changing conversation or reinitializing the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1354s"},{"start_seconds":1362,"text":"Practical the Practical concern is as a human user who hasn't trained the model or is not exposed to what the model is good at or what is bad at they don't","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1362s"},{"start_seconds":1371,"text":"necessarily know when they are switching tasks for example even when I move from asking about python to asking about HTML","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1371s"},{"start_seconds":1380,"text":"code I could for a specific model be switching tasks but I don't know as a user if this is a sufficient task switch for me to Mo go and create a new","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1380s"},{"start_seconds":1388,"text":"conversation I may naively continue in the same session and be getting poor results for no reason if I had instead a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1388s"},{"start_seconds":1396,"text":"model which could handle that kind of switch explicitly I don't know if that's like a fair answer in your opinion I find it very interesting right","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1396s"},{"start_seconds":1404,"text":"that people are getting people are learning very fast to use systems like for example you know Google uh like you can get way better well I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1404s"},{"start_seconds":1412,"text":"guess now Google is uh now it's very hard to get anything useful from Google but back in the day it was it was much","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1412s"},{"start_seconds":1420,"text":"it was uh like if you knew how to use Google you would find results much faster right than if you just like naively write a query especially in like","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1420s"},{"start_seconds":1429,"text":"earlier days but but I think the the other uh","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1429s"},{"start_seconds":1436,"text":"question is so so the PDF like I read through the through the PDF and it's it's doing a very good job of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1436s"},{"start_seconds":1445,"text":"uh sort of investigating the problem from a like there's a very principled approach right to to measuring it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1445s"},{"start_seconds":1452,"text":"uh on a variety of problems but that PDF at least the PDF I have doesn't doesn't look at all to the solutions right but","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1452s"},{"start_seconds":1460,"text":"my understanding is that you guys already have some progress towards the mitigating the tax task switch Yeah it's","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1460s"},{"start_seconds":1467,"text":"a good point so this paper just to put everyone on the same page Tas switch paper actually proba worth mentioning Beyond since you were provided the paper","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1467s"},{"start_seconds":1475,"text":"it's also been accepted to em L as well so if anyone happens to be in Miami in November please come check it out there as well so I guess the aim of the paper","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1475s"},{"start_seconds":1483,"text":"beyond that Shameless plug was that to um formalize and be the one of the first introductions to a problem I think people already knew but hadn't","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1483s"},{"start_seconds":1491,"text":"formalized in their heads and as you said there's only I in a way identify the problem and formalized in the principled way but hasn't sort to come","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1491s"},{"start_seconds":1500,"text":"to any solutions to it I guess that was me be future work but as you said since then we've been got a few ideas and how","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1500s"},{"start_seconds":1506,"text":"to solve it and so I think as AP we have a solution which works in an agenic framework but there are many players","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1506s"},{"start_seconds":1513,"text":"who'd also be interested in using this in a conversational interface so there's still a massive use case for llms beyond the agentic framework you actually just","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1513s"},{"start_seconds":1521,"text":"want to use them for normal conversation where you can still come across T switches so how do you resolve that kind","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1521s"},{"start_seconds":1528,"text":"of situation and to me to solve this problem it actually goes back to a fundamental machine learning problem the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1528s"},{"start_seconds":1535,"text":"reent task switch is a problem is because the input and when I say input I'm referring to both the conversation history and the latest question this","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1535s"},{"start_seconds":1544,"text":"entire input is in a way out of distribution for the model so what I mean by out of distribution is during training the models will actually never","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1544s"},{"start_seconds":1553,"text":"come across conversations where there exists a task switch if you think about the different training steps we have the pre-training Stage IT effectively just","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1553s"},{"start_seconds":1562,"text":"is is exposed to piing next most probable word in unsupervised way then we have the instruction tuning stage","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1562s"},{"start_seconds":1569,"text":"instruction tuning is a supervised task spe task specific training setup but you do include conversational data sets in","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1569s"},{"start_seconds":1577,"text":"this instruction tuning the conversational data sets included are for specific tasks it's only question answering or it's only entertainment","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1577s"},{"start_seconds":1585,"text":"conversation there's no stage at which during this training you're telling the model how to deal with the fact you have a task switch from the history to the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1585s"},{"start_seconds":1593,"text":"latest question so to me this almost comes out exactly the same problem as having an input which lies out of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1593s"},{"start_seconds":1600,"text":"distribution for the model so the model is not too confident and not too sure how to handle it during inference it'll do its best it's a strong grasp of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1600s"},{"start_seconds":1609,"text":"language so we'll try to answer this question the best it can but it hasn't been exposed sufficiently to assign the right way to the history as opposed to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1609s"},{"start_seconds":1616,"text":"the latest question so so in an ideal setting if it had been trained in this way it in a way would have automatically implicitly learned to assign attention weights of zero maybe to the entire","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1616s"},{"start_seconds":1625,"text":"history but right now because it hasn't seen it there's nothing telling it to and there's no there's no implicit bias on the weights of the attention weights","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1625s"},{"start_seconds":1633,"text":"on the history so then coming back to the question if we've established this is a problem of being out of distribution the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1633s"},{"start_seconds":1642,"text":"solutions we can kind of Leverage come from the whole area of outer distribution detection or resolution and to me this compos down to two things","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1642s"},{"start_seconds":1651,"text":"it's either you can develop a detector this detector is built on existing LMS so you try to detect out of distribution","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1651s"},{"start_seconds":1658,"text":"samples and you flag them so you explicitly tell the llms out of distribution and to deal with it in some way you can have some clever principled","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1658s"},{"start_seconds":1667,"text":"approach to that which kind of goes back to being a bit like an agentic framework because you now have not just the llm running naively on the input you have","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1667s"},{"start_seconds":1675,"text":"external systems working with the input to help inform the lm's decision but beyond the detector and I think this is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1675s"},{"start_seconds":1682,"text":"will be a neat solution it's going to what like we do in the aders robustness space so my PhD is in the area of adversar robustness which I would argue","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1682s"},{"start_seconds":1691,"text":"is exploring one very specific kind of a distribution setting basically where there's a malicious perturbation of the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1691s"},{"start_seconds":1697,"text":"input to deceive a model so going back to answering the question to make a model robust to out of distribution","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1697s"},{"start_seconds":1705,"text":"samples such as task switches we can just effectively do something called adversarial training we retrain the large language model but just give it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1705s"},{"start_seconds":1712,"text":"examples of what it fails at and it would learn perhaps implicitly to now handle task switching I think it could be a very simple solution you just","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1712s"},{"start_seconds":1720,"text":"update the supervisor fine tuning stage or the instruction tuning stage to include task switches and the conversational tasks and then suddenly I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1720s"},{"start_seconds":1727,"text":"think we we'd have a dramatic Improvement but that comes of course at a greater cost so it's the compromise between do we want to retrain these models versus building external","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1727s"},{"start_seconds":1736,"text":"detectors on top the ones more elegant but also more expensive uh so so I remember um in","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1736s"},{"start_seconds":1746,"text":"the uh when you do pre-training right when you do pre-training you have a lot of documents but the documents naturally are not like you know if your context","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1746s"},{"start_seconds":1755,"text":"size is 16,000 your documents are not necessarily you know a multiples of 16,000","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1755s"},{"start_seconds":1762,"text":"tokens and so they already sort of package multiple of them into the context right so so the so the model","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1762s"},{"start_seconds":1770,"text":"already has seen many times uh you know drastic test switch right because when you do pre-training like one of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1770s"},{"start_seconds":1778,"text":"them could be uh right and there's like a u there's usually end of sequence token right and so the model learned that if there is an end of sequence","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1778s"},{"start_seconds":1785,"text":"token it should ignore everything to the left uh and moreover something interesting is that of course on the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1785s"},{"start_seconds":1792,"text":"architecture side uh you can if you want to just math it out right uh because you know you already have this uh attention","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1792s"},{"start_seconds":1801,"text":"attention mask right you can as well say hey I know what the boundaries are let me just zero out the tension the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1801s"},{"start_seconds":1809,"text":"question is how would you know to do that you need external system yeah yeah that's that's a test time right but what I'm saying is that uh they found that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1809s"},{"start_seconds":1816,"text":"the training time it doesn't matter right so they they did experiments and they found that whether or not you mask out the previous context does not affect","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1816s"},{"start_seconds":1824,"text":"at all the model performance right and so they don't bother right because obviously easier not to affect the mask","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1824s"},{"start_seconds":1831,"text":"right so the model is already seeing a lot of task Swit just it has this token which obviously doesn't have inference time right that inference time if you do a task switch the token is not there the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1831s"},{"start_seconds":1839,"text":"mod is confused right but but you but something you mentioned is that now if we sort of extrapolate this idea to the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1839s"},{"start_seconds":1847,"text":"fine tuning instruction tuning phase right it doesn't have to be more expensive I think I think it's sufficient uh to just like you know if","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1847s"},{"start_seconds":1855,"text":"you have multiple examples annotated like you have 50,000 sequences instead of training on each sequence separately you just concate them right them","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1855s"},{"start_seconds":1863,"text":"together yeah right right and then you naturally with Without End of sequence talking right you just say hey this is a continuation of the the same","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1863s"},{"start_seconds":1871,"text":"conversation interesting point you actually mentioned at the end of sequence token I think what this what it suggests to me is the fact it made no","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1871s"},{"start_seconds":1879,"text":"difference there include to not actually mask out the contact and just leave it there and have that token it shows the model is very heavily relying on that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1879s"},{"start_seconds":1888,"text":"token to help informance decision and secondly remove that is kind of moving it off the manifold of data it's ever seen yes in a way actually I would even","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1888s"},{"start_seconds":1898,"text":"argue you can take you could improve the whole process by removing the Reliance on the end of sequence token as well","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1898s"},{"start_seconds":1905,"text":"maybe during the pre-training as well to just get the model used to their existing a sudden change in task change the flow of the conversation so again as","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1905s"},{"start_seconds":1914,"text":"you said the cheapest solution is L stacking them during instruction following but I think one add there is to added","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1914s"},{"start_seconds":1921,"text":"your instruction following You' have to retain the prompts or the instructions with previous task it's not just about","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1921s"},{"start_seconds":1928,"text":"shoving the context in yeah if you remove the the end of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1928s"},{"start_seconds":1935,"text":"sequence token during pre-training uh your model will learn that the moment it finish the sentence uh it can start you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1935s"},{"start_seconds":1942,"text":"know hallucinating absolutely arbitrary stuff right which is not because during inference what they do","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1942s"},{"start_seconds":1950,"text":"they they mask out like the end of sequence token cannot be produced right but but if you but but if you train without having end of sequence token","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1950s"},{"start_seconds":1958,"text":"then the model you will have no way of preventing model from switching to a completely different task when you don't want it to happen yeah agre I didn't","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1958s"},{"start_seconds":1967,"text":"mean remove the end of sentence sequence all together I meant um what if you sampled it from a range of different tokens so you don't rely heavily on a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1967s"},{"start_seconds":1975,"text":"single token so the model learns to have a method of not being dependent on that token existing during his training but","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1975s"},{"start_seconds":1984,"text":"even if it's arbitrary it can add noise during the process which you hope could be compensated by the instruction following stage but the sampling could","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1984s"},{"start_seconds":1991,"text":"help maybe override its dependence but I guess um it's too expensive a research research approach","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=1991s"},{"start_seconds":2000,"text":"perhaps to come with a robust solution in trying assing this approach uh so so the solutions you already mentioned right you mentioned uh","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2000s"},{"start_seconds":2009,"text":"pre-training uh on some examples of the task switching did you guys try any of them or is it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2009s"},{"start_seconds":2016,"text":"all right now it's I think it's usually easiest simply to go with open source models such as llama 3.1 so it's","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2016s"},{"start_seconds":2023,"text":"actually better to build in detected kind of approaches but for our application apter the agentic framework solves it anyway so the third solution I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2023s"},{"start_seconds":2031,"text":"guess I didn't even discuss is and I guess this is where the agentic framework Falls in even Chain of Thought in a way solves this problem the second","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2031s"},{"start_seconds":2039,"text":"you explicitly instruct the model to evaluate the answer it's coming to it is able through language maybe to discard","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2039s"},{"start_seconds":2047,"text":"its emphasis on history so even though this not a hard rule it is in a way a solution as well so this is an autocorrection kind of solution and you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2047s"},{"start_seconds":2055,"text":"can use Chain of Thought to do that but you could also use a agentic framework to do that the second you pass bits to agents the models been explicitly asked","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2055s"},{"start_seconds":2063,"text":"to work out what inputs it needs to put into each agent to answer the question and the second is giving the inputs required to answer the question for the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2063s"},{"start_seconds":2070,"text":"different agents it knows those inputs don't exist in the history so it won't even pass them to the model example would be in the history tell me the top","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2070s"},{"start_seconds":2080,"text":"synthesiz this emo for me and then the next question you ask llm is give me the correlation between apples and oranges it the inputs it passes to the agents","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2080s"},{"start_seconds":2088,"text":"are apples and oranges there's no reason it would try to pass anything about the synthesized email in the past so that kind of dependence is automatically","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2088s"},{"start_seconds":2096,"text":"through uh physical consideration and L they would discard yeah also one interesting thing","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2096s"},{"start_seconds":2104,"text":"to to measure here would be uh when you uh when you look at the cost of the task switch of the you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2104s"},{"start_seconds":2112,"text":"know the performance drop would be interesting to see what happens if right before you're doing a Tas task switch you just add one sentence saying let's","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2112s"},{"start_seconds":2119,"text":"switch to a different task right so so if you have a natural language q that there's a different task do we still see the performance degradation I wouldn't","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2119s"},{"start_seconds":2126,"text":"be surprised if do right because it still has you know yeah that's question so this I","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2126s"},{"start_seconds":2134,"text":"would say so if I was to summarize the solutions we're having so far is we had one which is retraining or ad training","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2134s"},{"start_seconds":2142,"text":"people say we have explicit external detectors and then you have a auto correction solution and this fourth solution is now requiring human","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2142s"},{"start_seconds":2151,"text":"collaboration this is the user knowing they're about to change a task sufficiently and informing the model model that be ready and I think that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2151s"},{"start_seconds":2160,"text":"would be very effective solution but to me perhaps falls in the same category as the human changing the session I don't","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2160s"},{"start_seconds":2168,"text":"know if you would argue that's the same thing no I think I think it's uh it's quite different um it makes more you think","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2168s"},{"start_seconds":2176,"text":"it's more natural yeah exactly yeah so I mean it becomes more like a human expert in that sense but you could say under","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2176s"},{"start_seconds":2184,"text":"the hood okay let's say if you're training users to use your system you say oh if you're changing tasks just let the model know and under the hood all","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2184s"},{"start_seconds":2192,"text":"the model does is actually reinitialize it itself but but if we want user to to inter intervene right the user might as","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2192s"},{"start_seconds":2198,"text":"yeah yeah it's uh we still require user to have the the main expertise of the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2198s"},{"start_seconds":2205,"text":"system they have to have when to intervene yeah yeah iation having the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2205s"},{"start_seconds":2212,"text":"automatic resolution and not the human collaboration yeah so so so you keep on mentioning agents","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2212s"},{"start_seconds":2221,"text":"right so under the hood APTA is still reliant on the on like interaction","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2221s"},{"start_seconds":2227,"text":"between agents right it's uh like architecture it is a uh multi-agent","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2227s"},{"start_seconds":2235,"text":"system maybe I can Define the agent better so agent can be anything from a very","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2235s"},{"start_seconds":2242,"text":"specific F un llm so a very complex thing to even just being a calculator so agent is literally just some specialized","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2242s"},{"start_seconds":2250,"text":"function and the llm is just something which is able to identify the relevant expert modules or agents we can call them to be activated what inputs to pass","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2250s"},{"start_seconds":2259,"text":"to them how to chain them together and once they've chained the outputs together how to feed it back to the llm so the answer can be synthesized for the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2259s"},{"start_seconds":2268,"text":"user yeah so so is it like under the hood yeah but but under the hood it is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2268s"},{"start_seconds":2276,"text":"still like multiple stru multiple specialized actors effectively and an orchestrator right yeah nice","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2276s"},{"start_seconds":2283,"text":"putting it yeah yeah uh so uh you mentioned uh go going back to the to the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2283s"},{"start_seconds":2290,"text":"earlier conversation you mentioned that uh like one of the problems is that if you don't have","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2290s"},{"start_seconds":2297,"text":"um like if you if you don't have a way to precompute some of the information that might be useful it is it it is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2297s"},{"start_seconds":2304,"text":"somewhat slow to get the results right so in the context of web three so so you're applying up today to the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2304s"},{"start_seconds":2312,"text":"hero uh uh offerings uh what are some examples of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2312s"},{"start_seconds":2320,"text":"something that is you know that uh that exists as that information that is computed and available to to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2320s"},{"start_seconds":2327,"text":"the I think a really good example is so give the general brief I would say we want to give insights which are in a way","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2327s"},{"start_seconds":2336,"text":"usually ring fenced for experts people who know how to do mathematical modeling people know how to do programming they know how","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2336s"},{"start_seconds":2344,"text":"to get these answers and even then it might take them a half an hour and hour so an example of a question someone might say is I am following","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2344s"},{"start_seconds":2353,"text":"influencer I would name someone to shame that for no reason influencer X okay on and on YouTube they've told me to go and","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2353s"},{"start_seconds":2362,"text":"buy Solana it's going to go up apparently now should I should I trust this influencer how often have they been","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2362s"},{"start_seconds":2370,"text":"correct in the past now on the Fly you could go and extract all the videos of this influencer transcribe them correlate them with what happened with","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2370s"},{"start_seconds":2379,"text":"the price and then go and answer the user oh this person's been correct 33% of the time that's way better than all the other influencers we've seen who are","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2379s"},{"start_seconds":2386,"text":"correct 1% of the time so go ahead and maybe trust them more than you would trust anyone else now a very simple expert agent you can imagine in the past","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2386s"},{"start_seconds":2395,"text":"in the background is for the top thousand influencers or top 10,000 influencers we could have pre-transcriptional","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2395s"},{"start_seconds":2428,"text":"in this case the example of the computation would be transcribing a lot of videos right ahead of time even retrieving I would say so preparing the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2428s"},{"start_seconds":2436,"text":"correct data streams to retrieve it transcribe it and Beyond transcribe it you have to use some clever natural language processing technology to identify what the content of what","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2436s"},{"start_seconds":2444,"text":"they're saying is which tokens do they refer to which direction are they saying those tokens are moving in so there some level of named entity recognition and","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2444s"},{"start_seconds":2453,"text":"then some kind of intent around those entities has to be extracted right so yeah that's an interesting U","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2453s"},{"start_seconds":2463,"text":"that's an interesting application one thing that is interesting is that if you if you like ask Gemini to look at some video uh it can immediately give you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2463s"},{"start_seconds":2470,"text":"answers about that video which is very that's like a mindblowing experience it almost looks like it already watched it right because for Gemini it can attemp","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2470s"},{"start_seconds":2478,"text":"to hold video at at once right so you can give it like a half an hour video and it will tell you oh yeah yeah that's what happens in it you know several","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2478s"},{"start_seconds":2486,"text":"seconds later which is fascinating but of course it doesn't scale like if you want to to watch the whole you know history of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2486s"},{"start_seconds":2493,"text":"someone's videos that's I think Beyond even the whole history of videos for a particular influencer what if you want to correlate","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2493s"},{"start_seconds":2501,"text":"it with 100 influencers or yes you want to do something which actually very cheap if You' already done it but very very","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2501s"},{"start_seconds":2509,"text":"expensive if you have to do it every time on the fly in a way MH I think another classic example is like anything","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2509s"},{"start_seconds":2516,"text":"which requires computation in terms of you to build an entire model around it or this long processing so the example I gave is long processing you get","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2516s"},{"start_seconds":2523,"text":"information you have to then go and break AP information prepare a database but there's also many examples where you have to train models so for example hero","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2523s"},{"start_seconds":2531,"text":"we're training a model to try and detect rug pools automatically you don't want to train this model on the fly but you can get this insights to inference on the Fly very easily I think that's like","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2531s"},{"start_seconds":2540,"text":"another classic example I would promote as an example of why this intermediary output structure is use when you extract","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2540s"},{"start_seconds":2548,"text":"deep insights in application yeah and so how far all of that so so is it in production or like","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2548s"},{"start_seconds":2554,"text":"is it in research stage where how far is it yeah good question so right now we're planning to launch the hero AI search in","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2554s"},{"start_seconds":2563,"text":"Q4 so end November December is the plan um we've in the phase of collecting feedback from internal testers and few","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2563s"},{"start_seconds":2572,"text":"select testers as well so as as you can imagine we came up with ideas of we thought was interesting but the idea of having a few testers also comes up with","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2572s"},{"start_seconds":2580,"text":"identifying what they think is interesting as well as questions to ask also a big part of this is when you just release a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2580s"},{"start_seconds":2588,"text":"product to someone they don't know what to ask so it's actually all about educating the users and how to exploit the platform best for the use case for","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2588s"},{"start_seconds":2597,"text":"example we could always have this influencer validator tool right if no one knows about it they would never even maybe ask the question which they could","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2597s"},{"start_seconds":2604,"text":"really benefit from but they don't know about it so that's a big part of how would you streamline the suggested queries or the interface of the system","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2604s"},{"start_seconds":2614,"text":"to allow users to exploit the most of the system right and so so here already has","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2614s"},{"start_seconds":2621,"text":"some of the like it has a wallet browser uh a payment platform right is it uh is it is it like part of the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2621s"},{"start_seconds":2630,"text":"bigger picture do they naturally feed to the search or was it more of uh uh creating early products the market yeah","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2630s"},{"start_seconds":2638,"text":"I think it's a very pertinent question so the idea here is that we want a single point of entry for all your web","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2638s"},{"start_seconds":2645,"text":"Three Needs or crypto needs in the space so even though the initial hero AI search is in a way allowing insights and","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2645s"},{"start_seconds":2653,"text":"information about cryptocoins or exchanges the next layer is looking to make it more executional so there are a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2653s"},{"start_seconds":2662,"text":"lot of like individual companies out there which have developed this kind of Technology anyway the advantage of having a single ecosystem with all the different tools it means your AI has","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2662s"},{"start_seconds":2671,"text":"ability to interact with all these tools as well because they're within the same ecosystem and so the idea would be then you could have your conditional","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2671s"},{"start_seconds":2678,"text":"execution such as can you always can you automatically go and buy me Bitcoin if it crosses 50k and sell it if it goes","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2678s"},{"start_seconds":2687,"text":"above 60k that's my strategy but I don't want to monitor it myself but you tell this execution agent not the kind of","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2687s"},{"start_seconds":2694,"text":"Agents we referring to right now to go and do this process for you and the whole point llm there is literally just an interface to what is very in a way","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2694s"},{"start_seconds":2703,"text":"very simple function under the hood and the advantage of having everything in one place is it makes it very easy to interface","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2703s"},{"start_seconds":2711,"text":"everything yeah so so like part of the idea there would be having access to the like effectively if the user is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2711s"},{"start_seconds":2719,"text":"doing it from within the browser that is integrated then there is natural access to the to the state","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2719s"},{"start_seconds":2727,"text":"uh to the current state of the user right yeah so in a way you can personalize it to the extent that user wants personalization if they want to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2727s"},{"start_seconds":2735,"text":"share their wallet address then they can go and do the transactions for them if they want to share other information and through conversations as well if they","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2735s"},{"start_seconds":2743,"text":"want to allow some level personalization they would have the ability to do so yeah and so given you investing in SE","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2743s"},{"start_seconds":2750,"text":"that heavily is is it your is it your vision that in the future people will be primarily interacting with the with the web through a conversation or like how","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2750s"},{"start_seconds":2759,"text":"do you envision you know like two years from now yeah I mean interfacing with so I I wouldn't say I would take","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2759s"},{"start_seconds":2768,"text":"credit for the vision of um there being a natural language interaction with surge I think that seems to be migration in the community anyways I think it was","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2768s"},{"start_seconds":2777,"text":"first really I guess um coined or really shown demonstrated through perplexity Ai and then I think everyone jumped on that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2777s"},{"start_seconds":2784,"text":"bandwagon like Gemini came out and you have their summary at the beginning every time you do Google search so I don't think it's necessarily A Vision","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2784s"},{"start_seconds":2792,"text":"but more of a need for how people will interact anything else will seem clunky and archaic you want to be able to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2792s"},{"start_seconds":2799,"text":"interact with your tools naturally your language because we have now tools such as LMS which can understand language very well there's no need for us to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2799s"},{"start_seconds":2808,"text":"inate code how we communicate even when you Google in know historically you're kind of coding your question this I would say um","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2808s"},{"start_seconds":2817,"text":"question top of my head okay say I want to know where is Prada in my location I say near me Prada why am I writing like","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2817s"},{"start_seconds":2824,"text":"that you know there's no need to exclude a filler words if I'm having a normal conversation and even though people got used to communicating that way in the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2824s"},{"start_seconds":2831,"text":"past I think people are very very quickly moving to a future where they want to communicate naturally with all the tools they","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2831s"},{"start_seconds":2838,"text":"have so again I would say it's not a vision it's a it's a basic need to have that interface","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2838s"},{"start_seconds":2847,"text":"yeah but I would emphasize this point it's very hard to make that expectation people have very robust natural language","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2847s"},{"start_seconds":2854,"text":"is so varied and so easy for machines to misinterpret what you're saying robustness is the key and that's why I really believe robustness cannot be","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2854s"},{"start_seconds":2862,"text":"achieved through a simple probabilistic framework you need deterministic functions under the hood or very specialized modules which do that thing","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2862s"},{"start_seconds":2869,"text":"extremely well and you only rely on the LM to do thing it does extremely well which is understand the language it shouldn't be doing much more than that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2869s"},{"start_seconds":2876,"text":"in my opinion y uh and so so out of curiosity why is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2876s"},{"start_seconds":2885,"text":"a um so so it seems like for hero the web three search is like one of the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2885s"},{"start_seconds":2891,"text":"core um offerings right uh is it like why why is it that you know like","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2891s"},{"start_seconds":2899,"text":"this core uh feature is developed in an external entity is it like have it happened historically or there is some motivation behind it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2899s"},{"start_seconds":2908,"text":"sure you mean like why are they partnering with APTA to get this developed question yeah yeah it's a good question I mean I think the simplest","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2908s"},{"start_seconds":2915,"text":"answer to that is it's extremely difficult I think today to make this work right now with the technology in place and as a team we at APTA so our","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2915s"},{"start_seconds":2925,"text":"research has been linked to this and previously to apter we ran an AI consultancy uh where we de develop products for lots of different companies","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2925s"},{"start_seconds":2934,"text":"around the world or the consulting or even due diligence for a lot of the VCS and based on that we already came across","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2934s"},{"start_seconds":2941,"text":"this narrative where we wanted to build something which allows deep insights through natural language without that latency and we had we didn't have a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2941s"},{"start_seconds":2949,"text":"partner to go and deploy that for we had the AI knowledge we didn't have the domain expertise to apply it so it just seemed like a very good fit for APPA to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2949s"},{"start_seconds":2957,"text":"work with hero they didn't necessarily have their tools or the expertise to build it or after had the tool and expertise but didn't have the domain","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2957s"},{"start_seconds":2964,"text":"knowledge to apply it so it just seem like a very synergistic setup in my opinion okay Al so after and here are","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2964s"},{"start_seconds":2972,"text":"completely unrelated I I thought it was the same team running feels like the same team nowadays but um two separate","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2972s"},{"start_seconds":2980,"text":"companies so Apon is actually developing this for the web 3 but in parallel we're also going to looking to apply this technology to multiple different","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2980s"},{"start_seconds":2987,"text":"Industries because as you can imagine it's very useful for any industry where you want a natural language access to deep information","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2987s"},{"start_seconds":2997,"text":"web the just seems like a really good an apt setting because it's so Rife with scams and confusion and noise it's a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=2997s"},{"start_seconds":3005,"text":"space where I think everyday users really benefit from having some transparency and some more informed decisions around the information out","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3005s"},{"start_seconds":3013,"text":"there and through natural language because they don't necessarily have the coding skills or programming skills to go and access that information","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3013s"},{"start_seconds":3021,"text":"otherwise and to to better understand UPA so so so the product of APTA is both","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3021s"},{"start_seconds":3028,"text":"the uh the infrastructure and all the agents on top of it right so all the agents are built-in house it's it's not","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3028s"},{"start_seconds":3035,"text":"like a API or a framework that others can can use to BU their right yeah so to build the actual specialized experts and","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3035s"},{"start_seconds":3044,"text":"functions under the hood we work very closely with the domain we're working in so in this case hero because we don't come from a native um crypto background","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3044s"},{"start_seconds":3053,"text":"to me I never knew people followed influences in the crypto space but as I've entered this space everyone follows influences a lot and seems to be a big","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3053s"},{"start_seconds":3061,"text":"driver of traffic and also a Big Driver of price based on what different influences say so this kind of domain expertise is proved by Hero so we","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3061s"},{"start_seconds":3069,"text":"developed the technology once we have provided the domain expertise in house and go and deploy it using our Air","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3069s"},{"start_seconds":3076,"text":"Technology but the source of what we build comes from the team we're working","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3076s"},{"start_seconds":3081,"text":"with Y and uh I guess besides the task switch uh paper are there any other","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3081s"},{"start_seconds":3091,"text":"interesting uh like research papers or any anything else like like maybe something about agents that is a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3091s"},{"start_seconds":3098,"text":"something people can look read and to get to better understand technology behind APTA yeah to understand technology","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3098s"},{"start_seconds":3107,"text":"behind APTA I think the AP end day because it's using so many different tools from large language models it almost comes down to coming summarizing","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3107s"},{"start_seconds":3116,"text":"the biggest limitations LMS and if you want to deploy them how do people solve those limitations so classic problems are like hallucination That's a classic","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3116s"},{"start_seconds":3125,"text":"problem everyone discusss and there's lots of solutions out there everything from rag to selfcheck GPT which is actually published by one of our","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3125s"},{"start_seconds":3132,"text":"co-founders then there's even agentic framework help solve that problem then there's a problem of bias then there's a problem of adversar robustness there's a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3132s"},{"start_seconds":3140,"text":"problem twitch as we give an example um these are things I'm going top of my head so if I you address this","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3140s"},{"start_seconds":3147,"text":"question you're saying I basically come say to yourself what problem am I having with llm and that kind of drives the research","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3147s"},{"start_seconds":3155,"text":"paper which is worth investigating because it's such a broad area each of these topics I would struggle to say this is the one paper to go and say it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3155s"},{"start_seconds":3162,"text":"solves everything it's more about incremental updates in each of these areas in sumary and and and as of today","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3162s"},{"start_seconds":3171,"text":"is it uh what is what is powering up is it like is it llama is it chat GPT what","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3171s"},{"start_seconds":3177,"text":"is the what are the underlying yeah so as of now um we aren't llm loyal as you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3177s"},{"start_seconds":3185,"text":"can imagine as as no one else is right now is the second we can get the fastest cheapest LM we swap do that so right now we're finding as most useful to use","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3185s"},{"start_seconds":3193,"text":"llama 3.1 because that gives the flexibility to find unit a bit more but as I said this is not what I would say is the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3193s"},{"start_seconds":3200,"text":"focus we're very agnostic to the llm um right now we're also actually investigating how we can get on to Gro","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3200s"},{"start_seconds":3208,"text":"to improve our speed as well under the hood M yeah but I mean in the future could like Sova they have also very cool","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3208s"},{"start_seconds":3215,"text":"chips being relaunched and released so as I would generally say this to everyone in the field there's no need","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3215s"},{"start_seconds":3222,"text":"for loyalty as a customer go for the best it's so easy to switch between them and everyone's providing access to very similar interfaces there's very little","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3222s"},{"start_seconds":3230,"text":"engineering work required to do the actual switch Yeah so uh uh speaking of loyalty","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3230s"},{"start_seconds":3237,"text":"I think some people have an opinion that the models are sufficiently different that you know switching the model is not just changing","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3237s"},{"start_seconds":3245,"text":"the API end point so so it's interesting I mean I think depends if you're talking about a coding model or","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3245s"},{"start_seconds":3252,"text":"you're talking about a race specialized function I would agree but the second you're using the models as we are as","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3252s"},{"start_seconds":3259,"text":"just an interface to understand language I think they all I as good as it need to be actually to be honest yeah so","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3259s"},{"start_seconds":3267,"text":"like I don't see for example I would never use a Chain of Thought model and that's heading there's no reason for us to do that we're already solving that problem through different functions or","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3267s"},{"start_seconds":3275,"text":"agents it's now literally only what inference uh speed and cost yeah the motivator","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3275s"},{"start_seconds":3283,"text":"in yeah so so you mentioned switching to Gro as of today are are you guys hosting them yourself or you using some other","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3283s"},{"start_seconds":3291,"text":"inflence provider we're using different Cloud providers right now I see yeah I think it's um comes down to hosting","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3291s"},{"start_seconds":3300,"text":"your own machines is very very expensive right now and doesn't seem like the most useful use of resources under you deployed and it becomes like a clear","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3300s"},{"start_seconds":3307,"text":"narrative you have also because we want to explore faster machines so if you host your own gpus right now and you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3307s"},{"start_seconds":3313,"text":"find the status quo is lpus through easy access in the future then there's no it's not worthwhile investing in that resource in my opinion right","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3313s"},{"start_seconds":3322,"text":"now yeah yeah and if there's someone else like hero who has a like a similar use case they want to build a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3322s"},{"start_seconds":3330,"text":"conversation agent they want to have a more robust uh specialized uh is like after already at","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3330s"},{"start_seconds":3338,"text":"a stage where people can reach out and and integrate or or like is it very much so but more in they have to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3338s"},{"start_seconds":3348,"text":"they have to guide the expert agents under the hood of course y because I guess the idea being we don't want to be the generalist the generalist is always","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3348s"},{"start_seconds":3355,"text":"kind of being done by the big companies out there we want to be robust for the specific business so the structure and framework is very much ready to be","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3355s"},{"start_seconds":3363,"text":"ported to different Industries but as you can imagine the teething process can be slightly complex sometimes you can have very finicky","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3363s"},{"start_seconds":3371,"text":"agents or very specialized functions which the LM do necessarily understand very well so you have to work with making sure it's robust as I say in 2","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3371s"},{"start_seconds":3381,"text":"hours anyone can build agentic framework it can take two months to two years to make it robust MH that's where all the clever AI tricks","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3381s"},{"start_seconds":3389,"text":"and research comes in yeah so so what is the interface like let's say I'm a company similar to Hero right uh will it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3389s"},{"start_seconds":3397,"text":"be my Engineers building agents on top of uh opta with like help or or is it or is","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3397s"},{"start_seconds":3405,"text":"it more of like me me sharing my domain knowledge but someone on the UPA side is ilding the agent I look forward to a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3405s"},{"start_seconds":3413,"text":"Utopia where we can almost provide like an eight endpoint to the framework and you can Define your own functions and your agents and they just feed into this API in a clever way I think that's a lot","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3413s"},{"start_seconds":3422,"text":"of like even like open AI a lot API are trying to do that right now with generalist llms or I've seen some Lang chain offering function calling that's a","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3422s"},{"start_seconds":3431,"text":"Utopia but from my experience this is not robust there are a lot of things you have to do to actually make it work well","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3431s"},{"start_seconds":3438,"text":"um I mean as a classic examples in context learning the llm has to be given a lot of examples of what kind of output","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3438s"},{"start_seconds":3445,"text":"from different inputs to give that specific agent it's a very small example that is and you don't know what unless","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3445s"},{"start_seconds":3454,"text":"you are building that agent yourself in a very close quarter to the llm you can't do a lot of these tricks which are required to make it robust so the","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3454s"},{"start_seconds":3461,"text":"current setup for us is at least they provide their domain expertise they may have even developed code or modules to do specific things we don't want to","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3461s"},{"start_seconds":3469,"text":"reinvent the wheel in any way we just want to interface it in many cases I think usually people have an idea what's important and they haven't developed it","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3469s"},{"start_seconds":3476,"text":"so then we work with them or if it's not necessary if we understand the problem well enough we can go and develop it ourselves yeah my uh my good friend IIA","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3476s"},{"start_seconds":3486,"text":"who's usually here with me but not today he I he had a child so congratulations to him yeah uh his uh his Utopia is that","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3486s"},{"start_seconds":3495,"text":"there's an agent that builds agents so you go to you know to an agent and you explain your use case and and the agent is sufficiently proficient in building","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3495s"},{"start_seconds":3504,"text":"agents um yeah we're gonna have a very very productive economy yep y okay almost out of time so","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3504s"},{"start_seconds":3513,"text":"uh um I would love to hear any closing thoughts about you know like how you envision the future what's uh what's next and","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3513s"},{"start_seconds":3521,"text":"uh yeah I think for my closing thought in one line summary would be let's use tools for what they're good at not","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3521s"},{"start_seconds":3529,"text":"overhype them and you can take that from it yeah but it's been really fun chatting thanks so much yeah okay thank you thank you","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3529s"},{"start_seconds":3538,"text":"for coming and I think thanks everybody for watching","url":"https://www.youtube.com/watch?v=meaoCJs5F2U&t=3538s"}],"youtube_url":"https://youtube.com/watch?v=meaoCJs5F2U","url":"https://near-pedia.xyz/videos/meaoCJs5F2U"}