AI in Production: Where Automation Meets Reality

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:10

   Thank you. Good morning. Hello, fellow refugees. Great to be here. I love that term. Let's uh let's start with some quick introductions and what you're focused on. Jessica, start us off. Sure thing. Hi, everybody. I'm Jessica Scrimmali. I'm a VP of product strategy and AI go to market at Oracle. Uh so my team drives awareness and adoption of the AI features that we build into our applications.

2. 0:32

   SU and I'm Swoon. Uh I'm a product manager at Robin Hood and I work on active trading products. But recently I've started working on uh AI products for active traders. And I'm Jeff Wiler. I host two podcasts, the people's AI and AI curious. My northstar is to try and bring clarity to the AI space. And one thing I love doing is talking to people who are actually using AI effectively in workflows like

3. 1:00

   not hype, not tomorrow, not in a whiteboard right now. I also love it when we're speaking to like red blood like true non kind of AI native startups, right? You guys obviously your companies have massive scale. So, I'm really curious to dive into what's working, what's not, where is our substance, where it's like, oh, lessons learned. Um, Jess, where have you found the most promising applications of AI solutions and workflows in your actual

4. 1:34

   day-to-day kind of what have you been able to do with that? Sure. Um, so Oracle, we are actually our own biggest customer when it comes to using the AI that we build into our back office software. And if you had asked me a couple years ago where I thought we'd see the most uh traction the quickest, I would have assumed a function like finance or supply chain, you know, structured data, um quantifiable ROI.

5. 1:57

   Where we've actually seen a lot of leadership is uh from our HR team and uh I think a lot of that has to do with risk tolerance. There's a bunch of different there's an array of AI features across um the risk spectrum especially in HR when you think about bias and regulatory concerns but there's also a bunch of kind of simple automation um and and AI use cases where pretty good is good enough and so that we've we've gotten a lot of traction with things like performance review kind of synthesis like goal setting um

6. 2:29

   populating your profile internally scheduling interviews when you're recruiting uh really simple use cases but very targeted use cases where um yeah, the leadership mindset, leadership mindset is let's just get out there, iterate um and we're okay if it's not perfect. So, one of the best um advice I've heard is folks saying is start small and start internal because if stakes are low, if

7. 2:54

   we make mistakes, we can kind of iterate. That makes sense. Swoon, I'm very that's a great segue though to you. Your your world is very different, right? like you're um when you have customer on the line that's not internal that's not starting small that's really high stakes how have you navigated that and what are the ways that you have injected AI into your systems yeah yeah so I actually think we're doing both in parallel so we are kind of like building out AI capabilities internally at the same time we're

8. 3:24

   building AI for customers and um I think we're learning a lot about AI through through that process Um, I'd say internally, uh, it's it's similar to what Jess mentioned, which is like we're starting with small various use cases across the company. So, like, you know, one thing recently that I did was I installed like an AI noteaker when I take interviews, which helps me a lot cuz that means I don't have to take notes. I can focus more on the

9. 3:46

   candidate. That's a small use case. I'd say more broadly, engineers are obviously using AI a lot. I've talked to a lot of engineers who say like they don't code anymore. I think that's a common thing a lot of you have probably heard. Um, for me as a product manager, I use AI uh really with every decision I make. So, it used to be that when I made a decision, a lot of that was on me. If somebody else around me had time to talk through that decision with me, I'd get a

10. 4:11

    second opinion. Otherwise, I would just make the decision myself. Now, I always get a second opinion. Uh, I always go to AI uh or or like just chat GBT, enterprise chatbt, always get like useful input. As for customers, um, yeah, I mean, I think it's it's a little bit of a different game. There's less like of a willingness for us to make mistakes. So, we have a long dog fooding process for any uh

11. 4:36

    All right, let's pause there. J back peel back the curtain. Jessica in the green room mentioned, oh, she asked, "Are you guys dog fooding?" And it's a term I How many folks here use the term dog fooding as a verb? And all right, I've seen uh maybe 10%. So, I didn't make it up. I was like, I didn't. So can can you unpack a bit what dog fooding means and how it's used in the Oracle context?

12. 4:57

    Yeah, so dog fooding is eating your own dog food or drinking your own champagne. So, you know, as product managers, we're building AI into the products that we sell and deliver to customers, but we're also using it to run our business. And so that was a little bit of what I was speaking to a minute ago uh about um yeah our ability our appetite to turn on the the features that we've built and and test them first and then see real value and go through an entire governance process just like our customers are doing.

13. 5:26

    I'd love to get a little more concrete on how this process starts with you guys when you're thinking about AI in a use case or for a product. Um are what's the what's kind of design flow like? Are you are you know whiteboarding first? Okay, here are all the things we can do which doesn't make sense and then we have kind of a robust kind of like you go through it and and figure out what to focus on or do you just like start vibe coding

14. 5:50

    immediately and and then kind of look at what's working, what's not. Uh talk about what the actual kind of um mechanics are on that front. Yeah, I can I can talk about it. So I I think the way we see AI at Robin Hood is that it's like a new technological primitive and any problem that we want to solve for customers, we now have one additional like really powerful tool that we can use to solve that problem.

15. 6:14

    And so like the set of problems we want to solve for customers is the same. But where we think about applying AI is like where do we think AI can make like a step change in solving this problem for customers where just using like 2.0 you know, or like traditional technology wouldn't create as good of a solution.

16. 6:30

    Jess, yeah. Um, I'd say at at the start it was a little bit like hammer nail syndrome, right? It's like, oh my gosh, AI, what can we build with it, which I don't I don't think was the right approach. But after that kind of settled down, um, we we really kind of take a a top down and bottom up approach to product uh development and strategy when it comes to AI just like we did our legacy

17. 6:52

    software features. So a lot of it is uh feedback we're hearing from customers. Um feedback from our internal team since we're customer zero. Um and then really um coming at the problem space with a deep understanding of the workflows and where there's actual friction um i.e.

18. 7:08

    problems to solve or jobs to be done where we can measure the output and feel really confident that if we move the needle with AI there well there is a needle to move and we can showcase that and and observe and and measure that for ourselves and for the customer. Um, so there's kind of a a two-pronged approach, but we we have some design principles. Um, we design for outcomes and measurement. We design um with integration in mind, meaning like we're

19. 7:33

    building it directly into the application. We're not doing some bolt-on Frankenstein thing with the AI. And then we design for trust and in the enterprise space that's huge. So explainability, transparency, all of that. So that's kind of the mindset we uh we have when it comes to AI product development.

20. 7:48

    I'm glad you mentioned measuring and metrics. Sorium, what kind of metrics do you guys use to quantify and track the effectiveness and ROI? How are you keeping track of that? Yeah, I think it's highly product dependent. So, I mean, consistent with like what I said before, which is we just see AI as a as a technological primitive, the metrics that we're measuring for AI success, at least as far as products are the same metrics that we've measured for product success.

21. 8:14

    And that depends on like what that metric is depends on on the product and what it's trying to solve. So for example, we have a digests feature in the Robin Hood app where for every stock you can see like a summary of of news like up to the minute on that stock so that you can explain why the prices move the way that it has. For that we might look at you know how often are users who are using the digest coming back to the

22. 8:36

    app. Like we have like a thumbs up thumbs down sort of feedback mechanism on that. We might look at what percentage of those are are thumbs up. Also we just kind of like do a vibe check on all of our products. though we read what customers are saying about it.

23. 8:48

    People talk about Robin Hood on on X and other platforms. They reach out to customer service and and they tell us about the product. So if customers love the products, we think it's successful. So yeah, that's that's sort of how we're measuring it. Yeah.

24. 9:00

    Jessica, what kind of metrics do you guys use? Yeah, so we look at um three dimensions really. Um adoption, efficiency metrics, and then some quality proxies. So adoption is like are people using the feature, but also are they coming back and using it again? um is it sticky?

25. 9:17

    Efficiency is you know hours or minutes saved, number of a reduction in tickets. There's a number of uh efficiency and productivity metrics depending on the use case. And then quality proxies are um some of we try to get into like business outcomes where we can measure them. Time to hire um amount of you know skills added to a profile uh things like that. Um what we're still figuring out

26. 9:39

    is kind of the trust dimension of um of ROI or or or of metrics. Um so we're starting to we collect feedback in the user flow, thumbs up, thumbs down um and some qualitative feedback from end users, but ultimately we want to get to a place where we can see, you know, did the user use the output that we suggested? Um did they have to course correct or um intervene in some way?

27. 10:03

    Things like that. So we're we're still on the journey. I can add one thing too. I mean what I mentioned was sort of like success metrics for the product. We're also measuring metrics to understand like correctness and trust of the AI solution as well. So we have evals for our AI products and we design them intentionally to communicate like did the AI succeed at what it was trying to do. So for example, if we're building a conversational AI product where it

28. 10:30

    builds you a technical indicator that that's one that I built. The evals might look something like did the AI fully understand the customer's prompt? So, was there any part of it that it left out? Or when the AI communicated what it was going to build for the customer, did the code actually match exactly what the AI said or were there any differences?

29. 10:47

    And then some of them are like some of those things are kind of, you know, a human will have to read it and judge it and eventually we can scale such that another LLM can make that. Well, I wonder who who is filling out these email forms, AI or humans.

30. 10:59

    So, so yeah, it's it's it's a good question. So AI, you're doing a good job AI. So at at first, you know, we actually, this was actually a big learning point for Robin Hood. At first, what we were doing was using LLM for those evals. So we would build an AI product like prompt another LLM to to check these things like, you know, did did the AI response match the customer's prompt? Did the AI code match the the like the spec that the AI laid out, etc. But we started realizing the we didn't trust the LLM's

31. 11:28

    response because we would read the the conversation. and we're like this didn't pass but the LLM passed it. And so what we learned is you know the first step now that we're doing when we launch an AI product is we are judging it and we are doing the evals and then we're sort of like training the LLMs on our responses and we're error correcting what the LLM say with what we say and over time that actually improves how

32. 11:51

    well the LLM as a judge does at scoring the AI responses and then we can kind of trust what they what they output. Then when we launch the product at scale and it's serving, you know, hundreds of thousands of customers, we can trust the LLM's response and we don't need to read every single uh uh customer prompt.

33. 12:10

    I love that. Before we get into the challenges and governance side of this, um I'm going to give you guys a brief moment to just brag. Uh talk about some of your favorite, coolest AI tools you've built. I think and I think the reason I ask is it's so easy to get lost in the abstractness of us all. Can you share a little more about some specific things you guys have built and the time it saves or the additional service and innovation it unlocks?

34. 12:36

    Do you want to go first? Yeah. Yeah. Yeah. Um I I think I think Robin has already shipped a few really awesome AI products. I mean I mentioned one that's the digest. Um I use that every single day. So, uh, I I mentioned it before, but you go on the on the stock page or crypto page, whatever, uh, on Robin Hood and you can see a digest that explains why the price moved today.

35. 12:59

    So, that's something that I think people had a question about why prices moved in a particular direction for years. Uh, you would go to the app, see a big price move, you'd have to like Google the stock and uh, sometimes you'd get a clear answer, sometimes you Why is it going down, SW? Can you Why is it going down? Can you let us know?

36. 13:15

    Yeah, I mean, it depends on the stock. Um, I think I looked at one I I often look at the Robin Hood stock one actually and and that's really insightful, but but I think that in that case I really love that product. It solved a problem that I think was really hard to solve for for many many years.

37. 13:31

    Um, but yeah, there's that's probably the best example I can think of. Jessica, how about you? Yeah. So um it's it's great be being uh or si sitting where I sit in the market and at Oracle because uh a lot of the AI we're building is not you know it's not the subject of a breathless LinkedIn post uh inspiring folks but it is actually creating real value. There's a lot of like very targeted almost internal um AIdriven automation uh that we're putting into place that saves real

38. 13:59

    minutes and hours. One of my favorites is uh is probably in the recruiting space where there's a number of different AI use cases and AI agentdriven workflows that can come together and make the end-to-end experience for both the candidate that's seeking a job at a company and the the company that's looking for talent um kind of find better better matches um and and and accelerate the time to hire

39. 14:23

    get butts and seat faster. So, it's not like sexy demo wear per se, but it's kind of these incremental innovations that we've woven in uh right where the work is happening uh that that is really showing um showing real value. So, I I find that exciting. I mean, I do work at Oracle, so that's that's what we do.

40. 14:40

    Well, for what it's worth, um, I've interviewed on the AI Curious podcast, head of AI for LinkedIn, IBM chief scientist, Upwork CEO, and it's a through line I've heard a lot and does not make headlines, but the answer when you really talk to folks is it's often the unsexy under the hood stuff that saves real time and solves real problems. So, totally agree with both what you're saying. Um, what do you find most challenging and what are the biggest still the biggest tech

41. 15:07

    challenges? Um, and what are any non- tech challenges? Where where have things gone sideways and where like, oh, this is not quite ready for prime time. Serum, what what is biggest challenge for you guys? Yeah, I guess I guess the tech challenges that I feel like I'm running into are that um, you know, AI I found is really great at making inferences on data that it has that it's been trained on, right? So it has an enormous corpus

42. 15:38

    of like text that humans have written that almost like comprises human knowledge and and whatnot and then it has an enormous corpus of you know Python and JavaScript and like you know very commonly written coding languages that it can write code in but when you ask it to write a language that's like domain specific which you know we have some of that at Robin Hood that's where it it struggles a little bit you know and we have to do a lot of work to like

43. 16:04

    error correct give it a lot of examples give it a a lot of documentation and eventually we can get it to work, but that's somewhere that I think it takes a little bit of time and it's not as like magical as people might expect when when when they expect AI to to code. So that that's like a technical challenge. I'd say a non-technical challenge is that um I think the direction that AI is headed

44. 16:26

    and how regulations uh you know especially in finance currently work are sometimes at odds with each other. So I think people expect AI to help them make decisions. So people expect that if they give AI a loose intent, for example, for a trading strategy, it can sort of help you define that really crisply and and say like this is the way that you would trade your hypothesis.

45. 16:51

    Uh but but regulation says uh that the customer needs to make every decision. And so if you just have a loose intent, somebody can't tell you this is the best way to trade that intent. You have to maybe it can help you formalize that. It can give you ways to turn your hypothesis into a trading strategy. It can't tell you exactly what the right way is. And sometimes giving customers

46. 17:12

    choice is a contrary to expectations because they expect the AI to think for them. And B, it can be overwhelming. Customers don't want choice. They just want something to tell them what to do. So, you know, we're figuring out how to build great products that are also compliant. And you know, I'm sure compliance will sort of evolve uh as AI does, but I I'd say that's one of the biggest nontechnical it's almost a design challenge without precedence in a way. This is dynamic fluid thing customer experience. We're

47. 17:39

    not sure what customers want. Jessica, biggest challenge for you guys. Yeah, I mean guardrails are a constant challenge for us given the nature of the underlying technology. Like AI is inherently probabilistic, right? and that is at odds with what the enterprise wants and needs especially in in specific functions or areas of the business. So trying to balance that um and deliver quality output you know it's like we're going to change the model on the back end because if if a new model performs better then we want to plug

48. 18:07

    that in but that inherently means that maybe the responses and each scenario may may shift and not be ex you know that's that's what probabilistic really is you're not getting the same outcome uh in a given scenario. So that's um a constant challenge from a technology perspective. Um but I actually think the more surprising thing that I've learned in this role is that uh the technology and the model you the model capability the technology challenges are actually

49. 18:31

    not the biggest blocker that we see. Um the the companies that have had the most success with AI have a fundamentally different mindset at the top when it comes to um how they treat the the technology. And and it goes back to that probabilistic kind of inherently uncertain nature of AI. The leaders that say, "Okay, I don't expect this new capability to be perfect are the ones that are able to get started quicker and see more traction." They don't have this zero defect mindset that I think was okay to have with legacy software, but

50. 19:04

    is just inherently not um doesn't jive with how AI works. They treat AI like they would a human employee in terms of okay, we're going to need to onboard this new thing or person and stay close to it, manage it more closely in the beginning like you would a new hire.

51. 19:18

    There's probably going to be a learning curve. There's going to be mistakes and we have um we're okay to we can accept those mistakes. So the leaders that have that um mindset create this psychological safety and trust uh across the business where the AI is brought to life and so folks can really experiment and and start to to learn and grow. So that's my been my biggest learning.

52. 19:41

    I I agree with that last part. I can add something on to there which is that I I think you know you sort of have to have that expectation Jess was was speaking to which is like AI is not going to be perfect 100% of the time but you know in financial services obviously that's you know problematic if the AI is not perfect and and you trust it to do something for you and so that's where I think we get around that problem by

53. 20:03

    being super transparent with what what the AI is doing and always giving you as a user a chance to kind of approve what what what it's doing. So if it writes you like an indicator script um we could abstract the code. So we could just make it such that you say hey this is what I want to see on on my chart. Can you like code that up for me? And it just shows you on the chart. Uh but what we do is we do expose the code and we give you a

54. 20:28

    chance to sort of like respond and and say hey something I I saw here wasn't quite right. And I think that sets the expectation with the user that this might not be perfect but if when it does work it can save you enormous time.

55. 20:40

    Couldn't agree more. And on the mindset front, I've my personal philosophy is that when leaders embrace a curiositydriven mindset when encouraging experimentation, exploration, and questions, and even play, that unlocks best results. We are out of time. Thank you both again. Please join me in thanking our panelists. Enjoyed. Thank you.