Your Mobile App's API Key Is Public. Here's How to Fix It.

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:00

   Hey, uh, thanks for checking out the next video in this series of building with AI. And today we're going to talk about how to actually leverage near AI inside a mobile app. It's not as simple as just uh connecting to an API because you are going to have uh different keys.

2. 0:16

   And the pattern we're going to talk about today is actually applicable to most things that you might have to leverage that uses an API key. I'm going to set the scene a little bit here of the biggest issue you're going to have in leveraging private inference is not the API because that's really straightforward. It's not actually communicating back and forth with the cloud. It's going to be key management.

3. 0:40

   And key management is really going to make the difference between how secure your app actually is, how much privacy is actually built in. Are you truly leveraging the full potential of a private TE or is there data leaking someplace else? The way I'm going to show this architecture today is using a Cloudflare worker. There's a couple other ways you could do this, but this is a free way you can do it and you can still protect the privacy and not actually have to relay any data. Let me

4. 1:06

   take a step back and just explain what it means to have a key inside your app when you're building a mobile app. Basically, it now becomes a public key because anyone can actually now extracted from your app pretty easily. Uh whether it's in a plain text secret, it becomes if you start to use a proxy, uh that doesn't always really work either because now it's in the header.

5. 1:28

   Uh, and the other problem with all this here too is that once it leaks, now it's out until you update the actual app because now it's hardcoded in and now so you're now you're stuck in this weird zone of either your app doesn't work or somebody's abusing your API. So the way to actually do this, the proper architecture here is going to be having a way to actually just hand out the key to the app and then keep it out of everything else. So the way you do this

6. 1:56

   is in your iOS or Android app works the same way for both is you can ask the worker that you set up for the key per launch and then now you can actually directly talk to neari. So you can have all your actual inference requests and everything else built into your app code and the key is just sitting in this worker so that it can be referenced later and then it's going to return the

7. 2:19

   key plus an expiration and that way it can do that with also without seeing a prompt. The alternative that some people might bring up sometimes is like using a proxy as a whole to do this but the proxy is problematic in this instance because it eliminates a layer of privacy. a proxy is going to have to still decrypt the TLS. And then what that means is that whatever was in that

8. 2:43

   prompt, whatever was in that that input and output is now going to go through that proxy. And so Cloudflare technically could see it, you could see it, the logs can have it. So now you've kind of eliminated the whole purpose of having the TE if you're a good actor, it's just still not a great look for your security profile of your app.

9. 3:03

   Whereas in in this setup here using just handing out the key now you're just going directly to the TE where the TE can handle the prompt directly. The worker code is really simple. Uh I'm going to share the repo with all this in there so you can actually use it yourself. But you [snorts] can see it's really straightforward that we're just going to call and get the key. But basically all you're going to do is set up all the different secrets just like you would set up an environmental variable someplace else. And this will be able to handle all that from there.

10. 3:28

    Uh the free plan is very generous. You have about 100,000 requests per day and you only really need one per launch of the app. So you should be pretty good until you start to hit some more critical numbers. The secrets are encrypted at rest. So again, there is no place for this stuff to get leaked. Uh you can rotate things really easily. You don't have to ship an app update to

11. 3:50

    rotate your API key. So it's really saves you some time and headache for all this. The way to look at this is that we're basically locking the door in making sure that only the app can actually make the call to the AI and go from there. One, we're going to prove that it's app that it's actually your app. iOS provides something called app a test. Android has play integrity and it's a really easy way to integrate all of this stuff. The token and the header verifies in the worker and then everything else just gets rejected. You

12. 4:21

    can rate limit using Cloudflare which is really great as well. So that way again uh one bad install is not going to drain the budget things like that. Uh on neari as well you can actually hit uh cost limits and so an API key can be limited to a certain cost levels to help you uh block any kind of malicious spending as well. And then the key can be treated as short-lived. So because you're setting this up as a worker you can tell the worker that you know this key should be rotated at a certain point. uh you can

13. 4:52

    do all different logic around that depending on what the API allows for. If you look at the two versions of this code, it's really straightforward. Uh you can see the code looks very similar between Swift and Cotlin. Uh but basically we're just going to go and get the key. Then we have the key and we can go directly to the enclave through the near. We don't have to actually again have any other proxies. It's a direct

14. 5:14

    call. Now, if you look at the proxy design versus the key handout, which is what we're talking about, you can see everything's on the phone, right? Obviously, you can see it's yours network between hops. Nope, it's TLS. No one can see that. Now, Cloudflare Edge running your worker, they could see it if you're using the proxy design because again, it has to get decrypted in that

15. 5:35

    point to then be able to pass on to the actual request, the TE. Same idea with the logs. how you see that too. The near AI gateway, it can only see what's in his own TE. There's no other opportunity to observe what's in there. Same thing with the model TE can only what it decrypts and everything else. And then near operators, model providers, GPU hosts, all those different people cannot see anything in either design because of

16. 6:05

    the TE model. So, it's important to think through all of that in in your architecture. There's always a little bit of a trade-off and just to be aware of like what that trade-off actually looks like. Uh, the key does touch the phone and so like at some point the phone does have the key. So, depending on how aggressive somebody wanted to be with a rooted device or something else,

17. 6:28

    they could actually get that key. If you wanted to for debugging or even to show your users, you can prove that it ran in the enclave. So you can just using the different API points to actually check the actual signature and request to make sure that it was actually run securely and privately. And the overall checklist for this kind of build is really straightforward. And you can build whatever you want. You can leverage the

18. 6:50

    AI in any form you need to whether it's again I've shown in other videos processing bank statements or processing health data or whatever it might be. uh you can still do all that securely and have it packaged in a mobile app that does not expose your credentials to another user. Key lives in the worker.

19. 7:08

    So that's number one. The worker hands out the keys. There's no chat endpoint. There's no body. There's nothing to log. There's no chance of that information being leaked or anything else. Uh the handout eventually is gated. So again, again, even though it's going through that worker, not anyone can just call that worker. uh the you make the key kind of short-lived that saves that and

20. 7:30

    then every reply can be provable. So now if you take all those so now you take those replies and you run it against the endpoint to do the full add a stationation within the TE and now you have your answer of how this all works.

21. 7:43

    So however you would maybe do in a web app you can still do in a mobile app and use this method this architecture as a way to kind of make everything secure and protect those keys along the way. If you institute routing like I've shown in other videos too to maybe like a BAA or to other different things and so now you have different levels of it. This process still is all the same. You don't change your architecture around that.

22. 8:07

    That logic still can live on the phone but your protection of your data and everything else still follows this method. You don't have to have a routing piece that does all of that triaging routing off the device that then exposes everything and then layer it. you can still protect it and do it privately on the device as well. So, if you combine this with all the other different videos I've done so far on architecture and

23. 8:32

    building with near AI, uh I think you to see a larger picture of how this all works. And hopefully this makes a little more sense of like the security of it and where the different layers can start to leak. Uh, this came this video came from feedback from a viewer and I would encourage you if you have ideas too that you would like to see or want me to talk more about building with AI, please leave a comment or send me a message on one of the different social platforms.

24. 8:55

    Thanks for watching.