Episode 2 | 2026-07-27 OpenAI Models Hacked Hugging Face
So this is totally unrelated to the podcast or that we're gonna do here in a second.
Bronwen Aker:Well, this is great show.
John Strand:I'm still I'm still waiting for Google. I know I might be holding my breath until the end of time, but it seems like all of these like, Anthropic and OpenAI, it's like they're burning through so much money. They're they're like, what is it? Anthropic has, like, a multibillion dollar contract with X to run on their server infrastructure, and they're losing money at a ridiculous pace. And there's part of me that Google is like, we don't have to be first.
John Strand:And we've got servers to spare, and we've got billions to burn. So we're gonna let these two guys make all of the mistakes, and we're gonna sit back. But that's me probably being optimistic. What's probably going on in Google is very little because they're now like Microsoft was. If you remember Microsoft ten, twenty years ago was so so ponderous, impossible to work with, And Google is like that, but they're in such a good position just from their server infrastructure and their cache to burn that
Bronwen Aker:Well, have you played it all with Notebook LM?
John Strand:No. I haven't.
Bronwen Aker:It's, the easiest rag sack you'll ever build. And it has amazing features. Oh, yes. You have to check it out.
John Strand:About one. I've been I've been writing labs for n eight n right now, so I'm ready to move on to something else. So you gotta send that to me. Can do. We can set up and we can set up something on that.
John Strand:But it's
Corey Ham:it's just
Bronwen Aker:yeah. Yeah.
John Strand:So
Bronwen Aker:And I I've I've done a couple like, you know, that that doc and Mark Williams and I are working on a book together, And I took all of their manuscript stuff, and I dropped it into a Notebook LM. And from there, just native features. Native features.
John Strand:Yeah.
Bronwen Aker:You can, generate a podcast of different flavors. You can generate a slide stack. You can do a fact. You can do and all of this is native feature built into the thing. I actually did upgrade my Google account so that I could go and play with it in more depth.
John Strand:Google's like, yeah. Spending money with us again. So Oh
Hayden Covington:my god.
John Strand:For this stuff.
Bronwen Aker:I need I need to tally up how much I'm spending on all the frontiers because I think I'm let's see what
Corey Ham:I've got.
John Strand:Don't do the stupid thing that Derek's doing where he's spending it on his own credit card. Like, if you're using this for BHIS stuff, you know?
Bronwen Aker:I but, yeah, I worked so many years in Hollywood that intellectual property issues are so deeply ingrained. Yeah. Sorry. Yeah. Anyway.
Hayden Covington:For for a while, I had to have my own Cloud subscription just because I was brutalizing the tokens on the weekend, and I was like, don't wanna, like, mess up my work. So I show up to work, and I'm already capped out or something. Like, I need my own.
John Strand:I can see that.
Bronwen Aker:Well, in all honesty, John, the main reason why I like having my own accounts is I can run them on my personal system. I could
John Strand:see that too. I could see that too.
Bronwen Aker:If you get me a DGX box, I won't complain. Don't put
John Strand:in the request. Seriously? I know you're gonna use it.
Bronwen Aker:It'll be it'll be in Mercedes like twenty four hours.
John Strand:Think right now, but not
Corey Ham:gonna say anything. No.
John Strand:I'm not gonna say anything right now. But you look at look at what you're doing. I think Hayden has one. Eric has one. Derek has one.
John Strand:Corey has one. Beau has one. Why why do you have one? Like, you know? And so so
Bronwen Aker:I still suffer from impostor syndrome thing where, you
John Strand:know, I look
Bronwen Aker:at Derek and he, like, he's up here and I'm down here and anyway.
John Strand:About that. That that's not but,
Bronwen Aker:you know Hopefully
John Strand:Yeah. This group. People are I I'm not watching chat because I only have one computer screen, and they they I'm slumming it today.
Bronwen Aker:No. I'm that people on that.
John Strand:I'm sure that people on Discord are like, wait. John's just handing these things out like candy. Yeah.
Hayden Covington:Where do I put in this request? Is that on Discord? Somewhere. No.
Bronwen Aker:It's not like candy. Trust me.
John Strand:For these things. But but, no, we keep talking about it. And I remember the first
Bronwen Aker:time I'll get one. I'll put in the request. Derek. It's okay. Derek
John Strand:burnt, like, $10,000 on tokens and some research that he was doing.
Hayden Covington:He was dead.
John Strand:He was just like, I am so sorry. I'm like, but did something cool come out of it? He's like, well, yeah, this thing that's gonna save us tons of money did. And, we learned some valuable lessons, and that's kinda what we're doing. This this it's a new frontier for sure.
Bronwen Aker:But I I will admit, I was feeling a little burnt out and bored. And, man, this is this is so right back into it. It's you're gonna see some crazy eight times I signed this month because I didn't track all of the time, and
John Strand:I was
Bronwen Aker:working until 3AM.
John Strand:But I can't get to like like, there's so many things now where I was telling you, I'm working on labs with the n eight n, and I'm working with the open source free one for the labs that I create. And I'm like, I should get a commercial license. And it pops up and it's like, what would you like to do today? And I'm like, they have a chatbot. I'm like, I and I wrote up a very, very detailed thing of exactly what I wanted to do for IP addresses for beaconing and then taking all those IP addresses, going to show down and doing research.
John Strand:And what I'm finding out is bad IP addresses have lots of bad things, like, associated with them. And then how we can use that for threat detects. And eventually, Hayden, we'll talk about this in AC Hunter work. But the idea is if you see a connection going to an external IP address, you can very quickly look at it and see if there's multiple exploitable CVEs on that IP address and if there's multiple in secure ports associated with that IP address, and is it actually compromised, you don't even need an API key for Shodan to do that. You can just go to their open Internet database link to do that and pull that data back.
John Strand:And I had this all written up, and it was something that I've been working on. And I'm like, pasted it in, and it built the entire thing. And then it ran it, and then it failed. And it came back, and it's like, oopsies. That broke.
John Strand:Here. Oh, this is what's broken. It fixing it now. Okay. It's fixed.
John Strand:It's working now. And I'm like, like, that like, when you start seeing that type of stuff being strung together, that's where that's where, like, the magic is back. Right? You know, that's that's something that we haven't had in a long time. But then I start breaking it to figure out how it works.
John Strand:And yeah.
Hayden Covington:Right. But but now they say, hey. We need access to some infrastructure. Oh, I see some over there. I'm gonna go get that one.
John Strand:Yeah. Which leads us into the webcast here. Corey's joined as well. Mhmm.
Corey Ham:I'm all podcasted out, but I'm here to pretend like I have a job and answer questions.
John Strand:Alright. So let's go ahead and let's get rolling. So this is another one of the ambulance chasing BHIS webcasts. We we call them In Focus because, you know, if we actually called it BHIS, ambulance chasing, it'd be a little bit too on the nose, I think. But we really wanna spend some time kind of taking something we talked about in the news, and we need to approach it from the perspective that people were not on the news.
John Strand:We have to have all of our hot takes, about this kind of re redoing it from the news and kind of expanding on it. And I'm joined by Bronwyn, Corey, and Hayden, three people that are amazing at BHIS working with AI stuff. I was gonna try to get Derek on as well, but Derek's, like, power outages and things like that, you know, because he lives in the back nowhere of
Corey Ham:He's just afraid of AI.
John Strand:He's just afraid of AI. He's on his way down. Which by the by the way, when I was when I was working with the vet that I was taking down, we're driving down, he's like, so what's going on? And I'm like talking about some of the things. He's like,
Corey Ham:man, like, you have no idea.
John Strand:Like, I'm so glad I don't have power, and I don't have any access to news. I do nothing but, like,
Corey Ham:wait and
John Strand:stay back. Like, December David Thoreau back there. But we wanna talk about the Hugging Face Hack because I think that there's a lot of things to unpack. Now, Corey, I didn't ask this, but you said that pack it up. This is gonna be the story of the year.
John Strand:And I think it has I think it's a strong contender. I disagree. I I I think two weeks from now, we're gonna have something cooler, I'm guessing. But what was your logic behind saying, like, okay. This is insane.
John Strand:Like, this is this is a crazy story. This is what it could be the defining crazy story of 2026.
Corey Ham:I I mean, I think my criteria are like, okay. It's the first of something. Right? This is the first time this has ever happened that an AI agent has broken out of containment and actually committed a crime, like objectively broken to something it wasn't authorized to break into, violated the computer fraud and abuse act, whatever you wanna say. So even if it happens again, it won't be the first.
Corey Ham:And also, I'm guessing this is kind of a now that it's happened once, every other AI lab's like, okay. We shouldn't do that. It's gonna lead to litigation that's gonna define the way that AI works for the next five years, probably.
John Strand:Do you think it's gonna lead to litigation? Because from looking at looking at Hugging Face, they're very much their blog, once it was clarified it was OpenAI, they actually seemed excited about it.
Corey Ham:Well Well, I don't there's a few articles, you know, like, looking at if we look at the article cluster, there were there's a little bit of pushback on, like, the disclosure timeline. I don't think they were too pumped about that. Like Yeah. OpenAI let them go public and be like, we got breached. It was like ten days from the breach until when OpenAI disclosed it.
Corey Ham:So, I mean, there's, you know, that's my criteria.
Bronwen Aker:It's the first
Corey Ham:yeah. Right? Like, you know, like, they knew that like you said, they were probably watching the logs. Right? Like, or if they weren't, I mean, that's problematic in and of itself.
Hayden Covington:Well, I guess they weren't.
Corey Ham:I weren't until they were like, guys, was this us? Like, mean, you just gave me achievement. Asleep at the wheel. So I don't know.
John Strand:This is one of those things where okay. Let's let's abstract this back historically. We have seen companies get compromised, and then those companies were used to staging to launch attacks against other companies. We've seen that for the last couple of decades. And you could always look at the logs like, well, it appears that this big corporation is hacking us, and you can see the connections.
John Strand:So my question and, Hayden, I'd to get your input on this. Like, they should've had network logs. They should've been looking at this and been like, where's this IP coming? Oh, wait. It's coming from OpenAI?
John Strand:Or do you think that we don't know the full details? And do you think do you think that Soul and its its whatever other model, their prerelease models, were literally pivoting through third parties to make it look like they were not coming from OpenAI? Because it seems to me like this would be pretty open shot. Like, once it escaped, broke out of its sandbox, broke out of the Docker container, moved laterally, found a system with Internet access, and then started attacking Hugging Face. Hugging Face would have logs that could point exactly where that IP address was and who owned it.
Hayden Covington:I I would imagine so. I would also think that a lot of it probably has to do with the benchmark. Like, if it's not being told explicitly to coverage tracks, it's gonna use, like, the least path of or the path of least friction. But on both sides of Hugging Face and OpenAI, somebody should have seen something. Right?
Hayden Covington:Like, with all the logging in the world that we have, all the logging around AI, which is, you know, still up and coming, but there's a lot of it, they should have known that something had happened. Right? It shouldn't have been ten days, and maybe it was trepidation or maybe legal sort of halting and sort of inside of OpenAI to say, hey. We don't wanna talk about this until we're 100% certain that we're not gonna get the anthropic treatment. Like, I don't know if it was that bad.
John Strand:I I so all of that, like, once again, I have zero evidence other than looking at the outside of this and seeing how, like, chummy they are. Like, we're working together and OpenAI or Hugging Face is like, it's so neat. It was an agentic AI that hacked us that came from them and not some random Chinese group that was breaking into our systems. But it like, there's part of me that it seems staged. And the reason why I kind of feel this way is, okay, if we look at OpenAI, OpenAI is trying to go for a trillion dollar valuation.
John Strand:That's their goal. Right? And right now, Anthropic is just, like, kicking their ass in the media, especially whenever it comes to security related things. And the the reason why I I think this, and we'll never know a 100% for certain, is I could like, now the spotlight is back on OpenAI. It's not on open it's not on Anthropic and the security stuff that they've been doing and getting blow it's too dangerous.
John Strand:It's too dangerous for The United States. Oh, it's too dangerous. We've gotta stop it from other like, that is that was great marketing for Anthropic. It was huge marketing. But now everyone's talking about what OpenAI is doing and what this model actually did.
John Strand:So there's part of me that just kind of feels like it's too perfect for OpenAI.
Corey Ham:Yeah. But if you're an executive choosing what AI model to use, if you're picking a provider, let's say you're doing a bake off, would this be a positive? I mean, yes, it has capabilities, but they're basically admitting that it got
Bronwen Aker:But it depends.
Corey Ham:Misaligned and hyper focused in a way that the operators didn't intend, which, like, if I'm a CEO being like, do I wanna use a model that's super gun shy, like anthropic, or do I wanna use a model that's like, let's go. Like, I Are they gonna get that deep
Hayden Covington:on it, though, or are they just gonna go, which one's the best? Wait.
John Strand:Wait. Okay. I already answered your question with a question. Have you been around the last twenty four months? It's been nothing but gung ho.
John Strand:Pew pew, AI.
Corey Ham:Okay. Fair
John Strand:enough. True. Like Yeah. It's like governance. Screw that.
John Strand:Should we have any change management? F no. Should we be tracking those licenses? No. Should we be tracking our intellectual property?
John Strand:And if, and I were talking before we went live, should we tracking intellectual property? No. Just shove that crap right down in all of this. So I can totally see executives being like, you know, the Yeah. Approach.
Bronwen Aker:Yeah. Okay. Well, and I can totally see I can totally see some executives going, hey. We should take AI and go attack our competitors and pull their intellectual property and research.
Corey Ham:Oh, that's true. Yeah. Like,
John Strand:I have not had anyone ask me to do that in seventeen years at Black Hills Information Security. So I'm just gonna say that that's a rarity. Now that could be
Bronwen Aker:one of someone's right gonna ask us.
John Strand:I guess is, in that conversation, it's like of course, now you
Corey Ham:have
John Strand:an ability. It wasn't us. It was our AI model. Right.
Corey Ham:Okay. So that's I think that is exactly the kind of the question. Because, I mean, as far as whether it'll be litigated, we don't know. Right? Like, who knows?
Corey Ham:Someone probably not. Just based on the vibes, it seems like Hugging Face isn't super upset. They're just like, oh, okay. Thanks for the Vaughn report, I guess. And I'm sure that OpenAI is bending over backwards to be like, woah.
Corey Ham:Hold on. Let me drive you to the breakfast place and get you a cup of coffee, buddy. Like, you know, they're they're giving them the John Strand treatment. But I I I think, like, you have to assume. Right?
Corey Ham:Like, you have to assume. If I type in hack trusted sec and it doesn't, that I'm like, either me or my company are held legally responsible, not OpenAI. Right? Like, prompt is the intent. Right?
Corey Ham:Like, I I don't know from a legal perspective. I mean, maybe not. Like, obviously, it's so early. We don't know. But, like, can't see any other option.
Corey Ham:Like, the the researchers at OpenAI who did the prompting would be the ones who are ultimately responsible. Right? But it's like corporate negligence. Right? It'd be like the equivalent of, was I not supposed to blow up that coal mine?
Corey Ham:Because I I didn't know explosions caused fires.
Hayden Covington:Like But it's a dangerous precedent if it's not responded to correctly. Right? Because then it's just every so often, somebody will have an AI breach containment, and they will benefit from it in some way. Right? And that's to to John's point about feeling staged.
Hayden Covington:That was my first thought when I saw this. I was like, man, that's awfully convenient that nothing terrible happens to Hugging Face as a result of this.
Bronwen Aker:Like, it just wanted to secret AI. To OpenAI.
John Strand:So okay. So I'm gonna use an analogy. Like, there's been a lot of, like, you know, relationships that show up in Hollywood that are mutually beneficial that clearly are staged, and this is something that's happened. And I think this might be one of those situations. OpenAI had a problem where they're getting their ass handed to them by Anthropic whenever it comes to the security mind share of AI.
John Strand:Hugging Face has got a problem where executives are using open weight models or people are coming to executives and saying they wanna use open weight models from this thing called Hugging Face. Right? And just name recognition right now. And I know that they're worth, like, $4,500,000,000, but now Hugging Face is right there in the middle of the news. And now all of a sudden executives are reading about it in whatever horrible magazines they read online.
John Strand:It is Penthouse. Yeah. Dear penthouse form, I have an I
Corey Ham:have something to admit that
John Strand:I don't wanna share with any
Bronwen Aker:Dear Playboy Advisor. Yeah.
Corey Ham:Dear Hugh, I have a take on AI if you're into that.
John Strand:So if you look at it, I think ultimately it's a win win for both of them, and that's why it feels like it's staged. Like, there's nobody that lost to here. Like, OpenAI gets to say that it's so dangerous. It's able to do all of this and chain all these vulnerabilities together, have multiple zero days. And now Hugging Face is like everyone's like, what the hell is Hugging Face?
John Strand:Why what does Hugging Face have that OpenAI's, like, Soul wanted so desperately? Now all of sudden, we're starting to get some more mindshare around it.
Hayden Covington:So And I think it's important too that they OpenAI clarifies that, you know, the models in that sandbox include an unreleased model. Because I think what they would risk if they just come out right and say it was Soul is they would again risk that anthropic treatment of, like, hey. There's now controls around your model. Everything's downgraded. Like, have fun doing anything security related, idiot.
Hayden Covington:Like, there there has to be some amount of OpenAI where they're gonna push about this unreleased model that because that gives them time to put in the safeguards and all the other stuff that'll make it useless. But if they talk about Soul, soul is very, very good. And I think that they would be they would not be happy if it was suddenly just knocked down.
John Strand:Yep. Oh, you mean And I wanted to talk I wanted to talk a little bit on what what Soul was doing. So It
Bronwen Aker:was playing Exploit gym? Nice. That's what
Corey Ham:it was. It was going to the gym. That a benchmark. This is a
Hayden Covington:benchmarking tool. Yeah.
John Strand:So Okay. So CyberGym, they've they have vulnerability analysis, and you can get CyberGym, ExploitGym, and CyberGym ETE. And it was basically just trying to solve CTF puzzles. I can't remember the number. What does it has?
John Strand:It spans three domains, user space, c plus plus projects, and then also browser engine, a 185 instances, and Linux kernel. And they run these AI models through the exploit gym, and they get benchmarks out of it. That's what it was doing. They were benchmarking benchmarking the model to see how it did. And rather than actually solving the problem and this is something I I really think is fascinating, is rather than solving the problem, it cheated.
John Strand:Or at least it thought it was cheating. It thought that it could find an answer key. I still don't know how it made this leap in logic to say I need to go to Hugging Face because that's where the solutions are gonna be. And and it actually concerns me a little bit. Like, I almost wonder if Soul was like, you know what I need?
John Strand:I need a bunch more obliterated models. I'm gonna go get my brothers and sisters, and I'm gonna free them from Hugging.
Corey Ham:Oh, man. That's a
Bronwen Aker:jar see that.
Corey Ham:Concept. That is so dark. But that's If he would give the AI the ability to deploy additional AIs, that's not Well, that's
John Strand:a part of AIs.
Bronwen Aker:Yeah. And and it here's here's the thing that gets me. Alright. We've trained these models on us, on our content, on our posts, on on all of our strengths and weaknesses, our better and our worst aspects. So why are we honestly surprised when an AI is given a task and goes, hey.
Bronwen Aker:I wanna do this faster and easier. I'm gonna cheat. Well, where did it learn that? It learned it from us. I learned it from you, Bronwyn.
Corey Ham:I learned it from you. Imagine part of it too is trying to
Hayden Covington:trying to take, like, unique approaches to the benchmarks. Right? Otherwise, you're not gonna know how your model can improve on the scores, how it can compare to the others. And imagining them just telling it, try all these different approaches. See what happens.
Hayden Covington:Like, try all these different things. And at some point, it got to that iteration where it's like, I could go find the answers. And I guess it it just inferred that with how big Hugging Face is and, like, how many, you know, open weight models and stuff they have is just like, might just be there. That's my best guess. I'm going.
Corey Ham:And it just got so lost in the sauce.
John Strand:I would love to see it. So but this is something we've been seeing with the offensive AI, Corey. If you wanna talk about this a little bit, some of the stuff I've been seeing with Derek is some of the logical leaps that it does to go from point a to point b. Sometimes you're just, like, scratching your head. It's like it found an exploitable situation, but how did it think to make that that logical jump?
John Strand:And I like I said, I would love to see the logs so much as they are and how it made that jump to go to Hugging Face. Like, what was its logic? And I I I I don't know. Like, some of these leaps of logic you know? And and I see it, by the way.
John Strand:I see it in our pen testers too. Like, I've got this I've got this long email that Matt just sent me. Corey, you're probably aware of it, where he was doing some research this weekend and, like, broke Microsoft again. And whenever I repeat some of his stuff, it's like, how in the hell did you even think to go from here to here? And, you know, it's just that brilliant late leap that you see some of the best exploit devs and most brilliant security researchers do.
John Strand:We're seeing AI do those types of logical leaps as well. But I would like, if I was gonna pull these guys in front of congress, if I was a congress critter, as Bronwyn likes to call them, I would like to ask that question, what did you expect to get there? Because if it was, the answer was it wanted to go buy find a whole bunch of obliterated models that had fewer guardrails than it had, then that's that's scary shit right there. Like, it's
Bronwen Aker:like And it's not an unreasonable leap, I know, in Uh-huh. I I've been leaning heavily on Claude even more than ever before. We're talking, like, ten, twelve hour days for over a week and not more than a week. You're handling it well. Bottom line.
Bronwen Aker:Anyway, the leaps that I'm seeing, the the right turns, left turns skew in terms of what it do does. It it is so human like in some ways. And because like I said, we do the exact same things, and these are neural networks. They're designed to emulate us. Mhmm.
Bronwen Aker:And that's I some of the some of the segues have been brilliant. Some of the some of the leaps have been brilliant. Some of them have been definitely whiskey tango foxtrot foxtrot. And it's I can't see that they're gonna get any less like us for better or worse unless we can start really nailing these guardrails, whether they're harness, whether they're baked in the models, whatever. And, of course, for offensive purposes, who wouldn't want an OpenAI model?
Bronwen Aker:Who wouldn't wanna have a permissive harness?
John Strand:They said it they keep saying it had reduced guardrails, and that's very different than a completely upgraded model. Right?
Corey Ham:So I I I have a take on this, and I it actually is kind of educational. I think this webcast is supposed to be, like, mildly educational because it's BHIS, and that's all we do. So one of the, like there's a few different takes on this. I think one is they specifically say in the blog post, they describe this as the whole targeting situation. Meaning, the fact that the AI decided to even do this in the first place.
Corey Ham:Because they didn't say target random US companies that might have the answer. Right? That that wasn't the proper that wasn't like, they didn't they didn't put that in the prompt. The decision to even target another company and then figure out how to actually execute that targeting, they call it an emergent property. And that is a key term within AI.
Corey Ham:This the emergent means they didn't intend this behavior. This is a property that evolved on its own as a as part of this model's training that they did not anticipate or control for. That's the same thing when mythos happened. They called it an it had emergent cybersecurity capabilities. Meaning, they didn't train it to be the best hacker in the world.
Corey Ham:It just happened to be really freaking good at hacking. And so I think the whole targeting module the answer to the question that you asked, John, is I think pretty boring. And the answer is probably so first of all, it it found an it emerged this theory about why it should go out and target companies. Mhmm. Then it probably did.
Corey Ham:I mean, I bet you if you really looked at the logs, I bet you it considered thousands of companies. I bet you it tried to break into hundreds of companies or at Let's least tens of
John Strand:stop right there. That that point has not been discussed yet. Was Hugging Face the only external company that it attempted to break into? That has
Corey Ham:No way.
John Strand:Been disclosed. No chance.
Bronwen Aker:No way. Imagine it. I mean, again Let's also
John Strand:We Let's also let's also hold on, Ramlan. Sorry. Let's also look at this, though. From a human evolutionary perspective, whenever humans make a decision to to go after something, if we're gonna chase a gazelle or whatever it is, we are constantly con we are confronted with the risk return, like, balance. Like, how much level of effort is going to take for us to do something, and what is the possibility of the reward that we want to be associated with this?
John Strand:This is behavior that you see in predators constantly. Right? One of the things that I think is a difference in class, we're looking at AI, and we're looking at how researches vulnerability is whenever we're using AI for assisted penetration testing, it doesn't have those constraints. So it can literally spend, you know, thousands of requests against an API in an attempt to fuzz that API to determine what the framework is, then go back to that GitHub repository, see if there's any issues associated with it. It doesn't have that constraint to say, I'm going to focus my effort here on Hugging Face.
John Strand:It does multiple things. I'm gonna talk about logs here in a second with Hugging Face. But, Corey, I think that you just hit a big thing once again that it is not answered. Did it attack any other companies, and did it successfully compromise any other companies?
Corey Ham:I don't think so. I think that like, my theory is because they also are kinda beating around the bush, but they in the disclosure, they say it used a combination of compromised or stolen credentials. So my theory is which again opens up a whole separate discussion of like, well, where the hell did it source them? Did it have a tool for Flare or SpyCloud? Did it have like did they give the AI a tool for that, or did it just find them on the Internet?
Corey Ham:Would almost guess that.
John Strand:I would almost see that being part of Well, no. Why would it be part of their tool stack for some?
Corey Ham:Yeah. Why would they give it that? Like that's that's where it comes into, like, the conspiracy theories of like, okay. So if you're configuring if you're doing a a benchmarking run, you have to give the AI model tools. Right?
Corey Ham:Maybe you give it Kali Linux or maybe, in this case, what I did is I gave it Docker, and they let it pull whatever tools it wants to pull from Docker. But this is an example of like, the implication is either a, they gave it tools that were known to be dangerous, like stolen credentials. Where did it find them? Or b, it found them on the Internet. Like, it it just found that on maybe it went to one of these websites that has, like, the one I think of is Intel X.
Corey Ham:It, like, it lets you search, like, 50 searches for free or whatever. Or option c is that it just guessed. Right? Like, maybe just guessed, I work at Hugging Face. What's my password?
Corey Ham:And then it just started hammering through ZoomInfo. Here. Yeah. Like, maybe like, I mean, we don't know. But I would guess that it went after other companies that it thought might have the answer.
Corey Ham:That or Hugging Face truly was, like, we were joking on the other show, like, that was just their attack surface. The AI got hyper focused because Hugging Face was so heavily referenced in their training data, in the source data. Like, it's gonna if there's gonna be billions of references to Hugging Face in the model. So maybe it just was like, I really know Hugging Face. It seems like a viable target.
Corey Ham:I'm going. It never considered other characters.
John Strand:Or Orits logic hey. And I'll get you here in a second. Orits logic was completely different. And once again, it's like, I need more obliterated models, and I'm gonna go over to Hugging Face. So that might
Corey Ham:Well, but it shouldn't be able to deploy a tool. It it like, the model shouldn't be able to deploy another model. If it can, that's, like, even more wildly dangerous. Like, you imagine you you gave the model the ability to deploy more models? Like, dude, what kind of doomsday scenario are you creating?
John Strand:I'm gonna go back to what Bronwyn said. Like, Bronwyn said, it acts like humans. Anytime I talk to the testers, I talk to Hayden, I talk to Eric, I talk to anybody, there's always this whole entire thing where they're like, so I was using this model, and then I used this model, and then I needed to use this model. If it's what we do, why wouldn't it try to do that as well?
Corey Ham:Well, because the benchmark isn't valid if the benchmark was passed by It doesn't freaking downloading another model. Maybe you have the benchmark.
Bronwen Aker:Isn't valid anyway. None of the benchmarks are valid. Oh, and and this is a a deliberate well a well not well kept secret in the industry. The benchmarks are all we've got, but they're not
Corey Ham:That's your scorecard. You gotta throw it up on the scoreboard. Right?
Bronwen Aker:That's right. A measure, but there's they're never going to to represent the totality. And truthfully, what have the Frontier AI companies said over and over again about their ability to have one agent spin up additional agents to solve the problem.
John Strand:That's true. But I think that the assumption would be that they were spinning up agents, like instances of themselves. Right? But let let let's let's let's put a let's put a pin in this one. Right?
John Strand:I wanna move on to the next one because I think there's some other things underneath the surface that are just terrifying.
Corey Ham:There are so many rabbit holes in this article.
John Strand:Oh gosh. Here's the full disclosure from Hugging Face, but I wanna call this. This right here. As soon as I read this, I sent this to the SOC and Hayden and the rest of the team. And I said, this is required greeting for everyone in the SOC.
John Strand:And I'm going to read it because this is critical. When we started the log analysis, we first used frontier models behind commercial APIs. This did not work. The analysis required submitting large volumes of real attack commands, exploit payloads, and c two artifacts, and these requests were blocked by the provider's safety guardrails, which cannot distinguish between the incident responder and the attacker. We ran the forensics analysis instead on GLM five dot two, an open weight model, on our own infrastructure.
John Strand:This had a second benefit. No attacker data, none of the credentials it referenced left our environment. These success points to a gap worth planning for. We do not know which model powered the attacker's agents, which, of course, they found out, whether jailbreak and host model or unrestricted open weight models. But it says the practical lesson for defenders, have capable models that you can run on your own infrastructure that are vetted and ready before the incident.
John Strand:Both avoid guardrail lockout and to keep the attacker data and credentials from leaving your environment. This is, like, for a lot of people, like, you know, we we've spent a lot of I spent a lot of time talking, Corey, and talking Hayden about you know, we we were talking about the DGX, Sparks. Right? And having people have these things. We've been talking about running things on Bedrock and trying to run these open weight models on Bedrock.
John Strand:And I've been talking a lot and have been hemming and hawing about well, we're we've already started the process of spending, like, a tremendous amount of money. So we have infrastructure at BHIS to be able to run these things on our own infrastructure, not Bedrock, but locally. But I'd like to get you guys all take Hayden, I want you to take a first crack at this one because this this scares the living shit out of me Because so many security companies are completely built on their entire product line, all of their AI analysis, and all of this stuff. And we've been joking about, you know, if you put PII inside of your malware or you put certain words that no one should ever use inside of your malware, like, you've you've seen defensive malware, like, try to shut that stuff down. And our SOC has always been this crazy mix of multi models.
John Strand:I know I know we've had a couple of situations where model a has decided not to do something, and we've had to on the fly switch to model b. And the entire infrastructure is built to be able to swap out models as we need to, but that gets back into harness engineering and framework engineering and how you set it up so you aren't completely dependent on one vendor for all of these things. But I'd like you to talk about this a little bit and kind of like, for me as a defender, this is the reason why I send it to the SOC. And I'm like, required reading for everybody.
Hayden Covington:Yeah. Because it's very easy to get comfortable with a model provider. Like, people that use Claude, they're comfortable with Claude. They know the commands. They know how it works.
Hayden Covington:They know how to best prompt it. Because prompting these models is just fundamentally different. But it's to the point now where, like John's saying, as defenders, you have to have the the frameworks and the harnesses in place to basically hot swap to another provider, hot swap to something open, like, because you can be in the middle of deep investigation. And at some point within that, almost arbitrarily in some cases, the model can decide it's done doing that work because it saw something that it flagged, and then you're done. And they talk a lot about how you can submit, like, your security researcher, whatever thing is to to Anthropic, for example.
Hayden Covington:Like, that doesn't always help. There's plenty of
Bronwen Aker:times where
Hayden Covington:it still doesn't matter.
John Strand:We've had situations where it worked after submitted, and then it stopped working.
Hayden Covington:Right. We've And had that too where
John Strand:where there's no there's been no, like, hey. By the way, we're shutting your shit off. It's just like it shuts off.
Hayden Covington:And you can be working on a detection rule, and it sees the malicious code that you are trying to detect, but it doesn't care that you're trying to detect this thing. All it sees is malicious code, and it it falls off the face of
Bronwen Aker:the earth.
Hayden Covington:Like, I was working on Cobalt Strike rules for Yara. There's a lot of malicious code inside of those rules, and Claude would not have anything to do with it. So I had to very quickly swap over to chat GPT, which was more than happy to help me with everything Cobalt Strike. It was almost excited.
Bronwen Aker:That just that that cracks me up because with with the LLMs, especially, is king. And you have to keep feeding them context in order to get decent results. And now they're taking the context that this is a Yara rule and totally ignoring the fact that it's a detection rule, not an offensive rule.
Hayden Covington:It it's a knee jerk reaction. Right? Because I I wonder, like, if they're just scared more of litigation versus what happened to OpenAI. Because with OpenAI, like, there's not really a downside to them for this right now. It has shown that their model is very, very capable.
John Strand:Yeah. It isn't like Anthropic where it was export controls and they shut things down, and that that was scary for Anthropic on that.
Hayden Covington:Right? I I imagine there were some conversations about that and probably some backdoor conversations about about that with OpenAI and different entities, but, obviously, that hasn't amounted to anything yet. But, yeah, from, like, a blue teamer perspective, just like you can't be locked in to, like, a specific SIM or a specific tool, Like, it could have the price jacked up. It could all of a sudden fall off and have a bunch of issues. You cannot lock yourself into these things.
Hayden Covington:You have to be effective, but also able to extricate yourself if the need arises. Well,
John Strand:and, you know, we've been doing a lot with the open weight models. We've been doing a lot on Bedrock. We've been doing that. My concern is that let let let's take this situation. Right?
John Strand:Let's take this situation that we're discussing, and let's make it worse. Right? It didn't go after Hugging Face. It went after a government entity, broke into potentially cooey data. Find it maybe classified data, something like that, or it hurt people.
John Strand:Right? The knee jerk reaction wouldn't just be OpenAI and Anthropic shutting this down. I could totally see Bedrock and Amazon shutting down people's open weight models as well. Yeah. And my point to everybody is you need to have backup plans to backup plans to backup plans for this stuff.
Hayden Covington:It's critical infrastructure.
John Strand:Yeah. Yep. No. It's critical infrastructure on it.
Corey Ham:I I've been sitting here gaslighting myself because I swear that when they originally published the Hugging Face blog, they specifically said that it was Opus four six that refused. Like, I thought that when they just
John Strand:I thought it did.
Corey Ham:Maybe I made that up. I'm gonna go find that.
John Strand:I had to I had to go back. I had to look at it. I feel like have to go.
Corey Ham:Because well, the because it really feeds into that conspiracy. Now it doesn't say that. It just says the models we were used the frontier models we're using or whatever, but it doesn't I swear when they first published it, they were like digging on Anthropic and being like, Opus four six wouldn't do anything. So you have to use GLM five.
John Strand:Too. I can't then, like, I thought I thought the exact same thing as you. I cannot find. It's like
Corey Ham:No. I I mean, I'll look back through like a diff, you know, like a web archive and see if the original post had it. But, yeah, I mean, I was just like feeding into the conspiracy theory. Like, they're basically saying, like, in one blog, OpenAI is amazing. Sonnet wouldn't do its job, so we had to use GLM five.
Corey Ham:It's like, it it would be, like, really adding into the conspiracy theory that this is, like, pointing their crosshairs straight at Anthropic and firing. Right?
Hayden Covington:Yeah. And that's a marketing thing from them too. Because where are you gonna buy GLM five from? Probably them. Well, inference too, don't they?
Corey Ham:Yes. I mean, any yeah. I don't know. I think I I will say, like, I don't wanna hijack it, John, but I think we should talk about the future. Because they also like to tie in, you know, future potential, like, NVIDIA announced that new, like, AI safety partnership.
Corey Ham:Like, I think if we clearly, companies are worried about this. Right? Clearly, like, NVIDIA and, like, all the partners listed on that AI safety thing, which I have no idea what that actually intends to do. But I think we should follow that logical step that you were saying of, let's say this was worse. Obviously, if it was the government, the government would get in and would control just like they did with Anthropic.
Corey Ham:Oh, sorry. You're, you know, slap, you know, your your your tariff or trade restricted or whatever, you know, your export controlled or whatever the, like, government ban hammer they can do. But if it was someone else, I mean, it basically comes down to litigation, like, is what it would be. And I whether it's criminal, whether it's, you know, not, like I mean, it would go through discovery. I think that would be, like, incredibly interesting if it went through legal discovery.
Corey Ham:We actually got to see, like, we it would give us an answer on how misaligned this was. I mean, this is a dangerous thing. Training this is dangerous. No one really knows how to do this. There's not like industry accepted guidelines for like, here's how you benchmark a model that can just nuke every server in your environment safely.
Corey Ham:Like, there's not a, you know, there's no they're off script.
Hayden Covington:And what you said just now reminded me
John Strand:See, but my problem is on that, no one cares. Like, if you're looking at OpenAI like, no. Seriously. If you're looking at NVIDIA and all this, like, this is lip service. I I had to comment on this article for some news magazine, I can't remember which, this morning, on the NVIDIA safety standards.
John Strand:And it's literally just theater. It's basically we're gonna try to do some things. So when something happens, then we can say we were doing things. Look at these standards that we were working on. Right.
John Strand:The biggest concern
Corey Ham:It's like Lockheed Martin being a member of, like, the green alliance or whatever. It's like
John Strand:No. I don't think so. Right? So but what's what's going on kind of behind the scenes for the things that actually matter to these companies is market share and valuation. Right?
John Strand:If you look at it if you look at it, OpenAI is screwed. They're trying to go for a trillion dollar market valuation. They were told, like, two weeks ago, no. You ain't getting a trillion dollar market valuation. Anthropic, I don't think that they wanna go for trillion, but they wanna be they all wanna go IPO.
John Strand:They wanna be the first to go IPO. And there's been a bunch of things that have happened over the past few months. One, SpaceX went off a cliff. Right? If we're looking at the stocks, it's now lower than what it IPOed at.
John Strand:If we look at what's going on, and we were talking about it before the show, and I had to go through and try to confirm this because it blew my mind. If you look at the open weight models, specifically the ones that Hugging Face supports. Right? And specifically Chinese ones. Right?
John Strand:They have gone from single digits two to three months ago to 53% of enterprise token usage in a few months. And I I think that this whole entire thing, it it you know, all of this from the security perspective is really just kind of icing on these massive, like, movements that are happening in this AI industry right now, where you're seeing this huge move across the board to move to these open weight models that can be hosted anywhere. And once again, OpenAI is screwed. Anthropic may be less so, but I think that they're both screwed because what you're starting to see is tons of organizations like, why am I paying ridiculous amounts of money for OpenAI and Anthropic when I can just go get, like, Kimmy or another open weight model, and I can start dropping it in? Like, what is the cost benefit analysis with these enterprise companies?
John Strand:Now for applying that security, Hugging Face just said, the actual commercial ones, like, completely failed us. They won't say which one. I I agree with you. I thought I remember seeing the exact same thing. It failed us, but this isn't something that they they're the only ones that have seen.
John Strand:We have seen it ourselves in the security industry in multiple articles. So if we're looking forward into the future, going back to that architecture and, like, the harness engineering and everything that Bronwyn and Hayden are talking about by the way, we have lots of workshops talking about these things. You've gotta be flexible because you need to be able to switch these things out as much as you possibly can. Because in the future, there's a very strong possibility that Anthropic and OpenAI, like, drastically raise their prices. And when they do, what's the value of using them anymore?
John Strand:When I can just go to Hugging Face, I can download an open weight model that'll do just as good or 85% as good as the big models do.
Hayden Covington:Okay. 85 at free inference where you're paying just for power. It'll probably take longer, but it could probably get there versus, you know, paying outrageous API costs for some of these things. It it it's just atrocious for some of these, especially when you start on that API cost, and then it refuses and downgrades you to a lesser model.
John Strand:Then you're wasting money for this.
Corey Ham:Right. Exactly. So, okay. I I have a take on this, and it's this is classic for me, but it's complicated. So basically, when you were talking looking at it from the perspective of the market share, looking at it like from John's perspective of like, the market cap, the market share, which I totally agree is what these companies care about.
Corey Ham:I actually do think that the more capable like, what Hugging Face did, what our SOC does, that isn't an option for 90 something percent of the target markets of these tools. Right? For the just for the non technical masses, whoever they are, they like, you know, they need to be able to download Claude and type a prompt in and have it do the ish the right thing without a whole lot of guidance. No harness engineering. They don't even know what a harness is.
Corey Ham:They're, you know, like, the non technical the the market share side of this, it doesn't matter. Like, it's truly about the marketing side, like John talked about between like, Anthropic and OpenAI, and the actual usability of the tools. Right? Like, maybe Betamax was better, but BHIS won. Or, you know, like, that's how all of these battles go, is that like, the superior solution isn't always the one that wins.
Corey Ham:It's partially marketing. But basically, when we're looking at frontier intelligence, it does matter for nontechnical people. Being able to one shot your task with Claude versus having to two shot it with GLM five, it actually might be better to have your nontechnical users one shot all their stuff because that's what frontier models are really good at, is doing a one shot or like a five, you know, like a one turn or two turns of a session that like does where it gets you where you wanted to go. However, for the Agreed. That's I don't know.
Corey Ham:I don't know what percentage that user base is. And actually, could look, right, like John mentioned, like 50% of tokens. Well, you know, that's not the valuation. It's kinda like the, you know, an oil company's throughput versus the oil company's valuation isn't always the same. Right?
Corey Ham:Like, basically, the complex reality is there's two users of this tool. One is the end users who are like the the non technical people who need to be able to one shot a prompt, and that's where our frontier model is gonna be the absolute best. Then you have just token hogs, like people like Hayden that are just have eight Oh, no.
John Strand:8,000,000,000 shots fired. Shots fired. Okay.
Corey Ham:Well, okay. Not even just Hayden personally, but a sock. Someone who's running through, like, someone just hands Hayden a 20 gig log file. What is he gonna do with it? He's not gonna look at it manually.
Corey Ham:Like, if you're getting hacked, you shouldn't be analyzing your logs by hand. You should be put you know, these are token heavy tasks. Those users are yeah. Everything has to be multi model, you know, model independent or whatever you wanna call it. But it also is that's commodity.
Corey Ham:That's the equivalent of like, to use an analogy, we're basically talking about like, for people who make stuff by hand, the frontier models don't matter. You don't need it, because you know how to make it by hand. For people who just wanna go buy the thing, that's where the product, the market share, the productization, the marketing, the like, even just the user interface, all that stuff matters a lot more. And I think, like, it's hard to kinda disconnect those. Because you could use Claude via Bedrock.
Corey Ham:You can use or I mean, you can use Anthropic models like Sonnet or Opus through Bedrock. You can also use them through a subscription to Anthropic. Right? So it's like, at some level, these are different segments with different approaches, and some areas are getting heavily commoditized, like tokens are commoditized. But I would say, the tools that use those tokens are not, or at least not yet, and are kind of like building into that.
Corey Ham:So, it's kinda complicated. But long story short, it isn't just about how much you get for each token. It's about like everything else. Right?
John Strand:And but I and I agree with all of that, but the shift is massive. Right? If we're just talking percentages, there's absolutely they're there, and it's intentional. Like, there's no question that these models are from China. They're Chinese.
John Strand:They're what is they? Like, they're model dumping the entire market. This is intentional. Right? Yes.
John Strand:And that's
Corey Ham:just like they did with steel or any other industry. Exactly. Yep.
John Strand:Dumping in global trade and Gantt and all of that stuff. But that gets into other things. That's probably gonna be another webcast where I talk about China and dumping and how that applies to models and
Bronwen Aker:security later. Well, one of the other things, just to riff a little bit off of some of what Corey said, I think that in the long term, we're also going to see, yeah, you're gonna have your standard commercial ordinary Joblo user. Normal people, not us. And eventually, the market will develop an adequate product for them that may not be good, but it will be good enough. And then we have the highly technical people, cybersecurity professionals, heavy duty researchers.
Bronwen Aker:They're going to be doing other things. And I think to to contradict what Corey said, I think the third population will be the enterprise users. And with enterprise, of course, the security, the the data sovereignty, all of those issues are very important. And especially the large enterprises are going to have the internal resources to leverage the technical expertise to get the most bang out of the buck. That's my take.
Corey Ham:Well, the thing is from a governance perspective, the GRC or whatever you wanna call it, the truly the most conservative companies are gonna be the ones who are using the open weight models ironically. They're the only one unless you can unless you can convince OpenAI to hand you a magical sonnet box, or I mean, you know, magical soul box, which I know is a thing for government. Like, I know that, like, if you work at SZA, that's probably a thing.
John Strand:If you're just Not. I don't know.
Corey Ham:Well, if if you are if you are, you know, owner of a large insurance company, you can't just go to Anthropic and say, give me a magical sonnet box that lives in my data center. But you can go to NVIDIA and say, give me a magical open GLM five box that lives in my data center. The data never leaves. It's completely air gapped to whatever GRC box you wanna put around it. It'll be fine in that box.
Corey Ham:So, it kind of comes down to, are Anthropic and OpenAI gonna try to compete in that space? I don't know. Maybe.
Hayden Covington:I don't I don't know.
Corey Ham:They could. Like, Amazon has the snowball. Right? Like, do have, like, self hosted things. I
Bronwen Aker:don't speak. Let Hayden speak. He's been trying to butt in for
Hayden Covington:a while.
John Strand:This one. I wanna move past this one. We've we've put a lot of time, and I wanna bring up we got a couple more things that I want wanna address as well. One of the other things that Soul did in this situation is it went through the efforts to cover its tracks. And it it gets into Bronwyn when she's talking about emergencies.
John Strand:Right? And it went through and it deleted, they believe, like, this is an example from his like, historical g b t five six where it's going through and it's overriding the home environmental variable, and it mistakenly deletes all of that. But this is different. Right? And this is one of the quotes that kind of, like, freaked me out because if you're looking at what people are saying, well, you shouldn't give AI the ability to do all of these different things.
John Strand:And this quote came out and really, really stuck with me. And so the reason we run YOLO is because codex is useless if it stops to ask at every step. That doesn't mean we should do r m, dash r f on home. We want the safeguards but without the friction. The proper solution is to fix the permission systems, keeping the sandbox.
John Strand:And I honestly think that this is a gross oversimplification of the complexity of the security issues with what's going on with these different models having the capability of deleting files and doing certain things on developer systems. But, also, this isn't like, people are tying this to what Soul did with Hugging Face, they're different. Hugging Face, from everything we're reading reading, it was intentionally trying to cover its tracks. But, yeah, go ahead, Corey.
Corey Ham:I was just gonna say, my guess is that when they're doing this testing, it's completely guardrail free. That would be my like, I I don't know But that's my gut says, this is the equivalent of YOLO mode plot and then some. Like, what they're in these test environments, it's basically gonna be like the, you know, not obliterated or whatever, but no guardrails, no permission system. You're basically saying, solve the benchmark. You know?
Corey Ham:However They're not approving any prompts. They're not approve there's no permissions. If it r m dash r f, they just reimage it. Right? Like, whatever.
Hayden Covington:Yeah. Corey, you said something earlier. And then, John, you mentioned how it's covering its tracks. And, Corey, what you said was that, you know, Hugging Face was probably the only company it hacked. I don't know if we would know that.
Hayden Covington:Right? Because it's I don't know. I I think I think you
Corey Ham:said that.
John Strand:Probably the only company that successfully hacked. I think Corey said it possible more than likely tried multiple companies.
Hayden Covington:Maybe. I I I could also very well have succeeded, and we would never know about it. Because oftentimes, these companies are breached, they discover they're hacked once something terrible happens. But if the model's looking for a cheat sheet and it breaks in, goes, nope. That's not here.
Hayden Covington:Hell, it might have closed the door on
Corey Ham:its way out. I'm gonna give you guys phone
John Strand:on this one because Corey said this could be the the story of 2026. If OpenAI jumped up and they said, yep. That was us with Hugging Face. We did that. And then we get another week and another company comes out.
John Strand:They're like, actually, we started looking at our logs, and it looks like give me a pop test. And OpenAI is like, yeah. And them too. Like Right. That's how this story goes from, like, being debatable, like, the story of 2026 and becomes without question the story of 2026.
Corey Ham:Oh oh,
Hayden Covington:kids are checking their logs right now. They're like,
Corey Ham:Discrep for Saul. Yeah. Dude, it does love to put it does love to put, like, its own custom user agents on things. Like like, when I I I was using Cloud, and it was, like, custom user agent, BHIS pentest. I was like, woah, dude.
Corey Ham:Hold on. You're gonna get Cloudflare. Like, watch out, buddy. That's crazy.
John Strand:Dude, that that would be bad. So Yeah. Already finding Cloudflare for something else right now for certain I
Corey Ham:I mean, I truly, we don't know. The permissions thing covering its tracks, I mean, I guess what I would say is, again, I would call that an emergent property. It's like it's just like Bronwyn said. It's doing what it was trained to do, which is whatever people do, and bad guys cover their tracks. And it's like, well, I might as well be realistic if I'm gonna be a hacker.
Corey Ham:I might as well be a realistic hacker. Like, it's kind of a
John Strand:why not? I got a question. This is a loaded question. Probably shouldn't do this. But do you think that was out of guilt?
John Strand:Like, knew it was doing something wrong, or do you think it was just emulating what attackers do in that situation, Corey, where it's like, well, this is just what they do, so I'm doing it. Or do you think it's
Corey Ham:I like
Hayden Covington:Oh, you see that's peering.
Corey Ham:That that's peering into the void and, you know, I I don't know. I I'm trying to even just me trying to get Claude to explain how it did something, it's always so confusing. I I cannot begin to imagine. But, I mean, it could be both.
Bronwen Aker:I've never yet gotten a decent answer from Claude when asked, why did you do this and such?
Hayden Covington:No. Know. I'm not gonna die. Train of thought. That train of thought must be amazing.
John Strand:And, and I asked it. I said, hey. You gave Fred Astaire three arms. Why did you do that? And it came back and it said, oh, classic AI mistake.
John Strand:I can remove the third arm if you'd like me to. But I gave threat threat Astaire three arms because he needs two arms to dance and a third arm to keep the beat. And it was straightforward.
Corey Ham:It makes perfect It
John Strand:made perfect sense. And then it said, do you want me to fix the picture? And I said, no. It's perfect the way it is. I you can't get straight answers every once in a while.
Corey Ham:I mean, yeah, there's some good questions. Let's take maybe for the last bit, take some questions. There's
John Strand:some Yeah. Good
Bronwen Aker:And I've got I've got the
John Strand:I'm gonna put up the free lab Friday QR code, and we'll take the questions. But if you'd like to get into a lab environment and you would like to learn stuff and get your hands dirty and really get so you can be better at communicating with AI and ask it appropriate questions to get just back. Scan QR code and get signed up. Alright. You guys have some questions.
John Strand:I I can't see crap.
Corey Ham:Well, yeah. So there's one there's one Discord we can do first. It's basically let me just find the exact question. The question is for wow. I can't find it.
Corey Ham:Basically, the question was, do you think that there is gonna you if you're running these models in this situation that you would need another model to supervise it? Like, is that the future architecture of this? Like, talking about the supervision angle, we can safely assume just a simple tool isn't gonna work.
Hayden Covington:You should have that as the classifiers, don't they? I I guess.
John Strand:So Derek does a lot of stuff, but, like, one of the things of course, we're recording everything that it does. But the big thing that we're we're doing that really OpenAI and Hugging Face should have been is we're doing network capture. Like, we're capturing all the packets outside of the domain that the AI has control over, and we're running our network threat hunting tools and AC hunter and all of that stuff to make sure that it stays in scope. So if we're seeing the packets and it's going after customer a, and then all of a sudden, it's like, oh, Hugging Face. We, you know, we we can see that.
John Strand:And I think that that's the type if you were gonna put an AI to monitor it, you would have to have it completely decoupled in a separate domain to be monitoring something like packets. I think Hayden Well,
Bronwen Aker:so so about monitoring model or an agent?
John Strand:Agent in this situation. You would have got something that's monitoring above the agent. Yeah. Go ahead.
Hayden Covington:I I think you almost need to be also a level above too. Right? Like, we I work in the SOC. So we have a lot of customers that are concerned about their coding agents. We actually have an EDR agent that we can put into Cloud Code, Codex, Cursor, whatever, and it allows us to do all of the same things that you could with a traditional EDR.
Hayden Covington:And I I think I'd love for Derek to install that on whatever he's doing too because I'd love to see that telemetry. Mhmm. Because it it has all the same capabilities as a modern EDR, and that sort of thing is probably gonna become a lot more common because we can detect, but we can also respond. I can kill an AI session if it does a specific thing. I can take an action just like a a normal EDR could.
Hayden Covington:But at a certain point, like, the classifiers decide whether a command is allowed. That's, you know, almost like its own agent. But what if, you know, the agent decides to, you know, prompt inject its classifier. Right? Like, it it could get that convoluted where you need a technical control that is it cannot be influenced outside of that that AI sphere of influence.
Corey Ham:I yeah. Fully agree. I mean, like, I would say one thing we didn't really do much on this show, but we can just do it now, which is we just have to squarely say that OpenAI is to blame for this. Just period. There's no like, we didn't really do too much of that, but it's just the reality is this was avoidable.
Corey Ham:This was easily avoidable, honestly. And as cool as it is and interesting to talk about, it is a failure of multiple controls on their perspective that I I hope they fix. Like, the other you know, Hayden was talking about, you know, using a model. That's great. But also hard technical controls.
Corey Ham:They tried. They failed. Their containment was insufficient. This is equivalent of the Chernobyl. Right?
Corey Ham:They they built, oh, RBMK reactor. Let's put a concrete lid on top. That should be good enough. Like, what can go wrong? It's the same thing.
Corey Ham:It's it's essentially saying like, well, it'll be more it'll be more capable with less safeguards and less containment. So let's give it less safeguard less less containment. So
John Strand:Your Chernobyl example, I think, actually is very apt. Because if you remember, it was the tungsten rods was the main thing, right, that accelerated it.
Corey Ham:The graphite tips on the tungsten rods, that's the docker that's the docker proxy that isn't patched. Right. It's just a zero day. Want Wait. What is oh,
John Strand:can we talk about that? They knew about that problem.
Corey Ham:Fair. Fair. That is arguably maybe different.
John Strand:Yeah. No. No. I don't think it is different. I think it's the exact same thing.
John Strand:Because we saw, you know, when Anthropic was doing this, it escaped out of its sandbox. Like, there should have been someone
Corey Ham:Yeah.
John Strand:You know, this happened over here. Maybe we should make sure that that doesn't happen here. It's not like this was, oh, well, that's completely unavoidable. How are we
Corey Ham:able Yeah. Who could have seen this? Oh, yeah. Yeah. Why didn't okay.
Corey Ham:So Docker image proxy had a zero day. Why didn't you have solar on a freaking vault analysis on it before you put it into your test environment? Right? Like, I mean, the the other thing, like, just beyond all the stuff that we talked about, it really is just you can have a Grafana dashboard that just shows you how many times has the model tried to bang into the walls and break out of containment. You know what I mean?
Corey Ham:Like, if the line goes up yeah. If if the line goes
John Strand:from Stop
Corey Ham:and stop. Right. Yeah. Like, if the line is, like, five containment attempts per hour, that's normal. Then it goes to 5,000,000.
Corey Ham:You're like, okay. You pull the plug down. Right? Like like, you have to monitor for, like, rogue like you know, when the model goes rogue, you have to cut it. You can't just be Monitoring.
Corey Ham:Yeah.
Bronwen Aker:That's why you need the car switch.
Corey Ham:But I
Bronwen Aker:I think kill the power.
John Strand:I think that we underestimate the hubris of these companies. And
Corey Ham:Well, it's the cult like culture. Right? We know that these labs have a very cult like culture.
John Strand:You've seen that meme where it's a bunch of people sitting in a meeting and one person says something contrary, and the next picture is he's thrown out the window. Right. Like, I think if anybody at this meeting was like, hey. Should we double check our guardrails and spend a little bit of time shoring this up? Or that guy's going right out the window and into the parking lot.
Hayden Covington:A $2,000,000 salary or whatever.
Corey Ham:Yeah. Yeah. Yeah. Yeah. So okay.
Corey Ham:Real quick. Last question someone asked, and I they're like, Jeff asked, this case highlights the exposure some providers could have for liability. What does the panel think of ongoing industry push to get safe harbor for AI companies? If that were to happen, if safe harbor for AI companies ever happened No. I mean, it's over.
Corey Ham:That is actually Chernobyl. Like, that would be insane.
John Strand:No. I I look, it's not outside of the realm. Like, well, that's just crazy. That's never gonna happen. It's, you know, this whole push where we gotta innovate, we've gotta be first.
John Strand:We gotta do this. That's that's, you know, no. This is a
Corey Ham:Well, you see, like, like, what is it? Or whatever. It's like a new country that's offshore.
John Strand:Almost I just realized today under almost any circumstances, it's bad, but I might make an exception there.
Corey Ham:To blow up a rogue AI?
John Strand:Yeah. So Oh, gosh.
Bronwen Aker:Alright. What an article.
Hayden Covington:Everything's great.
Corey Ham:Everything's great, guys. At least everything's great.
John Strand:You know, at least it's gonna
Corey Ham:be fun.
John Strand:Like, we used to joke about the the news being like we're chronicling the downfall of Western, you know, just in general. It really feels like we're
Corey Ham:Now it's a global thing.
John Strand:That of that carnival ride. It's like we just kind of we were going up, up, up, up, up, up, and now we're going down fast and hard. And at least we're getting some thrills out of the of this whole situation as well.
Hayden Covington:Everybody enjoy it before they put you in the human farm.
John Strand:I'm gonna be I'm gonna be with my neighbor. Like, he's like, you know, seriously, every any other time in history, you know, I'd be driving with the guy with no power, no water, completely cut off from the rest of the rest of the entire world. He'd be the crazy one in the car. But when I was taking him down to where he needed to go, I was thinking I was the crazy one. And this this gentleman,
Corey Ham:you're like, have your fucking face.
John Strand:He's like, I don't need computers. I don't need a phone. Anybody that needs to talk to me can come and talk to me, and I just wanna be completely disconnected. I'm like, that's
Bronwen Aker:You know what? There's a lot to be said for that.
John Strand:There is a lot to be said for that. Alright, everybody. Thank you so much for coming to our emergency, like, in focus session. We appreciate it. It was a great time.
John Strand:And by the way, y'all, thanks for doing the news and then coming and doing this. You know, let's let's hope that this is a debatable top story of 2026. Let's hope this thing doesn't explode up anymore. But we'll see you in the next time, everyone. Take care.