AI Is Not Magic: The Truth Behind the Technology Changing Everything | Diana Kelley

Hello, everyone.

Welcome back to another episode of the Security By Default podcast.

I’m the host of the show, Joe Carson, and it’s a pleasure to be here.

It’s my favorite time of the week.

I get to talk to amazing people and have fun while doing it.

So we live in a world of chaos, and what we’re really here to try and do is bring clarity and transparency and knowledge to that chaos so at least we can make informed decisions.

And this podcast is really to help you either educate yourself or choose a career path that will help you do the things you really enjoy doing and really get connected with people who’s making a difference in the industry.

And that’s really what I try to aspire to do, is bringing people that are really kind of making the world a safer place and really making a difference at the same time as doing it.

So I’m welcomed with an amazing guest who I’ve known for quite a long time.

We’ve worked, we’ve crossed paths many times in the past.

So, Diana, welcome to the podcast.

It’s so great to have you on.

If you can give the audience just a background, you know, how did you get in the industry?

What’s your origin story?

And a little bit about what you do today.

Thanks so much for having me on.

And it is.

It’s always, you know, it’s funny that you say, you know, you just.

It’s here trying to help people make the world because that is just so you.

That’s why it’s just been so wonderful to be able to know you and speak with you for so many years.

Is that you really do.

You just come.

You contribute so much to the industry in just this very humble and helpful and collaborative way.

So thank you for all.

Thank you that you do to me.

How did I get started?

I mean, if we go way, way back, we gotta rewind to the 70s.

I was actually on the DARPA Net and I got really fascinated and which was the precursor to the Internet.

If anybody’s like, what’s the DARPA Net?

It was the precursor.

I started out my.

My professional career as a sysadmin.

And I think I was even at that age, you know, a little sysadmin at heart because I wanted to know how the systems work.

And I couldn’t get to the manuals because I didn’t have access, the right privileges to do it.

So I actually figured out that there was a way to get passwords and to see what people were typing in for their password.

And I got the password of a Navy commander and got onto one of the dot mill sites and was reading the manuals there, which the next morning my dad came into my room.

Because I was a kid at the time, my dad came into my room and I felt bad for him because you could tell that he was very, very angry.

I had clearly done something wrong.

But also there was this tiny bit of like, huh, you know, a little impressed.

So of course the admins had had.

And the, the, the systems I had been on had seen what I had done.

They realized I had not done harm.

But they did say so they did actually didn’t take my account away.

They did allow me to read manuals.

But going forward.

But they did say if I ever did anything like that again and why it was wrong to steal passwords and all.

Obviously you don’t have that kind of plausible deniability at this point.

Um, but so interestingly, at the same time, the other thing I got to play with that I think is more salient to where I am now all decades later, was there was a.

And I played with it in the 70s and Eliza was one of the first pieces of.

Of AI actually.

And he.

Interestingly, it was.

It was supposed to emulate talking to a therapist.

So there was a lot of like, how does that make you feel?

That’s pretty impressive that you got to actually use that software.

Because I, I’ve been reading about it so many times over the years and it definitely is, you know, a.

Models was back was either training simulation models or it was basically therapy kind of trying to simulate human behavior.

And it was amazing that when we think about.

That’s 60 years ago.

Yeah.

And according to lore, from what I read, it was.

He wrote Eliza actually as a response to the AI conference that went on in Mass State here of New Hampshire at Dartmouth in the 50s.

And a lot of futurists were saying AI is going to replace human beings.

Does this sound familiar, anybody?

AI is going to replace human beings, and we’re not going to need to have.

There’s going to be basically everything’s going to be controlled by the machines.

And Wiesenbaum said AI isn’t quite like that and people may misunderstand what this technology does.

So he created Eliza partly to help people understand the illusion of intelligence, but not real Intelligence, but it ended up a lot of people thought it was true.

Eliza was truly intelligent.

And I was at IBM and IBM had Watson which.

Do you remember?

Watson beat Jeopardy.

And that was.

I remember, I remember.

I remember all that’s.

That was impressive.

But, but, but you think about it, it’s very logical because, because that’s what it’s been trained to do.

When you go through the machine learning and all of the aspects and the algorithms that you, you put the guardrails in place to give it that goal.

But that’s when we kind of for humans to be able to, to do that we’re, we’re unpredictable.

We’re, you know, we got frontal cortex which makes us, you know, look concentration and we got many goals that.

To be very focused on those.

You know, that’s, that’s, it’s, it’s sometimes it’s hard to compare, you know, a human versus machine when it’s, when it’s so targeted to do those tasks.

But it was amazing to watch and we also saw the, the alphago and we saw, you know, the chess and every, all of those many, many things has been played out over the years.

So it’s also amazing that you got to play with Watson as well.

That’s.

Which is impressive.

Well, it’s, it’s.

And you’re exactly right that it was trained to be because Jeopardy.

Asks questions or gives answers in a, in a very specific way.

And it’s sort of tra.

Yeah.

And tr.

The machine learning Watson to get the right.

So interestingly, this is kind of a fun thing.

Watson wasn’t actually faster or better at coming up with the answers than the humans.

It didn’t beat the humans on actual answer.

Where it beat the humans is that Watson didn’t have a, you know, the buzzer.

That’s where the humans.

So looking back on it, it was apparently it was the button because Watson had to.

Watson just had sort of, I get, you know, flipped a bit.

It was just like ready.

Right.

Whereas the human to.

So Watson was able to beat the humans on the speed because it was our thumbs sort of that slowed.

So it wasn’t our brains.

Which is sort of amazing.

Yes.

Which is going to be impressive because when I think about, I read the book about the battle of the brain, which is also then about talking about potentially the future.

And I did a TedX talk 11 years ago, I got reminded of it just recently, which is all about moving to the world of augmented reality.

And that our traditional way that we interact with the computers through mouse and keyboard is going to completely change, especially with voice.

And then if you get connected directly, if you’ve got a neural.

Right.

Yeah.

Then.

Then that speed completely changes or the human interaction between us and computers will completely evolve.

And I think that’s the next big wave of innovation is how do we make us move at computer speed with our decision making, not have to rely on our thumbs and fingers.

Yeah, yeah, exactly.

How can we get.

Cause the speed is very fast, even to the point that there’s been, you know, neuroscientists think that we make a decision before we can actually understand that we’ve made a decision.

And then we start to use language to rationalize why that.

But that it had already.

So yeah, the speed that things happen within us is just amazing.

But so anyway, so, so we had Watson and Watson was being used for healthcare and I was in IBM security.

Where else would I have been?

And we decided to train Watson for cyber.

And that was a really fascinating experience because there was, you know, understanding of the understanding words and Watson not being able to put context around it, which is if.

Or have the right attention.

Right.

Because where Watson would run into trouble is something like ip.

Does that mean intellectual property or does it mean Internet protocol or, you know, salt the hash.

Are we talking about cryptography or are we talking about breakfast or honeypot?

Honeypot was another one where Watson, you know, really wanted.

Wanted Winnie the Pooh to be going up for that honey pot and to explain what a honey pot or honey nut.

So these interesting.

And I kept thinking at the time, which ended up being.

I thought it was like really kind of naive and because I’m not a data scientist.

But when you look at what transformers are, which was looking at the context and the closeness to things that are nearby those words so that you can get attention on and start to really understand a little bit more about is this IP intellectual property or is it IP Internet protocol?

And that actually is a lot of what happened with what changed, what transformed with transformers and how we got from Watson to ChatGPT and Claude and Copilot.

So, yeah, so I do.

I feel pretty lucky to have been in around it.

I worked at Microsoft where I again was very interested in AI and what was happening with AI and the team, and then ultimately went on to be the ciso at Protect AI.

I also did the first AI security course for LinkedIn Learning, which was an absolute blast.

I based it on a lot of the research from Ram Shankar from, from Microsoft.

But yeah, so it’s.

So I’ve been really lucky to have been around this and near AI for a long time.

And now I am the CISO at Noma Security, which is an AI agentic AI security company.

Fast, I guess.

I mean that’s, that’s impressive that we can see that evolution of AI throughout the years, because a lot of people, a lot of people’s coming on the end of it with the last five years and they’re, they’re really jumping on and they’re.

And I think there’s a lot of assumptions and misunderstanding and interpretations about what it can do, about how to get value out of it, what the risks are.

And I think when you really look at the evolution, I always like to go back and understand about what decisions do we make at the time.

Why did people come up with Eliza, why did we do Watson, what was all the kind of evolutions throughout that.

And a lot of times when you go back, you start seeing the path itself and the decision making and about the improvements and the major evolution, things like the natural language processing and understanding.

I think that was the pinnacle, kind of massive ability for us to, with an AI algorithm in our language.

And that’s a massive difference.

And I think that’s sometimes a bit of a concern as well.

Remember I was actually on a panel just recently and we went through this whole discussion, it was about that we try to make AI in human.

We try to make it human, like by giving it a human voice.

And sometimes we make it a robot.

We try to make it look like a human in a robot style and that gives us this misrepresentation.

Then we start treating it as if it is human and as if it is intellect and has a frontal cortex and has contacts aware and can have creativity, but it doesn’t.

It’s based on maths, it’s based on algorithms, it’s based on machine learning, it’s a probability machine and all those things mix together.

And I think we really need to make sure.

Mike Aldous question is, should we have a prerequisite before people can actually start using AI models to make sure that they understand that this is not a human?

Don’t thank it because you’re going to cost it tons of tokens by just saying thank you and really thinking about, you know, the cost, the energy cost.

Modi said that, yeah, it was, the energy cost was just phenomenal by just humans, you know, thanking it.

Because we think that if we’re polite, it remembers that.

So the next time we ask it a question in future it will actually be kind to do it for us.

No, we’re always afraid that it might come up and say, no, no, no, I’m not doing that because you weren’t kind the last time.

So it’s interesting about our perception with it.

So tell me, in the recent years, what have you seen?

I mean, we’ve came a long way, especially with natural language processing, understanding a lot of the GPT models, the evolution we’ve seen with coding, with ability to make genealogy and there’s so much things that AI is getting involved into and there is that fear that it will take our jobs and replace us.

And I don’t believe, I mean, I still have positive outlooks that it’s there to empower us.

It’s to help us go faster and do things and maybe focus on the things that we can enjoy and spend the time on.

What’s your perception on all of that?

There’s a lot of the technology, evolution and innovation which is amazing, both in the good and the bad because all things can be used for both good and bad.

What’s the things that you’ve seen in the last couple of years that’s really been exciting, but also concerning, do you think it’s going to tick our jobs?

I loved what you said about should people have to take some training before they start using the AI?

Because it is true, they’ve been trained, the AI have been trained to interact with us like we’re humans.

And you sort of have to look no farther than did you see when the CLAUDE code, there was some source leak, open source code leak of some of Claude code.

Right.

And it turned out that they were using regular expressions to parse out sentiment before sending information into the context window, you know, and people were like, why would you use regex?

And it’s like, well, because it’s better than using inference to find.

But you could see from the actual, you know, words that were in there that people were clearly getting upset with Claude and saying things like, you’re terrible and you’re shitty.

Like it was just sort of funny because you sort of saw this microcosm of how people were clearly getting upset with the AI and swearing at it and that’s what that, that regular expression set was looking for.

But what, what are some of the biggest changes?

I think these models, and I loved how you were saying, you know, they’re not intelligent, they don’t quote learn the way humans do.

We train them, they’re prediction machines.

We’re getting a lot better at training them to do what it is we want them to do.

And one thing that’s shifted a lot and you’ll hear like, oh, it’s such a powerful model and it’s so much smarter.

But what’s really shifted, especially if you look in the software development, is how we’ve trained what we’ve trained these models on, especially around what camera about initially as chain of thought prompting.

So to try and get the model to solve a problem rather than to just know the answer.

Because that was traditional.

You look back at Watson and it was like at that place, at that point, you know, it was here’s Wikipedia, learn Wikipedia.

Then probably you’ve got all of the questions for thinking about it.

For example, like chain of thought was sort of the walkthrough and we talk about quote reasoning, but really it’s that these models are now being trained on how do you break down a problem.

So instead of just having like nine times nine, 81, right.

And then so memorize the multiplication table, don’t just memorize the multiplication table.

And that’s sort of how we were training originally is just have all the knowledge and you should be able to quote solve the problem.

But it’s okay if you have nine times nine, understand how to do location.

You know, what are nine nines?

Right?

Like that is, that’s how it stepping them through, you know, with like a word problems for math, you know, if you have three bushels of apples and there are four apples in each one, actually doing this chain of thought prompting.

But now they’re doing it in the training so that the models are much more likely to be able to break down, you know, instead of the problem and do quote reasoning through it.

Which has been really powerful for a couple of things.

For some research.

Anybody who’s used a model and sees it start to say you can see it quote thinking, right?

And what it’s really doing is it’s now being taught on how to do this chain of thought, how to break down the problem.

And then where that’s really been incredibly powerful is in software development.

And it’s.

And it really improved the capability of these models to write software and to also test for flaws and vulnerabilities and issues in the software, which has now led us to this latest model of anthropics that they’re not even releasing with sort of quite the name, their mythos, you know, I mean, like it’s, there’s, there’s feels like a little marketing involved with that.

But okay, so this model, they’re not even releasing out to the wild yet.

It’s in preview now with a consortium of organizations like AWS and Microsoft and Google and CrowdStrike who’ve agreed to look and understand at the capabilities so that when it is released that some of this power of being able to find vulnerabilities in software and vulnerabilities in systems will not lead to a more insecure state on the Internet.

We’re back in the vulnerability disclosure discussion again because ultimately we’ve always been doing this, you know, over, over from a long time.

And finally we now have a proper.

Will these algorithms and, you know, models adhere to vulnerability disclosure processes that companies are actually putting on their sites?

Will it read and be able to say, okay, I’m going to adhere to their policy disclosure, I will make their vulnerability disclosure to them and follow.

Will it follow those processes, you know, that we’ve set and we’ve got their guidelines and best practices that we’ve done for a long time.

And this was, this was one of the things that, you know, was the highlight, I think recently at the RSIC conference when I went to BSIDES and listened to Katie’s talk, that if we take, if we take that model and we point it to all the old code and legacy code that’s been written since the 60s and we’ll, and we’ll find tons of vulnerabilities, but there’s no one here to fix it.

I mean, how do we fix, you know, old stuff that, you know, unless.

I think, I think the good thing with Anthropic is that they’re doing it much slower because it used to be that had the, it was a race.

Release, release, release, release, release, and then, and then fix, fix, fix, fix later.

Now it’s kind of like, oh, maybe this one’s a bit too, too risky to make a public.

Yeah, too good at that.

Yeah.

Let’s, let’s think this Surya makes sure that we get additional, you know, eyes on it and feedback and, you know, make sure we had the right guardrails in place to make sure that it doesn’t make the world an unsafe place very quickly and we don’t have the speed to fix it it so that I Think it was a big realization if we took it and we pointed to, I mean, a lot of organizations are still using applications were built in the 90s banks still have, you know, mainframes that are still operating and we’re becoming this hyperbole.

COBOL out there still.

Yep, that’s where I started.

I’m a COBOL programmer.

That’s where like I started my, my background and you know, maybe there’s still, there’s still work for me in the future at some point.

Well, it’s sort of interesting because it goes back to your original point of is, you know, are there jobs for people?

And I think that there are certainly jobs for.

Because somebody might say, well, just let the AI actually fix it.

Well, AI is quite good at JavaScript, it’s quite good at Python, it’s quite good at Rust, but it’s apparently not so good at writing machine code, in part because it wasn’t trained on Latin.

We just don’t have the level.

Think of the sheer volume of Python code we have to train AI on versus the machine code.

So, and cobol, I suspect.

So there’s certainly for anything that’s legacy, there’s obviously there’s, there’s place for people.

We need that kind of insight.

And I do think that going forward, we’ve seen this throughout human history.

Jobs disappear and new jobs come.

Somehow as entities, as a species, we always find a way to keep ourselves busy and doing things.

And I don’t think is going to shift millennia of how we always find a new thing to do.

Also with AI, there’s going to be a lot of new jobs related to all the care and feeding of AI and the training of AI.

So some jobs will go away.

We have always had this evolution through times.

You mean the jobs that we’re doing today didn’t exist 60 years ago?

Right.

I mean, that’s what you have to look at.

And a lot of people’s jobs that they’re doing today didn’t exist in the previous generation.

There’s trades that have been around for a long time and those tend to stick because they’re not easy to replicate in machines and bots.

And maybe we’ll get there at some point, but we’ve always had to evolve through different industrial revolutions to find new things to keep us busy and to learn new skills.

I think though, is one of the things that I’ve seen is that the speed that we have to change jobs is faster now as well.

Where I’m finding that probably in my career, I probably had three or four different roles because I’ve had to keep retraining in order to stay relevant and also to progress my career as well.

I didn’t want to stick doing the one thing, and I looked at what’s the next evolution.

We’re always retraining, but I think that speed is accelerating.

You know, every time we replace our phone, we’ll probably have a new career.

Yeah.

And we just have.

We’re constant learners over time, we’ll just be learning new things to stay irrelevant and to keep busy and keep us focused because, you know, our minds, we need something to do and we need something to keep us active.

That’s right.

Yeah, we got it.

What if.

What is a Heroboro?

It says if you don’t use the little gray cells, they rust.

Yes.

So.

But it’s an amazing kind of direction, and I think we will.

You know, there will be lots of new jobs going forward.

You know, people will be learning, you know, becoming data analysts and, you know, trainers, and we’ll be coming observers.

We’re starting to watch these algorithms fight and argue, which I’ve had a lot of fun doing with watching agents fighting agents or Reddit.

Like, I.

There’s every now and then I’ll be in a subreddit where it’s like, it was clearly an AI posted it.

And then AI is mostly interacting.

And you just read it and you’re like.

Like, I thought multiple book was.

It was interesting that.

Who was it?

Meta.

So who bought multiple.

I forget.

But.

But I was like, we didn’t need multiple book.

We’ve got a lot of subreddits where it’s already like just AI chat with AI.

Yeah, but that was.

That was more of a.

It was an AI only.

Right, that’s true.

It was.

Red Reddit’s a mixture of, you know, humans plus AI human DNA.

So there’s some.

Some involvement in overlap, which is always funny.

But that was like a.

You know.

Which is always funny now.

Now when you’ meetings and online meetings, you know, it’s like, are you a bot?

You know, no, there’s a bot check.

And then the modbot was like, you know, are you human?

Like, you’re not allowed on human.

Yeah, yeah.

It’s a separation of communities going different directions.

So it’s always, always funny.

And that’s sometimes the comical side of things.

Always keeps us entertained.

Some of those decisions as well.

Yeah, well, and the conversations, like, if you looked at Mopoka, it was funny because if you had been trained on, you know, Reddit and substack and other places where humans get together and share medium and you could kind of understand a lot of what was being posted were things that humans do post.

It was predicting what was most likely to say.

And it just felt really weird for it to be coming from an AI.

And it does start to feel.

I understand why people are like, are they sentient?

Are they thinking on their.

Because it comes out with stuff that does feel so human that it can be.

It can be easy to question.

I think one of the things you brought up was really important as well is that in the last couple of years the learning part, we’ve been very good at understanding about how to do prompts, prompt engineering, looking at the guardrails, looking at how to interact.

So prompt has been a massive skill set that we’ve had to do.

And I think you also brought up another good point is that is a lot of the A models, they’re now starting to learn the context.

Because I think when you talked about hash, having breakfast and honeypots and IPs and stuff, all of that you need to have the context.

And I think that’s the next generation that we’re going through right now is that it’s no longer just about understanding the right prompt.

Because that was great for things like language translation or math based, you know.

Right.

The 9, 9, 9 times tables is that now we start to understand if we start training them on context aware then you understand about where does that nine times table apply?

Why?

Why?

You know, I would against.

One of.

One of the things I always use in a lot of my metaphors is that I’m going outside, I need eject.

And the question is that what’s the context?

Why do I need a jacket?

I had to look at all the other sensors and parameters.

Is it raining, is it cold, is it windy?

You know, what geographic location, what’s the forecast, why is that person making?

So you have to think about all of those different kind of perspectives, boundaries, environments, reasoning to make sure they understand the context.

And that’s what’s really.

I think the models are now kind of starting to have that as part of the trained model is the environmental mental understanding about what’s the situation awareness.

I think, I think with a mouth that’s the concern is that once it gets released it will start finding its way around by itself and we will just lose control.

So maybe there’s reasoning for keeping it a bit slower pace.

Well and there’s also, I mean there’s the model itself which when it’s done with training, the weights and biases are closed.

It can’t learn.

The model itself can’t learn when it’s not in learning mode, when the weights are closed.

But there’s a lot of work going on with these agent harnesses.

So even as we, we may not think, you know, when you hear of agentic, you think of like this little worker bee going out and doing something on your behalf.

But all LLMs are wrapped with some kind of a software.

So even when we’re working with Claude or we’re working with Copilot, you know, there’s an agent harness around it, there’s software around it.

That’s where that, this, the regular expression for sentiment analysis was coming from.

It was in the harness that surrounded.

So I think too to your point, that we’re getting smarter about what.

Because each time you go to the LLM, it’s like it doesn’t remember, like the weights are frozen because it’s data training, the context and the memory around it within the harness, I think we’re getting also much more sophisticated about what we remember, what we don’t.

And you mentioned skills.

And I think skills are interesting.

They’re these markdown files that can have code in them, but a lot of what the skills are basically if you go and you read a skill, it’s context to send to the LLM on behalf of that particular thing that you’re trying to do.

So there are these sort of pre canned prompts, complex prompts that have been tested to know that that’s going to lead the LLM to give you the inference that you want for what you’re trying to accomplish.

So I think that yeah, as we look at all of this work that’s going on, it’s very clear.

And again, this is why I think humans, we have jobs for forever.

It’s very clear that, that smart humans are figuring out how we take advantage, how we use these systems in a way that’s going to help us.

I hate the term force multiplier, but help us to be better at what we do and how we use them more effectively.

I just laughed.

I think it’s, I laughed at a friend, Dave has been talking about agentic AI agents being almost like gremlins where you add water and they multiply and that’s, that’s ultimately what we’re faced with, is that.

So we will start off with a few and we’re going to add data which is the accelerate and those things will just, they’ll just multiply.

The question is, can we make sure we stay in control.

I mean, that’s actually why I’m at the company I’m at because that’s.

It is we need a control plane for this.

Because what’s interesting about agent behavior is that when you say go do this, the system’s been trained to make that happen.

Right.

It doesn’t think about good and bad and should I be doing that the way that humans do?

And, and I mean there was.

I don’t know if it was anthropic or OpenAI, but one of the Frontier Model labs came out with it.

They were watching.

They sort of set this agent out with like, no matter what, get this done.

Right.

So the agent was getting stymied for things like, well, it couldn’t log into this system.

And so another agent had access to that system but couldn’t get the other agent the password.

So it actually hid it in a picture.

So use steganography to hide that password and get that out to the other.

Which again, a lot of people hear that and they’re like, oh my God, these gremlins are, you know, they’re like, we fed them after midnight and they’re gonna.

But I think it’s just more about, let’s understand how that technology works and that we have created software that, you know, is now quite good at getting stuff done and knows the ways around it.

So this is all the more reason to a train.

It responsibly put safety in the harness and also have controls like the control plane, so that if this stuff starts to go off the rails, not even because it’s trying to be bad, but because it’s trying to, you know, tcv.

It’s trying to do the.

It’s.

It’s doing what it was told to do.

Ye.

No matter what.

Okay, but not that.

So not, not, not using that method.

If you have problems, come, come back to the observer and we will help you.

Don’t try, don’t try to get around the safety.

The safety net that we’ve set up.

So which always kind of, you know, we always think of the MCPS as kind of the gateway, the control plane being there as well.

And I find that I’ve got to the point where I was always wondering should I create a single or a few agents and.

And have them become.

Become bigger digital versions of me.

But I’m starting to find this better approach to keep it microservices that are very focused at doing singular things and then having them work together in a swarm.

But with controls, yes, that becomes much More easier to maintain and control and allows you to also stop it from getting out of hand as well by having them very focused because you can.

What I find is I’ve always had the conversation a friend of mine, we did so we did an episode on AI last year, we talked about using AI for coding.

And the reason why is I wasn’t good at telling it what not to do by saying I don’t want you to do all of these error handling exceptions and stuff.

So it really gets into making sure we actually understand about how to keep it lean and how to keep it focused.

And if you have that, if it’s very specific task focused, then it will do it really well.

And if you give it these big goals, it’s going to find ways around your limitations in order to achieve because it’s very focused on those outcomes.

Yep.

No, I completely agree.

And also when it’s a sort of a tight goal.

Right.

You know what’s wrong and what you don’t want it to do.

And I think you’re right.

And I think it’s the same with.

I think we’re going to find more and more people training their own smaller models.

The frontier models are amazing.

I think everybody sort of partly wants to just like can’t just do it all with that.

But I do think that we’ll find not just chopping the goals and the tasks per agent, smaller and smaller so that we have more control over it.

But also we’ll see a lot of companies I think going back to training their own models and using their own data sets because you know, for certain purpose fit.

It’s very.

Everybody who’s used one of the frontier models has had some sort of experience with confabulation, you know, hallucination even just as much as like it’s not great at knowing what day it is is we know how to do that computationally, programmatically, really accurately and very, very easily.

But asking inference to do it not exactly even times.

Times and yeah, exactly so but so that’s why if you’re crossing time zones, it’s kind of like where are you?

Oh yeah, that’s true.

So a question that’s.

It’s been really intriguing and insightful for me.

How do you stay up to date?

Where do you get your knowledge from?

You know, especially when we’re Sitting on the frontier, you know, of, of technology and creating it.

Not just using it, but really creating how people are going to use it in the future.

How do you stay up to date?

What’s the knowledge?

Where do you go to for information?

There’s conferences, books that you’ve read, mentors that you have.

What’s the way that you say informed and knowledgeable?

Well, because I’m like, you know, eating the dog food, whatever, living, you know, doing my own yoga.

I actually do have a setup within both ChatGPT and Claude, within Claude cowork of give me an AI morning brief.

So I actually have both the AI.

I train them, I tune them to get better info.

There’s been like a lot of learning with that.

I have to say that still, even so, I’ve got both of them.

I hope someday they’re going to be better than the very smart, interesting people I follow on LinkedIn, which is where I really get some of the most, the latest, greatest, most interesting info.

But I, I am continuing to work every morning trying to tune my AI briefing bots to go out and be better than the humans right now.

Yeah, a great curated set of folks on LinkedIn I think is a really good follow who’s smart and interested in this.

They’re going to be posting about what’s going on.

In addition to maybe using your AI to do the searches, I’m also involved, very lucky to be involved with the OWASP Gen AI project.

Lots of smart people having lots of smart discussions.

There’s a CSA CISO group also.

CSA is doing a lot of work in this, so I think also looking at some of these big community projects because there’s a lot of activity and good knowledge getting shared in that there too.

Absolutely.

Fantastic.

And you’re also involved in the Women in Cybersecurity as well, because we definitely need more amazing people like you with your background in the industry.

And I believe that’s an area that you’re working into to make sure that there’s a path for awesome women to get into the industry as well.

Yes.

Yeah.

And I say it’s not just making sure that we don’t pull the ladder up behind us, but that’s not enough.

We gotta build the stairs so that it’s gonna be easy for all cohorts that may not have the same opportunities to make sure that we give that.

Because again, people are worried about losing their jobs, but also we need a lot of smart people in a lot of different areas and disciplines.

The attackers asymmetrically as we look at you said it earlier, you know, what can be used for good can be used for bad.

We’re going to need a lot more people on the defense side and thinking through these really tough, complex problems.

So, yes, having communities like women in cybersecurity whisis to help people find each other, find networking, find opportunities, find learning paths, I think is absolutely critical for defense going forward.

Fantastic.

Absolutely.

And if the audience has questions that they want to come to you later, what’s the best way to contact you?

I’m assuming I’m on LinkedIn.

Diana Kelly, LinkedIn well, I’ll make sure that we’ll add the link to the show notes so it makes it much easier for everyone to find.

It’s been fantastic.

This is, I missed you at RSI conference, unfortunately.

But we’ll definitely make sure that it’s one of those events that, you know, even when you do meet people, it’s like, you know, I was passing by, I saw Gary, I slept, I was like for five seconds on the street.

So you’re passing by, you’re seeing everyone.

Sometimes you don’t get enough time sometimes to enjoy and have a good conversation.

So we definitely have, we have to make, make that happen at blackhat.

I believe we do.

We have to have at least the hug.

It’s like, like, you know, I, I had done this a few RSAs with Itai Mayor.

I absolutely love him.

And it’s like for some reason we, we can’t have time to see each other, but we just, we have 30 seconds, we find each other and there’s a big hug and you know, it’s like that at least, at least you’ve had kids.

We’ll make sure, make sure it happens next time.

So it’s been awesome having you on and very insightful, very educational and amazing that your journey and what you’re doing again to make the world a safer place and everything you do is outstanding.

And thank you again for being you and everything you contributed.

Thank you.

So for everyone, this is the Security by Default podcast again, bringing you insights, interesting conversations, hopefully lessons for you to go back and play around in your day and maybe play around with Claude, look at GPT models and go and reading and learning, going back to some of the early concepts of the ELIZA models and really understanding but what was the reasons they were made, what was the contributions that we’re doing and really how it’s evolved and some of those pinnacle moments throughout those years.

So tune in every two weeks for new episodes.

Subscribe, Share, reach out to me if you have questions or topics you want in the future.

But again, everyone take care.

Stay safe until next time.

Thank you.