How Hackers Attack AI: The New Battle to Secure Intelligent Machines | Harriet Farlow

Posted by:

|

On:

|

This podcast episode delves into the intricate nexus of artificial intelligence and security, featuring an enlightening conversation with Harriet, the author of a newly released book Practical AI Security. We explore her compelling journey from a background in physics and anthropology to becoming a pivotal figure in the realm of cybersecurity, particularly focusing on the challenges posed by adversarial machine learning. Harriet elucidates the pressing necessity for organizations to comprehend and mitigate the security vulnerabilities inherent in AI systems, as well as the broader implications for national security. Our discourse also addresses the critical need for collaboration between cybersecurity professionals and AI developers to ensure that security considerations are embedded within AI design from the outset. Ultimately, we aim to provide our audience with a profound understanding of the evolving landscape of AI security and the imperative of safeguarding these transformative technologies.

🎙️ Security by Default Podcast

Practical AI Security: Attacking, Defending, and Securing the Future of AI

With Harriet Farlow — Founder of Mileva Security Labs & Author of Practical AI Security

Artificial Intelligence is transforming the way we build technology, automate decisions, analyze data, and solve some of the world’s biggest challenges.

But as AI becomes more powerful and more deeply embedded into our lives, one critical question becomes increasingly important:

How do we secure AI itself?

In this episode of Security by Default, host Joseph Carson is joined by Harriet Farlow, AI security researcher, founder of Mileva Security Labs, and author of “Practical AI Security: A Hands-On Guide to Attacking, Defending, and Securing Modern AI Systems.”

Together they explore the rapidly evolving world of AI security, adversarial machine learning, and why understanding how AI works is essential before we can protect it.

About This Episode

AI is often described as the next technological revolution, but securing AI requires us to rethink many traditional cybersecurity approaches.

Unlike conventional software, AI systems are built on data, probability, optimization, and learning models. They do not always fail in predictable ways, and vulnerabilities are not always solved with a simple patch.

Harriet shares her fascinating journey from studying physics and anthropology to working in data science, national security, and artificial intelligence, eventually discovering the world of adversarial machine learning — where attackers attempt to manipulate and disrupt AI systems themselves.

This conversation goes beyond the hype and explores what defenders, developers, and organizations need to understand as AI becomes a critical part of modern technology.

What You Will Learn

🤖 Why AI Security Matters More Than Ever

AI is becoming part of software development, business operations, healthcare, finance, critical infrastructure, and cybersecurity itself.

As adoption accelerates, organizations must move beyond simply asking:

“How can we use AI?”

and start asking:

“How do we secure AI?”

🧠 Understanding How AI Really Works

Harriet explains why machine learning systems are fundamentally different from traditional software.

AI systems are:

  • Probabilistic rather than deterministic
  • Dependent on training data quality
  • Designed around optimization
  • Continuously influenced by changing environments

Understanding these foundations is essential for anyone responsible for protecting AI.

🔓 The World of Adversarial Machine Learning

What happens when attackers stop targeting only applications and infrastructure…

…and start targeting the AI model itself?

The episode explores:

  • Model manipulation
  • Data poisoning
  • AI weaknesses
  • Training challenges
  • Unexpected behaviors
  • The difficulty of understanding model decisions

🛠️ How Do You Patch AI?

One of the biggest questions facing cybersecurity professionals today:

If AI learns something wrong, how do we fix it?

Traditional security follows a familiar process:

Find vulnerability → Apply patch → Reduce risk

AI changes that.

Sometimes protecting AI is not about fixing code.

It is about understanding and correcting behavior.

⚔️ AI for Security vs Security for AI

For years, organizations have focused on using AI to improve cybersecurity.

But now the challenge has expanded.

Cybersecurity needs AI.

But AI also needs cybersecurity.

As AI becomes part of everyday systems, security teams must understand how to protect the models, data, and decisions that organizations rely on.

🌍 Why AI Security Requires Different Skills

The future of AI security requires collaboration between:

  • Cybersecurity professionals
  • AI engineers
  • Data scientists
  • Researchers
  • Risk leaders
  • Policy experts

Building trustworthy AI means bringing these worlds together.

Security must be part of AI from the beginning.

Key Topics Discussed

🔹 Harriet’s journey from physics and anthropology into AI security

🔹 Working in data science and national security environments

🔹 Discovering adversarial machine learning

🔹 Founding Mileva Security Labs

🔹 Writing Practical AI Security with No Starch Press

🔹 Why AI vulnerabilities are different from software vulnerabilities

🔹 The importance of data quality and model training

🔹 Understanding probability and machine learning foundations

🔹 How attackers target AI systems

🔹 Why securing AI requires a new mindset

🔹 The future of AI safety and cybersecurity

🔹 Staying updated in a fast-moving industry

🔹 Building responsible and secure AI systems

Memorable Quotes

💬 “Before we can secure AI, we first need to understand how it works.”

💬 “AI security is not always about fixing a bug. Sometimes it is about correcting a behavior.”

💬 “Cybersecurity needs AI, but AI also needs cybersecurity.”

💬 “The future is not just about building smarter AI — it is about building safer AI.”

Episode Chapters

00:00 – Introduction to Security by Default

01:03 – Harriet Farlow’s origin story

04:28 – From data science to cybersecurity

08:48 – Creating Mileva Security Labs

10:51 – Conferences, community, and writing Practical AI Security

17:28 – How AI has evolved

19:43 – Understanding machine learning models

21:43 – The challenge of patching AI systems

23:37 – Training data, quality, and user impact

25:23 – Why AI models can be difficult to understand

27:36 – AI and cybersecurity coming together

30:18 – Why AI fundamentals matter

32:04 – Practical examples and real-world AI security

33:38 – Staying updated in AI security

36:27 – Learning from the AI security community

38:08 – Ethics and responsible AI development

Guest

Harriet Farlow

Founder — Malevra Security Labs

Author — Practical AI Security

🔗 LinkedIn:

https://www.linkedin.com/in/harriet-farlow-654963b7/

📘 Practical AI Security — No Starch Press

https://nostarch.com

🎓 AI Fundamentals Course

https://harriethacks.com/course/

Listen & Subscribe

🎧 Security by Default Podcast

Exploring the people, stories, and ideas helping make technology safer.

Because security should not be an afterthought.

Security should be by default.

#SecurityByDefault #AISecurity #Cybersecurity #ArtificialIntelligence #MachineLearning #AdversarialML #AI #ResponsibleAI #SecurityResearch

Takeaways:

  • The podcast episode discusses the importance of understanding AI security in the context of national security and its implications.
  • Harriet’s journey from a background in physics and anthropology to her current role in AI security demonstrates the interdisciplinary nature of the field.
  • The conversation highlights the necessity for collaboration between AI developers and cybersecurity professionals to ensure secure AI systems.
  • Listeners are encouraged to engage with various resources to stay informed about the rapidly evolving landscape of AI and cybersecurity.
  • The significance of addressing the ethical considerations in AI development is emphasized throughout the discussion, focusing on empowering rather than replacing human effort.
  • The episode underscores the idea that AI security is not merely about using AI for cybersecurity but also about securing AI systems from external threats.
Transcript
Speaker A:

Hello, everyone.

Speaker A:

Welcome back to another episode of the Security By Default podcast.

Speaker A:

I’m the host of the show, Joe Carson.

Speaker A:

It’s pleasure to be here and always my favorite time of my day, which is I get to talk to really cool, awesome, amazing people that actually really doing a lot of great things to make the world a safer place.

Speaker A:

And in the world we have, it’s.

Speaker A:

It’s sometimes chaotic, sometimes fuzzy, and sometimes we need, you know, that knowledge and education to help us see through that cloud and, you know, chaos that’s there.

Speaker A:

And I’m joined by an awesome person who I was introduced through nostarch, which is one of my favorite publishers.

Speaker A:

So nostarch Press do.

Speaker A:

Amazing book.

Speaker A:

And I was introduced to Harriet, who’s the guest on the podcast today to give us an update and insights into your latest book.

Speaker A:

So, Harriet, before we get started.

Speaker A:

So to give the audience a bit of a background, you know, who you are, what’s your origin story?

Speaker A:

How did you get into the industry?

Speaker A:

And, you know, is it something that.

Speaker A:

Did you choose it?

Speaker A:

Like, you know, is this the path that you decided, I’m going to go down this path and this is the one I want to do, or did you kind of come in through another angle, another direction that some people do, which is quite interesting as well.

Speaker A:

So if you can give the audience a bit of background about yourself.

Speaker B:

Thank you so much.

Speaker B:

Jo, first of all, thank you so much for having me.

Speaker B:

I’m really delighted to be chatting with you today.

Speaker B:

The origin story, where to begin?

Speaker B:

I’ll try not to make it too long.

Speaker B:

I don’t want to bore your listeners.

Speaker A:

No, they’re interested.

Speaker A:

They’d like to know how people get in because sometimes people starting their journey as well, they might kind of, you know, is this something.

Speaker A:

Is there something possible for me to go down this path as well?

Speaker A:

So sometimes it’s good for the people to hear, you know, the kind of detailed, detailed version.

Speaker B:

Yeah.

Speaker B:

Well, I hope, if anything, people feel inspired to go out and do their own thing based on my story, because I. Yeah, so I, I’ve spent the last 10 years at the intersection of artificial intelligence and security.

Speaker B:

But I didn’t start out in cyber security at all originally.

Speaker B:

So my degree was in physics and anthropology because I, I thought, you know, they’re two subjects that explain the world around us.

Speaker B:

You know, you have physics explaining the physical world, anthropology explaining us strange primates that occupy it.

Speaker B:

Neither of them were that employable by themselves.

Speaker B:

But something I did become really familiar with from those two disciplines.

Speaker B:

So I ended up working in data science, originally at Deloitte where mostly I was working on projects with the Australian Department of Defense.

Speaker B:

And like that was a really challenging environment.

Speaker B:

Deloitte definitely does not let you slack at all.

Speaker B:

But being able to work on defense projects was so interesting.

Speaker B:

Like I spent a year working in Darwin in Australia.

Speaker B:

As you can.

Speaker B:

I’m in Australia.

Speaker A:

Is that where you’re from, Darwin, or is it Canberra?

Speaker B:

Yeah.

Speaker B:

Not known for being that interesting.

Speaker B:

You know, I love it.

Speaker B:

It’s home.

Speaker B:

Yeah.

Speaker B:

Darwin for people not familiar is a very small town in the north of Australia known for mostly like British backpackers and crocodiles and including me.

Speaker A:

I spent a year in Australia.

Speaker A:

I did the Gibb River Road through three ways up to Darwin.

Speaker A:

Interesting.

Speaker A:

I have an interesting story from that, that time of my life.

Speaker B:

I look forward to hearing that.

Speaker B:

I think I’ve ever been out anyway before on a Monday night and seen so many people in the pubs than Darwin.

Speaker A:

Well, there was a reason why the backpack.

Speaker A:

So one of the things that I used to do was we’d go to the pubs and they’d have competitions, pub quizzes, football matches and different things and that’s what we did in order to try and win free trips or get free food.

Speaker A:

That’s why the pubs were a place where we went in order to actually uh, tried to get, I, I, I got free trips to Kakadu from going to the pub yet.

Speaker B:

Wow.

Speaker B:

There we go.

Speaker B:

Maybe I should just spend more time in the pub then.

Speaker A:

But Darwin is an amazing place.

Speaker A:

Absolutely.

Speaker B:

Yeah, it’s very cool and like obviously yeah, has a big navy presence.

Speaker B:

So yeah, I did like a, a year I’m doing FIFO work there.

Speaker B:

So fly in, fly out work and yeah, working on different boats, ships, the Australian Navy looking at data remediation and data quality issues.

Speaker B:

Um, I spent some time working with the Air Force.

Speaker B:implementing chatbots back in:Speaker B:

And yeah, from Data science I, after a few years at Deloitte and on those projects I, you know, I wanted to change.

Speaker B:

So I guess I did the sort of the opposite and I moved to the UK and I planned on working in a pub in the UK but on the, the way over there on my, my backpacking trip I ended up having a LinkedIn recruiter reach out about a job and so I was doing this Interview process.

Speaker B:

And then, yeah, something like the week I landed in London, I ended up getting a job offer for this job at a tech education startup in New York City.

Speaker B:

So I moved over there and it was a, it was a great opportunity to learn a lot about what different companies around North America, the challenges they were having in the tech space.

Speaker B:

We went into all these Fortune 500 companies and basically taught them about data science and machine learning, artificial intelligence, even crypto.

Speaker B:

But we did workshops on cybersecurity and that was my first real exposure to it.

Speaker B:

And I found it so fascinating.

Speaker B:

I thought it was such a cool area to get into because it not just focused on really interesting technical challenges, but there was a really fascinating geostrategic component, you know, national security implications too.

Speaker B:

So while I was working over there in New York City, I started a master, master’s in cybersecurity strategy and Diplomacy at a university in Australia.

Speaker B:

So I was doing it remotely.

Speaker B:

And then when Covid happened, unfortunately the US office of this startup closed down.

Speaker B:

I came back to Australia and continued the masters and I ended up working, getting a job at the Australian Signals Directorate, which is Australia’s equivalent of GCHQ in the UK or NSA in the usa.

Speaker B:

And I, yeah, I was fortunate enough to work on a few different teams there.

Speaker B:

I spent some time working in a team helping small businesses do better cybersecurity.

Speaker B:

Basically I worked with a team that was about applying data science skills to understand tech sigint, which is signals emitted from military platforms.

Speaker B:

And then my final team, my role was as acting Technical Director of the AI hub within asd.

Speaker B:

And the remit of the AI hub was basically to help inform the artificial intelligence strategy and direction of Australia’s national security and intelligence community.

Speaker B:

And there was a lot of work learning from and contributing to our five Eyes partners as well.

Speaker B:

That was, that was incredible.

Speaker B:

It was a really, it was really incredible to sort of peel like back that curtain as well that the national security engine.

Speaker B:

And I was, after doing my master’s, I moved on to doing a PhD.

Speaker B:

So I was doing my PhD in parallel to this work in the same.

Speaker A:

Topic or in something slightly adjacent.

Speaker B:

Yeah, I originally, originally the topic was something like translating tech SIGINT for military customers, something like that.

Speaker B:

It was based on the team that I was in when I started it.

Speaker B:

And then during the literature review process I found this field called adversarial machine learning, which is all about hacking and disrupting machine learning models.

Speaker B:

And I think I read my first paper on that topic and I was like, this Is it?

Speaker B:

This is what I’m doing for the rest of my life.

Speaker B:

This is so cool because it combined the two things I found most interesting, like my, my data science skills and the cybersecurity challenges.

Speaker B:

And so I’d been working on this PhD in parallel to that job and national security organizations at that time were taking AI security pretty seriously.

Speaker B:

But it wasn’t something I ever heard talked about outside of those institutions, except for my PhD, but they looked at a very different kind of niche scientific challenge, you know, disrupting machine learning models.

Speaker B:

It wasn’t really seen as something that could ever be a real threat.

Speaker B:

But yeah, the national security community was definitely really keen to the, you know, the challenge that it might pose in the future.

Speaker B:

I saw that there was this big challenge though, that the kinds of organizations that had access to this information was the national security organizations, maybe really big tech companies that had research labs in adversarial machine learning like Microsoft.

Speaker B:

But most of the organizations that would be using machine learning models and AI systems had no idea about it.

Speaker B:

And they were the ones most likely to be impacted if threat actors were trying to compromise their AI systems in some ways.

Speaker B:

So I very naively had the idea that maybe I should start a company to solve that problem.

Speaker B:arted Maleva security labs in:Speaker B:

Which was like a terrible premise to start a company.

Speaker B:

No market, no one had heard of it.

Speaker B:

And I really just spent the first couple of years traveling around to different conferences talking about AI security.

Speaker A:

Which type of conferences were.

Speaker A:

Because I’ve seen more of the security conferences, doing more AI now, but probably two, three years ago it was probably more AI specific conferences.

Speaker A:

So what type of conferences were you, Was it security conferences focused or was.

Speaker B:

It AI or they combined, they were usually security conferences.

Speaker B:

B sides is my all time favorite brand conference.

Speaker B:

I know your listeners will be late.

Speaker A:

I literally just, I’m just came from BSIDES rehear this morning.

Speaker A:

So I’ve been on a bus, a bus for four hours from BSides Riga.

Speaker A:

So it’s also one of my favorite because it’s focused on community.

Speaker A:

It’s the community aspect of things rather than the vendors and the tech, which is great, original knowledge sharing aspect.

Speaker A:

But yeah, BSIDES is awesome.

Speaker B:

Yeah, I agree.

Speaker B:I mean Definitely back in:Speaker B:

I’m sure that internationally there were probably lots of other AI speakers, too.

Speaker B:

Yeah.

Speaker B:

I mean, massive props to BSides.

Speaker B:eaking at my first B sides in:Speaker B:

And it just felt so inspiring and I felt confident and I was like, yeah, I’m going to start this company.

Speaker B:

So, yeah, I can credit BSIDES to that inspiration, I think.

Speaker B:ferences I spoke at Defcon in:Speaker B:

That was a really, like, incredible opportunity.

Speaker B:

Definitely the scariest thing I’ve ever done.

Speaker B:

Like, I don’t think.

Speaker B:

I mean, and I do a lot of public speaking, but I’ve never been so nervous to do a talk because it’s just such a big.

Speaker B:

Oh, yeah, biggest.

Speaker A:

It’s the biggest cyber security hacker conference in the world.

Speaker A:

So it’s.

Speaker A:

Yeah, to go to that.

Speaker A:

To go to that.

Speaker A:

That size and scale is fantastic.

Speaker B:

Yeah, I mean, it’s.

Speaker B:

It’s big.

Speaker B:

But like, all the previous speakers, they’re just so incredible.

Speaker B:

And, yeah, being part of the lineup was just so scary and intimidating, but it was.

Speaker B:

It was amazing because again, that was a crowd of people that took AI Security seriously.

Speaker B:

And within a couple of months of that talk, I think I remember I was on a plane at some point and I was thinking about how I could, I don’t know, talk to more people about AI Security, how I could get the message out faster.

Speaker B:

And I was like, no, I’ve always wanted to write a book.

Speaker B:

Maybe you should just write a book.

Speaker B:

I don’t know how that happens, but, you know, maybe I’ll write an outline and just, you know, I’ll start shopping around, I’ll start pitching it.

Speaker B:

And a week later, I got an email from.

Speaker B:

No Starch Press from.

Speaker A:

From Bill himself.

Speaker B:

No, I just.

Speaker B:

I hadn’t sent anything out, but it just, like, the opportunity came in.

Speaker B:

One of the amazing editors, I think, had seen the, like, the DEF CON talk and seen my YouTube channel and other things like that, and it just felt like really, really good timing.

Speaker B:

If I was someone who believed in manifestation, I would probably say it’s manifestation.

Speaker B:

However you define manifestation is definitely the right time and put the work in.

Speaker B:

It kind of happened.

Speaker A:

Sometimes it takes all of those different aspects to just happen at the right times.

Speaker A:

Speaking at DEFCON in the right fields or somebody’s there who, you know, heard the talk and thought, you know, this would be a great way to, you know, to turn into a publication that would be, you know, valuable for.

Speaker A:

For everyone.

Speaker A:

So when.

Speaker A:

When did the, the book writing start?

Speaker A:

And so you started with the outline and then you got discussed with no starts.

Speaker A:

So when did, when were you thinking about doing an AI security book?

Speaker A:

Or was it looking at, you know, did it, did it change over time or was it, you know, what you originally had planned to do at the beginning?

Speaker B:

Yeah, it actually didn’t change over time at all.

Speaker B:

Obviously the technology changed in certain ways.

Speaker B:

You know, agentic AI certainly became more of a topic of conversation over that period of time.

Speaker B:tarted writing it in December:Speaker B:

I really didn’t anticipate how much work it would be, which sounds like a really stupid thing to say, but it.

Speaker A:

Is a lot of work.

Speaker A:

It’s a lot of work.

Speaker A:

Especially when you get to a point where, you know, if you have a good copywriter, that makes your life a lot easier and if you end up getting a good, you know, visual designer.

Speaker A:

I don’t know who did the graphics for the book, you know, for the COVID but if you get somebody who’s really good at visualization, they also help lots sometimes.

Speaker A:

You know, it’s getting people who’s done it before and helping them navigate where you can just focus on the content.

Speaker A:

Makes life a lot easier.

Speaker B:

Yeah, yeah.

Speaker B:

No, the no such team was fantastic.

Speaker B:

I mean, everyone that I worked with, but especially Francis, so my editor, who I work with the most and yeah, for about nine months there, we’d probably email every day or every second day, I’d have, yeah.

Speaker B:

Different versions of chapters.

Speaker B:

She’d come back with feedback and yeah, it was a really, it was a lovely period of time.

Speaker B:

It was a lot of work.

Speaker B:

But yeah, it felt really, felt very exciting.

Speaker B:

But by the time it went in, like the full manuscript went in, I know it felt like it’s done.

Speaker B:

Like it’s kind of crazy that now the book is launching and it’s starting to come out.

Speaker B:

I feel like mentally, you know, it’s all been a long time ago.

Speaker A:

Yeah, that’s, it’s even.

Speaker A:

Yeah.

Speaker A:

The post, once you’ve done the, you know, the, the kind of initial drafts and the kind of, you know, the final manuscripts and everything’s coming together.

Speaker A:

The process of just, you know, going through that post production side of things and getting it all ready and you know, sometimes translations, sometimes just getting the prints and the formats that, you know.

Speaker A:

Right.

Speaker A:

Takes, Takes a lot of time.

Speaker A:

So I’VE got, I’ve.

Speaker A:

I’ve myself had, you know, one of the early reviews and I got to probably about close 40, 50% of the way through.

Speaker A:

And it’s very insightful.

Speaker A:

For me, one of the things, because I always find my background is purely cybersecurity, like security.

Speaker A:

Well, originally that was not, you know, a thing.

Speaker A:

Even when I started, it was just it.

Speaker A:

And security was just part of your job.

Speaker A:

And a lot of times we have had automation helping us when we’ve.

Speaker A:

We’ve been looking at data, using data to input.

Speaker A:

And I actually laughed because somebody asked me, it was like, can you remember in like recall, when’s the first time you really did anything that was like, resembled AI?

Speaker A:

And I laughed.

Speaker A:

When I used to work in HP in Concord west in Sydney, and what I remembered is like, years ago, we had a survey server that kept failing.

Speaker A:

It was a billing system.

Speaker A:

And the only way to get that system up and running was to go send somebody into the data center, into the cage, push the button and get it started again.

Speaker A:

And what I crafted was another server at the opposite side of it where I attached like a pencil to the CD tray.

Speaker A:

So what I could do, because of what we used to do, when you went into the data center, a cage, in order to find out which server you’re meant to work on, you’d eject the CD tray.

Speaker A:

So you went in and you looked around which server’s got the CD tray ejected?

Speaker A:

And you’re like, oh, that’s the one.

Speaker A:

And I laughed.

Speaker A:

One day when I went in to fix the server, I was like, oh, there’s the other, like the cd.

Speaker A:

So I got it.

Speaker A:

So I placed it opposite.

Speaker A:

So every time I needed to reboot the server, I ejected the CD tray and it would push the button to close the CD tray and it reboot the server.

Speaker A:

And then I actually, I started learning the data because the point is, is that when you start gathering and logging and creating analytical information from it, you can start becoming predictive.

Speaker A:

You start making assumptions about, you know, probably this is maybe a right time when we should probably restart the server.

Speaker A:

And I actually got at the point, because I would get a pager back then little pagers, and you get a pager saying, oh, the server’s not working.

Speaker A:

And then I would actually, you know, go and run the script to eject the CD tray.

Speaker A:

I push the button, put it back in and restart it, and everyone’s happy.

Speaker A:

And then I got to the point where actually the servers were started talking to each other and the CD tray.

Speaker A:

You know, when the server started realizing the other server wasn’t responding, it automatically ejected the CG tray.

Speaker A:

And I got the point where I could really start understanding the trends and knowing roughly when it was going to crash.

Speaker A:

And that was always makes me laugh.

Speaker A:

I’ve actually started creating a talk on that exact topic is, you know, AI and you know, in the old days, like retro.

Speaker A:

Retro AI, it’s quite.

Speaker A:

I’m in the process, I actually want.

Speaker A:

I’ve actually, I just went and found two machines where finding CDs, CD trace these days is not easy.

Speaker A:

So I’ve got two, two machines where I’m actually going to recreate it again and record it on video.

Speaker A:

But that was for me is like, you know, but what I found was that we kind of almost went down two separate paths, Security went down its path and AI is kind of taking its own trajectory.

Speaker A:

And a lot of it was many early systems back in the 60s for training models, the Eliza, the psychologist model and stuff, the therapist side.

Speaker A:

And that moved into a lot of probably the last, maybe 15 years before has been around visualization, image detection, you know, especially cameras and CCTVs in order to be able to detect things and understand what they are.

Speaker A:

But I think in the last probably, you know, seven years now with the improvements where I think for me probably the biggest change was the natural language processing and understanding the ability for us to communicate in a very different way that starts to understand what for what was the.

Speaker A:

What was for you, the, the thing that probably, you know, was what was the change that you saw make the biggest impact over the last years of AI has been evolving?

Speaker B:

That’s a really good question.

Speaker B:

I love the story you told because I think one of the challenges when people talk about AI is that they don’t really know what they mean when they use the term, or at least lots of different people use the term and slightly.

Speaker A:

It’s a very broad umbrella.

Speaker A:

When you say AI, it covers lots of things.

Speaker B:

Yeah, yeah.

Speaker B:

It’s a massive field of technologies.

Speaker B:

And I think there’s a technical definition that ISO gives and other organizations too.

Speaker B:

But in essence they’re all kind of along the lines of a machine that can do human like things.

Speaker B:e term was coined in like the:Speaker B:

So in terms of what I think the biggest change I’ve seen, I don’t know, I think a lot of the point of the book is really that there are a few things that are fundamentally different about the way that machine learning models work.

Speaker B:

That whatever kind of system they’re in, the system is comprised of a machine learning model or many and then lots of other things that have been in computer systems for ages.

Speaker B:

But the way that the integration of that machine learning component changes the system is really fundamental.

Speaker B:

And you can add things onto the system, you can add tools and stuff, but it doesn’t change the fact that machine learning models are probabilistic.

Speaker B:

They’re so reliant on that data and they can like.

Speaker A:

Yep, they can get better or worse and get better and worse again.

Speaker A:

So depending on that learning model and the deep learning and machine learning and the modification of the weights, you know, from basically if they skew off from, you know, data poisoning or other types of manipulation, then you can get a lot of bad results.

Speaker A:

And this is one of the things is, you know, as I was going through, I really enjoyed from my side as being a security practitioner, I think I really enjoyed, you know, the really first chapter was around, you know, the actually maths behind it.

Speaker A:

And that was for me is like, okay, I knew parts of it, but it really brought it all to kind of, you know, to clarity for me.

Speaker A:

It made much more clear for me as I went through.

Speaker A:

So that’s, that’s how it’s working because I kind of had some ideas and some pieces, you know, building my own agents and working on.

Speaker A:

But the mass and probability and the history behind that’s what kind of brought it kind of close.

Speaker A:

That piece of, of knowledge that I didn’t have, which was great.

Speaker A:

And then they’re moving into examples.

Speaker A:

Right now I’m in the section of the AI adversary part, exploiting it and, and, and the risks.

Speaker A:

One of the things that as I was going through, I love all of the references and associations back to, you know, some of the older models of the past.

Speaker A:

It was a highlight was as I was thinking about that was the part with the how do you patch AI was, was really kind of.

Speaker A:

And I started getting.

Speaker A:

Because a lot of my history is, is in patch management and you’ll have a, you know, system has a vulnerability and you patch it and that’s it.

Speaker A:

And it wasn’t really anything you needed.

Speaker A:

Sometimes you might need to go and fix the data because maybe the data got corrupted or some way or maybe it wasn’t compatible with the new patch and you had to make some, you know, slight modifications to it, but as I was going through and one of the things you highlight was about, you know, patching AI, especially when it’s all about the data, because the data is the fuel, you know, the algorithm, the learning model and everything that’s there, the LLM.

Speaker A:

But how would you patch AI if you find out that the data itself had, you know, got a bit of a, you know, it’s like a, you know, teenager that’s just got a bit of a bad habit.

Speaker A:

And how do you, how do you change the behavior back to the behavior that you expect?

Speaker B:

Yeah, it’s really hard.

Speaker B:

It’s a real challenging question because something I’ve noticed a lot of, of AI security professionals will refer to, instead of a traditional vulnerability associated with deterministic code that you can patch, like in AI systems and machine learning models, it’s more of a weakness.

Speaker B:

It’s something that’s inherent in the nature of how those machine learning models optimize.

Speaker B:

But at times isn’t always a bad thing.

Speaker B:

Like the probabilistic nature is a lot of the time a good thing.

Speaker B:

It’s why you get good pros and why you have some variety there.

Speaker B:

But yeah, I think something I’m conscious of during the book and in my practitioner life too is like depending on the kind of use cases and how it’s implemented in an organization that dramatically changes the technical answers that you have for a question.

Speaker B:

So, you know, most of the kinds of organizations I work with are big enterprises who, they don’t really build their own models most of the time using Microsoft products or aws.

Speaker B:

And they have almost no control over those machine learning components in the AI systems.

Speaker B:

But they do have control over contractual arrangements and they can contribute to discussions around governance and standards.

Speaker B:

That’s a very different kind of question where the, the, the solution is more process and systematic than if you’re a machine learning developer who has the slightly easier job, I guess, of deciding whether maybe they can like retrain a model from scratch, maybe they can fine tune it again.

Speaker B:

But yeah, it’s, it’s definitely not as simple as a, as a code solution.

Speaker B:

It’s, it’s essentially changing the fundamental property of what that model understands about the world.

Speaker B:

And thinking about how to solve those problems is in many ways similar and borrowing from the existing doctrine in cybersecurity.

Speaker B:

But it’s important to understand that there are pretty fundamental differences too, otherwise those kinds of challenges just get ignored.

Speaker A:

Yeah, my concern, as I was thinking about that part of the book and then that aspect, my mind started racing.

Speaker A:

Because I started thinking about and some of the things I’m seeing also in media at the moment with a couple of peers, mine, who’s using, using it to do coding.

Speaker A:

And I started thinking, you know, for the, for the first couple of years it’s been a lot of experts and you know, professionals in the field who’ve been using the models and their data has been contributing to training the models.

Speaker A:

So if you get a lot of people who knows what they’re doing and interacting with the models and it’s their interactions which are being used to train it.

Speaker A:

Now I got it to where it’s, it’s getting, it’s getting better and better and better and better.

Speaker A:

And then if you open it up to the wider public, you get a lot of people who don’t know what they’re doing, start using the model and the model gets, gets trained on, you know, the mass of people who have no idea.

Speaker A:

They’re just used to using Google searches and they’re using for the same thing and not understanding.

Speaker A:

And now that model is learning from people who are, let’s say, less experts or junior in the field.

Speaker A:

How does that impact the model going forward?

Speaker A:

If I’ve all of a sudden, you know, for the last two years I’ve got expert coders who’s been using it to improve their code and now you, you flood it with lots of junior coders, is it going to learn from the junior coders that that’s the right way to do things, do things and all of a sudden, you know, reverse back?

Speaker A:

Because I’ve seen some comments around, you know, with Claude recently where, you know, the quality, it started introducing vulnerabilities in the recent months.

Speaker A:

And if you’re not really understanding about quality and understanding about what it’s creating and not checking it, you’re just assuming that it’s right, then you know, it can create a lot of risks as well.

Speaker A:

Is that something also that you know, potentially is as who, who’s who’s been used the data being trained on.

Speaker A:

But do we quantify it?

Speaker A:

Do we, you know, check it for quality or do we just assume that it’s, it’s good?

Speaker B:

Yeah, that, that’s such a great question because one of the like, it’s both a technical and a systematic challenge is how opaque the models are when it comes to how they actually understand the world around them.

Speaker B:

I mean, so the base models that we’re using, like your, your Claude and your GPT family of models, you know, they were trained based on basically the entire Internet and unfortunately, as we now know, a lot of models are also trained on copyright material, but sort of everything that is available to humans are trained on and being able to like ensure that that is good quality data is impossible.

Speaker B:

And so they’ve obviously picked up lots of weaknesses in how they, you know, write code, for instance, and biases that they then propagate into language as well.

Speaker B:

But then the extent to which us interacting with those models and to what extent that retraining process actually fundamentally changes the representation underneath those models, I suspect is quite minimal actually.

Speaker B:

But we also don’t really know, like, we don’t know because there’s no mandate for AI labs to release any of that information beyond the kind of academic information they choose to release.

Speaker B:

And there’s also still a lot of questions among the researchers about how it actually how they hold information.

Speaker B:

You know, we understand a fair amount about how models work and we can interrogate them in some way and kind of try and understand their reasoning.

Speaker B:

But a lot of those approaches are still quite nascent and they’re not very reliable.

Speaker B:

So actually understanding like if you have a model that is billions of parameters, big, where is this certain idea located or how does this certain input lead to that kind of output?

Speaker B:

A lot of those are still unknowns.

Speaker A:

Efficiency and costs can also kind of have massive impacts.

Speaker B:

Yeah.

Speaker A:

So going back to the book, when you were doing the book, one of the things I found is that in CyberSecurity we need AI and AI needs security.

Speaker A:

It’s like, you know, it’s a double edged sword.

Speaker A:

You know, it can be good for bad, both directions, but we definitely need it to stay ahead.

Speaker A:

We need it to move fast because ultimately it’s an accelerant and data is the fuel to AI to make it move fast with accuracy and efficiency.

Speaker A:

When you were doing the book, I find that we need to bring both AI developers and architects with security together because we need to do with security in mind by design.

Speaker A:

So when you did the book, did you really target it for the AI side of the world who’s just looking at creating, you know, AI and doing it securely, or did you have it for cybersecurity to better understand AI more?

Speaker A:

What, what was the target audience you were thinking about when you were writing it?

Speaker B:

Yeah, the target audience is very much someone who is either an a cyber security practitioner already or adjacent in some related field or is interested in cybersecurity like a, like a student and really helping them go from like 0 to 60.

Speaker B:

In the intersection with artificial intelligence, it’s Challenging, because I also, in my network work with a lot of AI people who want to understand the security side better.

Speaker B:

But those fields are really quite separate and trying to bridge them both is quite hard.

Speaker B:

There’s a lot of implied, sorry, assumed knowledge that really isn’t common between the two fields.

Speaker B:

Even the term AI security when writing the book is quite challenging.

Speaker B:

We really workshopped the title quite a bit because among, like, different groups of people, AI security as a term means very different things.

Speaker B:

Like most of the time when I’m speaking with cyber people, AI security means using AI for security, either effectively or defensively.

Speaker B:

And there is one chapter about that in the book, but that’s not the focus.

Speaker B:

And then among AI safety researchers in my network, and they tend to come from more of an effective altruism leaning for them.

Speaker B:

AI security is securing the AI models in Frontier Labs and making sure that they have good, like normal cybersecurity so that, you know, threat actors don’t steal their AI models.

Speaker B:

That’s a totally different kind of problem.

Speaker B:

Although increasingly there’s, of course, overlap, as, you know, AI systems are doing things to other AI systems.

Speaker B:

But the way that AI security is framed within the book and sort of my field and paradigm is very much securing AI systems from external threats.

Speaker B:

They could be AI systems, but also humans.

Speaker B:

Just anything in the external environment that’s seeking to manipulate an AI system in some way.

Speaker B:

So in writing the book, I was really cautious or like, keen to make sure that, as you say in that first chapter, going through the math of machine learning, to me that’s really important because you don’t need to, don’t need to love math.

Speaker B:

You don’t need to do any more math than that.

Speaker A:

Well, even though it did take me a little, I was like, okay, I need to go back to my childhood to try and remember some of the mathematical algorithm.

Speaker A:

I was getting there eventually.

Speaker A:

But it did take me a moment of reading it over again.

Speaker A:

I was like, okay, what’s that symbol mean again?

Speaker A:

But I got there in the end.

Speaker B:

I certainly don’t want to bring back math trauma for people.

Speaker B:

I mean, we’ve all, we’ve all been there.

Speaker B:

But the math of optimization is so fundamental to machine learning because that is the core reason why people are either scared that AI systems are going to become super capable and try and take over the world, or even that, you know, errors in AI systems propagate unnecessarily.

Speaker B:

It’s all because of the way that they’ve been trained to relentlessly optimize on some Kind of objective or prop.

Speaker B:

The objective and to what extent that can actually work in a real environment.

Speaker B:

Most of the time, you know, most of the time it’s okay, but it’s certainly not a safe or robust solution for training large scale AI.

Speaker A:

And I have seen that transition.

Speaker A:

I did see, you know, the early years, a lot of security companies looking to use AI to accelerate their own capabilities and use cases and features.

Speaker A:

But in the past year it’s completely switched to realizing that actually it’s not AI for security, it’s, it’s, you know, security helping reduce the risk of AI AI systems and you know, because they’re going to be using a much broader scale and accounting and you know, in production and food and medicine and therefore, you know, it’s important for security to realize that it’s not just something they can use, but it’s something that they actually can reduce the risk and make safer as well.

Speaker A:

No problem.

Speaker A:

I do have some questions for you.

Speaker A:

You know, how has your movie Algorith improved and also your wine algorithm?

Speaker A:

Have you got those both prediction models down?

Speaker B:

Yeah.

Speaker B:

Yes.

Speaker A:

To choosing the best wine and the best movies.

Speaker B:

Yeah.

Speaker B:

For your listeners.

Speaker B:

I have a couple of examples in the book based on my real life hot takes on how like no movie should be longer than 90 minutes basically.

Speaker B:

I don’t know if they’ve improved over time.

Speaker B:

But yeah, it’s.

Speaker A:

Yeah, yeah, that’s.

Speaker A:

So that’s definitely, you know, it was part of the book for the audience.

Speaker A:

There’s a lot of, I think it was like 30, 30 different scripts or projects that they can leverage in order to understand and better in test and change the parameters and to learn, you know, how the output and impacts of each of the those.

Speaker A:

And I think they’re all Python scripts if I remember.

Speaker B:

Yep, yep.

Speaker B:

Correct.

Speaker A:

Yeah.

Speaker A:

And it did start off as fruit.

Speaker A:

Fruitcake was a cake in the beginning as well.

Speaker A:

And then it was the movies and then it moved to wine.

Speaker A:

I don’t know if that’s.

Speaker A:

Is that in the order that he started?

Speaker A:

But it was, it reminded me back when I, when I was.

Speaker A:

One of the reasons I started programming was to try and predict the football, you know, football league tables.

Speaker A:

That’s right.

Speaker A:

I was 8 years old writing basic code to do predict, you know, predictions of football.

Speaker A:

And that’s what we all try to do.

Speaker A:

We want good prediction models.

Speaker A:

We want to know, you know, what’s the future, you know, what’s, what’s, what’s the right, you know, where’s things going.

Speaker B:

Yeah.

Speaker B:

And I mean it Helps to have an intuitive model.

Speaker B:

Like you can kind of have an intuition about how some sort of classification or prediction might turn out.

Speaker B:

But really they’re in there.

Speaker B:

So that if someone meets me in the real world, they know not to make me watch a certain movie or to give me for instance.

Speaker A:

Because we want to optimize at the end, the most valuable thing we have in the world is time.

Speaker A:

And the more we can use these models to optimize our time wisely as possible, then that’s the great thing that comes out of them.

Speaker A:

So it is.

Speaker A:

So for yourself, the book’s been released and it’s available.

Speaker A:

Where can people get the book?

Speaker A:

Where’s the availability for them?

Speaker A:

Is it directly from those starts or is it like an available in bookstores or what’s the easiest way?

Speaker B:

As of, as of 9 June, it’s now available from all book retailers that stock it.

Speaker B:

No Starch Press website.

Speaker B:

You can go there directly and get a PDF version or the hard copy version.

Speaker B:

But yeah, it’s also now available on Amazon and other like Barnes and Noble.

Speaker A:

I’m gonna, I’m gonna add all the links into the show notes so it makes it probably, you know, easier for everyone to find it as well.

Speaker A:

So one of the things, I mean this is it’s such a fast moving industry and you know, sometimes just staying up to date.

Speaker A:

I’ve talked to a lot of startups or you know, AI startups and they’re looking about, you know, every six months, they don’t know where they’re going to be.

Speaker A:

So as the industry is moving really fast, you know, how do you stay up to date?

Speaker A:

What’s, how do you stay knowledgeable?

Speaker A:

You know, what’s, what’s your method of, you know, staying educated and informed in the advancements?

Speaker B:

Yeah, great question.

Speaker B:

I’m, I’m still so time poor that it can be really hard, but I think a couple of things I think are really important, especially in a field that moves as fast as AI security is.

Speaker B:

Firstly to have a really good breadth of understanding.

Speaker B:

So going to diverse sources I find really useful, especially in a field like AI security where it means so many different things and there’s different paradigms according to different people and for some people it’s just red teaming but really it should be more holistic than that.

Speaker B:

I think it’s, that’s really important to cover, you know, all the AI safety topics through to cybersecurity papers as well.

Speaker B:

So for example, I subscribe to a few email newsletters like Jack Clark for AI is really good.

Speaker B:

There’s a podcast I listen to the daily AI Breakdown, which is really helpful.

Speaker B:

Embrace the Red is a really good blog site, but there’s so many out there and I think people, if you search broadly and widely, that’s a really good way to start.

Speaker B:

I also am lucky that AI security is my day job.

Speaker B:

Like I, you know, I’m always, for example, doing projects where I’m helping build AI security frameworks for companies where I’m always learning something new about that industry or about that system.

Speaker B:

But also I run courses and stuff.

Speaker B:

So interacting with people in the courses is incredible.

Speaker B:

So learning what their real problems are and what’s happening on the ground in different industries.

Speaker B:

So more broadly speaking for your listeners who don’t get to work in AI security day to day, I still think that actually having those real human conversations with people, whether that’s at a B sides conference, the AI Security forum is incredible.

Speaker B:

If you want to go.

Speaker B:

They have one every year in Vegas around Hacker Summer Camp.

Speaker B:

They have others around the world as well, but that’s definitely the best my opinion, AI Security conference.

Speaker B:

But yeah, being able to connect with other people at that AI security intersection and hearing their thoughts, I think is a really valuable way of staying up to date.

Speaker A:

Fantastic.

Speaker A:

And for the training courses, I guess they can get it from the Mileva website.

Speaker A:

That’s where they can go on and find.

Speaker B:

Fundamentals.

Speaker B:

That’s what it’s called.

Speaker B:

And it’s a similar sort of trajectory for the book, but goes a bit deeper and some, I guess updates based on, I guess ways of teaching that work better for a course than for.

Speaker B:

But yeah, we get to work with people from.

Speaker B:

Yeah, no, when, when I first released the course, I thought if one person’s interested in this.

Speaker B:

And then we ended up having people sign up from like Microsoft, Palo Alto Networks, Google, like we’re training them, which is wild.

Speaker B:

So yeah, it’s a great, great community of people.

Speaker B:

But yeah, all the, like, the AI security niche is still so small that it’s such a great opportunity to get involved and even if you don’t come from AI or security that we need so much help that I really hope that even if people aren’t technical, even if they don’t want to read this book, even if they don’t want to code, they still feel like their contribution is worthwhile in this field because there are so many technical challenges we still have, but so many geostrategic challenges that we need help solving to make sure that AI security doesn’t become the threat that it needs to.

Speaker A:

Yeah, absolutely.

Speaker A:

We need to make sure that we think of it.

Speaker A:

One of the comments I said I was part of the subject matter expert group for the EU AI act and going back to.

Speaker A:And that was back in:Speaker A:

We were giving different problems that we had to.

Speaker A:

That was AI was being used in order to solve those problems.

Speaker A:

And one thing.

Speaker A:

So I said, you know, we need to embrace it, but we need to do it with responsibility that we are, we own it.

Speaker A:

And therefore that means any of the things that we want to, you know, make it.

Speaker A:

The goal is to make it our lives better with it.

Speaker A:

So it’s not about AI replacing people, it’s about AI empowering people.

Speaker A:

And how do we make sure that every time we think of a use case is it.

Speaker A:

How are we making people’s lives better by taking dangerous work away from people.

Speaker A:

You know, whether it be doing analyzing, you know, neural kind of, you know, mines and you know, dangerous places and forest fires and stuff, you know, all of those things where we can actually take humans away from the danger and let AI deal with it.

Speaker A:

And that was kind of one of the aspects.

Speaker A:

And so absolutely, I think, you know, we, we always have to make sure that from an ethical side of things.

Speaker A:

And that’s why I always say that, you know, when I heard about the book, I was like, oh, this is great because, because we’ve always looked how can we use it, but at the same time we also need to protect it.

Speaker A:

And that’s, that’s a fundamental kind of, I guess, responsibility that we have.

Speaker B:

Yeah, I definitely agree.

Speaker B:

And there are going to be some big conversations that societies have to make sure that AI can, you know, the, the incentive structures in place least help that happen too.

Speaker A:

Yeah.

Speaker A:

One of the concerns I have is that we’re trying to make it very human like, which I think is probably not the best thing to do is because then we, we misunderstand that when we’re inter interacting with AI is that we can misinterpret as being a human, as having intelligence.

Speaker A:

And I think we, we have to make sure that that and, and then people get, you know, psychology problems and stuff with inter interacting the more human like we make it it because that makes us more comfortable when we’re using technology, we’ll make it human like and again that, you know, we should also make sure that people are aware that, you know, what is human and what is not.

Speaker A:

So and it can be misinterpreted.

Speaker A:

That way as well.

Speaker A:

So if anyone has questions afterwards, they want to contact you.

Speaker A:

What’s the best way to reach out to you?

Speaker A:

What’s the best way to contact you?

Speaker A:

Is it through LinkedIn or to your website or otherwise?

Speaker B:

Yeah, people can absolutely reach out on LinkedIn.

Speaker B:

Harrietfarlow I also include my email address in the book, which was maybe a really dangerous to do, but it’s harrietarriathacks.com but if you look up Harriet Fowler, you’ll be able to find various ways of contacting me.

Speaker A:

Well, emails are meant to be public.

Speaker A:

The question is that, do you filter them enough?

Speaker A:

And the, you might have to have an agentic AI agent that just filters the incoming messages into things that you need to read and things that, you know is maybe, you know, something of interest.

Speaker A:

So yeah, it’s a challenge.

Speaker A:

We all have, you know, when email is getting, getting flooded with a lot of messages.

Speaker B:

So, so many.

Speaker B:

Yeah, I definitely, yeah, definitely want people to reach out and feel like, fantastic.

Speaker A:

I’ll also make sure that those links and everything’s in the show notes as well, so it’s easy for people to find.

Speaker A:

So, Harriet, it’s been fantastic speaking to you.

Speaker A:

I’m really excited, I’m excited to, to continue my, my goal is to go through this weekend and, and spend more time reading the book, but I also want to get to the point where in the next couple weeks I’m going to get more the practical side as well.

Speaker A:

You’re playing around with the scripts as well?

Speaker A:

Because I want to, I want to play around with the, the, the scripts, the Python scripts and I want to optimize the wine list as well with my own inputs.

Speaker A:

That’s one of the, that’s one of my fun things I’m going to play around with and then also get it into my, my own agent code as well to see how it works for that.

Speaker A:

So it’s been fantastic having you on, really enjoyed and looking forward to the future work and thanks for creating the book.

Speaker A:

It’s definitely something we all need because for me it’s really, it’s, it’s closed the gap and where I didn’t have knowledge in certain areas and it’s, it’s made it possible for me to better understand it.

Speaker A:

So thank you for everything you do.

Speaker B:

No, thank you so much, Joe.

Speaker B:

Thank you so much for having me.

Speaker A:

It’s a pleasure.

Speaker A:

So everyone, this is the Security By Default podcast.

Speaker A:

I’m the host, Joe Carson and it’s been awesome having Harriet on talking about the Practical AI Security book and we’ll make sure that you’ve got the links in the show notes to go and get your own copy so you can actually get to understand better about how you can make AI security and make the world a safer place.

Speaker A:

So for everyone, tune in every two weeks.

Speaker A:

Take care.

Speaker A:

Stay safe until next time.

Speaker A:

Thank you.

Posted by

in