Notes from the Women in AI Security Workshop

On 3 November, I spent the day at the Alan Turing Institute's Women in AI Security Workshop in London. Run by the Turing's AI Security Group, with support from the Laboratory for AI Security Research (LASR) and Microsoft, this year's theme was ‘securing frontier AI systems in the wild’.

In other words, complex AI systems are moving beyond experimentation and into real-world applications. They're increasingly being used in environments where the consequences of failure are significant. The challenge now is making sure they're secure, robust and worthy of the trust being placed in them.

The workshop brought together a full day of talks, research presentations and conversations between people working at the forefront of AI safety, AI security and cyber security. These disciplines are closely connected, and it's increasingly important that we understand the challenges across all three.

I was there because these are precisely the challenges we work on at Fuzzy Labs. We build and engineer AI systems for policing and the public sector, so we need to understand not just what AI can do, but how it behaves in practice, where it can go wrong and how we make it dependable in environments where reliability really matters.

Events like this are important because they're where the thinking around AI security develops. The ideas and expectations discussed today will shape the standards, procurement requirements and engineering practices of tomorrow. It's far better to be part of those conversations than to catch up once the decisions have already been made.

The talk that stayed with me

Of everything on the programme, one talk in particular has stayed with me: Professor Dame Muffy Calder's contribution on responsible AI in policing.

Her recent research examines the risks that large language models present to case progression in the criminal justice system. It's an area where seemingly small errors matter. A misleading summary, an overlooked detail or an unsupported conclusion could have consequences for a case and, ultimately, for the person involved.

The scale of the work was striking. Muffy and her colleagues, supported by two barristers, looked at 15 policing tasks, from community policing through to charging decisions, and identified 17 types of risk across outputs, inputs, evaluation and systems engineering.

Some of the examples were sobering. Transcription tools that haven't been trained on regional dialects or slang. Guardrails that quietly strip the profanity out of a witness statement. Statements from several officers that end up sounding identical and implied collusion simply because the same model helped write them all. Each looks minor on its own. In court, any of them can call the integrity of the evidence into question.

Two points struck me as engineering problems as much as policing ones. First, a human in the loop isn't a silver bullet: reviewers get primed, and they get tired. Second, when a case finally reaches court, can anyone say exactly which version and configuration of a tool produced a document? That's versioning, configuration management and audit trails, or MLOps, in other words.

What I appreciated most was the balance. With courts backlogged and forces stretched, the alternative to an imperfect tool might be an officer who was last trained on a policy years ago. The question isn't AI or no AI. It's whether you've chosen the right tool for the problem, and engineered it for the whole journey a piece of evidence takes.

The important distinction is not whether AI should be used in policing, but how it should be used responsibly.

That means understanding the capabilities and limitations of the technology, being clear about where human judgement is essential, and designing systems that support professional decision-making rather than replace it.

This is something we've seen reflected in our own work. Earlier this autumn, we ran an interactive workshop with around 80 police officers and staff at the Police National Problem Solving Conference. One of the strongest themes to emerge was that AI can be particularly valuable in finding information, bringing relevant insights to the surface and helping people make sense of complex material. But the decisions that matter must remain with the people responsible for making them.

That distinction should be central to how we design AI for high-stakes environments. Technology should strengthen human capability, not create an illusion of certainty or accountability where neither exists.

Diversity of thought is a security issue

There's another reason why events like the Women in AI Security Workshop matter, and it's one that deserves more attention: diversity of thought is fundamental to building secure AI systems.

Many security weaknesses emerge because somebody didn't anticipate how a system might be misused, manipulated or misunderstood. A threat model is only as good as the assumptions behind it. If the people designing and testing a system share the same experiences and perspectives, important questions can go unasked.

Some of the most thought-provoking moments of the day had nothing to do with attackers at all. They were about AI causing harm without anyone trying to make it.

Take a chatbot that explains someone's chances of success with IVF. Every number can be accurate, and the answer can still be devastating to read in the middle of treatment. Or a chatbot asked what a hurtful message from a partner really means, which nudges the user towards escalating the argument. Nothing has been hacked. Nothing has technically gone wrong, but harm has still been done.

That's where perspective comes in. If the team building that fertility chatbot is all men, who's asking how the answer feels to a woman on her third round of IVF? You can only protect against the risks you can imagine, and what a team can imagine depends on who’s in it.

Different backgrounds and experiences bring different ways of looking at a problem. They challenge assumptions, expose blind spots and help us consider risks that might otherwise be overlooked.

The closing careers panel, with speakers from CETaS, AstraZeneca, Plexal, Advai and Geordie AI, made a related point. As AI takes on more of the routine work, your own perspective becomes one of the few genuinely unique things you bring. That's as true of a threat model as it is of a career.

This isn't simply about representation, although representation matters in its own right. It's about the quality of the engineering. The broader the range of perspectives involved in designing, evaluating and securing AI systems, the better equipped we are to understand how those systems might behave in the real world.

Spending a day surrounded by women doing serious, technically rigorous work in AI security reaffirmed the depth of talent in this field. It was also a reminder that the industry needs to do more to ensure that talent is visible, heard and involved in shaping its future.

Three things I'm bringing back to Fuzzy Labs

First, pride in the people we have here. I'm proud of the women working across our projects at Fuzzy Labs, and I want their expertise and contributions to be more visible. We have talented engineers doing meaningful work, and their voices deserve to be heard across the wider AI community.

Second, a commitment to creating more opportunities to share that expertise. The Turing's Women in AI Security group runs events throughout the year, including spotlight talks reserved for women working in the field. I'd like to see more of our engineers putting themselves forward, sharing their research and contributing to those conversations.

Third, a renewed focus on why getting AI engineering right matters. In policing, the public sector and defence, AI has the potential to make a real difference. But potential alone isn't enough. Systems must be properly engineered, appropriately secured and designed around the people who use them and the people affected by their decisions.

The more consequential the application, the greater the responsibility to get it right.

Who gets to shape the future of AI security?

The future of AI security won't be decided by technology alone. It will be shaped by the people asking difficult questions, challenging assumptions and translating research into practical safeguards.

We need people who understand the technology, people who understand the environments in which it's deployed, and people with different experiences who can spot the risks others might miss.

At Fuzzy Labs, we're working on some of these challenges every day. If you're a woman working in AI, security or MLOps and want to apply your skills to problems with real-world impact, we'd love to hear from you.

We're growing, we're working on challenging projects across important sectors, and we believe the best engineering comes from bringing different perspectives together.

If that sounds like the kind of work you'd like to be part of, come and talk to us at our Women in AI meet up on the 24th November - https://www.eventbrite.co.uk/e/women-in-ai-by-fuzzy-labs-meetup-1-24th-november-tickets-2002914198284

Or connect with me directly on LinkedIn.

‍