WorkAI at Scrut

AI at Scrut: from a Q&A assistant to agents

Role
Head of Design, then Director of Design
Team
2 Designers on AI Features, Partner with AI PM + ML Engg
Timeline
January 2025 → Now
Scope
Scrut Teammates, five AI features inside workflows, compliance agents, testing AI output

I led design for three stages of AI at Scrut: a Q&A assistant, AI features inside everyday workflows, and a set of agents that check compliance continuously. Compliance customers need to trust what the product tells them, and AI can be wrong. At each stage my job was to work out how far users could rely on the AI, and to design the product so they could always check its work.

3Stages of AI, one at a timeAssistant, workflows, agents
5AI features inside workflowsQuestionnaires to cloud fixes
1Reusable review patternBehind all five features

The decision

Position it as a junior expert

We built a compliance Q&A assistant and deliberately positioned it as a junior infosec expert. People already know how much to trust a junior colleague: helpful for a first answer, never the final word, always worth checking.

Ask users to learn a new kind of tool

Users work out for themselves how far to trust it.

A junior infosec expert Chosen

A familiar idea. Every answer links to its sources, and with no source the assistant does not answer.

What I did

Trust designed in at each stage

  • Designed Teammates to show its work: sources on every answer, and a carefully designed "I can't answer this".
  • Designed for review across five workflows: accept, edit and reject, the evidence, and a confidence score.
  • Fixed gap analysis after it flagged gaps in our own policy templates, by teaching the system about policy layers.
  • Decided when agents speak up, reusing the notification and task system I had built.

The study in one picture

Figure 1Three stages of AI at Scrut: what the AI does, what the user does, and the main safeguard at each step.

Read the full case study

Password protected. It has the trade-offs in full, the results and how they were measured, and what I would do differently.