Skip to content
AgentFix

Adoption LessonsGovernance4 min read

Four questions our AI policy didn’t answer

Jason Lawrence

Our AI policy answered the questions the people who wrote it had thought of. In its first five months, our team asked four it hadn’t. They asked in the open, in the Slack channel where we answer AI questions, and each answer changed what the policy or the practice said.

The number: four questions in five months

Do we tell clients? In April someone asked whether we had a standard for disclosing AI use in client-facing documents, and suggested a line for the footer. Nobody answered it in the channel. The answer now sits in one of our five rules, “say so”: clients are told when AI use is material. We tell them once, at the start of an engagement, rather than stamping every document.

What does de-identify mean? In June a consultant asked, apologising first for “a silly question”, whether the rule meant renaming a spreadsheet, a set of transcripts and every mention of the client in them before using them, then changing it all back so the files could be found again. It didn’t. The policy meant client data, not client documents, and its worked example had pointed people the wrong way. A second consultant had read it the same way. A third thought the rule might be different in a chat and in a Project. The wording changed in the next version, seven weeks later.

Can I install someone else’s skill? In July a consultant found a useful skill on GitHub, had Claude check it for security problems, then asked whether there was a policy on installing external skills. They had already worked out the data answer for themselves: the skill is fine, as long as you are deliberate about what goes in and where the output lives. The policy still doesn’t name skills. That one is open.

Can we use AI on this client’s data? In September a delivery lead asked before starting. They wanted AI to help analyse a client’s address data, and they listed exactly which fields and why. A director answered in the thread: metadata and patterns across records are within our rules, and to be safe, email the client and ask for written approval. The draft email that came with the answer said what the analysis would look at, what it wouldn’t touch and how the data was protected, and asked for a one-line reply. It’s a good model for that email.

What went right

All four were asked where everyone could see them, so each answer reached the next person with the same question. The policy now sends tool approvals to that channel too. Three policy versions in five months is what this looks like from the outside. These four questions are what it looks like from the inside.

What went wrong

“Sorry if this is a silly question” is the line a leader should worry about. The question was sensible. The policy made a reasonable reading look silly, and the consultant was about to spend hours renaming files to comply with it. Another person had read it the same way and said nothing until someone else asked. We don’t know how many quietly worked around it.

And the first question never got an answer in the channel. A question left unanswered teaches people not to ask the next one.

What we would do now

Expect the questions, and decide in advance where they go. Every question your policy doesn’t answer will be asked within weeks of a rollout. Whether it is asked in the open or worked around in private is the only part you control.

Answer each one in the channel, and put the answer where the next person will look. Treat any question asked twice as a change for the next version. And when someone apologises for asking, fix the wording, not the person.

One thing to try this week

Search your team channel for “policy” and “allowed”. Count the questions that never got an answer. Each one is a gap someone is working around.

If you don’t have a one-page version of your rules to point people at, here is a template to start from.

The Adoption Diagnostic’s interviews ask your people what they have done with Claude that the policy doesn’t cover. About the Adoption Diagnostic

More lessons

Get new lessons by email

Short lessons from a real Claude rollout, each backed by a number.

A new lesson every Tuesday. Unsubscribe any time. Handled as set out in our privacy policy.