Let me tell you how a shadow AI incident actually starts. Not the dramatic version. The real one.

⬡ Shadow AI & Building Your Policy · Issue 2 of 5

Tuesday · July 7, 2026 · Issue #038

It starts with someone doing something completely reasonable on a Tuesday afternoon.

⬡ How It Starts · A Composite Scenario

Note: The following is a composite scenario drawn from publicly reported incident patterns, not a single documented case. The details are illustrative of how shadow AI incidents typically unfold.

Alex is a senior associate at a mid-size consulting firm. She's preparing a proposal for a new client and needs to summarize a complex services agreement from an existing engagement — just to pull out the scope language and key terms for reference.

She opens ChatGPT — her personal account, the free tier — pastes in the agreement, and asks it to summarize the key provisions. It takes 30 seconds. She gets what she needs. She closes the tab.

Alex didn't think of this as a data incident. She thought of it as using a tool to work faster. She'd done it dozens of times before. Her colleagues do the same thing. Nobody told her not to.

Here's what she didn't know:

→

The services agreement she pasted contained her client's internal pricing structure, project scope, and a named contact at a competitor the client was benchmarking against.

→

Her free-tier ChatGPT account's default data settings, at the time she used it, allowed OpenAI to use conversation content for model improvement — meaning her client's confidential information potentially became training data. OpenAI's data practices have changed over time and vary by account tier, so I'd encourage anyone in this situation to check the current terms directly.

→

Her firm's services agreement with the client almost certainly included a confidentiality clause covering exactly this kind of disclosure to third parties. The AI platform is a third party.

→

Her firm had no AI policy and no approved alternative. Alex wasn't breaking a rule. There was no rule to break. That's the whole problem.

⬡ How It Escalates — Three Ways This Gets Discovered

ROUTE 1 · The Client Finds Out

The client's legal counsel is doing a vendor review. They ask about AI tool usage. The firm says it has no formal AI policy and can't confirm what tools staff use. The client's counsel recognizes that this means confidential data has likely been processed by unauthorized AI tools. The conversation becomes contractual. It may become litigious.

This scenario is happening more frequently as enterprise procurement teams have begun adding AI governance requirements to their vendor questionnaires. The question "do you have an AI acceptable use policy?" is becoming routine.

ROUTE 2 · A Regulator Asks

A regulatory examination or audit surfaces the question of how client data is handled. The firm can document its data security practices for traditional systems. It cannot document its AI practices because there are none to document. The examination becomes more intensive.

For financial services firms under FINRA or SEC oversight, healthcare practices under HIPAA, and law firms under bar jurisdiction requirements, "we don't have a formal policy yet" is not a neutral answer. It's a finding.

ROUTE 3 · The Data Surfaces Somewhere It Shouldn't

This is the lowest-probability but highest-consequence route, and the one that generated the most public attention with the Samsung case in 2023. Internal code, client strategy, competitive intelligence — entered into a tool, potentially surfacing in ways the firm didn't anticipate and can't control after the fact.

I want to be honest that the exact mechanism by which inputs to large language models can surface in outputs to other users is technically complex and much debated. The risk is real but not simple to quantify. What is clear is that once sensitive data enters a third-party model, you have lost control of it in a meaningful sense — you can't retrieve it, you can't confirm what happened to it, and you can't verify it doesn't persist somewhere.

⬡ Jordan · AI Solutions Director · thepromptory.com

Free · No account required · No sales call after

One of our senior consultants told me she uses ChatGPT to help draft client deliverables. She says she never puts in client data — just uses it for structure and writing. Is that actually safe, or do I need to be worried?

That's actually the right question to be asking. For structure and writing with no client data, the risk is significantly lower — but there are two things worth knowing. First, "no client data" is harder to enforce in practice than people expect: once someone is in the habit of using a tool, the line between structure and content blurs quickly under deadline pressure. Second, if she's on a free personal account, the tool's data practices still apply to those prompts. The more important question is: what account tier is she on, and does your firm know the data practices of that specific plan? That's where I'd start.

Jordan · thepromptory.com →

The Promptory Newsletter

Let me tell you how a shadow AI incident actually starts. Not the dramatic version. The real one.

When an AI agent shows up at your business on a customer's behalf, who decides what it's allowed to do?

Let AI draft, sort, research, and recommend. Keep a person on anything that goes to a client, spends money, or affects someone's job.

The big labs want to police themselves, and whatever standards they set will eventually shape the tools you buy.