⬡ Shadow AI & Building Your Policy · Issue 2 of 5

Note: The following is a composite scenario drawn from publicly reported incident patterns, not a single documented case. The details are illustrative of how shadow AI incidents typically unfold.

Alex is a senior associate at a mid-size consulting firm. She's preparing a proposal for a new client and needs to summarize a complex services agreement from an existing engagement — just to pull out the scope language and key terms for reference.

She opens ChatGPT — her personal account, the free tier — pastes in the agreement, and asks it to summarize the key provisions. It takes 30 seconds. She gets what she needs. She closes the tab.

Alex didn't think of this as a data incident. She thought of it as using a tool to work faster. She'd done it dozens of times before. Her colleagues do the same thing. Nobody told her not to.

Here's what she didn't know:

ROUTE 1 · The Client Finds Out

The client's legal counsel is doing a vendor review. They ask about AI tool usage. The firm says it has no formal AI policy and can't confirm what tools staff use. The client's counsel recognizes that this means confidential data has likely been processed by unauthorized AI tools. The conversation becomes contractual. It may become litigious.

This scenario is happening more frequently as enterprise procurement teams have begun adding AI governance requirements to their vendor questionnaires. The question "do you have an AI acceptable use policy?" is becoming routine.

ROUTE 2 · A Regulator Asks

A regulatory examination or audit surfaces the question of how client data is handled. The firm can document its data security practices for traditional systems. It cannot document its AI practices because there are none to document. The examination becomes more intensive.

For financial services firms under FINRA or SEC oversight, healthcare practices under HIPAA, and law firms under bar jurisdiction requirements, "we don't have a formal policy yet" is not a neutral answer. It's a finding.

ROUTE 3 · The Data Surfaces Somewhere It Shouldn't

This is the lowest-probability but highest-consequence route, and the one that generated the most public attention with the Samsung case in 2023. Internal code, client strategy, competitive intelligence — entered into a tool, potentially surfacing in ways the firm didn't anticipate and can't control after the fact.

I want to be honest that the exact mechanism by which inputs to large language models can surface in outputs to other users is technically complex and much debated. The risk is real but not simple to quantify. What is clear is that once sensitive data enters a third-party model, you have lost control of it in a meaningful sense — you can't retrieve it, you can't confirm what happened to it, and you can't verify it doesn't persist somewhere.

⬡ Jordan · AI Solutions Director · thepromptory.com

Free · No account required · No sales call after