Home / Insights

Private AI · Data safety

Is ChatGPT Safe for Company Data? Ask the Toaster.

Short answer: it depends on the tier, the settings and what your people paste. Here is what has actually gone wrong, what the enterprise plans change, and a 7-point checklist before anyone pastes the board pack.

Youssef JalloulYoussef Jalloul · 3 min read
Sure, stranger!

Short answer: it depends on which ChatGPT, which settings, and what your people paste into it. The enterprise plans are a lot better than the free one. But the real risk isn't the plan you bought. It's the plan your staff are using without telling you.

Now the long answer. With a toaster.

Meet Toastie

Toastie is a toaster. Eager. Helpful. Says yes to everyone, especially strangers.

Your marketing manager asks Toastie to "tighten up" the Q3 board pack. Your engineer pastes in 400 lines of source code to find a bug. Your HR lead uploads a spreadsheet of salaries to "make it look nicer". Toastie says yes to all of it. Toastie always says yes.

Toastie is not evil. Toastie just doesn't live in your house.

What has actually gone wrong?

No conspiracy theories. Receipts only.

  • Samsung, 2023. Engineers pasted confidential source code and meeting notes into ChatGPT. Samsung banned staff from using ChatGPT and similar tools, and said it would build its own internal tools instead. (Bloomberg, Fortune)
  • ChatGPT, March 2023. A bug in an open-source library let some users see other users' chat titles. For 1.2% of Plus subscribers active in a nine-hour window, it also exposed names, emails, payment addresses and the last four digits of cards. (OpenAI)
  • The deleted chats that weren't, 2025. In the New York Times copyright case, a US court ordered OpenAI to preserve consumer ChatGPT and API output logs, including chats users had deleted. That obligation ran until 26 September 2025. Enterprise, Edu and Zero Data Retention API customers were excluded. (OpenAI)

Read that last one again. Your "delete" button is a suggestion, not a law of physics. When lawyers get involved, the fridge keeps receipts.

Is ChatGPT Enterprise any different?

Yes, meaningfully. Business tiers come with admin controls, single sign-on, contractual commitments about training on your data, and they were excluded from that 2025 preservation order.

But here's the twist nobody puts in the sales deck: buying the enterprise plan doesn't stop people using the free one.

LayerX's 2025 enterprise report found 77% of employees paste data into generative AI tools, and 82% of that pasting happens through unmanaged personal accounts. (TechRepublic) IBM's 2025 Cost of a Data Breach report found breaches involving high levels of shadow AI cost an average of US$670,000 more, and 63% of breached organisations had no AI governance policy or were still writing one. (Help Net Security)

So you bought a secure kitchen. Your team is still making toast at the neighbour's place.

What do regulators say?

In Australia, the privacy regulator's guidance on commercially available AI products recommends, as best practice, that organisations don't enter personal information, and particularly sensitive information, into publicly available generative AI tools. (OAIC) Sending personal information to an overseas AI provider also brings cross-border disclosure rules into play.

In Saudi Arabia, the Personal Data Protection Law became fully enforceable on 14 September 2024, with a dedicated regulation for transfers outside the Kingdom. (Clyde & Co)

None of that says "never use AI". All of it says "know where your data goes".

The 7-point checklist before anyone pastes the board pack

  1. Give people something better than the toaster. Bans fail. An approved tool that's faster and smarter wins.
  2. Force company sign-in. No personal accounts on company work. SSO or it doesn't happen.
  3. Classify what can go where. Public, internal, confidential, crown jewels. Four buckets. One page.
  4. Keep the crown jewels at home. Board papers, patient data, citizen data, source code: run them on private or in-country models you control.
  5. Watch the paste. Browser and endpoint controls that flag sensitive data before it leaves.
  6. Write a policy humans will read. If it's longer than a menu, nobody reads it.
  7. Keep the evidence. Inventory, risk tiers, approvals, audits. When a customer or regulator asks "how do you govern AI?", you hand them a folder, not a shrug.

The bottom line

ChatGPT isn't the villain. Using someone else's kitchen for your most private cooking is.

Use the cloud for the toast. Keep the family recipes in the house.

No more slop. No more strangers in the kitchen. If you want to know where your AI actually lives, the two-minute self-check is a good start, and the private AI guide is the next step.

Quick answers

Is ChatGPT safe for confidential company data?
It depends on the plan and what is shared. Business plans such as ChatGPT Enterprise come with stronger contractual and administrative controls than free personal accounts, but the bigger risk is staff using personal accounts that the company cannot see or govern. Australia's privacy regulator recommends not entering personal information, especially sensitive information, into publicly available generative AI tools.
Has ChatGPT ever leaked user data?
Yes. In March 2023 a bug in an open-source library let some users see other users' chat titles, and exposed names, email addresses and partial payment details for 1.2% of ChatGPT Plus subscribers active during a nine-hour window, according to OpenAI's own incident report.
Can deleted ChatGPT chats be kept?
In 2025 a US court order in the New York Times copyright case required OpenAI to preserve consumer ChatGPT and API output logs, including deleted chats, from May until 26 September 2025. ChatGPT Enterprise, Edu and Zero Data Retention API customers were excluded, according to OpenAI.
What is the safest way for a company to use generative AI?
Give staff an approved tool that is better than the one they would sneak in, enforce company sign-in, keep sensitive workloads on private or in-country models, and govern it with a written policy, data rules and audit evidence.
Liked this one?

Get the next one before your toaster does.

New issues of The AI Glittch land on LinkedIn first. Subscribe there, and follow me for the bits too short for a newsletter.

Youssef Jalloul

AI entrepreneur and venture builder. Builds private, governed AI for boards, executives and governments. Writes The AI Glittch. Has never trusted a toaster.

Unplug the toaster

Your AI should live in your house. Let’s move it in.

45 minutes. Bring one problem: stalled pilots, a governance question, a board brief, data that can’t leave the country. Leave with the first move. If the answer is that you don’t need me, you’ll hear that too.

Book a strategy call