Somebody in Your Business Is Pasting Customer Data Into ChatGPT

The receptionist had thirty referral letters to turn into diary notes and about forty minutes before the doors opened. So she did the sensible thing. She opened a free chatbot in a browser tab, pasted the first letter in, and asked it to cut the whole thing down to two lines. It worked beautifully. She did the other twenty-nine in under ten minutes and got on with her morning.

That is a composite rather than one specific incident, assembled from a conversation I have now had with several owners this year. The version I keep returning to is a physiotherapy clinic in Cavan, because referral letters carry names, dates of birth, GP details and medical history. Special category data, in the language of the regulation. It left the building thirty times before the clinic opened, and nobody involved did anything they believed was wrong.

That is the part worth sitting with. Nothing broke.

The incident that never announces itself

Almost everything my team deals with tells you it happened. A site goes down and monitoring pages someone. Logins fail repeatedly and fail2ban writes a line. There is a timestamp, a log file, and a person to ring.

This one has none of that. No alert fires when an employee pastes a customer file into a chat window. No dashboard turns amber, no backup runs, no rollback is available. The first time most owners find out is months later, when somebody asks a question they cannot answer: where did that information go, and who holds a copy of it now.

The scale is easy to underestimate. The CSO's Information Society Statistics found that just over a fifth of enterprises in Ireland used AI in some capacity during 2025, up from a little over 15% the year before, with the small-enterprise figure sitting around 17% ^[1]. Natural language generation, which is the formal name for asking a machine to write something for you, was among the two most common uses.

Read those numbers with one caveat, though. The survey asks what the business uses. It does not ask what individual staff quietly opened in a browser tab on a busy morning, and that is a different, almost certainly larger figure.

Where the text actually goes

The single most useful thing you can understand about this is that the account tier decides what happens to your words. Not the prompt, not the wording, not how careful the person was being.

OpenAI publishes this plainly. On consumer plans, ChatGPT improves by further training on the conversations people have with it unless the user opts out under Settings and then Data Controls. ChatGPT Business, Enterprise and Edu accounts, and the API, are excluded from that training by default ^[3].

So the practical position in most small businesses is this. The tool was signed up for on a personal free account, in a personal name, using whatever email address that person happened to use. The customer's referral letter, or the client contract, or the complaint that arrived through your website contact form, now sits on a third party's infrastructure under terms nobody in your business has read. If that member of staff leaves in November, the account goes with them and you have no way to retrieve or delete anything.

None of this requires a careless employee. It happens to organisations that take security seriously. In 2023, Samsung restricted staff use of generative AI tools across one of its largest divisions after discovering engineers had pasted internal source code and meeting notes into ChatGPT ^[5]. Samsung employs security teams larger than most Irish towns. This is a defaults problem, not a discipline problem.

Why the responsibility lands on you

This is the bit that surprises owners. The AI company is not carrying this for you.

Under GDPR, whoever decides why and how personal data is processed is the controller. If your business decides that summarising client files with a chatbot is a good idea, that is your decision and your processing. The Irish Data Protection Commission's guidance on AI and large language models says it directly: where an organisation uses an AI product supplied by a third party, additional data protection risks arise from the personal data that employees input to the tool, and as a user of such a product your organisation could be a data controller, in which case a formal risk assessment should be considered ^[2].

There is a second gap underneath that one. GDPR requires a written contract with anyone processing personal data on your behalf. A personal consumer subscription does not give you that contract. Business and enterprise tiers exist partly because they do.

The moment this stops being abstract is a subject access request. A customer writes in and asks for everything you hold on them, and who else has seen it. You have thirty days to answer. "Some of it went into a chatbot, on an account we do not control, and we are not sure how much" is not an answer you want to write down and sign.

The obligation that quietly changed last month

There is a training duty here too, and most owners have never heard of it.

Article 4 of the EU AI Act has required providers and deployers of AI systems to take measures supporting the AI literacy of their staff, and of anyone else operating AI on their behalf, since 2 February 2025. It applies regardless of headcount. What changed recently is the strength of it: the Digital Omnibus on AI entered into force on 27 July 2026, and the European Commission's own summary of it states that the previous AI literacy requirement for companies is simplified, with the Commission and the member states taking a stronger role in promoting AI literacy ^[4].

In plain terms, you are no longer expected to guarantee a level of competence in each individual. You are still expected to take reasonable measures, weighed against your size and your resources. For a five-person practice that means a short written guide and a conversation, not a course.

Keep this separate in your head from the disclosure rules that landed on 2 August. Those govern telling visitors when they are talking to a machine on your own site, which is a narrower set of duties than the headlines suggested. This one is about your team, your tools and your customers' data, and it has been live for eighteen months.

Getting AI use under control in four steps

The good news is that this is an afternoon of work, not a compliance project. If you rang me about it tomorrow, this is the sequence I would give you.

• Inventory what people already use. Ask the team, without any suggestion of blame, which AI tools they have used for work in the last month and what for. Ask it as curiosity, because the honest answer is the whole point of the exercise.
• Consolidate onto an account the business owns. Move any use that touches customer information onto a business-tier account paid for by the business, with the owner as administrator. That account carries a processing agreement and does not walk out the door with a staff member.
• Redact by default. Agree one rule that people can recall under pressure: no names, no addresses, no account numbers, no health details and no payment details go into a prompt. Anonymise first, ask second. Most tasks work just as well on a stripped-down version.
• Document it on one page. Write down the tools, the account, the identifier rule and who to ask when someone is unsure. Date it. A policy nobody can find or remember is decoration.

Notice what is not on that list. No consultant, no audit, no ban. Banning these tools outright is the option that reliably fails, because it does not remove the pressure that made the receptionist open the tab in the first place. It just moves the activity somewhere you cannot see it.

What none of this fixes

I would rather tell you the limits than let you find them later.

Moving to a business tier stops your text being used to train a model. It does not mean the text vanishes. Content can still be retained for a period for abuse monitoring and safety review, and genuine zero-retention arrangements are usually something an organisation has to request rather than something that arrives with the subscription. "Not used for training" and "not stored" are two different sentences, and vendors are precise about which one they are saying.

The other limit is blunter. There is no rollback here. If a customer file went into a consumer chat window in March, no policy you write in August retrieves it. You can stop the next one. You cannot undo the last one. That asymmetry is exactly why the inventory step comes first.

And one honest boundary on my own advice: this is operational guidance from someone who runs hosting infrastructure, not legal advice for your particular business. If you handle health records, financial data or anything covered by professional confidentiality, twenty minutes with your solicitor is money well spent.

When the free tier is genuinely the right call

I am not going to pretend every business needs a paid plan.

If you are a sole trader with no employees, and you use a chatbot to draft product descriptions, tidy up a marketing email or suggest a few subject lines, and no customer identifiers ever enter the prompt, the free tier is genuinely adequate. A business subscription would buy you a data processing agreement covering data you are not putting in anyway. Spend the money somewhere it does something.

The line is not the price of the plan. It is whether another living person's information goes into the box. That is the question to hold on to.

The same question applies to your website

Worth turning this on your own site while you are at it, because the same principle governs both: know what leaves, and know who holds it.

The AI in a website builder is a good example of the distinction. When you describe your business and an AI assembles a complete WordPress site in under a minute, the input is a description of what you do, your services and your tone. Not your client list. That is a one-time generation from information you were going to publish anyway, which is a very different exposure from feeding a customer's file into a chat window every Tuesday. If you want the wider picture of where AI genuinely fits on an ordinary business site, that groundwork is worth covering first.

The same audit applies to the tools already running on your pages. Every analytics script, chat widget and form plugin is a route by which visitor data leaves your site for somebody else's servers, which is why analytics that work without cookie consent are worth understanding properly. On our platform that data stays on Irish infrastructure, and support comes from people in the same timezone as you, which matters more than it sounds when you have a thirty-day clock running on a data request.

The takeaway

Nobody in that clinic set out to disclose patient information. They were short on time, the tool was free, it was already open, and it worked. That combination beats a policy written on an intranet nobody visits, every single time.

So the useful move this week is not a decision about whether AI is safe. It is a five-minute conversation asking your team what they are already using, and for what. Whatever comes back, you will know more than you did this morning, and you will be able to make the next call with your eyes open.

Frequently Asked Questions

Is it against GDPR to put customer data into ChatGPT?

It is not automatically unlawful, but it is processing, and processing needs a legal basis, a risk assessment and a contract with whoever handles the data on your behalf. A personal consumer subscription gives you none of those things. The DPC's guidance notes that where staff input personal data into a third-party AI product, your organisation could be acting as a data controller. The practical answer for most small businesses is to strip identifiers out of prompts, or move business use onto a business-tier account that carries a processing agreement.

Does ChatGPT train on what I type into it?

It depends entirely on the account. OpenAI's policy is that consumer plans improve the model by training on conversations unless you opt out under Settings and Data Controls, while ChatGPT Business, Enterprise, Edu and the API are excluded from training by default. The tier decides, not the wording of your prompt. Most staff signed up on a free personal account, which is the training-on default.

Turning off model training means my data is deleted, right?

No. Training and retention are separate things. Content can still be held for a period for abuse monitoring and safety review even when it is excluded from training, and zero-retention arrangements are generally something you request rather than something you receive automatically. "Not used for training" is not the same sentence as "not stored".

Do I legally have to train my staff on AI?

Article 4 of the EU AI Act has required providers and deployers to take measures supporting staff AI literacy since 2 February 2025, and it applies regardless of company size. The Digital Omnibus on AI, in force since 27 July 2026, simplified that duty rather than removing it. You are still expected to take reasonable measures, judged against your size and resources. For a small firm that is a short written guide and a conversation.

What should an AI policy for a small business actually say?

One page is enough. Name the tools people may use and the account they should use them under. State one memorable rule about identifiers, such as no names, addresses, account numbers, health details or payment details in a prompt. Say who to ask when someone is unsure. Date it, and say when you will review it.

Is a free chatbot ever fine for business use?

Yes. If you are a sole trader with no employees, using it to draft a product description or tidy a marketing email, and no customer identifiers ever enter the prompt, the free tier is genuinely adequate. The line is not the price of the plan. It is whether another person's data goes into the box.

Sources