Run · founder · 5 min read

The Fireworks AI Breach: Your AI Chats Went to a Vendor You Never Chose

A breach at AI provider Fireworks exposed Notion AI chat transcripts. What happened, who is affected, and how to limit what hidden vendors hold.

On November 12, TechCrunch reported that hackers who broke into Fireworks AI had accessed chat transcripts belonging to Notion customers. Notion has confirmed it.

Most Notion users have never heard of Fireworks. It’s an infrastructure company that runs AI models on behalf of other products. You typed into Notion, and part of what you typed was sitting on someone else’s systems. That is the real story here, and it applies to nearly every AI tool you use.

What happened at Fireworks

This timeline comes from Fireworks’ own October bulletin and its November 9 update.

  • October 6: Fireworks spotted someone using its internal credentials without permission. The attacker read settings attached to jobs in a limited number of customer accounts, including API keys and tokens those customers had stored as secrets.
  • October 9: Fireworks published its first bulletin. It said it had found no evidence that customer models, datasets or prompts had been accessed.
  • October 23-24: a separate cloud provider that hosts Fireworks’ research machines had its own security incident. A day later the attacker used that to reach Fireworks development machines and run an unauthorized job. Fireworks says it caught this the same day. It has not named the provider.
  • November 9: Fireworks updated its account. Billing details (addresses, invoice and payment history, though not card numbers) were accessed, and for certain customers so were a subset of datasets and unfinished model files.

So the October reassurance about datasets did not hold. Fireworks says its main inference service, which keeps no customer data by default, was not affected, and that its investigation is still running.

Fireworks also says the evidence points to a financially motivated criminal group that sells stolen access to AI models through underground resellers. The industry calls this LLMjacking. It’s the same motive behind the Railway breach, where attackers went straight for AI provider keys.

What Notion users lost

Notion’s security advisory and help page fill in its side.

Fireworks first told Notion on November 2 that some data had been accessed, and at that point Notion believed it was only test data. Notion disconnected from Fireworks the next morning and deleted what it had stored there. On November 8, Fireworks confirmed the final list of what was taken, and it included real Notion AI chat transcripts. Notion contacted affected customers on November 9.

The details that matter:

  • Scope: Notion says fewer than 1% of workspaces are affected. It hasn’t given a number.
  • What was in the transcripts: anything a person typed into the chat and anything the AI wrote back. The transcripts came from a feature called Insights, which Notion ran through Fireworks and has now switched off.
  • Attribution: for the vast majority of affected workspaces, Notion says the transcripts carried no label linking them to a user, workspace or company.
  • Meeting notes: AI Meeting Notes audio, transcripts and summaries were not affected, unless someone pasted them into a Notion AI chat.
  • Leaked keys: on November 11, Notion revoked Notion API keys it found inside exposed messages. People had pasted them into the chat.

There is one loose end. Fireworks says the services in question keep no data by default. Notion says its investigation found data had been retained from the Fireworks endpoints it used. Neither company has explained the gap.

Are AI coding tools affected?

Yes, at least one. On November 13, Cursor said a key it had issued to Fireworks was exposed by Fireworks, and that an attacker used it to copy coding agent traces from a storage bucket the two companies shared. Credentials found in those traces led to October’s trojanized Claude Code extension. Cursor says fewer than 0.05% of users had traces exposed, none of them Privacy Mode users, and that it has moved all inference off Fireworks. Our breakdown of Cursor’s report has the details.

TechCrunch also names the legal AI company Harvey as a Fireworks customer but does not say it was affected. Fireworks has not said how many customers were hit.

If you or your developer have your own Fireworks account, follow the notice Fireworks sent: replace any revoked keys, rotate every third-party credential you stored there as a secret, and check your usage logs for activity you don’t recognize.

What to do now

If you use Notion AI

  1. Check whether Notion contacted you. If it didn’t, Notion says your workspace hasn’t been identified as affected and you don’t need to do anything.
  2. If it did, review what was exposed. Workspace owners can open the ”?” menu in the sidebar and choose “Security incident review.” It only works in a desktop browser.
  3. Rotate anything secret that went into a chat. Passwords, API keys, database connection strings. If anyone on your team pasted one into Notion AI, change it today.

If you build with AI tools at all

  1. Stop pasting secrets into AI chats. Any of them. A chat box feels private. It is a form that sends your text to at least one company and often several.
  2. Read the subprocessor list. Every serious software vendor publishes a page listing the other companies that handle your data. Search for the tool’s name plus “subprocessors.” Five minutes there tells you who else is in the chain.
  3. Ask about retention. “Do you and your AI providers keep my prompts, and for how long?” is a fair question for any vendor. Zero retention is the answer you want, in writing.
  4. Hold less yourself. If your own app sends customer text to an AI model, don’t store raw conversations forever just because you can. Data you didn’t keep can’t leak.
  5. Cap spending on your AI keys. Stolen keys get resold and run up bills. A monthly limit on each key turns a disaster into an annoyance.

The bigger picture

Two things stand out.

First, the chain is longer than it looks. You chose Notion. Notion chose Fireworks. Fireworks chose a cloud provider, and that provider’s incident is how the second intrusion happened. Your data’s safety depended on a company three steps away that still hasn’t been named.

Second, early breach statements are provisional. Fireworks said on October 9 that it had no evidence datasets were touched. A month later it said some were. Notion was first told the exposed data was synthetic. Six days later it wasn’t. Nobody appears to have lied; investigations simply find more over time. When a vendor you rely on discloses a breach, act on the worst plausible reading in week one rather than the most comforting one.

None of this is a reason to stop using Notion AI or any other AI tool. It is a reason to treat every AI chat box as a place where secrets don’t go.

Related guides

Recommended next step

Was this helpful?