What Training Data Should a Live AI Agent Be Allowed to Remember?

A live AI agent should remember almost nothing by default. After a run, keep only what an owner explicitly approved for learning — usually playbook version ids and coarse outcome labels — and treat customer details, VIP notes, discounts, and raw transcripts as ephemeral or blocked. If the agent “gets smarter” by silently keeping yesterday’s customer text, you do not have continuous improvement. You have an unapproved training set sitting next to production.

This is different from the audit trail. What Audit Trail Should a Live AI Agent Leave? covers immutable events you need to reconstruct a run. Memory policy decides what may survive after those events are written — and what must never become future training fuel. It is also different from How Should You Version the Playbook for a Live AI Agent?: versions name the rules; retention names what the agent may keep about past work under those rules.

Pro Logica’s AI agents solutions page shows the same loop across trades: open the screen, follow the playbook, pause for a human. That loop stays safe only if “learning from production” is a named retain list, not a vague promise that the system will get better over time.

What counts as training data for a live agent?

For operational agents, “training data” is broader than a model fine-tune dump. It includes anything that changes future behavior:

• Retained examples: past drafts, screenshots, or field values kept so the agent can mimic them later.
• Outcome labels: approve, edit, hold, escalate, stop — especially when those labels steer sampling rates.
• Preference snippets: tone notes, VIP exceptions, discount patterns copied from real tickets.
• Retrieval stores: vector indexes or “memory” folders that surface prior customer language into new runs.
• Shadow fine-tunes: overnight jobs that retrain or adapt prompts from production traffic without a version cutover.

If it can change tomorrow’s draft without a human noticing, it is training data — even when engineering calls it “cache,” “memory,” or “context packing.”

Why is silent retention dangerous?

Silent memory creates three failure modes owners feel before they can name them.

First, privacy and trust. A follow-up agent that keeps raw phone numbers, insurance IDs, or chart fragments “for quality” turns every run into a growing customer dossier. The audit trail may show the send. It will not show that the model later reused a VIP’s private note on a different account.

Second, policy drift without versioning. If retained examples quietly change how the agent phrases discounts, you get playbook drift without the cutover discipline from versioning. Review trays look fine in aggregate. VIP failures look random. Nobody can say which remembered scraps caused the new tone.

Third, contaminated learning. One bad approve, one sarcastic edit, or one edge-case VIP exception becomes durable “truth.” The agent starts teaching itself from outliers. Throughput can rise while trust falls — the opposite of healthy measurement in How Do You Measure Whether an AI Agent Is Working?.

What should stay ephemeral after every run?

Default to forget. After the audit events are written and the human review (if any) is complete, delete or seal:

• Raw screen scrapes and full CRM page dumps
• Customer PII: names in free text, phones, emails, addresses, account numbers
• Message bodies and chat transcripts used to draft replies
• Temporary tool outputs that are not required for the immutable audit record
• Redacted working copies once the reviewer finishes Approve / Edit / Hold

Ephemeral does not mean “no logs.” It means the learning path cannot re-read those bytes later. Pair this with write-back limits from What Should an AI Agent Be Allowed to Write Back?: if the agent may not permanently change a field, it also should not permanently memorize that field for training.

What may you retain on purpose?

Keep a short, boring allowlist that owners can read without a researcher:

• Playbook version id for every completed item (already required for reconstructability).
• Coarse outcome labels approved by a human: approve, edit, hold, escalate, stop — without the full customer payload.
• Aggregated counters for measurement: pause rate, edit rate, escalate rate, stop events — not free-text reasons that contain PII.
• Synthetic or owner-authored exemplars that never came from a live customer record.

If you need richer examples, create them the way you create a new playbook version: draft, approve, date, and pin. Do not harvest them from production chat because it felt convenient.

Who decides retention, and how do you prove it?

Name the same class of owner you already use for live agents in Who Should Own an AI Agent After It Goes Live?. Retention is an operating policy, not a model lab hobby. The owner approves the retain list, the delete-after-run jobs, and any exception that keeps a redacted snippet longer than the default window.

Prove it with controls that match the rest of the series:

• Review trays from How Should You Review Work From a Live AI Agent? should not offer “save this draft to memory” unless that path is on the allowlist.
• Escalate packs from How Should You Escalate Work From a Live AI Agent? should carry evidence for humans, then expire — not feed a permanent training bucket.
• Stop criteria from When Should You Stop a Live AI Agent? should include “unauthorized retention detected” as a freeze reason when memory jobs write outside the allowlist.

Governance themes in the NIST AI Risk Management Framework map cleanly here: map data flows, measure what is retained, manage deletion, and govern who can expand memory. A slide that says “we learn from production” is not a control.

How do you stand up memory policy this week?

Take the live first job from What Work Should an AI Agent Handle First?. Inventory every store the agent can write after a run: logs, object storage, vector indexes, prompt caches, spreadsheet “memory” tabs. Mark each store ephemeral, retain-allowlisted, or forbidden. Wire delete-after-run for ephemeral stores. Confirm the audit trail still reconstructs the story without needing the forgotten bytes.

Then refuse second-job expansion from When Is an AI Agent Ready for a Second Job? until the first job’s memory policy is written, owner-approved, and tested with one deliberate retain and one deliberate forget drill. Pause rules in When Should an AI Agent Pause for a Human? stay honest only when the agent cannot “remember” a VIP exception that a human never approved as lasting policy.

AI agent development work at Pro Logica scopes structured execution so retention is an explicit control, not a side effect of logging. For broader production patterns, see AI systems. If you need an engineer beside the team while those forget jobs and allowlists take shape, that is closer to forward-deployed AI engineering.

Watch the nine-trade demos on the AI agents page if you need the loop in motion. The trade changes. The retention rule does not: ephemeral by default, named retain classes only, hard blocks on PII and raw chat, no silent fine-tune from live tickets.

If you want help designing memory retention for a live agent — forget jobs, allowlists, and owner proofs — book a call. Bring one production store you cannot prove is deleted after review.

What should you read next if this issue sounds familiar?

If this topic matches what your team is dealing with, these pages are the best next step inside Prologica's site.

• What Audit Trail Should a Live AI Agent Leave? for immutable events that must not double as an open training dump.
• How Should You Version the Playbook for a Live AI Agent? for named rule ids when retention policy changes.
• Forward Deployed Engineer (FDE) Services for delivery context.