Retention, deletion, and we might need it someday

Someone asks for a five-year pull of every support ticket “just in case.” Someone else refuses to delete a sandbox dump of customer emails because “we might need it someday.” The warehouse bill grows. The risk grows faster. Nobody wrote a villain memo. The default was keep forever, and forever is expensive in money, attention, and breach surface.

This is Part 5 of Data stewardship at work. Parts 1 through 4 covered roles, truth maps, access, and incidents. This part is retention and deletion as weekly habits, not a 40-page policy rewrite. It is practical guidance for analysts and domain stewards. It is not legal advice. Your counsel, privacy officer, or compliance lead owns formal obligations for your company and jurisdictions. What you can own is reducing accidental hoarding and making keep-or-delete decisions legible.

Link the architecture posts when you need program language: Data governance, Master data management, and when sharing constrained datasets across parties, Data clean rooms. Quality and pipelines still matter because retention without lineage is guesswork: Data quality, How data actually moves. Paths: Learn / Tutorials.

What you’ll learn

• Why “we might need it someday” fails as a policy
• Hot, warm, cold, and delete-or-anonymize as a simple spectrum
• Five questions that force a keep / archive / delete decision
• How deletion, anonymization, and aggregation differ in practice
• SQL sketches for inventory and aged rows (examples, not production gospel)
• Weekly steward habits that shrink hoards without a big-bang project
• Common mistakes and a practice checklist

Retention is a decision, not a hard drive setting

Retention means how long you keep a class of data, in what form, for what purpose, and what happens when the clock runs out. Storage platforms will happily keep rows forever if you let them. Forever is not neutral. It increases cost, slows tables, widens who can accidentally export sensitive history, and makes deletion requests harder because you cannot find what you kept.

Many privacy regimes expect storage limitation: keep personal data no longer than needed for the purposes you stated. The EU’s GDPR frames this as a principle (storage limitation) and pairs it with rights such as erasure in defined cases. Other laws and contracts add industry-specific clocks (financial records, health, employment). Your company may already have schedules. Your job as a steward is to notice when analytics copies quietly ignore those schedules.

Rule of thumb: If nobody can name the purpose and the end date for a dataset, you do not have retention. You have a landfill with SQL access.

The retention spectrum: hot, warm, cold, then decide

Think in temperature, not only in “prod versus archive folder.” Temperature is about access speed, cost, and who still needs row-level detail.

Hot is not “better.” Hot is “expensive and sharp.” Cold is not “safe forever.” Cold still holds risk if it is full personal data with weak access control. The spectrum’s job is to force a move toward delete or anonymize when warm has no remaining purpose.

Delete, anonymize, aggregate: three different endings

People say “anonymize” when they mean “hide the email column in a spreadsheet.” Be precise.

• Delete: rows or fields are removed from systems you control (and eventually from backups per policy). Use when you no longer need the record and law or policy allows or requires disposal.
• Anonymize / de-identify: transform so individuals are not reasonably identifiable, alone or in combination with other data you hold. Harder than dropping a name. Quasi-identifiers (ZIP + birth date + gender, rare attributes) can re-identify. If re-identification is easy, you did not anonymize; you masked poorly.
• Aggregate: keep counts, rates, and sums at a grain that does not point at a person (and watch small cells: five customers in a rural ZIP can still identify). Aggregates are often what leadership needed all along.

Pseudonymization (replace names with tokens you can reverse with a key) is useful for operations, but it is still personal data in most privacy frameworks because the key exists. Do not call tokenized CRM dumps “anonymous” in a slide deck.

Five questions: keep, archive, or delete

When someone asks to keep a table “just in case,” run this card. Write answers in one page, not a novel. If answers are missing, default toward shorter retention and fewer copies until Legal or the data owner sets a longer clock.

Example answers for a marketing experiment export:

• Purpose: measure uplift for campaign Q1; report to leadership once.
• Audience: growth analytics only; not sales ops.
• Form: need user-level for 30 days post-campaign; after that weekly aggregates are enough.
• Clock: user-level delete 90 days after campaign end; aggregates keep 2 years for year-over-year.
• Exit: scheduled job drops user-level partition; aggregates remain in a mart without identifiers.

That is retention as stewardship. No 200-page wiki required.

Legal holds and “delete everything” are different jobs

Sometimes counsel says: do not delete. That is a legal hold. It freezes disposal for specific matters or time ranges. Holds override your normal cleanup clock until released. If you get a hold notice, stop automated deletion for the covered systems, document what you froze, and do not freestyle “I’ll just delete the old stuff outside the hold” without instruction.

Conversely, when a customer or employee exercises a deletion or erasure right, or when policy says dispose, you need a path that finds copies: warehouse tables, BI extracts, shared drives, notebooks, vendor tools. Stewards who map copies (Part 2 energy) make those requests survivable. Stewards who only know one dashboard make those requests a scavenger hunt.

Worked example: the “temporary” support export

Scenario: Support asked Analytics for a one-time CSV of the last two years of tickets, including customer email, phone, and free-text body, to “look for themes.” File lands in a shared drive. Six months later it is still there, copied into two laptops and a vendor prototype.

Five-question card, filled honestly:

Inventory sketch (warehouse side). Toy names; adapt to your catalog:

-- Find aged support extracts staged for "temporary" analysis
-- Example only: replace schema/table names with your inventory

SELECT
  table_schema,
  table_name,
  created_at,
  last_queried_at,
  row_count,
  has_email_column,
  has_phone_column
FROM stewardship.dataset_inventory
WHERE domain = 'support'
  AND purpose_tag IN ('ad_hoc_export', 'sandbox', 'vendor_poc')
  AND created_at < CURRENT_DATE - INTERVAL '30' DAY
ORDER BY created_at;

-- Age profile of a hot table that should roll off
SELECT
  DATE_TRUNC('month', created_at) AS ticket_month,
  COUNT(*) AS tickets,
  COUNT(DISTINCT customer_email) AS distinct_emails
FROM support.tickets_raw
GROUP BY 1
ORDER BY 1;

-- Example deletion of a staged extract past its clock
-- Run only with approval, backups policy understood, and no legal hold
-- BEGIN;
-- DELETE FROM analytics_staging.support_export_2025_q4
-- WHERE export_batch_id = 'tmp-2025-11-02';
-- COMMIT;

For free-text themes, prefer aggregate outputs early:

-- Keep what leadership needs without retaining customer contact fields
CREATE TABLE analytics.support_theme_monthly AS
SELECT
  DATE_TRUNC('month', created_at) AS month,
  product_area,
  theme_tag,
  COUNT(*) AS ticket_count
FROM support.tickets_tagged
WHERE created_at >= DATE '2024-01-01'
GROUP BY 1, 2, 3;

Then delete the CSV, rotate drive links, and ask the vendor to confirm deletion of their copy. Confirmation in writing beats “they said they would.”

Weekly habits that beat a yearly purge panic

• Monday inventory slice: list new sandbox tables and shared exports created last week. Tag purpose or schedule delete.
• Column minimization default: when building extracts, start from the metric or question, not SELECT *. Personal fields need a reason.
• Replica tax: every extra copy needs an owner and a clock. If you cannot name them, do not create the copy.
• Dashboard vs dump: if the ask is a trend, ship a chart or aggregate table, not a PII-laden CSV.
• Quarterly cold review: for each domain table older than policy, confirm purpose still exists. Link to metrics specs so “we use it for the board” is checkable (Metrics that matter).
• Incident learnings: after a wrong-number incident (Part 4), ask whether stale copies confused the picture. Retention hygiene is also quality hygiene.

What good looks like in a small team

You do not need a full data lifecycle product on day one. You need:

• A one-page schedule for the top ten sensitive tables (purpose, hot/warm/cold windows, exit action).
• A named steward per domain who can answer “do we still need this?”
• A ticket type for deletion or anonymization so work is visible.
• Agreement with Security and Legal on who approves exceptions and holds.
• Pipeline jobs that enforce clocks where safe, with human review for holds and edge cases.

Master data and golden records (see MDM) still need retention rules. A perfect customer golden record that never expires can violate storage limitation just as loudly as a messy CSV.

Common mistakes

How to practice this week

1. Pick three datasets you touch often. Fill the five-question card for each. Share with the domain owner.
2. Find one “temporary” export in a drive or sandbox. Either attach a clock and owner or delete it with approval.
3. Replace one recurring PII CSV with an aggregate mart or dashboard tile.
4. Ask Legal/Security (politely) where the official retention schedule lives. Bookmark it. Part 6 is about making that conversation calm.
5. Add retention notes to your catalog cards (Part 2): purpose, clock, exit. Even three sentences beat silence.

Quick recap

• “We might need it someday” is not a purpose, clock, or exit plan.
• Move data along hot → warm → cold → delete or anonymize on purpose.
• Delete, anonymize, and aggregate are different endings. Use honest words.
• Five questions force keep / archive / delete decisions in plain language.
• Legal holds freeze disposal; normal hygiene still needs weekly attention.
• This is practice guidance, not legal advice. Partner with counsel for obligations.

Next in the series: Part 6 covers working with Legal and Security without panic, including PII basics for analysts. Keep governance and quality links nearby: Data governance, Data quality, Data pipelines.

Sources

Research and further reading used for this article:

• GDPR.eu overview of Article 5 principles (including storage limitation; orient to official text for your counsel)
• EUR-Lex: Regulation (EU) 2016/679 (GDPR) official text (storage limitation, purpose limitation, integrity and confidentiality, erasure provisions)
• NIST Privacy Framework (privacy risk management outcomes useful for retention and disposal discussions)
• NIST: De-identification (why removing a name is not always enough)
• IAPP: Glossary of privacy terms (shared vocabulary for personal data, anonymization, pseudonymization)
• Analytics Made Simple: Data governance
• Analytics Made Simple: Master data management

Merch on Etsy

Open store →

Imbalanced Classes Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Imbalanced Mouse Pad View on Etsy → Naive Bayes Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Bayesian Mouse Pad View on Etsy → You're On Mute White Ceramic Coffee Mug, Data Science Mug, Data Mug, Analytics Mug, Statistics Mug, Programming Mug, Gift Mug, Coffee Mug View on Etsy → Y Hat Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Y Pred Mouse Pad View on Etsy →