Reproducibility for solo analysts

Three months ago you delivered a crisp churn chart. Leadership loved it. Today they ask why April looks different in the new deck. You open six notebooks named final, final2, and final_USE_THIS. One depends on a CSV that lived on a laptop that was wiped. Another used a warehouse table that was rebuilt. You are not careless. You were solo, moving fast, and nobody asked for a re-run kit until the number became political.

This post is a one-shot guide to reproducibility for solo analysts. Not a lab science manifesto. Not a demand that you build a platform team in a backpack. A practical set of layers, a minimum kit, and a worked example you can adopt in a single afternoon. No series membership required.

What you’ll learn

• What reproducibility means when you are the whole analytics team
• Four layers: data snapshot, environment, code path, and narrative parameters
• A solo kit (folders, pins, seeds, README) that fits real job pressure
• Notebook vs script tradeoffs without religious wars
• Common failure modes and a 45-minute retrofit for an old analysis

Reproducibility in solo language

Academic definitions talk about bit-for-bit recreation by strangers. Solo analyst reality is closer to this:

Working definition: Future-you (or a peer) can re-create the same decision-relevant numbers from documented inputs and steps within a reasonable time, and can explain any intentional differences.

You may not freeze the entire warehouse. You should freeze the inputs that define a published claim. You may not containerize everything. You should pin the language and library versions that change results. You may still use notebooks. You should not rely on hidden cell order and unreplayed widgets.

This sits next to quality and pipeline habits on this site. Quality asks “is the number trustworthy?” Reproducibility asks “can we get back to that number on purpose?” See the data quality series and data pipelines series when you outgrow pure solo workflows.

Four layers that actually matter

Layer 1: Data inputs

If the inputs move silently, nothing else saves you. Solo-friendly options:

• Export a dated extract (CSV/Parquet) for the exact analysis window
• Save SQL that materializes a snapshot table with a date suffix
• Record query time, filter set, and row counts in inputs.md
• Hash large files when practical so you know the file did not change

Live warehouse queries are fine for exploration. Published claims need a pinned input story.

Layer 2: Environment

Python analyses rot when pandas upgrades change defaults. SQL is more stable but still depends on engine version and session settings.

• Use a virtual environment per project (or per year if you must simplify)
• Pin versions in requirements.txt or environment.yml
• Note the warehouse dialect and any non-default session parameters
• Record OS only when it matters (paths, Excel drivers, locale)

Layer 3: Code path

There should be one obvious way to re-run. Hidden steps are debt.

• Prefer a script or a notebook that runs top-to-bottom without manual clicks
• Parameterize dates and file paths at the top
• Avoid absolute paths that only work on your machine
• Keep “manual Excel fix” notes if you truly cannot avoid them (then try to eliminate them next time)

Layer 4: Narrative parameters

The memo is part of the analysis. Definitions, thresholds, and exclusions must live with the code, not only in a slide that got overwritten.

• definition.md: grain, inclusions, exclusions
• Thresholds as named parameters (CHURN_GAP_DAYS = 28)
• A short “how to re-run” section in the README
• Output folder with dated exports of tables and charts used in the deck

The solo reproducibility kit

You do not need enterprise metadata platforms on day one. You need a boring folder and a few non-negotiables.

Suggested layout:

project-churn-2026q1/
  README.md
  definition.md
  inputs.md
  requirements.txt
  params.yaml
  sql/
    01_snapshot_customers.sql
    02_churn_labeled.sql
  src/
    run_analysis.py
  data/
    raw/           # gitignored if sensitive
    snapshot/      # dated extracts you used
  outputs/
    2026-04-02/
      churn_by_segment.csv
      chart_churn.png
      memo.md

And a tiny params.yaml so magic numbers are not buried:

as_of_date: "2026-03-31"
lookback_days: 90
churn_gap_days: 28
min_tenure_days: 14
segments:
  - smb
  - midmarket
  - enterprise

Sensitive data stays out of public git. The structure still works on a private repo or a shared drive with access control. Stewardship and access habits matter here; the data stewardship series covers role clarity when more people join later.

Notebooks without regret

Notebooks are excellent for thinking. They are dangerous as the only source of truth.

If AI helped write notebook code, keep the same verification discipline you would use for SQL. The model does not remember your warehouse quirks next quarter. You need the pins and checks. Pair with how to check AI-written SQL when queries are involved, and Python series practice when scripting becomes the main path.

Worked example: re-running Q1 churn

Sam is a solo analyst at a 40-person SaaS company. In April, Sam published “Q1 churn by segment.” In July, the CEO asks for the same definition applied to Q2, plus an explanation of a Q1 slide that Finance disputes.

Without a kit, Sam is stuck. With a kit, Sam opens project-churn-2026q1/README.md:

# Q1 churn by segment

## Claim
Monthly logo churn for customers with tenure >= 14 days,
using a 28-day inactivity gap after last paid activity.

## Re-run
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python src/run_analysis.py --params params.yaml

## Inputs
See inputs.md. Snapshot tables:
- analytics.snap_customers_20260331
- analytics.snap_activity_20260331

## Outputs
outputs/2026-04-02/ used in the April 3 exec deck.

Core labeling logic in SQL (simplified):

WITH base AS (
  SELECT
    c.customer_id,
    c.segment,
    c.first_paid_date,
    c.last_paid_activity_date
  FROM analytics.snap_customers_20260331 c
  WHERE c.first_paid_date <= DATE '2026-03-31' - INTERVAL '14 days'
),
labeled AS (
  SELECT
    customer_id,
    segment,
    CASE
      WHEN last_paid_activity_date
        < DATE '2026-03-31' - INTERVAL '28 days'
      THEN 1 ELSE 0
    END AS churned_flag
  FROM base
)
SELECT
  segment,
  COUNT(*) AS customers,
  SUM(churned_flag) AS churned,
  SUM(churned_flag) * 1.0 / COUNT(*) AS logo_churn_rate
FROM labeled
GROUP BY 1
ORDER BY 1;

Python runner sketch that keeps parameters outside the SQL string soup:

from pathlib import Path
import yaml
import pandas as pd

PARAMS = yaml.safe_load(Path("params.yaml").read_text())
OUT = Path("outputs") / PARAMS["as_of_date"]
OUT.mkdir(parents=True, exist_ok=True)

# In real life: read from warehouse snapshot or local parquet
df = pd.read_parquet("data/snapshot/customers.parquet")

as_of = pd.Timestamp(PARAMS["as_of_date"])
min_start = as_of - pd.Timedelta(days=PARAMS["min_tenure_days"])
gap = pd.Timedelta(days=PARAMS["churn_gap_days"])

eligible = df[df["first_paid_date"] <= min_start].copy()
eligible["churned_flag"] = (
    eligible["last_paid_activity_date"] < (as_of - gap)
).astype(int)

summary = (
    eligible.groupby("segment", as_index=False)
    .agg(customers=("customer_id", "count"), churned=("churned_flag", "sum"))
)
summary["logo_churn_rate"] = summary["churned"] / summary["customers"]
summary.to_csv(OUT / "churn_by_segment.csv", index=False)
print(summary)

When Finance disputes Q1, Sam does not argue from memory. Sam re-runs, compares to outputs/2026-04-02/churn_by_segment.csv, and checks whether Finance used a different gap (30 days) or included sub-14-day customers. That is reproducibility paying rent.

For Q2, Sam copies the project folder, updates params.yaml and snapshot names, and keeps definitions stable unless a deliberate change is logged. Same method, new window, honest comparison.

How much freeze is enough?

Not every exploratory chart needs a museum archive. Use a simple tiering rule:

If you are unsure, upgrade one tier. The cost of a folder is tiny compared with reconstructing a disputed number under time pressure.

Lightweight automation for one human

Solo does not mean manual forever.

• A shell alias or Make target: make churn runs the pipeline
• Scheduled snapshot SQL for metrics you know will return
• Simple assertions after runs (row counts, rate bounds)
• Changelog file when definitions change

Example assertion block:

assert summary["logo_churn_rate"].between(0, 1).all()
assert summary["customers"].sum() > 0
assert set(summary["segment"]) <= set(PARAMS["segments"])

These checks will not catch every conceptual error. They will catch the embarrassing ones: empty frames, rates above 100%, surprise segment labels.

Common mistakes

• Only saving the chart image: images are not re-runnable methods.
• Live queries as the record of truth: tables evolve under you.
• Unpinned packages: the same script, different month, different defaults.
• Magic numbers in deep functions: thresholds nobody can find later.
• Personal absolute paths: /Users/you/Desktop/... is not a process.
• Notebook archaeology: 80 cells, 12 unused, 1 critical filter midway.
• No definition file: code without the business sentence still fails executive review.
• Overbuilding: three weeks of platform work for a one-off chart. Match tier to stakes.

Practice: 45-minute retrofit

Pick one analysis from the last quarter that might come back.

1. Create the folder skeleton above.
2. Write definition.md in ten lines or fewer.
3. Move or re-export the input you actually used; note row counts in inputs.md.
4. Pin packages you remember; if you cannot, pin what you use today and note the uncertainty.
5. Export the tables and charts that appeared in the deck into outputs/<date>/.
6. Add a README re-run section even if the first re-run is still partly manual.

Imperfect retrofit beats perfect intentions. Next project starts clean. For broader skill building, browse Learn.

Quick recap

• Solo reproducibility means future-you can re-create decision numbers on purpose.
• Four layers: data inputs, environment, code path, narrative parameters.
• A boring project kit beats hero memory.
• Notebooks are fine with restart-run-all, parameters, and exported outputs.
• Tier effort to stakes. Board claims need more freeze than throwaway EDA.
• Retrofit one old analysis this week so the next dispute is calmer.

Sources

• The Turing Way, Guide for Reproducible Research: https://the-turing-way.netlify.app/reproducible-research/reproducible-research.html
• Sandve et al., Ten Simple Rules for Reproducible Computational Research (PLOS Computational Biology): https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285
• Python Packaging User Guide, installing packages and environment practices: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/
• pandas documentation (IO and version awareness for analyses): https://pandas.pydata.org/docs/
• Software Carpentry, reproducible research lessons and good practices: https://software-carpentry.org/lessons/
• Git documentation (versioning analysis code and configs): https://git-scm.com/doc

Merch on Etsy

Open store →

You're On Mute White Ceramic Coffee Mug, Data Science Mug, Data Mug, Analytics Mug, Statistics Mug, Programming Mug, Gift Mug, Coffee Mug View on Etsy → Imbalanced Classes Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Imbalanced Mouse Pad View on Etsy → Naive Bayes Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Bayesian Mouse Pad View on Etsy → Boxplot Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Box Plot, Whisker Plot View on Etsy →