Export results and hand off

You cleaned the data. You validated the grain. You even restarted the kernel and the numbers held. Then you emailed a screenshot of a DataFrame and wondered why Finance rebuilt the table by hand in a new sheet. Delivery is part of the analysis. If the handoff is sloppy, the work was unfinished.

This is Part 9 of Python for analytics. Part 8 built a light pipeline. Now you export results people and tools can trust: clean CSV, a nod to SQL tables and Parquet, plus the metadata notes that stop “what does one row mean?” threads at 5 p.m.

What you’ll learn

How to write CSV exports that survive Excel, BI tools, and re-import into pandas
When Parquet or a database table is a better destination than CSV
What to include in a handoff note for humans and dashboards
A destination-by-checklist table for real workplace exports
How export choices protect the reproducibility you earned in Part 8

Export is a product surface

Example:

Export destinations checklist

Your notebook is for you. Your export is for strangers with calendars. They will open it in tools you do not control. They will sort it, join it, and paste it into slides. The file must carry enough structure to survive that journey: stable headers, honest types, no decorative totals row, and a short explanation of grain and filters.

This is the same handoff discipline as the spreadsheets series on From spreadsheets to real data, just with pandas writing the file instead of “Save As.” Foundations still matter: if you cannot state the decision and the population, a prettier file will not help (Analytics foundations).

Think of three consumers: a human skimming in Sheets, a BI tool refreshing a dashboard, and a future pipeline reading your output as the next input. Good exports serve all three without three different “special” files when one well-described table will do.

CSV: still the universal adapter

CSV is imperfect and everywhere. Use it when partners need a file they can open without installing your stack. Write it deliberately.

import json
from datetime import datetime, timezone
from pathlib import Path

import pandas as pd

# Assume clean_df came from your Part 8 pipeline
clean_df = pd.DataFrame(
    {
        "order_id": [101, 102, 103],
        "customer_id": ["1", "2", "2"],
        "order_date": pd.to_datetime(["2024-06-01", "2024-06-15", "2024-06-20"]),
        "amount": [40.0, 12.5, 30.0],
        "region": ["East", pd.NA, "West"],
    }
)

out_dir = Path("outputs")
out_dir.mkdir(exist_ok=True)

csv_path = out_dir / "orders_clean_2024-06-30.csv"

# Good defaults for analytics handoffs
clean_df.to_csv(
    csv_path,
    index=False,           # avoid a mystery index column
    encoding="utf-8",      # play nice across locales
    date_format="%Y-%m-%d",
    na_rep="",             # or "NULL" if your warehouse load prefers a token
)

print("wrote", csv_path, "rows", len(clean_df))

Example output:

Example output: export + meta handoff

Why these choices:

index=False prevents an unnamed first column that breaks the next person’s header assumptions.
UTF-8 reduces “mojibake” when names include accents or non-Latin scripts.
ISO dates (YYYY-MM-DD) survive Excel and SQL loaders better than locale-specific formats.
No totals row in the data file. Totals belong in a separate summary table or in the BI tool.

If a partner lives entirely in Excel and needs a double-click experience, you can also write .xlsx with to_excel. Still keep a CSV (or Parquet) as the canonical machine-readable artifact when possible. Excel formatting is a presentation layer. It is a weak system of record.

Metadata beside the file

A clean CSV without context becomes a rumor. Ship a tiny sidecar note. JSON is easy to generate; Markdown is easy to read. Pick one and be consistent.

meta = {
    "dataset": "orders_clean",
    "as_of": "2024-06-30",
    "generated_at_utc": datetime.now(timezone.utc).isoformat(),
    "grain": "one row per order",
    "filters": [
        "dropped rows missing order_id, customer_id, order_date, or amount",
        "excluded spreadsheet total rows",
    ],
    "columns": {
        "order_id": "unique order identifier (string-safe)",
        "customer_id": "customer key as text to preserve leading zeros",
        "order_date": "order date, UTC calendar day, ISO format in CSV",
        "amount": "order amount in USD, null if unknown (not zero-filled)",
        "region": "sales region; null means unknown (not 'Other')",
    },
    "row_count": int(len(clean_df)),
    "source_pipeline": "run_pipeline() in analyze_orders.py",
    "contact": "analytics-team@example.com",
}

meta_path = out_dir / "orders_clean_2024-06-30.meta.json"
meta_path.write_text(json.dumps(meta, indent=2), encoding="utf-8")
print("wrote", meta_path)

That file answers the questions dashboards cannot: grain, exclusions, as-of, and who to ping. Paste a shortened version into the PR, ticket, or Slack thread when you deliver. Future-you is also a stakeholder.

Parquet in plain language

Parquet is a columnar binary format popular in analytics engineering. It stores dtypes more faithfully than CSV, compresses well, and plays nicely with tools like DuckDB, Spark, and many warehouses’ external tables. For intermediate outputs inside a data team, Parquet often beats CSV.

# Requires pyarrow or fastparquet installed in the environment
parquet_path = out_dir / "orders_clean_2024-06-30.parquet"
clean_df.to_parquet(parquet_path, index=False)
print("wrote", parquet_path)

# Round-trip check
back = pd.read_parquet(parquet_path)
print(back.dtypes)

If your consumer is non-technical and only has Excel, CSV still wins for the last mile. Many teams keep Parquet for machines and CSV for people. That is not waste if each file has a job.

to_sql conceptually

pandas can write to databases through SQLAlchemy (or similar) with DataFrame.to_sql. Conceptually you are doing three things: choose a table name, decide whether to replace or append, and ensure dtypes map to database types without mangling keys.

# Conceptual pattern (needs a real SQLAlchemy engine and network access)
# from sqlalchemy import create_engine
# engine = create_engine("postgresql+psycopg2://user:pass@host:5432/analytics")
#
# clean_df.to_sql(
#     name="orders_clean",
#     con=engine,
#     if_exists="replace",  # or "append" for incremental loads
#     index=False,
#     method="multi",
#     chunksize=1000,
# )

print("Prefer staging tables + warehouse tests over silent replace in production.")

In production analytics, raw to_sql(..., if_exists="replace") on a shared table is a sharp edge. Prefer writing to a staging table, running row-count and null checks, then swapping, or use your team’s existing load tool. The pandas call is fine for personal sandboxes and prototypes. Shared warehouses deserve contracts, not surprise drops.

If SQL is already your comfort zone, push heavy filtering upstream (Part 10) and treat Python exports as curated products, not as a second warehouse without governance. The SQL series covers the query side of that partnership.

Handoff notes for BI

Example:

Handoff pack contents

Dashboard tools (Looker, Power BI, Tableau, Metabase, and friends) reward boring tables:

One header row, snake_case or stable labels, no merged cells.
One grain per table. Do not mix daily and monthly rows in one file.
Dimensions and facts clear enough that a calculated field does not reinvent the metric.
Nulls are nulls. Do not encode missing as zero unless the metric definition says so.
Publish an as-of and refresh expectation (“daily by 9:00 local” or “manual after pipeline run”).

If you deliver both a detail table and a summary table, name them so nobody unions them by accident. orders_clean and orders_by_region is clearer than final and final2.

Destination × what to include

Destination	Primary artifact	Must include	Usually skip
Email to a human	CSV or XLSX + short note	Grain, as-of, filters, owner	Full code dump
Shared Drive folder	Versioned CSV/Parquet + meta.json	Stable filename pattern, row count	Ad-hoc “final_final” names
BI import	Tidy table, one grain	Clean headers, typed dates, no totals rows	Pivot-shaped presentation tables
Warehouse table	Staging load + tests	Schema, uniqueness, null rules	Silent replace on prod tables
Next Python job	Parquet preferred	Preserved dtypes, partition if large	Screenshots
Slide deck	Chart + one sentence grain	Caveats that change the decision	Raw dumps as tiny unreadable tables

Reproducibility note that travels with the file

A one-paragraph README is enough for many teams:

Example handoff blurb: orders_clean_2024-06-30.csv is one row per order as of 2024-06-30. Built by run_pipeline() from data/orders_raw.csv. Dropped incomplete rows and spreadsheet totals. Amounts are USD; null amount means unknown, not zero. Region null means unknown. Contact analytics-team for regeneration.

If legal or security constraints apply (customer emails, health data), state the redaction rules in the same note. Handoff is also a privacy moment, not only a formatting moment.

Filename and version habits that save weekends

Pick a pattern and refuse to improvise under deadline pressure. A simple scheme works:

# dataset_asof_status.ext
# orders_clean_2024-06-30.csv
# orders_clean_2024-06-30.meta.json
# orders_by_region_2024-06-30.csv

Include the as-of date in the name, not only in a cell inside the file. People forward files without opening them. Avoid final, latest, and USE_THIS in the basename. If you must overwrite a “latest” pointer for a BI tool that cannot handle dates in paths, keep the dated file as the archive and copy or symlink to orders_clean_latest.csv in a controlled step your pipeline owns.

When legal holds or audits matter, store the generating commit hash or script version in the metadata file. You do not need a full data catalog on day one. You need enough breadcrumb that next quarter’s you can regenerate or defend the number.

Common mistakes

Leaving the index in the CSV so every reload invents a column.
Writing floats for IDs and destroying leading zeros or precision.
Locale-dependent dates that swap month and day across regions.
Embedding filters only in a chart title instead of in data and metadata.
Overwriting the same filename with no as-of, so nobody can reconstruct last week.
Shipping a pivoted “report shape” as the only artifact, which is painful to join later.

Practice and next step

Take the clean frame from your Part 8 practice. Export CSV with the options above, write a meta.json, and send both to a teammate (or to yourself on another machine). Ask them to state the grain without asking you. If they cannot, the handoff failed even if the bytes are perfect.

Part 10 closes the series by putting Python and SQL in the same workflow: when each wins, how to push filters down, and how to review generated code with adult skepticism. Keep exploring paths on the Learn hub.

Quick recap

Exports are products: stable headers, honest nulls, ISO dates, no decorative totals.
CSV for universal human access; Parquet for dtype-friendly machine handoffs; SQL loads with care.
Ship metadata: grain, as-of, filters, row count, owner.
Match artifact shape to destination (BI wants tidy; slides want a chart plus caveats).
Version filenames and keep the pipeline that regenerates them.

Sources

pandas DataFrame.to_csv: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html
pandas DataFrame.to_parquet: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_parquet.html
pandas DataFrame.to_sql: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_sql.html
Apache Parquet overview: https://parquet.apache.org/docs/overview/
Python json module: https://docs.python.org/3/library/json.html
Analytics Made Simple, Learn hub: https://analyticsmadesimple.com/learn/