Pandas DataFrames explained: rows, columns, and column types

A DataFrame is just a table: rows that each mean something, columns with names, and a type for each column that decides what math is allowed. Once you see it that way, pandas, the Python tool that holds these tables, becomes much easier to learn, because you already know the shape from spreadsheets. Imagine opening a sales export in Google Sheets: one row per order, a column for the date, a column for the amount. A DataFrame is that same sheet, held in Python.

The previous post in this series got a working Python setup onto your computer. This one treats DataFrames as tables you can inspect with confidence. If you are not yet used to asking what one row stands for, the tutorials on moving from spreadsheets to real data and on framing the problem will help. If you know SQL, the standard language for asking a database questions, your instincts about its tables, taught in the SQL tutorials, transfer almost one-to-one.

What a DataFrame is

Here is what a small DataFrame looks like as a table:

A DataFrame is a two-dimensional table with labeled columns. Each column is a Series, which is a single named list of values that ideally share one type. One row is one observation at a level of detail you choose, such as one order, one customer on one day, or one support ticket. That is the same mental model as a well-built range in a sheet or a SQL table, and pandas adds the ability to program it: you store the table in a variable, transform it, and keep the steps in code instead of in a fragile history of mouse clicks.

You will also meet the index, which is the set of labels for the rows. By default, reading a CSV (a plain-text table with commas between values) (a plain-text table with commas between values) gives you 0, 1, 2, and so on, which is fine. Later you might set an order id as the index. For learning filters and totals, it is enough to treat the index as row labels sitting quietly in the background, so do not let it stop you from loading a file.

Spreadsheet idea to DataFrame idea

If you think in grids first, this translation table shows what each spreadsheet idea is called in pandas:

Notice what is missing from the table: merged cells, colors used as data, and “notes in column Z.” Those sheet habits do not translate, while tidy rectangular data translates cleanly. If your export has two header rows and a title in cell A1, clean that up in the source export or tell pandas which rows to skip. A later post in the series builds a full pipeline (a set of steps you run in order to clean or move data) for messy files, and this post assumes a simple header.

Load a CSV the way professionals start

Most analytics sessions in Python begin by reading a file. Use the sample file from the setup post or any tidy CSV of your own.

import pandas as pd

orders = pd.read_csv("hello_orders.csv")

print(orders.head())
print("shape:", orders.shape)
print("columns:", orders.columns.tolist())

Example output:

The call head() shows the first five rows by default, and you can pass a number for more or fewer, as in head(10). The shape attribute returns the pair (rows, columns), which is your first sanity check. If you expected 5,000 orders and shape says 50, something went wrong upstream, long before any pivot or total.

Look at the column names early. Spaces, odd capital letters, and units tucked into the header, such as “Revenue (USD)”, become annoying later. Renaming is cheap, but living with bad names for twenty steps is expensive.

Data types: the quiet rulebook

A dtype (short for data type) describes what kind of value a column holds, such as a whole number, a decimal, text, true or false, or a date. Types decide whether sum makes sense, whether a join key matches, and whether “2024-01-01” is treated as a date or as a pile of text.

print(orders.dtypes)
print(orders.info())

The call info() works like a quick medical chart. It lists the column names, how many values are filled in, the types, and the memory used. If a revenue column shows up as object, which usually means text, because of a stray dollar sign or comma, the sums will misbehave. A later post goes deep on converting types and handling missing values, so for now just train your eyes to glance at the types after every load, before you trust a total.

These are the common first-day mappings between the kind of information and its type:

• Counts and ids that never need decimals are whole numbers, when the file is clean
• Money and rates are decimals, with the usual rounding caveats, and finance systems may use exact decimals later
• Categories such as region are text, which pandas may label object or string
• Dates should ideally be real date values and not free text

If SQL is your home language, dtypes are cousins of the column types you declare in CREATE TABLE. The names differ, but the discipline is the same, because types are part of the contract and not an afterthought.

Rows set the level of detail; columns describe it

Before you reach for fancy methods, write one sentence: one row means _____. For hello_orders.csv, one row means one order. If you later group the data into region totals, one row will mean one region. The shape changes with the level of detail, which is normal, and confusion only starts when you mix levels without noticing.

Columns are attributes of that row: identifiers, dimensions such as region, and facts such as revenue. This is the same vocabulary people use when designing a data warehouse (a central database built for reporting) (a central database built for reporting), only lighter. If a column does not describe what one row stands for, you may have a layout problem, such as a wide sheet that should be tall, or leftovers from a join. Spotting that early is good analytics practice and not pedantry.

Rename columns without drama

Readable names make every later line of code cheaper to write and to read. Prefer snake_case, which means lowercase words joined by underscores, because it plays well with code: order_id, region, revenue.

orders = orders.rename(
    columns={
        "order_id": "order_id",  # already good; shown for pattern
        "region": "region",
        "revenue": "revenue_usd",
    }
)

# Or rename many messy headers at once after a bad export:
# orders.columns = ["order_id", "region", "revenue_usd"]

print(orders.head())
print(orders.dtypes)

Example:

Example output:

There are two styles. A dictionary passed to rename fixes only a few labels, while assigning a full list to columns replaces every name at once when the export order is fixed and the names are hopeless. The dictionary is safer when columns might be reordered, and the full list is faster when you control the file layout.

Assign the result back to orders, or use inplace with care. Beginners often call rename and wonder why nothing changed, because they forgot to keep the returned DataFrame. Many pandas methods return a new object instead of editing the old one, which is a deliberate feature that keeps pipelines safe.

Worked example: inspect, then tidy names

Imagine an export that is slightly messier but still rectangular:

Order ID,Region Name,Revenue USD
1,East,4200
2,West,8100
3,East,6900
4,South,1500
5,West,3200

This is a full inspection pattern you can reuse at work:

import pandas as pd

path = "hello_orders_messy_headers.csv"
orders = pd.read_csv(path)

print("=== head ===")
print(orders.head())

print("=== shape ===")
print(orders.shape)

print("=== columns ===")
print(orders.columns.tolist())

print("=== dtypes ===")
print(orders.dtypes)

print("=== info ===")
orders.info()

orders = orders.rename(
    columns={
        "Order ID": "order_id",
        "Region Name": "region",
        "Revenue USD": "revenue_usd",
    }
)

print("=== after rename ===")
print(orders.head())
print(orders.dtypes)

You should see five rows, three columns, whole-number revenue if the file is clean, and friendly snake_case names afterward. If Revenue USD was imported as text, info() will show it. Do not average a text column and then blame pandas. Fix the type, which a later post covers, or fix the export.

Here is the small result table after the rename:

Series versus DataFrame, without the confusion

New learners lose time wondering why a method has disappeared. Very often they are holding a Series when they thought they were holding a DataFrame.

col = orders["revenue_usd"]          # Series
tab = orders[["revenue_usd"]]        # DataFrame with one column
print(type(col), type(tab))
print(col.mean())
print(tab.mean())  # still works; result shape differs

A Series is closer to the result of selecting one SQL column, and a DataFrame is closer to a whole table. Many operations exist on both, but the selection rules differ, and some table helpers expect two dimensions. When an error message mentions Series, check whether you used single brackets by accident.

You do not need to memorize how pandas organizes its object types. You need a reflex: when you are confused, print type(...), then shape if it exists, then head. Those three prints solve more early bugs than rereading a chapter.

The index, lightly

Print orders.index and you will see a RangeIndex starting at 0, which is normal. Some tutorials immediately set a meaningful index, but you can wait, because filtering with clear column conditions, the topic of the next post, is easier for SQL-minded people to read than clever index tricks.

There are a few cases where you might care about the index sooner:

• Data over time that is labeled by date
• Joins that accidentally repeat index labels
• Combining two Series, where pandas lines values up by their labels

For now, remember that columns are your main handles and the index is only the row labels. If a method returns something that looks like a table but behaves oddly, check whether you are holding a Series, a DataFrame, or a GroupBy object, which is the half-finished result of grouping that a later post explains.

Peek at values without drowning

Beyond head, a few cheap peeks help you learn how a table behaves before you transform it.

print(orders.tail(3))          # last rows; useful for time-sorted files
print(orders.sample(3, random_state=1))  # random peek; set seed for reuse
print(orders["region"].value_counts())   # category frequencies
print(orders["revenue_usd"].describe())      # numeric summary

The call value_counts answers the quick question of what is in a column. If you expected three regions and see twelve spellings of East, you have found a cleaning job early. Running describe on a numeric column gives the count, average, smallest and largest values, and quartiles. It is not a full statistical report but a flashlight.

When a column should be numeric but describe fails or shows no real statistics, check dtypes again. Text that looks like numbers is still text until you convert it, a topic that returns in a later post. For now, the skill is simply noticing the mismatch without panic.

One more habit is to compare len(orders) with orders.shape[0], which should match. If you ever filter and assign wrongly, these checks catch an empty table before you email a blank summary. An empty table is a valid intermediate result, but an empty table nobody noticed is how weekends get ruined.

How this connects to SQL and Sheets day to day

In Sheets, you stare at the grid. In SQL, you run SELECT * with a LIMIT. In pandas, head() plays the role of that limit, and info() is a quick glance much like SQL’s DESCRIBE. They are not identical, but they answer the same question of what you are holding.

The professional habit across all three tools is to profile before you transform. Count the rows, list the columns, check the types, and spot the missing values, and only then filter, join, or total anything. People who skip profiling ship confident dashboards that are wrong. People who profile look slower for five minutes and faster for the rest of the week.

Common mistakes

• Assuming the CSV is tidy because it opens in Excel. A file with two header rows and blank spacer columns still opens, so opening proves nothing.
• Never checking shape. You filter wrongly and then analyze 12 rows while thinking you have 12,000.
• Ignoring the types until a sum looks insane. Make dtypes a reflex right after read_csv.
• Leaving spaces in column names. A name like df["Revenue USD"] works, but it is annoying to type, so rename early.
• Thinking the index is the data itself. Your keys should usually be real columns until you have a reason to do otherwise.
• Calling rename and throwing away the result. Keep the returned table, because the original is not changed.
• Mixing levels of detail in one table without labeling them. Order rows and region totals are different tables and should stay apart.

Rule of thumb: After every load, run head, shape, and dtypes (or info) before you calculate anything you would put in a slide.

Quick recap

• A DataFrame is a labeled table, and a Series is one column of it.
• Sheet headers and SQL columns map to columns, shape, and dtypes.
• Start each session with read_csv, then look at head, shape, and info or dtypes.
• Say the level of detail out loud by finishing the sentence “one row means…”
• Rename early, and keep the objects that methods return.
• Treat the index lightly until you truly need it.

Practice and next step

Take any small tidy CSV you own, or the hello file, and write a short table card in comments or in a note that answers these questions:

• What one row means
• Which column could act as a unique key, if any
• The column list and the type of each column
• Anything surprising in the first rows shown by head()

Then rename at least one column to a clearer name, and print head() again to confirm it worked.

The next post covers selecting, filtering, and sorting. It maps SQL SELECT, WHERE, and ORDER BY onto clear pandas patterns without cleverness for its own sake.

Sources

• pandas, “Intro to data structures”: https://pandas.pydata.org/docs/user_guide/dsintro.html
• pandas, “10 minutes to pandas”: https://pandas.pydata.org/docs/user_guide/10min.html
• pandas read_csv: https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html
• pandas DataFrame.rename: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.rename.html
• Analytics Made Simple, Learn: https://analyticsmadesimple.com/learn/
• Analytics Made Simple, Analytics foundations: https://analyticsmadesimple.com/series/analytics-foundations/
• Analytics Made Simple, From spreadsheets to real data: https://analyticsmadesimple.com/series/spreadsheets-to-data/
• Analytics Made Simple, SQL series: https://analyticsmadesimple.com/series/sql/

Merch on Etsy

Open store →

You're On Mute Coffee Mug – Data Science Gift View on Etsy → Standard Deviation T-Shirt View on Etsy → Mean Squared Error Shirt View on Etsy → Standard Deviation White Ceramic Mug View on Etsy →