Data Scientist · Step 3 of 8 · 10 min
Exploring data before you trust it
Run a basic exploratory analysis and spot the problems that would mislead a model.
Look before you model
Exploratory data analysis, or EDA, means getting to know a dataset before you draw conclusions from it. How many rows? What does each column mean? What are typical values, and which values are strange? It is detective work, and it is where you find most of the problems that would quietly ruin a model later.
Write down what you expect before you look. Surprises are the interesting part, and you only notice them if you had an expectation.
Distributions and outliers
A distribution shows how values are spread. A histogram of house prices might show most homes in a middle range with a long tail of expensive ones. Averages hide this, so look at the median and percentiles too: df['price'].describe() gives them in one line.
Outliers are values far from the rest. Some are real, like a genuine mansion. Some are errors, like an age of 999. Do not delete them automatically. Find out which kind they are, and note your decision.
Missing data has a story
df.isna().sum() counts missing values per column. The count matters less than the reason. If income is missing mostly for younger customers, dropping those rows changes who your analysis is about. Filling gaps with the average can hide the same problem.
Sometimes “missing” is itself information. A blank “cancellation date” may simply mean the customer has not cancelled. Ask the people who collect the data before you decide.
Plot it, and watch for leakage
matplotlib is Python's basic plotting library, and seaborn builds on it for quick statistical charts. A few lines give you histograms, box plots and scatter plots. One chart often reveals what ten summary numbers hide.
While exploring, watch for leakage: a column that would not be known at the moment you need the prediction. A “refund issued” column in a dataset for predicting complaints is a classic. If one feature seems almost too predictive, ask when it gets filled in.
Try it
A ten-minute EDA on a public dataset
- 1.In Colab or a Kaggle notebook, load any small public dataset, such as the California housing file from the last step.
- 2.Run df.describe() and df.isna().sum().
- 3.Make one histogram and one scatter plot with seaborn: import seaborn as sns, then sns.histplot(df['median_house_value']).
- 4.Write three sentences: one surprise, one possible data error, one question for whoever owns the data.