◢◤ ARKANE_
SYS_TIME UTC+0
Work with me

Chapter 9 quiz · 25 questions

Exploratory Data Analysis & Visualisation

Tests whether you can read a distribution, pick the right chart, interpret correlations without over-reading them, and spot confounding and data snooping before they reach a model.

Based on chapter 9. Every answer comes with an explanation and a link back to the section.

/Q9

Quiz

Question 1 of 25

The EDA progression and mindset Level 1
Progress

The chapter's EDA progression ends with "What surprised me?". Why is that question asked last?

Choose one
/02

练习From the book

8 exercises

The chapter's own exercises are open questions. Try one, then compare with the model answer.

Exercise 9.1 [W]

List the five questions this chapter's EDA progression asks, in order, and explain in one sentence why "what surprises me?" is placed last rather than first.

Show the model answer

The five questions, in order: (1) what do I have (shape, types, nulls, duplicates); (2) what does each variable look like alone (distributions, outliers, skew); (3) how do variables relate to each other (correlations, conditional distributions); (4) how do variables relate to the target; (5) what surprises me. "What surprises me?" is placed last because it can only be answered after the first four questions have established a baseline of expectations. You can't recognise a surprise until you know what "normal" looks like for this specific dataset.

Exercise 9.2 [W]

A dataset's age column has a mean of 29.7 and a median of 28.0. What does the gap between them suggest about the distribution's shape, and which single chart would confirm it?

Show the model answer

A mean of 29.7 against a median of 28.0 means the mean is pulled upward relative to the median. That indicates a right-skewed (positive skew) distribution: a small number of unusually old passengers/customers/etc. stretching the upper tail. A histogram with a KDE overlay (Listing 9.4) would confirm the shape directly and show whether the skew is mild or extreme.

Exercise 9.3 [A]

Using Listing 9.2 as a starting point, run it against a dataset of your choosing and report which columns, if any, have more than 10% missing values.

Show the model answer

Answer depends on the dataset chosen. The deliverable is a quick_profile() run and a reported list of columns exceeding 10% missing, following the exact output format of Listing 9.2. For example, on the Titanic dataset used in this chapter, only deck (77.2%) and age (19.9%) exceed that threshold.

Exercise 9.4 [A]

Using Listing 9.8's grouping pattern, compute the Titanic survival rate broken down by embarked instead of sex. State whether the spread across categories looks large enough to be a useful feature.

Show the model answer

df.groupby('embarked')['survived'].mean() on the Titanic dataset returns roughly 55% survival for passengers who embarked at Cherbourg (C), 39% at Queenstown (Q), and 34% at Southampton (S). The spread (~20 points between C and S) is large enough to be worth investigating as a feature. It is confounded with class, though: Cherbourg passengers were disproportionately 1st class, so some of this apparent effect is the class/fare effect from Listing 9.8 showing through a correlated variable.

Exercise 9.5 [A]

Reproduce Listing 9.5's "near-redundant pair" check on the Titanic numeric features (survived, pclass, age, sibsp, parch, fare) and confirm whether any pair exceeds the 0.95 threshold.

Show the model answer

Running Listing 9.5's near-redundant-pair check on survived, pclass, age, sibsp, parch, fare finds no pair above the 0.95 threshold. The strongest pairwise correlation among the features (excluding the target) is pclass/fare at roughly −0.55-0.55, well below the redundancy cutoff. This dataset has essentially no redundant numeric feature pairs, so no columns need to be dropped purely on multicollinearity grounds.

Exercise 9.6 [E]

The chapter found that age's marginal correlation with survived was only −0.08-0.08, yet a clear "children first" effect appeared once the data was split by class. Propose one other pair of Titanic features where you'd expect a similar effect, hidden in the overall numbers until the data is split. Explain your reasoning before checking it.

Show the model answer

A plausible candidate is fare split by pclass and survived jointly. The marginal correlation between fare and survived (0.26) is real, but could hide a stronger within-class effect, since fare varies enormously within 1st class, from a low-end 1st-class ticket to the most expensive suites. A passenger paying a top fare within 1st class might show a bigger survival boost than the marginal correlation suggests. The general pattern to watch for: any feature whose marginal relationship to the target looks weak but which is known to vary systematically with a third, stronger predictor (here, class) is a candidate for the same hidden effect.

Exercise 9.7 [E]

A colleague proposes running full EDA, including test-set rows, before splitting the data, arguing "we're just looking, not modelling yet." Using this chapter's data-snooping discussion, write two or three sentences explaining what could go wrong.

Show the model answer

Running EDA on the full dataset, test rows included, contaminates every pattern it finds. An outlier threshold, a "this feature looks predictive" judgement, or a bucketing scheme for rare categories has then been chosen with knowledge of the test set's actual values. Even without directly touching the labels, decisions like "drop rows where fare > 400" or "bucket embarked's rare category into 'other'" become choices tuned to the specific data points that will later be used to evaluate the model. That inflates the model's apparent generalisation performance. The fix is to split first, then run full EDA only on the training partition, treating the test set as if it doesn't exist until final evaluation.

Exercise 9.8 [R]

Read about Anscombe's quartet (or its modern descendant, the "Datasaurus Dozen"). Pick one of the four Anscombe datasets. Describe, without plotting it, what summary statistics alone would have told you, and what they would have hidden.

Show the model answer

Discussion only, no single answer. Each of Anscombe's four datasets shares an identical mean, variance, and Pearson correlation coefficient (~0.816). Summary statistics alone would report the same numbers for all four, and suggest the same linear relationship strength. What they hide: one dataset is a genuine linear relationship with noise. One is a clean non-linear (quadratic-like) curve that Pearson's linear correlation badly misrepresents. One is a near-perfect linear relationship broken by a single outlier that shifts the fitted line. And one is a vertical cluster of points entirely determined by one high-leverage outlier. Only plotting each one reveals which of these four very different stories the identical statistics were describing.

online /books/classical-data-science/quiz/9 Newcastle, UK v0.1.0 // rss