Tests whether you can pick the right layer of the stack and predict what NumPy, pandas and scikit-learn code does, including broadcasting, NaN handling and leakage traps.
Based on chapter 6. Every answer comes with an explanation and a link back to the section.
/Q6
Quiz
Question 1 of 24
The layered stack and what each library is for Level 1
Progress
A colleague wants to run a two-sample t-test comparing mean order value between two customer segments. Which part of the stack is built for this?
/02
练习From the book
8 exercises
The chapter's own exercises are open questions. Try one, then compare with the model answer.
Exercise 6.1 [W]
Using the layered-stack idea from this chapter's Exposition, explain in one or two sentences why learning pandas well requires understanding NumPy, but not the other way around.
Show the model answer
pandas DataFrames store each column as a NumPy array under the hood, so pandas' speed and much of its behaviour (vectorised operations, dtype handling) come directly from NumPy. But NumPy arrays have no concept of labelled rows, mixed column types, or an index, so NumPy has no need to understand pandas at all. The dependency runs one way, matching the layered-stack diagram in the Exposition.
Exercise 6.2 [W]
State in one sentence each what .fit(), .transform(), and .predict() do in the scikit-learn API pattern. Identify which of the three should never be called on test data.
Show the model answer
.fit() learns parameters from training data. .transform() applies a learned transformation to produce new data, used for preprocessing. .predict() applies a learned supervised model to produce predictions on new data. .transform() and .predict() are safe to call on test data. .fit() should never be called on test data, since fitting on it leaks test-set information into the transformation or model.
Exercise 6.3 [A]
Using Listing 6.6 as a template, measure the speedup of np.sqrt over an equivalent Python loop (using math.sqrt) on an array of 500,000 elements, and report the ratio you observe.
Show the model answer
Timing np.sqrt against an equivalent [math.sqrt(x) for x in py_list] loop over 500,000 elements gives roughly 22.5 ms for the Python loop and roughly 0.46 ms for the NumPy call: a speedup of approximately 49x. This is smaller than Listing 6.6's ~117x multiplication speedup, because math.sqrt is a fast built-in per call, narrowing (but not closing) the gap. The qualitative result, NumPy wins by well over an order of magnitude, holds regardless of which elementwise operation is timed.
Exercise 6.4 [A]
Using Listing 6.7's approach, compute survival rate by passenger class andembark_town instead of class and sex. Report which single (class, port) combination has the lowest survival rate.
Show the model answer
Pivoting survival rate by pclass and embark_town on the Titanic dataset shows the lowest survival rate is third class passengers who boarded at Southampton, at roughly 19%, noticeably lower than third-class Cherbourg (38%) or Queenstown (38%) passengers. This is a case where a single grouping variable (class alone, or port alone) would have hidden the compounding effect visible only in the two-dimensional pivot.
Exercise 6.5 [A]
Reproduce Listing 6.8, but compare passenger age between survivors and non-survivors instead of fare. Is the difference statistically significant at p<0.05? Which group is older on average?
Show the model answer
Comparing age between survivors (mean ≈ 28.3 years) and non-survivors (mean ≈ 30.6 years) with scipy.stats.ttest_ind gives t ~ -2.07, p ~ 0.039. That's below the 0.05 threshold, so the age difference is statistically significant, with survivors being slightly younger on average. The effect is much weaker than the fare difference in Listing 6.8 (p ≈ 6e-15). Age separates survivors from non-survivors only modestly, while fare (as a proxy for class) separates them dramatically.
Exercise 6.6 [E]
Procedure 6.4's common mistake describes fitting a scaler on test data. Write two or three sentences explaining what would go wrong with a reported accuracy number if this mistake were made silently on a real project, and why no exception would be raised to catch it.
Show the model answer
If a scaler were fit on test data, or on the combined train+test set, rather than only on training data, the reported test accuracy would be inflated. The scaler's mean and standard deviation would already "know" something about the distribution of the test set. That gives the model an information advantage it wouldn't have in production, where future data is by definition unseen at fit time. No exception is raised because nothing about calling .fit() on the wrong slice of data is syntactically or semantically invalid. scikit-learn has no way to know which rows are supposed to represent "the future." So the leak is silent, and only shows up later as a mismatch between validation performance and real-world performance.
Exercise 6.7 [E]
A colleague proposes dropping pandas entirely and working with raw NumPy arrays "for speed" on a project with twenty columns of mixed numeric and categorical data. Using this chapter's Trade-offs section, write the two or three sentence argument against this.
Show the model answer
Twenty columns of mixed numeric and categorical data is close to a worst case for raw NumPy. NumPy arrays are single-typed, so mixed columns would need to be split across multiple arrays, or coerced into a single dtype, usually object, which forfeits NumPy's speed advantage entirely. Every join, filter, or groupby that pandas expresses in one readable line would need to be hand-written with manual index bookkeeping. The "for speed" framing is backwards here: pandas' groupby and filtering operations already delegate to vectorised NumPy underneath. The actual speed cost of raw NumPy on this kind of data would be development time, not runtime.
Exercise 6.8 [R]
Read the NumPy quickstart mentioned in this chapter's Further Reading. Find one broadcasting rule this chapter's Exposition did not discuss. Describe a data science situation where getting that rule wrong would silently produce an incorrect result rather than an error.
Show the model answer
Discussion only, no single answer. Candidates worth investigating in the NumPy quickstart: the rule that dimensions are compared from the trailing (rightmost) axis, and either must match or one of them must be 1 or absent. Or that broadcasting can silently succeed on shapes the author didn't intend to be compatible: e.g. a (3,) array broadcasting against a (3, 1) array to produce a (3, 3) result via outer-product-like expansion, rather than the elementwise operation the author expected. Consider a concrete case: adding a "per-row mean" array of shape (n,) back onto a (n, m) matrix without reshaping it to (n, 1) first. Broadcasting rules make this fail loudly in some shapes and silently produce the wrong answer in others, depending on whether n happens to equal m.