Tests why extra columns hurt, how filter, wrapper and embedded selection differ, how to read PCA eigenvalues and loadings, and when t-SNE and UMAP are the wrong tool.
Based on chapter 15. Every answer comes with an explanation and a link back to the section.
/Q15
Quiz
Question 1 of 23
The curse of dimensionality Level 1
Progress
A dataset keeps the same number of rows while the number of features grows from 5 to 500. What happens to the ratio between a point's nearest-neighbour distance and its farthest-neighbour distance?
/02
练习From the book
8 exercises
The chapter's own exercises are open questions. Try one, then compare with the model answer.
Exercise 15.1 [W]
Explain in your own words why feature selection preserves interpretability while feature extraction does not, using the "income is still income" framing from this chapter.
Show the model answer
Feature selection removes features but keeps the survivors as-is, so "income" in the output is still literally income. Its scale, units, and meaning are unchanged, and a domain expert can interpret it directly. Feature extraction (PCA) creates new features as linear combinations of many originals. A principal component has no inherent real-world meaning of its own, only a mix of loadings across original features.
Exercise 15.2 [W]
Classify each of the following as a filter, wrapper, or embedded method, and give one reason: (a) dropping features with variance below 0.01; (b) RFECV; (c) keeping features with Random Forest importance above the median.
Show the model answer
(a) Variance threshold: filter (no model involved, judges only the feature's own spread). (b) RFECV: wrapper (repeatedly retrains a model and cross-validates to pick a feature count). (c) Random Forest importance above the median: embedded (importance is a byproduct of fitting one model, not a separate model-agnostic step or repeated retraining).
Exercise 15.3 [A]
Using Listing 15.3 as a starting point, rerun mutual-information selection with k=5 instead of k=10 on the synthetic dataset from Listing 15.8, and report whether the resulting 5 features are a subset of the 15 reported in the Worked Example.
Show the model answer
With k=5, SelectKBest(mutual_info_classif, k=5) keeps the 5 highest-MI-scoring features. Mutual information scores are computed once and ranked, so the top 5 are necessarily a subset of the top 10, which is itself a subset of any top-15 selection. The ranking doesn't change with k. Only how far down that ranking you cut does. This holds only because the scores are seeded. mutual_info_classif is randomised, which is why Listing 15.3 passes random_state=42.
Exercise 15.4 [A]
Using Listing 15.9's logic, find how many components are needed to reach 99% cumulative explained variance on the same 50-feature training set, and compare the result to the 37 needed for 95%.
Show the model answer
Running the same cumsum(pca_full.explained_variance_ratio_) logic with the threshold raised to 0.99 gives 39 components, only 2 more than the 37 needed for 95%. That's a smaller jump than the "climbing the last few per cent gets expensive" intuition might suggest. It's a direct consequence of this dataset's structure: 20 of the 50 features carry genuine signal (10 informative, 10 redundant combinations of them) and 30 are pure noise. Once PCA has captured the signal-carrying components, the remaining components are splitting up noise variance that's spread almost evenly across the leftover dimensions, so each one adds only a sliver past the point where the real structure is already represented. A dataset with a longer tail of informative-but-individually-small directions would show a bigger 95%-to-99% jump than this one does.
Exercise 15.5 [A]
Reproduce the digits-dataset PCA vs. t-SNE timing comparison from the Worked Example, but with perplexity=5 instead of perplexity=30 in the t-SNE call. Describe qualitatively how the resulting 2D plot's cluster tightness changes.
Show the model answer
Lowering perplexity from 30 to 5 tells t-SNE to weight a much smaller local neighbourhood per point. The resulting plot typically shows tighter, more numerous, more fragmented small clusters. Local structure within digit classes becomes more visible, but the plot loses some of the smooth global separation between whole digit classes that a larger perplexity provides.
Exercise 15.6 [E]
The Worked Example shows RFECV and embedded Random Forest importance both beating the 50-feature baseline. Using the curse of dimensionality from this chapter's Exposition, explain why removing features can improve accuracy rather than merely preserving it.
Show the model answer
In high dimensions, many features are noisy or redundant. A model can fit spurious patterns in that noise, which is overfitting, and irrelevant dimensions make distance-based reasoning less meaningful, the curse of dimensionality. Removing such features doesn't just save compute. It removes the substrate the model could otherwise overfit to, which is why validation accuracy can rise, not just hold steady, after selection.
Exercise 15.7 [E]
A regulator requires that a credit-scoring model's predictions be explainable in terms of the original applicant-provided variables. A colleague proposes using PCA to cut a 200-feature dataset down to 20 components first. Write two or three sentences explaining why this is likely to cause a problem, and what you'd suggest instead.
Show the model answer
PCA components are linear combinations of many original applicant-provided variables: income, age, credit history, and so on, all blended with different weights. A regulator asking "why was this applicant denied" cannot be given an answer in terms of "component 7," not in any way a human applicant could act on or an auditor could verify against the original variables. Feature selection, keeping a subset of the 200 original variables via RFECV or embedded importance, preserves that direct explainability and should be recommended instead.
Exercise 15.8 [R]
Read the Wattenberg et al. "How to Use t-SNE Effectively" piece cited above. Identify one t-SNE pitfall it describes that this chapter did not mention, and explain how it would change the interpretation of the digits-dataset t-SNE plot in the Worked Example.
Show the model answer
Discussion only. Wattenberg et al.'s piece emphasises that t-SNE cluster sizes and inter-cluster distances in the resulting plot are not reliably meaningful. Only relative local neighbourhoods are. Applied to the digits plot: a viewer should resist reading "digit class 3's cluster is bigger than class 7's" or "class 1 is closer to class 4 than to class 9" as quantitative facts, even though the plot invites exactly that reading.