◢◤ ARKANE_
SYS_TIME UTC+0
Work with me

Chapter 12 quiz · 25 questions

Ensemble Methods

Tests why combining models works: bagging and Random Forests for variance, boosting for bias, and stacking with out-of-fold predictions. Includes hand calculations of correlated-average variance, bootstrap coverage, majority votes and gradient-boosting updates.

Based on chapter 12. Every answer comes with an explanation and a link back to the section.

/Q12

Quiz

Question 1 of 25

Why ensembles work: diverse errors and correlation Level 1
Progress

Averaging many models beats a single model only under one condition. Which?

Choose one
/02

练习From the book

7 exercises

The chapter's own exercises are open questions. Try one, then compare with the model answer.

Exercise 12.1 [W]

Explain in your own words why bagging reduces variance without increasing bias, using the σ2/B\sigma^2/B relationship from this chapter's Exposition.

Show the model answer

If BB models each have variance σ2\sigma^2 and their errors are uncorrelated, averaging them gives variance σ2/B\sigma^2/B, strictly smaller as BB grows, with no term in that formula affecting bias. Bagging doesn't change what each individual model is trying to predict: no change to the systematic error, i.e. bias. It only reduces how much the ensemble's average prediction fluctuates around that target across different training samples.

Exercise 12.2 [W]

State which of bagging or boosting each of the following targets: (a) a model that is systematically wrong on a specific subgroup of examples; (b) a model whose predictions change wildly with small changes to the training data.

Show the model answer

(a) Boosting: a model systematically wrong on a subgroup has high bias on that subgroup, which boosting directly targets by reweighting/refitting on exactly the hard cases. (b) Bagging: wildly changing predictions from small training-data changes is the definition of high variance, which bagging's averaging over bootstrap samples is designed to reduce.

Exercise 12.3 [A]

Using Listing 12.5's setup, explain why Random Forest's RMSE barely changes between n_estimators=200 and n_estimators=500 ($45,502 vs. $45,514) while LightGBM's RMSE at the same two settings keeps improving slightly, from $28,323 to $28,159. What does this suggest about each method's sensitivity to over-provisioning trees?

Show the model answer

Random Forest's RMSE barely moving from 200 to 500 trees ($45,502 → $45,514, marginally worse, well within noise) indicates bagging's variance reduction has already saturated before 200 trees: additional trees add averaging stability with diminishing, essentially flat returns. LightGBM keeps improving slightly over the same range ($28,323 → $28,159), because boosting is still reducing bias round by round at 200 trees. Each new tree is still correcting real signal the ensemble hasn't captured yet, not just fitting noise. That's the core asymmetry this chapter's Exposition draws between the two. Bagging's averaging effect plateaus once the ensemble is large enough to cancel variance. Boosting keeps extracting value from additional rounds for longer, up to the point, not yet reached here, where it starts overfitting instead.

Exercise 12.4 [A]

Using Listing 12.4 as a base, add XGBoost as a third base model to the stack. State, from this chapter's own measured numbers, whether you'd expect the resulting stack to beat solo XGBoost's $28,873 RMSE.

Show the model answer

LightGBM, not XGBoost, is the single best individual model in this chapter's benchmark: $28,323 RMSE, versus XGBoost's $28,873. It's already one of the two models in the existing stack. That makes beating solo XGBoost's $28,873 specifically a fairly low bar: LightGBM alone already clears it, so the three-model stack would need the meta-learner to badly mis-weight things for the combined prediction to land worse than $28,873. The more interesting, harder bar is solo LightGBM's own $28,323. This chapter's own measured result is a reason for caution there: the RF+LightGBM stack's meta-learner assigned Random Forest, the weaker base model, a negative weight, and the stack still ended up worse than solo LightGBM ($29,535 vs. $28,323). Adding XGBoost, weaker than LightGBM but stronger than Random Forest, could get down-weighted the same way and land close to solo LightGBM, or dilute it enough to land worse. Expect the three-model stack to beat solo XGBoost comfortably. Don't assume it beats solo LightGBM without checking.

Exercise 12.5 [E]

A customer's data science team reports "we stacked five models and it's more accurate than any of them individually." Using this chapter's Worked Example, write the two or three sentences you'd say before accepting that claim.

Show the model answer

"Which models were in the stack, and did you compare against the single best individual model before concluding the stack is better? This chapter's own measured result shows a stack can be worse than the best model it was built from, even with that model inside the stack. 'Stacked and improved over all five individually' needs to be checked against the best of the five, not just an average or the worst of them."

Exercise 12.6 [E]

A colleague argues Random Forest should always be tried before XGBoost "because it's simpler." Using Procedure 12.3 and this chapter's measured accuracy gap (Random Forest's RMSE roughly 58% higher than XGBoost's), write the case for when that advice is right and when it isn't.

Show the model answer

Random Forest-first is right when time-to-baseline matters more than squeezing out the last accuracy, since it needs almost no tuning and gives a solid, honest number quickly (per Procedure 12.3). It's the wrong default when the roughly 58% RMSE gap this chapter measured (Random Forest $45,502 vs. XGBoost $28,873) matters to the business outcome. At that point, the extra tuning effort XGBoost/LightGBM demand is worth paying for, and "simpler" shouldn't be conflated with "good enough."

Exercise 12.7 [R]

Read the XGBoost paper's (Chen & Guestrin, 2016) abstract and introduction. Identify one engineering optimisation it describes that this chapter's Trade-offs section does not mention. Explain what problem it solves.

Show the model answer

Discussion only. Likely candidates include XGBoost's cache-aware block structure for out-of-core computation (enabling training on datasets larger than memory) or its weighted quantile sketch algorithm for approximate split-finding on weighted data. Either addresses a scalability problem (very large datasets, or weighted training samples) beyond what this chapter's Trade-offs table, which focuses on bias/variance/overfitting risk, covers.

online /books/classical-data-science/quiz/12 Newcastle, UK v0.1.0 // rss