Tests whether you can tell which lifecycle stage a project is really in, spot how it is failing, and judge when it may move on or must loop back.
Based on chapter 10. Every answer comes with an explanation and a link back to the section.
/Q10
Quiz
Question 1 of 25
The eight lifecycle stages and the CRISP-DM / TDSP mapping Level 1
Progress
Which of this book's eight lifecycle stages has no explicit phase in CRISP-DM?
/02
练习From the book
8 exercises
The chapter's own exercises are open questions. Try one, then compare with the model answer.
Exercise 10.1 [W]
Name the eight lifecycle stages in order. For each, name one chapter (or block of chapters) in this book where that stage's skills live.
Show the model answer
Problem framing: Chapter 2. 2. Data collection & understanding: Chapters 7-9. 3. Data preparation: Chapters 7 and 14. 4. Modelling: Chapters 11-20. 5. Evaluation: Chapters 13 and 16. 6. Deployment: Book 2, Chapter 23. 7. Monitoring: Book 2, Chapter 25. 8. Iteration: loops back to Chapter 2's problem framing.
Exercise 10.2 [W]
Explain in two or three sentences why the textbook left-to-right pipeline diagram is misleading, using at least one concrete example of a backward loop from this chapter.
Show the model answer
The left-to-right pipeline diagram implies each stage is finished once and handed off to the next, but real projects loop backward constantly: evaluation (stage 5) routinely reveals the success metric itself was wrong. A model hits a strong AUC, the team celebrates, and the business owner then explains they cared about precision at a specific operating point nobody optimised for, sending the project back to stage 1. Similarly, modelling (stage 4) routinely sends a project back to data preparation (stage 3) when a model's errors reveal a feature bug or missing signal. The two stages cycle many times rather than completing in sequence. A diagram with a single feedback arrow tacked on at the end can't represent a process where nearly every stage has an edge back to nearly every earlier one.
Exercise 10.3 [A]
Using Listing 10.3, construct an ask dictionary describing a hypothetical stakeholder request where historical_labelled_data_exists is False. Report what the function returns, and why that's the correct triage outcome.
It describes, say, a stakeholder who wants to predict equipment failures but has never logged which equipment failed. It returns "Not ready -- this is a data collection project first, an ML project second". This is the correct outcome because Listing 10.3 checks simple_rule_suffices first (false here, so it doesn't short-circuit to "ship a rule"), then whether this is a one-off analysis (it isn't), then whether historical labelled data exists. It doesn't, so no amount of modelling skill can produce a supervised model yet. The correct next step is building a labelling/data-collection process, not starting a lifecycle project that has no label to train against.
Exercise 10.4 [A]
Using the logic of Listing 10.6 as a model, compute what accuracy the naive "predict majority class" baseline would score on a dataset with a 3% positive rate. Explain why that number is a poor argument for using the naive baseline in production.
Show the model answer
On a dataset with a 3% positive rate, the naive "predict majority class" baseline (always predict the negative class) scores 97% accuracy by construction. It is the negative class's share of the data. That number is a poor argument for production use because the baseline achieves it while catching zero positives. Precision and recall on the positive class are both 0.00. The model is useless for any business action that depends on identifying positives. This is the exact scenario the tutorial's dummy classifier shows at 93% accuracy and 7% churn. A high accuracy figure on an imbalanced dataset says nothing about whether the model does the one thing it's being built for.
Exercise 10.5 [A]
The tutorial's logistic regression achieved AUC-ROC of approximately 0.638 but failed to meet the precision/recall target from Stage 1. Using this chapter's Stage 5 discussion, name the two specific stages the project should loop back to. For each, give one concrete change that might close the gap.
Show the model answer
The two stages to loop back to are stage 3 (data preparation) and stage 1 (problem framing/success metric). For stage 3: add richer features. The tutorial notebook works from only six raw features: support tickets, payment failures, login recency, tenure, plan type, product count. Engineering additional signals, such as usage trend over time, interaction-weighted engagement scores, or billing-cycle-relative features (Chapter 14), could give the model more to separate churners from non-churners than a modest AUC of 0.638 suggests is currently available. For stage 1: renegotiate the success metric. Precision ≥ 0.40 at recall ≥ 0.60 may be unattainable with the signal currently available. The business owner may need to accept a different operating point, such as lower recall in exchange for usable precision. Or they may need a different intervention design, such as a broader, cheaper retention outreach that tolerates lower precision. The alternative is asking the model to hit a target the data can't support yet.
Exercise 10.6 [E]
Rewrite the tutorial's Stage 6 deployment plan for a scenario where the business owner instead wants real-time scoring at the moment a customer contacts support, rather than nightly batch scoring. What changes about the serving pattern, the latency budget, and the rollback plan?
Show the model answer
Moving from nightly batch scoring to real-time scoring at the moment of a support contact changes all three elements the tutorial's Stage 6 plan assumed. The serving pattern shifts from a scheduled batch job over the full customer base to a request-response endpoint invoked synchronously (or near-synchronously) when a support interaction begins. The model needs to be behind a low-latency API, e.g. a SageMaker real-time endpoint, rather than a nightly Batch Transform job. Features need to be computable on demand rather than pre-joined the night before, which likely requires a feature store with fresh, queryable feature values, rather than a batch feature table. The latency budget tightens dramatically. Batch scoring can tolerate minutes to hours per run. A support-interaction-triggered prediction needs to return in well under a second, so it doesn't stall the support agent's workflow. This rules out any model or feature pipeline with expensive real-time joins or slow feature computation. The rollback plan also changes. A bad batch run can be discarded and rerun before anyone sees the output. A bad real-time model is affecting live support decisions immediately. So the rollback plan needs an instant traffic-shifting mechanism, e.g. an endpoint alias or canary rollback, rather than "rerun the job." It also needs a fallback response, such as defaulting to the old rule-based flag, if the real-time endpoint is unavailable or too slow.
Exercise 10.7 [E]
A team tells you they "follow CRISP-DM," but their retrospectives show every project has shipped a model that was never revisited after deployment. Using Appendix A's Framework Comparison, explain which stage CRISP-DM under-specifies that likely explains this pattern. Explain what TDSP would have required instead.
Show the model answer
CRISP-DM under-specifies monitoring. Per Appendix A's Framework Comparison, it has no explicit monitoring phase at all, ending effectively at deployment. A team that "follows CRISP-DM" literally has no process step that requires anyone to watch the model after it ships. No dashboards, no alerts, and critically no named owner for production health. That explains exactly the pattern described: models get shipped and never revisited. TDSP would have required an explicit deployment-as-ongoing / customer-acceptance stage with monitoring dashboards, drift alerts, and, per Appendix A's RACI table, a named individual (not "the team") accountable for receiving those alerts and triggering retraining. The gap isn't a modelling failure. It's a process framework that never asked "who watches this after ship day."
Exercise 10.8 [R]
Read the original CRISP-DM paper (Wirth & Hipp, 2000) or the Microsoft TDSP documentation. Compare its treatment of "iteration" to this chapter's Stage 8. Does the source treat iteration as a distinct, named stage with its own deliverable, or as an implicit property of the whole process? What does that difference suggest about how seriously each framework takes the "loop back" problem?
Show the model answer
Discussion only, no single answer. Worth examining directly: the original CRISP-DM paper treats deployment as its sixth and final phase, with only a brief note that the process is "cyclic." It offers no dedicated deliverable, checklist, or owner for what an iteration requires. Iteration is a property asserted of the diagram, not a phase with its own exit criteria. TDSP's documentation, by contrast, explicitly draws the lifecycle looping back into Business Understanding, and ties that loop to concrete artefacts: updated project charters, retrained model versions. The difference suggests CRISP-DM, written in 1996 for data-mining projects that often ended at a delivered analysis or report, didn't anticipate that "the model runs continuously in production and needs to be revisited" would become the normal case. TDSP, written in the cloud/MLOps era, treats the loop-back as the expected steady state of a live system, not an edge case.