Scaling a small trial before examining failures spreads errors with the service. Case count alone does not establish representativeness.
The decision to examine
Start with narrow boundaries and non-personal training data, then vary cases and conditions. Define acceptance and rejection in advance and keep an anonymous error log.
Worked example
A hypothetical 20-case test finds two minor errors and one serious error. We do not hide the serious failure in 85% accuracy; we pause this use and investigate.
From idea to practice
- Describe the problem as a need, not a product name.
- Specify evidence needed before a decision and who reviews it.
- Test a non-personal example and log failures as well as successes.
- State whether to continue, improve or stop and why.
Your exercise
Design five ordinary and five difficult cases, including one failure that requires stopping regardless of the average.
Review rubric
A useful answer identifies a context, checkable evidence, a boundary and an owner for the next step. If it fits any organisation unchanged, add a detail specific to the case. This is a learning rubric, not automatic marking or official assessment.
Further reading: AI Risk Management Framework — NIST · 2023; page reviewed 2026 · Checked: 8 October 2026
Your next step
Open the readiness notebook · Explore policy sources
Institutional DJAIRI material prepared with AI assistance. Learning examples are hypothetical; analysis and recommendations are Mustanef’s. Citing a source does not imply partnership or endorsement.