Build an AI-assisted triage flow
Compare a simple baseline with simulated predictions and review uncertain work. This module has 3 lessons and takes about 1h 50m. Each lesson has a practice exercise that runs real Python in your browser and a short quiz.
Start module 08 in the Learning LabStart with a rule baseline
Return repair if that substring appears, otherwise training if it appears, otherwise other. Compare without case sensitivity.
Before using AI, measure a simple approach. A keyword rule can provide a useful baseline for a small topic classifier. It will miss synonyms and can misread context.
This toy rule matches substrings in lowercase text. Repair takes priority over training when both occur. Everything else becomes other.
This is a deterministic simulation, not a language model. Its purpose is to provide a baseline that an AI system must improve on.
See it in action
def solve(text):
text = text.lower()
if "repair" in text:
return "repair"
if "training" in text:
return "training"
return "other"
print(solve("Need REPAIR help"))
This is a deterministic simulation, not a language model. Its purpose is to provide a baseline that an AI system must improve on.
Set an explicit review threshold
Route repair or training only with a valid numeric score at least 0.8. Return review otherwise, including boolean scores.
A triage workflow can reserve uncertain suggestions for review. The score threshold here is a teaching policy, not a recommended production setting.
A prediction goes to its queue only when its label is repair or training and its numeric score is at least 0.8. All other proposals go to review, including invalid scores.
Measure review volume and wrong routes together. A higher threshold may reduce automatic mistakes while increasing manual work.
See it in action
def solve(label, score):
valid_score = type(score) in (int, float) and 0 <= score <= 1
if label in ("repair", "training") and valid_score and score >= 0.8:
return label
return "review"
print(solve("repair", 0.8))
Measure review volume and wrong routes together. A higher threshold may reduce automatic mistakes while increasing manual work.
Practical: Measure routing coverage
Return automatic routing coverage rounded to two decimals. Count only repair and training; use 0.0 for an empty list.
Coverage is the share of items handled by an automatic route. It is different from accuracy. A system can cover every item and still be wrong.
Given final routes, count repair and training as automatic. Divide by the total and round to two decimal places. An empty batch has zero coverage in this exercise.
Report accuracy on labelled examples and coverage on the same population. Neither number alone describes the whole workflow.
Your experiment plan
Use the browser editor or the notebook. All records and AI responses are fictional. No network calls, credentials, or real messages are used.
- Make six fictional enquiries with expected labels and simulated scores.
- Run the example and predict its result before changing it.
- Complete the function in Practice and run the checks. Add your own normal, empty, and failure cases.
- Compare coverage at two thresholds and count wrong automatic routes.
- Save the notebook with your observations and explain one limitation of the simulation.
What to hand in
- A triage table with labels, routes, coverage, errors, and review workload.
- A test table with inputs, expected outputs, actual outputs, and one revision.
Reflect on your results
Which error costs more for this workshop?
The automatic exercise checks the core code behaviour. Review observations and written deliverables yourself or with a mentor.
See it in action
def solve(routes):
if not routes:
return 0.0
return round(sum(route in ("repair", "training") for route in routes) / len(routes), 2)
print(solve(["repair","review","training","review"]))
Report accuracy on labelled examples and coverage on the same population. Neither number alone describes the whole workflow.
Free, open and run by a nonprofit.
This page is part of AI Automation, a free course of 20 modules and 60 lessons in the engineering.fit( ) Learning Lab. The Learning Lab is built and run by engineering.fit( ), a nonprofit founded by Abdullateef Opeyemi Bakare that helps African engineering students gain research experience. You do not need an account or pay a fee to learn.