Build an honest ML workflow
Define a prediction task, prevent data leakage, and compare your model with a simple baseline. This module has 3 lessons and takes about 1h 15m. Each lesson has a practice exercise that runs real Python in your browser and a short quiz.
Start module 10 in the Learning LabSplit before you learn
Separate training data from unseen test data for a synthetic maintenance task.
A feature is information available when you make a prediction. A target is the value you want to predict. Here, hours of operation is the feature and synthetic wear is the target. X is a two-dimensional feature table. y is a one-dimensional target array.
Train on one set of rows and evaluate on separate rows. A fixed random_state makes the split reproducible. Random splits suit independent examples. For future forecasting, split by time; for repeated machines or patients, split by group. The same entity should not leak across both sides.
See it in action
import numpy as np
from sklearn.model_selection import train_test_split
# Synthetic independent machines.
hours = np.array([10, 20, 30, 40, 50, 60, 70, 80])
wear = np.array([2, 5, 5, 9, 10, 13, 13, 17])
X = hours.reshape(-1, 1)
y = wear
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=42
)
print("Training shapes:", X_train.shape, y_train.shape)
print("Test shapes:", X_test.shape, y_test.shape)
print("Keep test labels separate until final evaluation.")Choose a split that matches how the model will be used.
Keep preprocessing inside a pipeline
Fit missing-value replacement and scaling only on synthetic training data.
Preprocessing changes raw inputs before learning. An imputer fills missing values; a scaler adjusts feature scales. If either learns from test rows, the model has indirectly seen the test set. This is data leakage.
A Pipeline fits each step using training data and applies the same learned steps during prediction. The example uses invented appointment workload data, not clinical decisions. A pipeline helps prevent leakage but cannot detect a feature that contains future information.
See it in action
import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge
# Synthetic workload and wait-time data. Not patient data or medical advice.
X_train = np.array([[10, 2], [20, 3], [30, np.nan], [40, 5], [50, 6]])
y_train = np.array([8, 12, 15, 20, 23])
X_test = np.array([[25, np.nan], [45, 5]])
model = make_pipeline(
SimpleImputer(strategy="median"),
StandardScaler(),
Ridge(alpha=1.0)
)
model.fit(X_train, y_train)
print("Predicted wait minutes:", model.predict(X_test).round(1))
print("Training medians:", model.named_steps["simpleimputer"].statistics_)Learn every preprocessing step from training data, then reuse it.
Set a baseline worth beating
Measure a simple constant forecast before adding a learning model.
A baseline gives you a reference. For a numeric target, a model that always predicts the training mean is easy to build. Mean absolute error, or MAE, is the average distance between predictions and actual values, in the target units.
Evaluate your baseline and candidate on the same unseen rows. A complex model that fails to improve the relevant metric has not earned its complexity. For seasonal energy data, also try a practical baseline such as the previous day or the same hour last week.
See it in action
import numpy as np
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
# Synthetic independent buildings: area and daily energy.
X_train = np.array([[20], [40], [60], [80], [100]])
y_train = np.array([12, 21, 31, 39, 52])
X_test = np.array([[30], [70], [90]])
y_test = np.array([16, 35, 46])
for name, model in [
("Training mean", DummyRegressor(strategy="mean")),
("Linear model", LinearRegression()),
]:
model.fit(X_train, y_train)
error = mean_absolute_error(y_test, model.predict(X_test))
print(name, "MAE:", round(error, 2), "kWh")Measure a useful baseline before spending time on model complexity.
Free, open and run by a nonprofit.
This page is part of Python to AI, a free course of 20 modules and 60 lessons in the engineering.fit( ) Learning Lab. The Learning Lab is built and run by engineering.fit( ), a nonprofit founded by Abdullateef Opeyemi Bakare that helps African engineering students gain research experience. You do not need an account or pay a fee to learn.