Turn language into numbers
Represent maintenance text, compare vectors, and inspect a small attention calculation. This module has 3 lessons and takes about 1h 30m. Each lesson has a practice exercise that runs real Python in your browser and a short quiz.
Start module 17 in the Learning LabRepresent maintenance notes
Build a token-count matrix and inspect what the representation loses.
Maintenance notes are text, while most models need numbers. A bag-of-words representation creates one column per vocabulary word and records how often it appears in each note. It is simple enough to inspect by hand.
This example tokenises words, fixes a vocabulary from the reference notes, and transforms a new note with that same vocabulary. Unknown words contribute nothing. In a supervised project, learn the vocabulary only from training text.
Counting loses order. 'Pump is not hot' and 'Pump is hot' look similar unless your representation captures negation or word sequences. Inspect these limits before using text models for engineering decisions.
See it in action
import re
import numpy as np
notes = ["pump vibration high", "pump temperature high", "valve pressure low"]
def tokens(text):
return re.findall(r"[a-z]+", text.lower())
vocabulary = sorted(set(word for note in notes for word in tokens(note)))
def vectorise(text):
words = tokens(text)
return [words.count(word) for word in vocabulary]
matrix = np.array([vectorise(note) for note in notes])
print("Columns:", vocabulary)
print("Reference rows:", matrix)
print("New note:", vectorise("HIGH pump vibration today"))
Start with a representation you can inspect, then test its blind spots.
Compare documents with vector similarity
Use cosine similarity and understand the limits of hand-built embeddings.
An embedding is a numeric representation of an item. Similarity search compares representations to find related items. In this exercise, each document has three hand-chosen coordinates for solar equipment, storage, and maintenance.
Cosine similarity compares direction rather than raw length. Divide the dot product by both vector lengths. This can make a short and a long vector similar when their proportions match.
These coordinates are an illustration, not learned language embeddings. Real embedding models learn from data and can miss meanings, languages, and domain-specific terms. Test retrieval on representative local queries before relying on it.
See it in action
import numpy as np
titles = ["Panel cleaning guide", "Battery inspection", "Solar and battery system"]
# Hand-built teaching vectors: solar, storage, maintenance.
documents = np.array([[1.0, 0.0, 0.8], [0.0, 1.0, 0.9], [0.8, 0.8, 0.2]])
query = np.array([1.0, 0.0, 0.6])
scores = (documents @ query) / (np.linalg.norm(documents, axis=1) * np.linalg.norm(query))
order = np.argsort(-scores)
for index in order:
print(titles[index], round(float(scores[index]), 3))
Similarity depends on what the representation captures, not just the scoring formula.
See how attention mixes information
Calculate scaled dot-product attention and inspect its weights.
Attention lets a query combine information from several items. Each item has a key used for matching and a value used for the output. Dot products between the query and keys produce match scores.
Divide scores by the square root of the key dimension, then apply softmax to turn them into non-negative weights that sum to one. The output is a weighted sum of the values. Subtract the largest score before exponentiation to improve numerical stability.
The tiny vectors below are chosen by hand for three maintenance tokens. A transformer learns its query, key, and value projections from data. This calculation explains one attention operation; it does not build a large language model or prove that attention weights explain a decision.
See it in action
import numpy as np
tokens = ["pump", "hot", "valve"]
query = np.array([[1.0, 0.0]])
keys = np.array([[1.0, 0.0], [0.8, 0.2], [0.0, 1.0]])
values = np.array([[2.0, 0.0], [1.0, 3.0], [0.0, 2.0]])
scores = query @ keys.T / np.sqrt(keys.shape[1])
shifted = scores - scores.max(axis=-1, keepdims=True)
weights = np.exp(shifted)
weights /= weights.sum(axis=-1, keepdims=True)
output = weights @ values
print("Token weights:", dict(zip(tokens, weights[0].round(3))))
print("Combined information:", output.round(3))
Attention matches with keys and combines values using weights from the query.
Free, open and run by a nonprofit.
This page is part of Python to AI, a free course of 20 modules and 60 lessons in the engineering.fit( ) Learning Lab. The Learning Lab is built and run by engineering.fit( ), a nonprofit founded by Abdullateef Opeyemi Bakare that helps African engineering students gain research experience. You do not need an account or pay a fee to learn.