# Compare variants carefully

AI Content Creation | Module 18: Measure and learn

## Your goal

Plan a fair test and avoid overclaiming a small result.

Compare one meaningful difference, such as the opening line, while keeping the offer and action stable. Decide the success metric before seeing the result.

If two posts run at different times for different audiences, the difference may come from timing or audience rather than the opening. Report those limits instead of declaring a winner with certainty.

A small fictional test with two registrations versus three is a weak basis for a broad conclusion. Use it to choose a next test, not to promise future performance.

## Source pack

FICTIONAL PRACTICE BRIEF: Green Bench is a student repair club. It runs a free notebook-cover repair workshop on Saturday, 14:00 to 16:00, in Room B. There are 12 places. Learners bring one worn notebook; tools and scrap card are provided. Register using the club form. No date, form URL, access arrangements, impact figures, or testimonials have been supplied. Ask the organiser about missing details before publishing. This is a creative exercise, not a real event.

## Worked example

Plan a comparison of two workshop hooks and state its limits.

### Test plan

Compare a problem-led hook with a materials-led hook while keeping the workshop facts, visual, and registration action unchanged.

### Measure

Use registrations per landing-page visit in the same observation window, and record how traffic was allocated to each version.

### Limits and next step

Different audiences or small samples could explain the result. Repeat a controlled comparison before making a strong claim about the hook.

## Your work

Plan a comparison of two workshop hooks and state its limits.

### Test plan

Write at least 8 words.


### Measure

Write at least 8 words.


### Limits and next step

Write at least 8 words.

## Review your work

- [ ] I changed one main variable.
- [ ] I chose the metric in advance.
- [ ] I named plausible alternative explanations.

Completeness checks do not assess creative quality or factual accuracy. Review the actual work yourself or with a mentor.

## Quiz

1. Two posts get different results at different times. What can you conclude?
   A. The opening caused the difference for certain.
   B. The results differ, but timing may explain the difference.
   C. The better post will always win.

2. Variant B gets more clicks but fewer registrations than A. What should guide your decision?
   A. The campaign goal and rates, with their limitations.
   B. Clicks alone regardless of the goal.
   C. The more colourful design automatically wins.

## Quiz explanations

1. B: An uncontrolled comparison does not isolate the cause.

2. A: The chosen metric should match the intended outcome.

## Further reading

[Anthropic: prompt engineering overview](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview)