Clarify the task and decision
This guide turns case study evidence framework into a reviewable operating workflow. It connects domain decisions, ownership, evidence, and acceptance so the result continues to work in production.
Plan evidence before rollout so the published case study distinguishes baseline, intervention, outcome, attribution, and limitations.
Practical workflow
- 1
Establish a baseline from observed content volume, effort, delay, quality, support demand, and risk.
- 2
Define the target operating model, audiences, channels, ownership, integrations, and review standard.
- 3
Model cost and benefit with named data sources and separate confirmed values from assumptions.
- 4
Run a representative pilot with agreed acceptance measures and a decision date.
- 5
Approve scale-up only after owners accept the operating process, evidence, budget, and reporting cadence.
Worked example or tool
A publication template links every claim to a metric definition, source, owner, period, and approval. In the tool, also record the baseline, owner, decision, evidence, open issue, and approval date. Use a real page or transaction so the team sees dependencies, exceptions, and the maintenance work that follows release.
| Decision point | Record | Acceptance criterion |
|---|---|---|
| Baseline | Observed current state | Source and date recorded |
| Decision | Selected option and rationale | Risk and audience considered |
| Evidence | Test, document, or measure | Reviewable and version-specific |
| Approval | Name, role, and date | All mandatory criteria met |
Plan evidence before delivery
State the decision the case study should inform, the audience, intervention, expected outcomes, and plausible alternative explanations. Register measures, sources, collection dates, owners, and publication permissions before implementation.
Choose representative content and journeys, not only the strongest example. Record exclusions and failed attempts so the publication reflects normal operation.
Write the evaluation question.
Freeze baseline measures and definitions.
Secure data and quotation permissions.
Record selection and exclusion rules.
Build a comparable baseline
Measure time, effort, quality, corrections, task success, confidence, support demand, and accessibility where relevant. Preserve raw observations and calculation code or formulas.
Compare like with like across content complexity, team, channel, and season. If a controlled comparison is impossible, use repeated measurements and explain external changes.
- 1
Define every metric precisely.
- 2
Collect enough cases to show variation.
- 3
Keep source records and timestamps.
- 4
Review baseline quality independently.
Analyse outcomes with limitations
Report absolute values, change, sample size, range, and missing data. Separate observed association from a causal claim. Check whether adoption, training, staffing, policy, or demand influenced results.
Include adverse outcomes such as extra review, rejected text, incidents, or groups that benefited less. A balanced account is more useful and credible than perfect-looking averages.
Reproduce every published number.
Explain missing and excluded data.
Test alternative explanations.
Publish relevant limitations beside findings.
Publish a reusable, governed case study
Use a clear structure: context, baseline, intervention, implementation, results, limitations, lessons, and next steps. Link claims to evidence and label customer quotations distinctly.
Obtain factual, legal, privacy, accessibility, and participant approval. Date the study and define review triggers when the product, workflow, measures, or customer situation changes.
- 1
Run independent fact checking.
- 2
Confirm consent and anonymisation.
- 3
Provide accessible tables and definitions.
- 4
Archive evidence and schedule review.
Create a publication record that survives scrutiny
Prepare a claim register before drafting the narrative. For every proposed statement, record the exact source, calculation, sample, time period, responsible analyst, verification status, and wording limitations. Keep screenshots, interview notes, exports, survey instruments, scripts, and approval messages in a controlled evidence folder. Give files stable identifiers so a reviewer can move from a chart or sentence to the underlying record without relying on the original project team.
Use an analysis table that shows numerator, denominator, missing cases, exclusions, range, and comparison period for each metric. Report both counts and percentages when small samples could make a percentage look more impressive than it is. If qualitative evidence is coded, publish the coding categories and have a second reviewer examine a meaningful sample. Reconcile quotations against recordings or approved notes and never alter their meaning to make the story smoother.
Design the case study for reuse by another organisation. State context such as service type, team size, content volume, audience, starting maturity, implementation duration, review model, integration method, and constraints. Describe the mechanism that produced each result, not only the result itself. Readers can then judge whether their conditions are similar enough to expect the same outcome. Include practical artefacts such as a measurement table, acceptance criteria, and review cadence rather than turning the publication into promotional testimony.
Before release, conduct a challenge review with someone who did not deliver the project. Ask which claims are unsupported, which alternative explanations remain plausible, where selection bias may exist, and whether adverse results are represented fairly. Record the response to each challenge. If evidence changes after publication, correct the web edition transparently and retain a dated revision history so citations remain understandable.
Link every public claim to a stable evidence record.
Show counts, percentages, exclusions, and missing data together.
Describe operating context so readers can judge transferability.
Complete an independent challenge review and retain revision history.
Make the published method reproducible
Add a reproducibility note describing available data, protected material, calculation checks, and a contact for methodological questions. When raw records cannot be shared, publish aggregated values, blank collection instruments, field definitions, and a worked calculation. This lets readers inspect the method without exposing participants or confidential operations.
State how later corrections will be handled. Preserve dated versions, explain material changes, and identify whether an update affects data, method, interpretation, or wording. A stable revision record keeps earlier citations understandable and prevents a corrected result from silently replacing the evidence originally reviewed.
Publish definitions, instruments, and a worked calculation.
Explain protected evidence and available substitutes.
Provide a methodological contact and correction process.
Retain dated versions after material updates.
Roles, evidence, and approval
A credible business decision remains useful after the presentation. Store assumptions with an owner, source, date, range, and sensitivity. Report quality and service outcomes alongside cost. Do not count benefits twice, and do not treat generated volume as reader value. The accountable owner should review actual results against the baseline after the pilot and at regular operating intervals.
Operations and maintenance
The work does not end at publication. Link the language version or configuration to its source, monitor quality and service measures, and define concrete review triggers. Triggers include source changes, legal changes, new audience needs, recurring support questions, technical changes, and incidents. A named owner evaluates the trigger, opens a new revision when needed, and records renewed approval.
Release checklist
The baseline uses observed data.
Audience and service outcomes are measurable.
One-time and recurring costs are separated.
Assumptions have owners and sensitivity ranges.
Quality, accessibility, security, and integration work are included.
Pilot acceptance criteria are agreed in advance.
Benefits are not double-counted.
The scale-up decision and reporting cadence are assigned.