Experiment Decision Memo You Can Download (2026)
By William Zhu & the InfiniSynapse Data Team · Published: 2026-08-22 · Last updated: 2026-08-31 · Last verified: 2026-08-31 · Next review: 2026-11-30 · About · Privacy policy · Editorial standards · Corrections
Table of Contents
- TL;DR
- What an Experiment Decision Memo Must Contain
- A Memo-Field Framework You Can Replay
- How Teams Compare Artifacts and Screenshots
- Tool Landscape for Downloadable Memos
- Implementation Steps from Tables to a File
- Accuracy and Experience Record: Illustrative Memo Pack
- Evidence Boundaries and Independent Validation
- How to Cite This Page
- Selection Scorecard for Memo Stacks
- Failure Modes That Leave Only a Chat
- Frequently Asked Questions
- Conclusion
TL;DR
We evaluate these patterns at the InfiniSynapse desk on sanitized composites; sample figures on this page are illustrative, not customer uplifts.
Direct answer: An experiment decision memo is the downloadable artifact: assignment integrity, primary, interval, guardrails, CUPED on or off, attached SQL, and a human ship / hold / iterate line. The p-value is one row, not the product.
What you'll learn: why the experiment decision memo is the object you keep; a field table; how chat paragraphs fail audit; how to bind metric notes; an illustrative hold pack; and the breaks that leave only a screenshot.
Download evidence: illustrative memo · decision CSV · verification script · source check · reproduction protocol. This package is first-party and illustrative—not customer, production, randomized-trial, benchmark, or third-party evidence.
An experiment decision memo is not a Slack verdict. It is a file a skeptic can reopen next quarter. If the SQL is missing, you have a slide.
What an Experiment Decision Memo Must Contain
Key Definition: An experiment decision memo is a signed, downloadable record of a frozen test—unit, primary, guardrails, peeking, CUPED state, queries, and a recommended action—that a human still accepts or rejects. It is not a chat paragraph that says “ship.”
OECD materials (retrieved 2026-09-04) provide official-publication context. Pairing a definition with a number is an analogy, not endorsement of this memo.
For direct statistical interpretation, consult the ASA Statement on Statistical Significance and P-Values and NIST/SEMATECH e-Handbook of Statistical Methods (retrieved 2026-09-04). Neither reviewed this page or run.
An experiment decision memo sits under the parent method in A/B test analysis. This page is narrower: the artifact, not the assignment-first order. For that order, use experiment analysis. For the optional cut, use CUPED explained.
The p-value is a line
A significance badge answers a narrow question. An experiment decision memo answers a broader one: did we measure the unit we randomized, did we harm a kill, did we peek, and who ships? The badge can be in the table. It cannot be the title.
Eurostat materials (retrieved 2026-09-04) provide official-statistics context. Vintage and revision notes are a publication analogy only.
Downloadable means inspectable
If you cannot download the experiment decision memo with the SQL that produced it, you cannot audit it. IEC materials (retrieved 2026-09-04) provide standards context but do not prescribe this memo.
A Memo-Field Framework You Can Replay
Every experiment decision memo should fill this table before anyone treats the last sentence as a launch.
| Field | What you lock | Typical source | Failure if skipped |
|---|---|---|---|
| Assignment | Unit, variant, time, SRM | Assignment log | Dual-exposed users |
| Primary | One metric, one window | Outcome + note | Metric shopping |
| Guardrails | Named kills + availability | Adjacent tables | Silent harm |
| Peeking | Horizon or look log | Design note | First-green-day |
| CUPED | Covariate or off | Pre-period | Hidden toggle |
| Query | SQL or equivalent | Task trail | Slide-only |
| Owner | Human action | Signature | Chat as approval |
An experiment decision memo is the filled table, not a narrative that hides the gaps. arXiv (retrieved 2026-09-04) hosts preprints; repository presence alone is not peer review.
NSF materials (retrieved 2026-09-04) provide research-program context. The reopenable packet is an analogy, not NSF approval.
How Teams Compare Artifacts and Screenshots
Teams argue tools. They should argue what remains after the meeting. An experiment decision memo beats a screenshot because the query travels with the claim.
| Artifact family | Works when | Breaks when |
|---|---|---|
| Downloadable memo + SQL | Skeptic can replay | File is a screenshot of a chat |
| Vendor UI export | Export includes query + vintage | Project ends, UI disappears |
| Notebook | Pinned revision, named owner | Local-only, no metric note |
| Chat verdict | Never, as the record | Used as the ship order |
Chat is a draft
An agent can draft the experiment decision memo. That draft is not the decision. AI for data analysis is a professional read, not ChatBI. The human still names the action.
Screenshots age badly
A badge without the join will not survive a finance review. An experiment decision memo that attaches the query and the metric note will. Explainable AI data analysis is the habit for opening the trail.
Tool Landscape for Downloadable Memos
You do not need a new warehouse to produce an experiment decision memo. You need assignment, outcomes, a bound metric note, and a workspace that keeps the file. A dated CSV is valid if the memo stamps the file date.
A data agent can write the memo and keep the SQL in the task trail. InfiniSynapse does that on sources you authorize. It does not auto-write the ship decision into production, and it does not ship a prebuilt experiment warehouse.
Tables plus a knowledge-base note
“Conversion” and “refund” are sentences. Bind them. An experiment decision memo that lets each teammate redefine conversion will fork. Data knowledge base is the hop when the note is the missing object. Data governance at experiment grain is the same bind.
Files you can keep
Markdown or PDF plus the query is enough. An experiment decision memo can live in a task workspace next to charts and extracts. If you need the join itself, continue in analyze experiment results in SQL. If you need kill lines inside the file, use guardrail metrics.
Implementation Steps from Tables to a File
Start from the design note, not from a winner adjective. An experiment decision memo that starts at “ship” will omit the gaps.
Freeze fields, then ask for the file
Confirm unit, primary, kills, peeking, and CUPED state. An experiment decision memo cannot invent a missing refund column. Write “not measured.” Ask for the downloadable memo with SQL. Open the joins. If assignment time was replaced by first-event time, reject the file.
Reuse the same field list on the next test so conversion does not drift. If the extract is a file, stamp the vintage in the header. If the extract is a live database, write the query time. Those two stamps are how a later reviewer knows which rows you saw.
Separate the paragraph from the signature
The last line is a human action: ship, hold, or iterate. An experiment decision memo does not ship itself. Write the owner. If marketing wants a public claim, raise the evidence bar before copy ships.
For horizon math behind the file, see A/B test sample size. For durable reuse of the same note, organizational analysis memory is the hop when the next test should not redefine conversion.
Accuracy and Experience Record: Illustrative Memo Pack
The following numbers are an illustrative desk composite, not a customer result or uplift claim. Run ID: EDM-HOLD-20260823. Run date: 2026-08-23. Operator: InfiniSynapse Data Team. Objects recorded: assignment split, primary, refund guardrail, named but unreproducible CUPED covariate, five aggregates, and two held items.
| Item | Desk composite (illustrative) |
|---|---|
| File | Markdown memo + SQL, dated extract |
| Units | 86,000 users; 50.1% / 49.9% |
| Primary | Checkout completion +1.8 pp (illustrative) |
| Guardrail | Refund rate +0.4 pp (illustrative) |
| CUPED | Prior 28-day orders; both lines shown |
| Peeking | No unofficial looks recorded |
| Decision | Hold; owner named; iterate retry copy |
An experiment decision memo on this pack is useful because the hold and the SQL are in the same file. A chat that said “ship the +1.8” would have dropped the kill. Keep the file next to the extract. If someone later asks why the launch waited, the refund line and the owner name are already there.
Figure. Desk composite from this page: 86,000 users; +1.8 pp primary, +0.4 pp refund; CUPED both lines in the memo. Published context: oecd.org; ec.europa.eu; iec.ch. Not a customer experiment, SLA, or official benchmark.
| Evidence class | What you can cite | What you cannot claim |
|---|---|---|
| Desk composite on this page | Fields, file shape, hold | Customer uplift or vendor bake-off |
| Published public sources above | Definitions, vintages, records | That those bodies ran this desk pack |
Desk composite: +1.8 pp primary, +0.4 pp refund → hold in the downloaded file. Context: OECD indicators, Eurostat vintage, IEC records, arXiv methods, NSF packets.
The downloadable illustrative memo records Hold. The CSV exposes five aggregates, a CUPED calculation held as unavailable, and an adverse-refund hold.
Evidence Boundaries and Independent Validation
This is not customer, production, randomized-trial, peer-reviewed, benchmark, representative, or causal evidence. Assignment/outcome rows, metric definitions, SQL, interval bounds, p-values, variances, CUPED calculations, look log, and real signature evidence are unavailable.
The 86,000 users are not a disclosed sampling frame. The 50.1%/49.9% split and 1.8-/0.4-point deltas cannot be independently recomputed. “No unofficial looks recorded” means no look log is disclosed; it does not prove peeking was absent.
The source check distinguishes statistical references from publication analogies. The open protocol specifies an external test. As of 2026-08-31, no qualifying independent report, statistical peer review, customer validation, or media investigation exists.
The output checker confirms displayed labels and values only. It does not establish randomization integrity, significance, CUPED validity, causation, authorship of SQL, signature authority, or commercial impact.
An experiment decision memo records assignment. An experiment decision memo names metrics. An experiment decision memo exposes guardrails. An experiment decision memo declares peeking. An experiment decision memo names CUPED. An experiment decision memo attaches queries. An experiment decision memo preserves limitations. An experiment decision memo requires ownership.
How to Cite This Page
Page: Zhu, W., & InfiniSynapse Data Team. (2026). Experiment decision memo you can download. InfiniSynapse. https://infinisynapse.com/en/blog/experiment-decision-memo
Artifact: InfiniSynapse Data Team. (2026). Experiment decision memo EDM-HOLD-20260823 (illustrative). https://infinisynapse.com/blog-media/experiment-decision-memo/downloads/experiment-decision-memo-EDM-HOLD-20260823.md
Neither is an independent audit, customer experiment, randomized trial, peer review, benchmark, signed production decision, or proof of lift. Cite unavailable rows, SQL, intervals, look log, signature evidence, held CUPED calculation, and first-party limitations.
Selection Scorecard for Memo Stacks
Score from 1 to 5. An experiment decision memo that cannot be downloaded with SQL should not win on a prettier badge.
| Criterion | What “5” looks like | Disqualifier |
|---|---|---|
| Completeness | All contract fields present | Primary-only paragraph |
| Attachments | SQL + vintage + note | Screenshot of a chat |
| Guardrails | Availability written | Missing = fine |
| CUPED honesty | Named or off | Hidden toggle |
| Replay | Skeptic can rerun | Local-only notebook |
| Decision rights | Named human owner | Model “recommends ship” |
An experiment decision memo scores well when finance can reopen it next quarter. It scores poorly when the stack implies a prebuilt experiment warehouse you do not operate.
Failure Modes That Leave Only a Chat
Write the break in the memo if it happened. Reviews go faster when missing fields are explicit.
Chat as the record
A winner paragraph in a thread will disappear. An experiment decision memo that was never downloaded is not an artifact. Keep the file in the task workspace.
SQL omitted
A claim without a join is a slide. An experiment decision memo must attach the query or an equivalent inspectable plan. If the agent drafted SQL, read the window. If you wrote it, still attach it.
Owner left blank
The last line without a name is not a decision. An experiment decision memo needs a human who can accept residual risk. The agent does not ship.
A fourth pattern is attaching a screenshot of the badge and calling it the memo. The badge is a line. The file is the product.
Before you open a workspace, check four things: field list, metric note, assignment timestamps, and a named owner slot. If those four are missing, a tool will still produce a confident paragraph.
Route the same diagnosis to the live guide that owns the next object. Each row is a single hop.
| Live guide | Open it when |
|---|---|
| AI data report generator | you need a downloadable analysis pack |
| dashboard | someone wants tiles instead of a memo |
| self-service data analysis for business | a non-analyst will request the file |
Download the decision memo from the last run
Upload a sanitized assignment-and-outcome extract, bind the metric note, and ask to download the decision memo with SQL from the last run. This check uses only sources you authorize.
Commercial association: You do not need the workspace to complete the educational diagnosis on this page.
Open InfiniSynapseHow this page is sourced. William Zhu is cofounder of InfiniSynapse (GitHub @allwefantasy); no personal LinkedIn, statistics credential, experiment-platform affiliation, or independent reviewer role is claimed. His profile establishes authorship, not statistical qualification. Desk decisions are recorded in artifact EDM-HOLD-20260823. Reviewed internally by analytics engineering · data platform · LLM security · editor. Editorial standards · corrections · publishing principles. COI: InfiniSynapse sells an AI-native Data Agent. ASA, NIST, OECD, Eurostat, IEC, arXiv, and NSF did not validate this artifact.
Frequently Asked Questions
Is a p-value enough if the memo is short?
Bottom line: No. An experiment decision memo is the artifact; the p-value is a line. You still need assignment integrity, guardrails, peeking, CUPED state, SQL, and a named owner. A short memo that omits a kill is a slide.
Can I keep the memo as a chat transcript?
Bottom line: Not as the record. An experiment decision memo must be downloadable with the query. Chat can be the draft. Promote the file. Freeze the extract date so later appends do not silently rewrite history.
What if a guardrail column is missing?
Bottom line: Write “not measured” in the experiment decision memo. Do not infer zero. A hold is allowed. Bind the definition when the column arrives and rerun. The file should show the gap, not hide it.
Who signs if an agent wrote the draft?
Bottom line: A human with launch authority. An experiment decision memo can be drafted by an agent. It cannot accept residual risk. Write the owner. If marketing wants a public claim, raise the evidence bar before copy ships.
Can readers reproduce the figures in the downloadable artifact?
Bottom line: No. Rows, SQL, intervals, and CUPED calculations are unavailable. The artifact and CSV expose assumptions, five aggregates, and two held items—not a reproducible experiment.
Has an independent statistician audited this artifact?
Bottom line: No qualifying external report is published as of 2026-08-31. The open protocol defines the data, diagnostics, calculations, and review required.
Conclusion
An experiment decision memo is the file you keep: fields, SQL, vintage, and a human line. The p-value is not the product. Hold when a kill moves or a join is missing. Keep CUPED named or off. Do not let a chat paragraph ship the variant.
When the assignment table and the metric note are ready, download that memo from an authorized run at https://app.infinisynapse.com/. Open the SQL, keep the file, and reuse the same fields on the next test.