LabCheck

A checker for liquid-handling protocols.

A liquid handler does what the protocol says. If the protocol asks a 20–300 µL pipette for 15 µL, or takes 60 µL from a tube with 40 in it, the robot just goes ahead. Usually you notice when the plate looks wrong.

LabCheck runs the protocol in software before the robot does. It keeps count of the liquid in every tube and well and lists the steps that can't work as written. You don't need a robot or any lab data to use it.

What it checks

1. PIPETTE_RANGE: the volume is within the pipette's range.
2. REAGENT_EMPTY: the source still has enough left when the step runs.
3. WELL_OVERFLOW: no well ends up over its capacity.
4. WELL_MISSING: the well exists on that plate (there is no I1 on a 96-well plate).
5. CONTROL_MISSING: the negative and positive control wells get filled.
6. DEAD_VOLUME: optional. A source isn't emptied below the volume the pipette can't reach. Off in the figure above.

Each failure is listed with its step number and a one-line reason.

Where this comes from

I'm building this over four weeks. It starts from the Laboratory Automation Protocol format in this ACS Synthetic Biology paper. First I reproduce the paper's format, then I add the rules, then I break protocols on purpose and count how many the checker catches. The plan is on the report page, sample protocols and reference numbers are on data, and the benchmark so far is on results.

To make your own protocol, click wells on the Build page, or paste JSON into Try it. The glossary explains the lab words.

Method

Protocol in, verdict and list of failures out. See the report page for the full methodology including protocol schema, simulation algorithm, rule trigger conditions, and evaluation design.

1. Parse. Read reagents, volumes, labware and steps from YAML or JSON, using the LAP format.
2. Simulate. Go through each transfer and update the volume in every source and well.
3. Check. Test every rule after each step.
4. Report. List the step, the rule and why it failed.

Protocol schema (JSON)

{
  "plate": 96,
  "reagents": {"buffer": 1000, "dye": 40, "sample": 120},
  "pipette": {"min": 20, "max": 300},
  "well_capacity": 200,
  "dead_volume": 0,
  "controls": {"A1": "negative", "A12": "positive"},
  "steps": [
    {"from": "buffer", "to": "A1", "ul": 100},
    {"from": "dye", "to": "B1", "ul": 20}
  ]
}

Simulation algorithm

stock = reagents.copy()
wells = {}
for step in steps:
  stock[step.from] -= step.ul
  wells[step.to] = (wells[step.to] || 0) + step.ul
  check_rules(step, stock, wells)
check_controls(wells, controls)

Rules

An example step

reagents:
  buffer: {labware: reservoir, well: A1, ul: 1000}
  dye:    {labware: tubes,     well: A1, ul: 40}
controls: {negative: A1, positive: A12}
steps:
  - id: 4
    transfer: {from: dye, to: B1, ul: 15}
# step 4 fails: 15 µL is below the 20 µL minimum

Evaluation design

Error injection: start from valid generated protocols, inject one known mistake per protocol, measure whether the corresponding rule fires. Two injection types (WRONG_REAGENT, WRONG_WELL) have no dedicated rule—any detection is incidental.

Benchmark generation: 8–24 steps, 3 reagents (buffer 4000µL, sample 800µL, dye 400µL), random well/reagent/volume respecting all constraints.

Metrics: detection rate per error type (Wilson 95% CI), multi-error detection (all injected codes reported), false alarm rate on clean protocols, timing.

Real-protocol panel: four transcribed OT-2 protocols (serial-dilution subset and full run, restriction digest, PCR distribution) with the libraries' own volumes, labware and pipettes, checked as transcribed — any report counts as a false alarm on real data. Retrospective: two shipped library faults — the 649b67 example input overdraw (caught: REAGENT_EMPTY) and the 052d03 diluent-planning fix (miss: reservoir-well planning has no rule yet). Sources and transcription notes are on the data page.

Statistical methods

Rates are binomial proportions (caught / n). Intervals are Wilson score intervals at 95% (z = 1.96):

(p + z²/2n ± z·√(p(1−p)/n + z²/4n²)) / (1 + z²/n)

I use Wilson rather than the textbook Wald interval (p ± z·√(p(1−p)/n)) because Wald collapses to zero width at 0% and 100% — exactly where most of these rates sit — and under-covers near the edges. Wilson holds ~95% coverage across the range. No p-values are reported; with n = 300 the intervals speak for themselves.

Reproducibility: base protocols come from one pseudorandom stream; each injected fault type draws from its own independent stream derived from the seed. Same seed and count replay the same tables bit-for-bit. The Atlas page re-runs five seeds to show the numbers do not depend on a lucky draw.

Complexity

Checking is linear: each step does constant work (update two counters, test six rules), so time is O(S) in the number of steps and memory is O(W) in filled wells plus O(R) reagent stocks. The full cost table is on the atlas page. Timing uses the median of repeated runs to resist garbage-collection pauses.

Threats to validity

• Generator realism. Protocols are uniform-random subject to constraints, not lab-like: no dilution series, no replicates layout, no plate-edge avoidance. Rates measure code correctness, not field performance.
• Fault realism. Faults are synthesised by someone who knows the rules. A mistyped well that lands on another valid well (no rule) is under-represented relative to real typos.
• Single-fault bias. Real protocols can carry several faults; multi-error runs are covered but only up to five, all of distinct types.
• Oracle problem. There is no independent oracle: the generator guarantees validity by construction, so "false alarms on clean protocols" tests the generator as much as the checker.
• Transcription fidelity. The real protocols are hand-transcribed subsets: well-to-well serial transfers are outside LabCheck v1's reagent → well model, per-sample tube identities are collapsed into one stock, and a two-pipette run is modelled as one combined range. Each approximation is documented on the data page, but a mistranscription would look exactly like a false alarm or a miss.
• Browser timing. Millisecond timings depend on the machine and tab state; treat them as order-of-magnitude.

What it will not catch

It knows nothing about viscosity, evaporation, collisions on the deck, timing, or whether the experiment makes sense biologically. A pass only means the protocol agrees with itself. The assay could still fail.

Planned stack: Python, pytest, YAML/JSON protocol files, and the Opentrons API where it fits.

Report

LabCheck: a static checker for liquid-handling protocols. Version 1.0 — October 2026.

Abstract

Liquid handlers run protocols exactly as written, so a mistake in the protocol becomes a mistake in the experiment. LabCheck simulates a structured protocol, checks it against six rules before it runs, and explains each failure. I measure how well it works by injecting known errors into valid protocols. The browser implementation validates protocols in under 2 ms for typical workflows (≤25 steps) and catches >95% of injected errors for the five rule-covered failure modes.

Background

The Laboratory Automation Protocol (LAP) format, described in ACS Synthetic Biology (2023), gives automated experiments a shared structure: reagents with volumes, labware definitions, pipette specifications, control wells, and an ordered list of transfer steps. LabCheck adds a validation step on top of LAP. It is meant for lab automation and assay development, where the run only works if the protocol was right to begin with.

Methodology

Protocol representation

A protocol is a JSON or YAML document with these fields:

• plate: 96 or 384 (defines well grid and default capacity)
• reagents: object mapping reagent name → starting µL (e.g., {"buffer": 1000, "dye": 40})
• pipette: {min, max} in µL (default 20–300 for P300)
• well_capacity: maximum µL per well (default 200)
• dead_volume: µL the pipette cannot reach at bottom of source (default 0, optional)
• controls: object mapping well → "negative" or "positive" (default A1 negative, A12 positive)
• steps: array of {from, to, ul} where from is a reagent name, to is a well (e.g., "B3"), ul is volume

Simulation engine

The simulator maintains running state for every source and well:

1. Initialize stock[r] = starting volume for each reagent
2. Initialize wells[w] = 0 for each well on the plate
3. For each step (from, to, ul) in order:
   1. Record pre-step state (stock, wells)
   2. stock[from] -= ul
   3. wells[to] += ul
   4. Run all six checks against the new state
   5. Record any violations with step index, rule code, and human-readable message
4. After all steps, check that every control well in controls received ≥1 µL

Validation rules

Error injection for evaluation

To measure detection rates without real-world error data, I start from valid generated protocols and inject one known mistake per protocol:

• PIPETTE_RANGE: change a step's volume to 1–18 µL (below min) or 301–400 µL (above max)
• REAGENT_EMPTY: reduce the dye stock so total draws exceed it by 1–20 µL
• WELL_OVERFLOW: set a buffer transfer to 300 µL into a single well
• WELL_MISSING: change a well to I1, A13, J4, H0, K12, or Z9
• CONTROL_MISSING: remove all steps targeting A1 or A12
• WRONG_REAGENT (no rule): swap the reagent in one step to a different one
• WRONG_WELL (no rule): move one step's dispense to a different valid well

The last two have no dedicated rule; any detection there is incidental (the swap happens to cause overflow, empty source, or missing control).

Benchmark protocol generation

Valid protocols are generated by randomly sampling transfers that respect all constraints:

• 8–24 steps per protocol (uniform)
• Reagents: buffer (4000 µL), sample (800 µL), dye (400 µL)
• Each step picks a random well (A1/A12 forced for first two steps as controls), random reagent, volume 20–120 µL in 5 µL increments, respecting remaining stock and well capacity
• If a random choice would violate a constraint, it is rejected and re-sampled (max 600 attempts)

Evaluation

Error injection gives a detection rate for each error type, and clean protocols give the false alarm count. Full results with 95% Wilson intervals are on the results page. Visualizations of detection rates, timing, and limit sensitivity are on the analysis page. The real-data evaluation transcribes published OT-2 protocols and shipped library faults: four valid protocols with zero false alarms, one in-scope historical fault caught, one out-of-scope miss kept visible; sources and transcription notes are on the data page.

Limitations

The checker only finds what its rules describe. It doesn't model liquid properties (viscosity, evaporation, foaming), deck layout (tip rack positions, labware coordinates, collisions), timing (incubation, settling), or biology (whether the assay design makes sense). A pass only means the protocol is internally consistent with respect to the six rules—nothing more.

Plan

Log

Glossary

Lab words used on this site.

Assay

A lab test that measures something, such as how much of a molecule is in a sample.

Dead volume

Liquid left in a tube or reservoir that the pipette cannot pick up.

Error injection

Adding one known mistake to a correct protocol to see whether a checker notices it.

False alarm

The checker flags a protocol that was actually fine.

LAP (Laboratory Automation Protocol)

A structured way to describe automated experiments, from the paper this project starts with.

Liquid handler

A robot that moves measured volumes of liquid between containers.

Negative control

A well that should show no signal, so you can see the background level.

Pipette range

The smallest and largest volumes a pipette can measure reliably, for example 20–300 µL.

Plate

A tray of small wells. A 96-well plate has rows A–H and columns 1–12. A 384-well plate has rows A–P and columns 1–24.

Positive control

A well that should show a signal, so you know the assay can work.

Protocol

The ordered list of steps an experiment follows: what to move, how much, and where.

Reservoir

A trough holding a bulk reagent that the pipette draws from.

Well

One small cup in a plate, named by row and column, such as B3.

Boundary test

A protocol sitting exactly on a limit (20 µL, 200 µL, stock exactly spent). Must pass; one unit beyond must fail.

Calibration

Checking that a measurement tool reads correctly at known values. Here: the checker at known protocol limits.

Confidence interval

A range that plausibly contains the true rate. All intervals here are Wilson 95% intervals.

Coverage

How often a confidence interval contains the truth over repeats. Wilson intervals hold ~95% coverage; the simpler Wald interval does not at extreme rates.

Incidental detection

A fault with no dedicated rule that still gets flagged because it causes a different rule to fire.

Jaccard index

Overlap of two sets divided by their union. Used for protocol similarity on step tuples.

Median of runs

The middle timing of repeated runs, robust to one slow run from garbage collection or tab throttling.

Must-pass case

A protocol variant exactly on the allowed side of a limit. Any report on it is a false alarm by construction.

Seed

The starting number of a pseudorandom generator. Same seed replays the same protocols and faults.

Specificity

Share of clean or must-pass cases correctly left unflagged. The complement of the false alarm rate.

µL

Microlitre, a millionth of a litre.