One dataset.
Three databases.

Workload fit

Score 1–5 from the project's access-pattern analysis.

The dataset

One logical dataset, three physical forms.

• Generated from a fixed seed, so every run sees identical data.
• Skewed on purpose: a few popular products and heavy buyers, like real shops.
• Several sizes from 1M to 100M rows to test scalability.
• Same facts everywhere: orders as tables, products as documents, follows and purchases as edges.

Benchmark protocol

Rules applied to every measurement.

• Discard the first runs (warm-up), then repeat each query many times.
• Report median, p95 and p99, never the average alone.
• Measure cold-cache and warm-cache separately.
• Change one variable at a time: index, size or isolation level.
• Declare indexes before timing and confirm the plan with EXPLAIN, .explain() or PROFILE.
• Use the same machine, same client and same dataset for all three.

Why warm-up runs are discarded

Illustrative: the first runs pay for cold caches. The pink points are excluded.

Mean vs percentiles

Illustrative: one cold run drags the average up; median and tails describe the steady state.

Cold-cache vs warm-cache

Illustrative: first touch pays for disk; repeats run from memory (ms, log scale).

Step 1 · Train

Predictors + calibration, SHAP, distributions.

python train_models.py --fast

Step 2 · Distill

In-page calculator weights (≈3 KB).

python export_js.py

Step 3 · Launch

Predictor workbench, seven tabs.

streamlit run streamlit_app/ml_app.py

Step 4 · Open

Calculators work offline; the frame embeds the live app.

open index.html

Step 5 · Verify

Calculators answer; charts filled; app boots clean.

checklist in §Methodology

Step 6 · Publish

Fill test-env rows; swap sample numbers; deploy + paste app URL.

git commit -am "publish" && git push

Schema (BCNF)

Keys and relationships.

Revenue by category

Output of the window-function report (sample data, £k).

Index strategy: latency

Same query, before and after indexing (ms, log scale).

Rows examined (EXPLAIN ANALYZE)

Query rewrite + composite/covering index.

Normalization to BCNF

One wide table, decomposed along its functional dependencies until every determinant is a key.

-- Before: orders_wide
(order_id, cust_id, cust_city,
 prod_id, prod_name, qty)
-- Functional dependencies
order_id → cust_id
cust_id → cust_city
  -- transitive: violates 3NF
prod_id → prod_name
  -- partial: violates 2NF
(order_id, prod_id) → qty
  -- full dependence on the key
-- After: BCNF decomposition
customers(cust_id, cust_city)
products(prod_id, prod_name)
orders(order_id, cust_id)
order_items(order_id,
 prod_id, qty)

B+-tree lookup

Point lookup for key 41 traverses root to leaf; linked leaves keep range scans sequential.

Height vs table size at fan-out ≈ 500: level count barely moves as rows explode. Doubling the table rarely adds a level, and linked leaves turn ranges into a sideways walk.

EXPLAIN ANALYZE plan

Cost-selected plan for customer 42's orders; estimates track actuals.

Numbered in execution order: ① scan, ② lookup per row, ③ join. Row estimates feed the choice above them — a wide est/actual gap means stale statistics, so run ANALYZE TABLE and re-check.

Cardinality: estimated vs actual

Fresh histograms keep the optimizer honest (rows per plan node).

Materialized view vs live query

Dashboard query time (ms). MySQL has no native MV, so a summary table is refreshed by an event.

Optimizations applied

Rewrites that changed the plan.

-- push filters before joins
SELECT ... FROM (
  SELECT * FROM orders
  WHERE placed_at>='2024-01-01'
) o JOIN ...
-- subquery → join
-- IN → EXISTS
-- covering index skips
-- table lookups entirely
CREATE INDEX idx_cov
ON orders (
  customer_id,
  placed_at, status);

Join algorithms by table size

Time to join two equal-size tables (ms, log scale).

Storage by normal form

Same data, fewer duplicated values (MB).

Measured: indexing impact

Average time for 300 customer lookups on 200k rows, measured live (ms per query).

Try these

Things worth experimenting with.

-- 1. Run the plan with and without an index:
CREATE INDEX i1 ON orders(customer_id);
-- 2. Compare SCAN vs SEARCH in the plan.
-- 3. Rewrite the window query as a self-join
--    and compare the time.

Product document

Embedded specs and top reviews.

{ "_id": "p_1042",
  "name": "Trail Runner X",
  "category": [
    "Shoes", "Running"],
  "specs": {
    "weight_g": 280,
    "drop_mm": 6 },
  "reviews": [
    { "user": 88,
      "stars": 5 } ],
  "price": 89.0 }

Aggregation pipeline

Top-rated products with 50+ reviews.

db.products.aggregate([
  { $unwind:
    "$reviews" },
  { $group: {
      _id: "$_id",
      avg: { $avg:
        "$reviews.stars" },
      n: { $sum: 1 } } },
  { $match: {
      n: { $gte: 50 } } },
  { $sort: { avg: -1 } },
  { $limit: 10 } ])

Documents surviving each stage

Sample run over 120k products.

Index impact on query time

Collection scan vs index (ms).

Reviews per product

Most products have few reviews; a few have many (sample).

Time per pipeline stage

Where the aggregation spends its time, ms (sample).

Embedded vs referenced reads

Read latency as the parent document grows (ms).

Storage by collection

Share of the 7.6 GB MongoDB footprint.

Cypher

Recommendations within 3 hops.

MATCH (me:Customer
 {id:$id})-[:FOLLOWS*1..3]
 ->(f)-[:BOUGHT]->(p)
WHERE NOT (me)
 -[:BOUGHT]->(p)
RETURN p.name,
 count(*) AS score
ORDER BY score
DESC LIMIT 10

Top customers by PageRank

Computed live from the hero graph.

Degree distribution

A few hubs, many leaves.

Traversal depth: SQL joins vs Cypher

Time to find friends N hops away (ms, log scale).

Network reach within N hops

Share of the 60 customers reached (sample).

Recommendation hit-rate

Hits in top-10 by method (sample).

Shortest path and centrality

Pattern matching and graph algorithms.

MATCH p = shortestPath(
  (a:Customer {id:1})-[:FOLLOWS*..6]-(b:Customer {id:42}))
RETURN length(p), [n IN nodes(p) | n.id];

MATCH (c:Customer)<-[:FOLLOWS]-(f)
RETURN c.id, count(f) AS inDegree
ORDER BY inDegree DESC LIMIT 10

Graph model

Nodes, relationships, properties.

Degree vs PageRank

Each dot is a customer, computed live. Rank rewards who you are connected to, not just how many.

Customers per hub neighbourhood

Each customer is grouped with its nearest top-5 hub.

Degree distribution, log–log

A heavy tail is the signature of hub-dominated networks. Computed live.

Throughput by isolation level

Transactions per second, 32 clients (sample).

Two-phase locking

Grow, then shrink. Never lock after the first unlock; that guarantees conflict-serializable schedules.

T1: lock-X(A) read(A)
write(A) lock-X(B)
read(B) write(B)
COMMIT unlock(A,B)
-- grow: lock, read,
-- write; shrink: unlock
-- strict 2PL holds
-- X-locks until commit

MVCC read view

Readers see a snapshot and never block writers.

-- Session 1
-- (REPEATABLE READ)
START TRANSACTION;
SELECT bal
FROM acct
WHERE id=1;
-- reads 500
-- Session 2 commits:
-- UPDATE sets bal=400
SELECT bal
FROM acct
WHERE id=1;
-- still reads 500
COMMIT;
-- next read: 400

Deadlocks per 1,000 transactions

Rises with contention (sample).

Throughput vs concurrency

Transactions per second as clients increase (sample).

Abort rate heatmap

Percent of transactions rolled back, by isolation level and clients (sample).

Injection: try it

Type a name. The unsafe query is built by string concatenation; the safe one is parameterised.

Unsafe

Safe (parameterised)

Injection attacks that succeed

Concatenated SQL vs parameterised queries (% of test payloads).

Query latency by workload

Median ms, log scale. Lower is better.

Overall profile

Normalised 0–100. Higher is better.

Scalability

Point-lookup throughput as data grows (ops/s ×1000).

Storage overhead

On-disk size for the same data (GB).

Throughput per GB

Point-lookup kops per GB stored, derived (sample).

Workloads won

Times fastest of the six benchmarked workloads (sample).

Latency percentiles

Tail latency tells a different story than the median (ms).

Where the time goes

Per-query breakdown (ms).

Slowdown vs the best engine

How many times slower each database is than the fastest one for the same workload.

bestdeeper orange = further from best

Latency distribution

Share of queries finished within a given time (empirical CDF, 400 queries each).

Reading guide: steep middle, flat slow tail. Median and p95 mark where half and 95% of queries have finished.

When an index stops helping

Time to fetch a growing share of a 1M-row table. Cost model: random index reads cost about 10x a sequential row.

The model behind the chart above, zoomed out: past about a 10% share the index loses and the scan wins.

Tail penalty: p99 ÷ p50

How much worse the tail is than the median, derived (sample).

Worst mismatch per database

Max slowdown vs the best engine, log scale, derived (sample).

Tail gap in ms

p99 minus p50 per database, derived (sample).

Calculator: high-value customer

Distilled linear model, test AUC . Defaults are population medians.

Predict

Calculator: order amount

Distilled ridge model, test R² , RMSE £.

Predict

Calculator: review rating

Distilled ridge model, test accuracy . Output rounded to 1–5 ★.

Predict

Choose MySQL when

Correctness matters most: money, inventory, multi-row invariants, ad-hoc analytics over normalised data.

Choose MongoDB when

Records vary in shape and are read whole: catalogues, content, events. Scale-out comes built in.

Choose Neo4j when

The question is about relationships: paths, recommendations, influence. Cost follows hops, not table size.