Irrational Solution

Sample report

Production Triage report

Covers one system: the GreenRoom V1 to V2 migration, reviewed before cutover.

System
GreenRoom V2, in private beta release
Timing
Before cutover
Engagement
Three business days, one system
Decision owner
Founder or CTO
Prepared by
Eric Henschel
Question
Is the V1 to V2 migration safe to execute?

SampleNot a client engagement. This report reviews GreenRoom V2, a professional network for the music industry built by GreenRoom Technologies Inc.

The findings describe the migration plan as it stood at review, and section 5 shows the fixes built since. V1 is still live and the cutover is pending. Vendors are named generically, as in any report shared beyond its client. Client reports follow this structure, scoped to your system and your decision.

Summary

Do not cut over on the plan as it stood at review.

The target design, a Postgres graph in four layers, is sound. The way there was not. Two critical risks put data that cannot be re-created at stake: sign-in continuity rested on an identity bridge nobody had checked, and the ETL could not safely be re-run.

The fix plan has four items. F1 to F3 must land before cutover. F4 ships alongside them, so discovery, the feature V2 is built around, is protected from day one.

Risks by severity

  • 2critical
  • 2high
  • 1medium

System snapshot

Two systems run side by side until cutover.

V1 (live)

Serverless API

Document storeidentity, payments, bookings

Hosted auth provider

Payments

V2 (private beta release)

Web app and API

Postgres graphfour layers, entities to intent

V2 auth

Discoverynatural-language search over profile data

Account history and sign-in continuity both have to survive the cutover.

Findings, ranked by risk

Five risks, ranked by blast radius and likelihood. In a client report each one cites file paths, queries and run output; here the evidence is summarized.

Risk register
IDRiskSeverityFixed by
R1 Sign-in continuity could break at cutover Critical Fixed by F1
R2 The ETL was not safe to re-run Critical Fixed by F2
R3 No rollback once V2 accepts writes High Fixed by F3
R4 Discovery has no evaluation harness High Fixed by F4
R5 No per-stage visibility in the migration Medium Fixed by F2
R1

Sign-in continuity could break at cutover

Critical

Where it lives Identity bridge, from the hosted auth provider to V2 auth

Evidence

The plan assumed the auth provider's sessions and user IDs would carry over for legacy V1 accounts. Nothing checked that mapping, and those sessions do not transfer to the V2 auth system.

Blast radius

Legacy users locked out of their paid history, a support backlog, and lost trust in the cohort V2 most needs to keep.

R2

The ETL was not safe to re-run

Critical

Where it lives ETL, from the document store to Postgres

Evidence

Rows were inserted without deterministic keys, so a partial failure followed by a re-run could write a second copy of the same record.

Blast radius

Duplicate or conflicting records across every table the ETL writes, and manual clean-up of each one.

R3

No rollback once V2 accepts writes

High

Where it lives Cutover plan

Evidence

No reverse migration was designed, and nothing prevented V1 from changing underneath a load.

Blast radius

Any defect found after cutover becomes a forward-fix emergency instead of a controlled revert.

R4

Discovery has no evaluation harness

High

Where it lives Discovery, the natural-language search layer over the graph

Evidence

No fixed set of queries with expected matches existed, so a change to a prompt or model could not be compared with a baseline.

Blast radius

The feature V2 is built around gets worse without anyone noticing until users stop trusting it.

R5

No per-stage visibility in the migration

Medium

Where it lives ETL pipeline

Evidence

Success was inferred from the absence of errors, not from checks showing that the loaded data matched the source.

Blast radius

Silent data loss found weeks later, when it is expensive to trace.

Fix plan

Four fixes, about 15 days of work in all. Each is written so any senior engineer could carry it out, not only the one who wrote it.

F1

Read-only dry run of the identity bridge

Fixes R1, about 3 days

Run a read-only pass over every legacy V1 account against the V2 identity data, and produce an exceptions list before any cutover.

Use a lazy credential bridge: a V1 user signs in with their existing password and the V2 account is created at first sign-in. Accounts that fail mapping get a re-link flow designed now, not during an incident.

Acceptance check

Every legacy account either maps cleanly or appears on the exceptions list with a re-link path. Conflicting rows are held back and reported, not loaded.

F2

Deterministic keys and a verify suite

Fixes R2 and R5, about 4 days

Give every migrated record a deterministic uuid5 key derived from its source identity, and write with ON CONFLICT so a re-run changes nothing.

Add a dry-run mode and a read-only verify suite that reports counts per stage, so each run can be audited and resumed.

Acceptance check

Field-level parity: after running the full ETL twice, every field of every migrated row matches the source mapping, and the second run changes no rows. Row counts alone do not pass.

F3

Freeze V1 during loads, with manifest-based rollback

Fixes R3, about 5 days

Hold V1 read-only for the length of each load so the source cannot move underneath it. Record a manifest of every row a load writes, so a rollback removes exactly those rows and nothing else.

Acceptance check

A rollback rehearsed on a copy of the data returns it to its pre-load state, confirmed by a field-level comparison with no differences.

F4

A minimal evaluation tripwire for discovery

Fixes R4, about 3 days

Freeze a set of real discovery queries, each with its expected profile matches, into a regression suite that runs on every merge.

Pin the model version, set temperature to 0, and cache retrieval results, so a change in score means the system changed and not the noise around it. This is a tripwire, not a research project.

Acceptance check

The suite fails the build only when the pass rate drops by more than 5 percentage points against the frozen baseline, averaged across 3 runs.

Where the system stands

The migration is not done: the cutover is still pending. Since this review, the fixes for the migration itself have been built:

  • F1, the credential bridge: a V1 user signs in with their existing password, and accounts that do not map cleanly are held back, not loaded.
  • F2, safe re-runs: every record has a deterministic uuid5 key and every write uses ON CONFLICT, so running the load twice changes nothing. A dry-run mode previews each load, and a field-level audit checks the loaded data against the source.
  • F3, freeze and rollback: V1 is read-only during each load, and a rollback removes exactly the rows the load wrote.

F4, the evaluation tripwire for discovery, is being brought online.

Start with fifteen minutes.

A free call about your system. You leave knowing which engagement fits, or that none does. No preparation needed.

See all prices
Book a call