---
title: Product pipeline
description: The fixed stages every product passes through when it is imported, created or corrected (validate, avatar, evaluate the details, refined avatar, review and watch), what each reads and writes, the stage interface, the statuses, and who owns what.
section: platform
order: 6
updated: 2026-10-21
---

# Product pipeline

Every product passes through the same stages, in the same order, whenever it is imported (store sync, CSV, feed), created or edited in the admin app, or corrected in the partner app. Each stage records what it read and what it decided, and is skipped next time when nothing it reads has changed. Every stage is safe to run again.

The code is in `packages/core/src/server/pipeline/`. The avatar function is `packages/core/src/shared/avatar.ts`.

## The stages

| # | Stage (`name`) | Reads | Writes | On failure |
| --- | --- | --- | --- | --- |
| 1 | Validate (`validate`) | Name, category, size, price, photo address, stored colour and colour name | Issue codes, status `ok`, `warn` or `blocked`. The codes in the table below also go into `products.issues` | Recorded as `failed`; the next stages still run |
| 2 | Avatar, first pass (`avatar`) | Category, real size, look attributes, details (`specs`), what was corrected | An avatar (below) | Never fails: missing details give a plain avatar of the right kind and size |
| 3 | Evaluate the details (`evaluate`) | Photo, name, category | Look attributes and details with a confidence each, and a coherence verdict | **Planned, owned by another session.** Today a slot that records `noop` |
| 4 | Avatar, refined (`avatar_refined`) | What stage 2 reads, plus stage 3's output | The avatar again, with confident photo readings merged in | Same as stage 2 |
| 5 | Review (`review`) | The flags from stages 1 to 4 that need a staff decision | `open` when there are flags, `ok` when there are none; staff set `approved` | Stays in the queue |
| 6 | Watch | Changed products, newer stage versions | Re-runs only the stages that are out of date | |

The avatar runs twice because it depends on the details, but every product should have one at once. It is the same pure function both times.

### Stage 1: validate

Version 1. Deterministic rules; nothing is fetched.

| Code | Severity | When | Data |
| --- | --- | --- | --- |
| `no_dimensions` | block | Width, depth or height missing. Already hides the product (`isVisible()`); not written by this stage, `productIssues()` adds it | |
| `no_price` | block | Price missing, zero or not a number. A store sync never saves such a product (it skips it); one entered in the admin app is kept but flagged | |
| `size_implausible` | warn | A measure outside the category's range in `shared/sizes.ts` (`SIZE_RANGE`), for example a "sofa" 69 cm wide or a dining table 8 cm high | `fields` (the measures that are off), `range` |
| `kind_mismatch` | warn | The name says another category. Read with `detectCategory()` on the name alone, plus two rules it lacks: "tool" (chair) in a sofa's name when the size fits an armchair, and "öökapp" (bedside cabinet) filed as a side table | `suggested`: the category to use |
| `no_colour` | warn | No colour from the store, or only the placeholder brown (`#B5865A` or `#C49A62`) with no colour name. The same code the rest of the platform already uses | |
| `no_photo` | warn | No photo address | |
| `photo_invalid` | warn | A photo address that is not an `http` or `https` address. The photo is not fetched here | |

"Blocked" is used only for these two hard failures; it hides nothing that was not hidden before. Warnings never hide a product. Validate owns the codes above in `products.issues`: it adds them and removes them when they stop applying, and leaves the codes other code writes there (the sync's notes, such as `colour_from_name`).

A wide, low chest of drawers ("Kummut Softoak", 129 × 39 × 84 cm) is often a sideboard, but 124 of the 173 real dressers in October 2026 had that shape: Estonian shops call sideboards "kummut" too. Size alone cannot tell them apart, so this is left to stage 3's coherence check.

### Stages 2 and 4: the avatar

`avatarFor(piece, { corrected, details })` returns an `AvatarSpec`:

| Field | What it is |
| --- | --- |
| `kind` | The category, or a room piece's kind |
| `builder` | Which 3D builder draws it: `storage`, `table`, `chair`, `lounge`, `recliner`, `sofa`, `bed`, `plant`, `lamp`, `rug` or `box` |
| `size` | The size it is drawn and placed at, from `visualSize()`: a corner sofa's reach is the store's `corner_reach` when given, otherwise estimated from its length |
| `look` | Colour, material, wood, tone, finish, style, handles, handle colour, upholstery |
| `params` | What the builder needs: legs (after defaults), the front layout of storage (doors, drawers, open, or doors with drawers and where), rounded ends, a table's round top, base, lower shelf and drawer, an armchair's frame and back, a sofa's extension and corner side, the seat height |
| `sources` | For every look field and parameter: `correction`, `store`, `photo` or `default` |
| `approx` | True when a default changes the outline (no legs or fronts given) or the corner reach is estimated |
| `builderVersion` | `AVATAR_BUILDER_VERSION` |

Values are merged with this precedence: **a correction from the store or staff > the store's own data > a photo reading at or above 0.7 confidence (`PHOTO_CONFIDENCE`) > a default.**

The 3D room (`buildPiece` in `core/client/room3d.ts`) and product cards (`toScenePiece` in `packages/ui/src/ProductCard.tsx`) draw from the avatar; no builder reads raw attributes or picks its own defaults. A scene piece may carry a stored avatar; it is used when its `builderVersion` is current, and otherwise worked out again from the piece's data. Today the shop works the avatar out in the browser from the product's data, which gives the same result as stage 2. Drawing the stored stage 4 avatar in the shop is **planned**, for when stage 3 adds photo readings. Staff see the stored avatar on the review page and the product page.

Where details come from is one function (`facts()` in `avatar.ts`): look columns, `products.specs` and its corrections, and stage 3's readings. A new source of details plugs in there, without touching the builders.

### Stage 3: evaluate the details (contract)

Planned; another session builds it. Until then the stage records `noop` with no output, and stage 4 equals stage 2.

It is given a function through `PipelineDeps.evaluate({ name, category, imageUrl, product })` (`PIPELINE_DEPS` in `run.ts`), so tests can fake it and no network call happens in other stages. It must return `EvaluatedDetails`:

```ts
{
  fields: {                     // keys: look attributes (VOCAB keys, "colour" as #RRGGBB) or detail keys (shared/taxonomy.ts)
    [key: string]: { value: string | number | boolean; confidence: number }   // confidence 0–1
  },
  coherence: {                  // one call also answers "does the photo show what the name and category say?"
    matches: boolean | null,    // null: can't tell
    category: string | null,    // the category the photo shows, when it differs
    confidence: number,
    note: string | null         // short, for staff
  } | null
}
```

- Values must be valid for the product's category: a vocabulary value, or a detail value `parseSpecs()` accepts. Anything else is ignored.
- Readings below 0.7 confidence are kept in the record but not drawn; the stage notes `low_confidence` with the field names.
- A confident `matches: false` notes `photo_mismatch` with the category and note.
- The stage's input hash covers the name, category and photo address, so it re-runs only when one of them changes or its version is bumped.
- It never writes products columns; corrections stay with stores and staff.

### Stage 5: review

The admin app's **Products → Review** page (`/products/review`, permission `products.manage`) lists products whose review is `open`, oldest first. Each shows the store photo, the stored avatar drawn with `productThumb`, and the flags. Staff can:

- **Approve** ("Looks right"): the review becomes `approved` and the product leaves the queue until its flags change.
- **Use the suggested category** for `kind_mismatch`: saved as a correction (`products.overrides.category`), never in the synced column. The pipeline then runs again.
- **Edit** the product in the admin form, whose changes to a synced product are corrections too.

Every decision is audited (`review_approve`, `review_set_category`). Only flags that need a decision open a review: `size_implausible`, `kind_mismatch`, `no_price`, `photo_invalid`, `low_confidence`, `photo_mismatch` and `stage_failed` (`REVIEW_CODES` in `stages.ts`). Missing data a store can add (`no_colour`, `no_photo`, `no_dimensions`) shows on the product and in the partner app instead: two in five real products had no colour. Example products are left out of the queue.

### Stage 6: watch

The runner itself. A stage runs when it has no record, when its input hash changed, or when its version is newer than the record's; otherwise it is skipped. Because each stage's hash includes what the earlier stages decided, a change flows on to the stages that depend on it and no further. A forced run keeps an approved review while its flags are the same.

## The stage interface

```ts
interface Stage<O> {
  name: StageName;                   // "validate" | "avatar" | "evaluate" | "avatar_refined" | "review"
  version: number;                   // bump to re-run it for every product
  inputHash(input, ctx): string;     // hash of everything the stage reads
  run(input, ctx): { output: O; issues: StageIssue[]; status: StageStatus } | Promise<…>;
}
```

- `input` is `{ product, corrected, colourHex }`: the product as the shop sees it (`rowToProduct`), the keys of its corrections (details as `specs.<key>`), and the colour as stored.
- `ctx` is `{ earlier, stored, deps }`: the records of the stages before it in this run, the stage's own last record, and the injected side effects.
- A `StageIssue` is `{ code, severity: "warn" | "block", data? }`.
- Stages are pure where possible. Anything with side effects (network, Claude) comes in through `deps`.

### Statuses

| Status | Meaning |
| --- | --- |
| `ok` | Nothing to note |
| `warn` | Issues that stay on the product |
| `blocked` | A hard failure that already hides the product, or keeps it out of the shop |
| `failed` | The stage threw. Its issue is `stage_failed`; the next stages still run and nothing is hidden |
| `noop` | The stage is not built yet (stage 3) |
| `open`, `approved` | Review only: waiting for staff, or decided |

## The runner and the record

`runPipeline(productIds, { from?, only?, force?, deps? })` in `run.ts` runs the stages in order for each product. `from` starts at a stage, `only` runs the named stages, and `force` (true or a list of stage names) runs them even when nothing changed. It never throws to the caller; a product whose run breaks is counted as failed. `runPipelineSafely()` is the version for saves and syncs. Products are read and their records written 200 at a time, one statement per batch: on 10,000 products a first run took 30 seconds with a write per record and takes 6 seconds now (in-memory database; a database over the network gains more, since about 60,000 queries became about 150).

Each run is recorded in `product_pipeline`, one row per product and stage (migration `021_product_pipeline`):

| Column | What it holds |
| --- | --- |
| `product_id`, `stage` | The primary key |
| `status` | One of the statuses above |
| `input_hash`, `version` | What decides whether the stage runs again |
| `output` | The stage's output: validate's codes, the avatar, stage 3's readings, the review's flags |
| `issues` | The stage's issues, with their data |
| `run_at` | When it last ran |
| `decided_by`, `decided_at` | Review only: who approved it and when |

## Triggers

| When | What runs |
| --- | --- |
| A store sync saves products (`runSync()` in `integrations/sync.ts`) | Every saved product, after the sync is recorded |
| An admin creates or edits a product | That product, after the response is sent (Next's `after()`), so the save never waits |
| A store corrects a product in the partner app | That product, after the response is sent |
| A staff member changes a category on the review page | That product, before the response, so the page shows the result |
| The backfill job | Every product; only out-of-date stages run |

Pipeline errors never break a sync or a save: they are logged, and the product is saved as before.

**The backfill job** is `POST /api/cron/pipeline` in the partner app (GET also works), with `Authorization: Bearer $CRON_SECRET`, the same secret as the scheduled syncs. `?force=validate,avatar` or `?force=all` re-runs those stages for every product. A call goes through the products in id order and stops after `?seconds=` (default 240, under a hosted function's time limit); its answer then has `next`, and `?after=<next>` goes on from there. `next` is null when every product has been through. `npm run pipeline:backfill` calls it on a running partner app until it is done (`-- --force <stages>` passes `force`). Call it after every deploy and once a day.

## Adding or changing a stage

- **A new rule in validate, or a change to a stage's output:** change the stage and bump its `version`. The next backfill runs it for every product.
- **A change to the 3D builders that changes how existing products look:** bump `AVATAR_BUILDER_VERSION` in `shared/avatar.ts`. It is the version of both avatar stages, and drawings cached under the old version are not reused.
- **A new stage:** add it to `STAGES` in `stages.ts` at its place in the order, with a unit test in `packages/core/tests/pipeline.test.ts`, and a row on this page.

## Who owns what

| Part | Owner | State |
| --- | --- | --- |
| Stage interface, runner, record, triggers, backfill job | This pipeline | Built |
| Stage 1, validate | This pipeline | Built |
| Stages 2 and 4, avatar, and the builders reading it | This pipeline | Built |
| Stage 3, evaluate the details (reading photos with Claude) | Another session | Planned; the slot and contract are built |
| Stage 5, review page | This pipeline | Built |
| The details' vocabulary | [Furniture details by category](../product/furniture-details.md) | Built |
| Finding the same product sold by two stores | Not decided | Out of scope |
| Real 3D model files | Not decided | Out of scope |

## Related pages

- [Platform architecture](architecture.md)
- [Product import, management and search](product-flow.md): diagrams of the whole path, from a store's data to search
- [Furniture details by category](../product/furniture-details.md)
- [How a piece is drawn](../product/how-it-works.md#how-a-piece-is-drawn)
- [Partner integrations guide](../partners/integrations.md)