Skip to content

Code reference

One section per module of brickwork.core, rendered from the docstrings when the site is built. For calling Brick Work over HTTP, the API v2 reference is the contract; the web layer, the API routers and the store are internals and are not listed here. Set scoring, candidate sets and Build Scout are explained in Set scoring.

Brick Work's services on the 4.0 store: the collection, the catalogue, prices, planning and undo. The REST API and the screens call this layer, and the MCP server's adapter is part of it; its modules are listed in the docs' code reference (ADR 0030).

add

Add on the 4.0 store (M.8c, issue #110; design 4.3, 3.5, 6.3, 6.5, 5.2): search, review proposals, atomic receipts and their undo.

Every row speaks an item id. A person types or pastes a number (items.find: the Rebrickable number first, the BrickLink number as well), the review resolves it to an item or, when nothing matches, to an unknown number row with two offers (design 6.5): Fetch from Rebrickable (a set only; queues the store4 rebrickable_set job) and Add as unidentified (a local item of the chosen kind named as typed, with the typed number as a to_check BrickLink link).

confirm writes inside the app's transaction, one transaction per review row: an owned set becomes quantity copies through store4.copies.create_copy_in with design 4.3's preselection (the set's set_preference, then the global switch with a key, then the latest Rebrickable version; BrickLink without a fetched list is decision F's pending copy); every other owned or loose kind becomes stock (core.stock); a wish becomes a wish row. The materialisation happens in the same transaction, so there is no later step. Each row ends with one pipeline job (core.pipeline) and ONE receipt event add:receipt (entity id = the batch) which undo reverses by its registered undoer (core.undo): it archives what the row created and refuses when the rows changed since.

A choice of part list (design 4.3's picker) has no screen yet (M.10): the preselection decides. Photo mode (M.8h) recognises a photo through core/identify.py, offers its candidates, and keeps the photo as the new item's own photo only when asked.

require_catalogue(db)

Add needs an activated catalogue build (design 6.1 step 3). Without one it says what to run; it never fails with a 500.

search(db, kind, query='', sort='number')

Items of one kind by number or name: {items: [...first 100, decorated], count}.

A number matches the Rebrickable number and the BrickLink one (items.find: born key, the catalogue's search index over every linked number, confirmed BrickLink links); words match names by prefix (catalogue.item_fts); a size ("2x4", "x2") filters by the size key of part names (#41); local items (BrickLink-only or unidentified) are searched by name and number.

lookup(db, item_id)

The identity of an item id (a merged item as the item it was merged into), else None when neither file knows it.

colors(db)

Every colour of the collection (the catalogue's are projected at activation), by name.

picture_order(db, kind)

The picture source order for a kind: the setting source_priority.pictures (design 6.8), else the default (core.pictures.order).

decorate_rows(db, rows)

Rows with an item_id (and optionally color_id, list_id) gain the numbers to show (label, number, also), the best picture, what is owned and wished already and whether a link waits to be checked. A handful of queries for any number of rows.

brickset_rows(text, kind, where)

Rows of a Brickset My Sets CSV: the sets marked owned (Add to Owned or Loose) or wanted (Add to a wishlist), as Number-Variant. An owned count split into new and used becomes a row per condition; the rest has no condition, as in any file. UserNotes is the note.

brickset_minifig_rows(text, kind, where)

Rows of a Brickset minifigure CSV: the figures owned loose (QtyOwnedLoose; Loose or Owned) or marked wanted (one each; a wishlist), by Brickset's minifigure number, which is BrickLink's. UserNotes is the note.

file_rows(data, filename, kind, where='owned')

Rows of a BrickLink wanted list or inventory (XML: BrickLink numbers and colour numbers), a Rebrickable CSV (Rebrickable numbers and colour ids), a Brickset My Sets CSV (set numbers; where picks its owned or its wanted sets) or Brickset's minifigure CSV (minifigure numbers: the loose ones or the wanted ones). Nothing is looked up here.

offers(db, kind, number)

What a row for an unknown number offers (design 6.5): add it as unidentified (always), fetch a set from Rebrickable (a set, with a key). help says what would help when there is no way to get the real item.

resolve(db, rows, where, *, list_id='default', map_wanted_ids=True)

Review proposal for input rows: {items, unresolved}. Writes nothing.

part_list_options(db, set_item_id)

The Add review's Part list picker for a set (design 4.3): None unless there is a choice, else {'options': [{'key', 'label', 'note'}], 'preselected': key} in the order of the design's preselection (copies.part_list_choices).

picture_warning(db, row)

True when no picture is known for a part row in its colour (design 6.8): the review warns before confirming.

validate_edits(db, proposal, edits, *, batch=None)

The review rows as the person edited them, validated against the proposal (identity never comes from the edit).

receipts(db, batch)

The saved receipts of a batch, in review order, each with the event_id that undoes it.

confirm(db, rows, *, batch, via=None, actor=None, photo=None, keep_photo=False)

Write the reviewed rows, one transaction each, and return their receipts (a row already saved under this batch returns its saved receipt, so a lost response never doubles a row). A row that fails stops the batch; rows before it stay saved. photo is the picture a photo-mode Add recognised; it is kept as the new item's own photo only with keep_photo.

has_unsettled_fetches(db)

A fetch job of an Add receipt is done (or finally failed) and the receipt has not been settled yet: the next drain settles it.

settle_fetches(db, *, via=None, actor=None)

Create what the receipts of a finished rebrickable_set job asked for, once each: the quantity of copies of the fetched set (with the latest Rebrickable list, the same preselection as any Add), or its wish, and the set's own pipeline job. Each is one add:fetched event, undoable like the receipt's rows; rows that cannot be made are recorded as add:fetch_failed and not tried again. A job that failed leaves its receipt with nothing to create (the job view says so; Retry queues the job again and a later run settles it). Returns the number of receipts settled.

fetch_outcome(db, job_id)

What became of the Add row that fetched this job's set, for the Add page: {'state', 'message', 'copies'}. state is added (rows made), waiting (the job is not done), failed (the job failed: nothing was added), refused (the fetch worked but the rows could not be made) or unsettled (done, rows follow with the next drain).

undo(db, batch, *, via=None, actor=None)

Undo every row of a batch that is not undone yet, last row first, all or none. Returns the number of rows undone.

colour_ink(rgb)

Readable text for the shared catalogue colour chip.

admin

New-store admin queries and atomic, app-audited mutations. No legacy adapters.

origin()

(via, actor) for an event written now: the request's audit origin, else the app.

begin_operation(key=None)

Start a new operation in this context (a web request, a CLI command, a worker pass) and return its key.

Every event written after this call shares the key (event.operation_key, design 5.2). Call it once at the entry point; end_operation() clears it. Code with no operation (a test, a script) writes events with a NULL key.

operation(key=None)

begin_operation for the length of a block; the previous key is restored on exit.

split_entity(entity)

The 3.3-style entity string as the 4.0 pair: split at the first colon (copy:<id> -> ('copy', '')); no colon, or nothing after it, gives an entity_id of None.

audit(db, entity, field, old=None, new=None, *, via=None, actor=None)

Write one event on the 4.0 event table. The signature is the 3.3 one; entity is split into entity_kind and entity_id (split_entity), operation_key is the current operation's. old/new are stored by store4.audit.value_text, the one rule every event writer uses (None as NULL, strings as they are, dicts and lists as sorted JSON).

value_write(db, group, action, identifier=None, name=None, confirmed=False)

Add, rename or archive a value; returns the event id of the change (Undo takes it back).

seed_freshness(db)

The freshness rules a new collection starts with (3.3's: prices 30 days, metadata 365, pictures never). Only into an empty table.

seed_new(db)

What a fresh collection starts with (user journey § 1): the freshness rules, the three system status flags and the database/setup event, in one transaction. Only on a fresh file (no freshness rules yet); returns whether it seeded. A flag of that name already live (from an import) is kept as it is.

ensure_collection(path)

The collection file at path, created and seeded when missing (an existing one keeps its own). It opens without a catalogue: onboarding builds the catalogue next (design 6.1) and brickwork setup says where it goes.

effective_priority(raw)

The source order per step as the pipeline uses it: the stored source_priority where it names the step, else design 6.8's default.

patch_settings(db, data)

PATCH /settings (#47): any subset of the whitelisted settings, in one transaction.

freshness: [{source, kind, max_age_days}] (as GET /settings returns it); import_order: {item kind: [the six steps]}; repair: {shipping, fees, min_order, use_loose_stock}; prefer_bricklink_inventory: true or false (new copies take BrickLink's part list, design 4.4); candidate_sets_common_max: a whole number of 25 or more (Lot Detective ignores lines held by more sets than this, #200; default 300).

save_settings(db, country, priority, freshness, part_list_grouping=None)

Save the retail country, the source order and the freshness rules from the Settings page; returns the event id (None when nothing differs from what is stored). Refused whole on a value that does not fit.

stale_prices(db)

The held or wanted (item, colour, condition) whose market price is not fresh from some enabled source of its kind's order.

One row per line the collection holds (a set copy, loose stock, a wish) with item_id, color_id, condition (words; None for "any"), name and, for display, the BrickLink item_type letter and item_no. Fresh means a successful, unvoided sold or stock observation younger than the source's prices freshness rule (7 days when there is none).

live_import_job(db, item_id)

The id of an item's pipeline job that has not finished (queued, running or waiting for a budget), else None.

queue_refresh(db, rows, steps=('market_price',))

Queue steps for each item of rows (dicts with item_id) that has no unfinished pipeline job; returns the job ids. A bulk action: the jobs are one batch (#127), shown as one row in Jobs, cancelled together, and overtaken by interactive work.

queue_missing_booklets(db)

Queue the identity step for booklets without a PDF. A PDF belongs to a set, and a booklet shares the files of the set with its number (collection.instruction_files). The identity step is local in 4.0 and the pipeline has no instruction-PDF step, so this checks the item's number only; it never downloads a PDF. Returns (queued job ids, skipped items: the ones with no number).

jobs(db, limit=300, *, batches=True)

The newest jobs, newest first, each with its steps and a heading (the Jobs list): pipeline jobs and the two fetch kinds. count_jobs says how many there are in all. With batches=False the jobs of a bulk action are left out: Jobs shows each batch as one row (#127, pipeline.batches).

statistics(db)

The collection counted by completeness, theme, storage and purchase place (live copies), and spent against retail.

check(db)

The database check: SQLite's integrity and foreign-key checks of the collection (and of the attached catalogue, which is a file of its own), and the rows of every table of the collection.

check_all(db)

check plus the cross-file checks of the pair (store4.crosscheck): what brickwork check and the Database page run. API v2's /database/check stays check, so its response keeps its shape.

last_check(db)

The stored result of the last explicit database check, or None. Never runs a check.

run_check(db)

Run the full check, keep its result with a date (setting maintenance:database_check), and audit the run (one transaction, not undoable).

instruction_target(db, kind, number)

The set a PDF is attached to, from a BrickLink type letter and a number typed in the form. A PDF belongs to a set (the store allows no other item); a booklet (type I) shares the files of the set with its number, so it names that set.

instruction_upload(db, root, kind, number, label, content, *, undoable=False)

Install immutable, content-addressed booklets; roll back a new file on failure.

instruction_set_of(db, item_id)

The set a PDF is attached to, from an item id (API v2): a set is its own target, a booklet names the set with its number (as instruction_target); anything else cannot hold a PDF.

attach_instruction(db, root, item_id, label, content, *, undoable=False)

instruction_upload for a set already named by its id (the form names it by a typed number).

instruction_overview(db)

The Instruction files page: one row per booklet item with the PDFs of the set that has its number (a PDF belongs to a set), its picture, and the PDFs of sets that no booklet item names. Counts canonical booklet identities only: a set-level PDF does not say how many booklets the set has.

display(value, secrets=None)

Redact strings before serialization, including quoted credential values.

The secret list is read once per call and shared by every nested string.

history(db)

Saved identifications (newest first) with the candidates each offered, for Identify history. A candidate names its item when it has one (item_id: the number shown is the item's, the picture its cached one); older or foreign rows are read as they are.

build_scout

Build Scout (#9): which catalogue sets can be built, or nearly built, from the loose stock. Read-only except build_set and wish_set.

It sits on the set-scoring engine (core/set_scoring.py, ADR 0026) but answers a different question. The engine ranks by rarity-weighted evidence; Build Scout ranks by how much of a set the stock covers, and its numbers are the buildability ones (ADR 0027):

  • Parts are a set's own part lines (sub-sets expanded, spares left out). A minifigure's components are not part lines of the set: the figure is one whole line, counted apart. So parts_available = sum over the set's part lines of min(have, need), percent = parts_available / parts_total, missing = parts_total - parts_available (pieces) and missing_lines (lines not covered in full).
  • Figures are the set's whole minifigures, figures_available = sum of min(have, need) over its figure lines. A loose whole figure counts; loose parts of a figure never do.
  • Swaps (another colour, an alternate mould) are the engine's, listed apart and never in a number.
  • Cost is the missing lines (parts and figures) priced from price_current where a price is known, in the one currency most of them are priced in, with the number of missing lines left unpriced. It is worked out for the page shown, never for the whole catalogue.

The stock is one eligibility rule (stock_rows) shared by the ranking and by Build it, so the pieces on the screen are the pieces that move: live loose parts and minifigures, with workbench.available (damaged rows give nothing unless asked for, a figure with defective parts none, a row's missing and damaged counts are taken off). A source selection is the union of the stock kept in no lot or storage (loose), the chosen lots and the chosen storages with the storages inside them; none chosen means all loose stock.

Two indexes are built from the engine's loader with the engine's build_index, once per catalogue build and process: the parts one (the sets' own parts only) and the figures one (whole figures only), so sub-sets are expanded by the same code as everywhere.

Scout

What one catalogue build gives: the parts index, the figures index and the sets' year and theme.

theme_ids(wanted)

The theme ids a theme filter means (an id, or a name in any case) with every sub-theme, or ValueError.

stock_rows(db, *, lots=(), storages=(), loose=False, damaged=False)

The live loose rows the chosen sources hold, each dict(id, item_id, color_id, kind, have, lot_id, storage_id), have being what can be given (workbench.available). With no source chosen: all loose stock. A row in a chosen lot and a chosen storage is one row.

pile_of(rows)

{line key: quantity} of the rows (a figure is a line with colour 0, as in the engine).

sources_text(db, lots=(), storages=(), loose=False)

The words of a selection: 'Loose stock · 2 lots · 3 storages'.

scout_for(db)

The Scout of the attached catalogue build (built once per build and process; the engine's key says when it is a new build).

covered_by(index, lines)

{position: pieces covered} and {position: lines covered in full} of a pile against an index: the engine's min(have, need).

ranked(scout, pile, *, min_percent=DEFAULT_MIN_PERCENT, max_missing=None, theme=None, year_from=None, year_to=None, size_from=None, size_to=None)

Every set that shares a part line with the pile and passes the filters, best first: [(position, covered, complete lines)] and the count of sets that share a line. Sorted by percent available, then fewest missing pieces, then set id.

figures_of(scout, pile)

{set item id: (figures covered, figures needed)} for the sets that have figures.

scout(db, *, lots=(), storages=(), loose=False, damaged=False, limit=20, offset=0, detail=0, cost=False, **filters)

The Build Scout answer: dict(catalogue, source, input, total, limit, offset, results); results are the page's sets, each dict(rank, set_item_id, year, theme, parts_available, parts_total, percent, missing_parts, missing_lines, figures_available, figures_required, cost, ...). With detail above 0 each set lists that many of its biggest missing lines (gaps), its missing figures (figure_gaps), its cost and its swaps; cost alone adds the cost. Read-only; ValueError for a refused argument.

missing_lines(chosen, position, pile)

The set's part lines the pile does not cover in full: [(key, need, have, missing)], the biggest gap first.

figure_lines(chosen, item, pile)

The set's whole-figure lines as [(key, need, have, missing)] (all of them, covered or not).

price_map(db, pairs)

dict(prices={(item, colour or 0): Decimal}, currency) from price_current for the (item id, colour id or None) pairs: the used price before the new one, a sold average before a stock listing, the newest observation last; a price in another currency than most are is left out.

cost_of(missing, prices)

dict(amount, currency, priced_lines, unpriced_lines) for [(key, quantity)] missing lines against price_map's answer; amount is None when no line is priced.

set_detail_in(db, chosen, position, pile, *, lines_shown=MAX_LINES, prices=None)

What a set adds to a row: its biggest missing part lines, its missing figure lines, the cost to complete and its swaps.

set_detail(db, set_item_id, *, lots=(), storages=(), loose=False, damaged=False)

One set's full Build Scout numbers for the chosen stock (whether or not it passes a filter), or ValueError for a set that is no candidate.

missing_export(db, set_item_id, **sources)

The BrickLink wanted-list XML of a set's missing parts and figures for the chosen stock, by the exporter the copies use (gaps.export_lines): dict(xml, lines, skipped).

wish_set(db, set_item_id)

Put the set on the default wishlist, marked WISH_NOTE in the wish's note (by the Add flow, so the receipt is its own and Undo is Add's). Returns dict(event_id, wish_id, already); a set already wished is already and nothing is written.

plan_takes(db, set_item_id, rows)

How a Build it takes from rows (stock_rows): [(stock id, amount)] so that every part line gets min(have, need) and every figure line min(have, need), the largest usable row first and then the lowest id (the Matchmaker's order), and the totals taken.

build_set(db, set_item_id, *, lots=(), storages=(), loose=False, damaged=False, note='Built from loose stock')

Build it: one copy of the set, the matching parts and whole figures moved out of the chosen stock into it, the rest recorded as missing on the copy, as one transaction and one undoable event (found_copy.make_copy_in). Returns dict(event_id, copy_id, pieces, figures, missing_parts, missing_figures); a ValueError is a refusal that wrote nothing (a set with no part list or made of sets, nothing in stock for it).

candidate_sets

Lot Detective, the service (#200): which catalogue sets could a lot have come from? Read-only, over the scoring engine of set_scoring.py.

The lot's live loose parts and figures are scored (ADR 0026); this module adds what a person reads on top of the engine's evidence:

  • Ignore common parts (on by default): lines held by more than N sets are dropped from the pile before scoring. N is the stored setting candidate_sets_common_max (default 300, at least the engine's distinctive limit); it is not an environment variable.
  • Set filters: theme (id or name, with its sub-themes), year range and the catalogue's part count range.
  • Score: 100 * evidence / the best candidate's evidence, a whole number 0 to 100. It says how strongly the found parts point to a set relative to the best candidate; it is not a probability. The best candidate is the first set that passes the set filters (theme, year, size), so the top set shown is always 100; min_score then cuts the list.
  • Reasons: plain sentences built from the engine's counts and the catalogue (see reasons_of).
  • Distinctive parts: found and not found, the engine's distinctive lines (held by at most DISTINCTIVE_MAX sets).
  • Stickers (#201): a line flagged Sticker is never dropped as common (its base part is usually a tile or a brick many sets hold), and the engine's sticker rule (set_scoring.apply_stickers) is applied to the cached ranking before the set filters, so the best candidate and the scores include it. A set a sticker is confirmed for says so in its reasons.
  • Swaps: the engine's swaps (another colour, an alternate mould) are listed per set and never count in the score.

The ranking is the engine's cached one (per catalogue build and input), so a lot that did not change costs only the filters; a changed lot is a different input and ranks again.

common_max(db)

N, the most sets a line may be held by and still count: the stored setting, else 300 (an invalid stored value is the default).

valid_common_max(value)

The setting's value as a whole number of at least COMMON_MAX_LOWEST, else ValueError.

theme_ids(db, theme)

The ids of a theme and the themes below it: theme is a catalogue theme id (digits) or a name (case-insensitive). ValueError when unknown.

printed(db, items)

The part items among items that are printed: the part number has a print suffix (pr and digits, as in 3626bpr0001 or 973pr1234c01) or the catalogue lists the part as the printed child of another part (relationship type P). Derived from what the catalogue holds.

reasons_of(found, total, only_printed, only_other, covered_lines, set_lines, stickers=())

The sentences that say why a set is a candidate. found of total distinctive parts; only_printed and only_other the found parts that exactly one set holds (printed or not); then how much of the set's part lines the lot holds. stickers: the numbers of the sets a sticker in the lot is confirmed for, one reason each.

sticker_flags(flags)

The engine's flags from the lot's stock flag names: a flag named Sticker (any case) is passed on as sticker, the others are not used.

candidates(db, lot_id, *, min_score=0.0, theme=None, year_from=None, year_to=None, parts_min=None, parts_max=None, ignore_common=True, limit=20, offset=0)

The ranked candidate sets of a lot, dict(catalogue, input, total, limit, offset, results); ValueError for no such lot or theme.

Each result: rank, set_item_id, score, evidence, coverage, reasons, covered_lines, set_lines, distinctive_found, distinctive_total, distinctive_found_parts and distinctive_missing_parts (up to PARTS_SHOWN each, heaviest first; each line is item_id, color_id, quantity, need, weight, only_in_set, printed) and flags.

set_numbers(db, items)

{set item id: its number, as1682-1}.

distinctive_of(index, position, lines)

(found, missing): the distinctive lines of a set (held by at most DISTINCTIVE_MAX sets) the lot has and has not, each dict(key, have, need, weight, only_in_set), heaviest first. A common-part drop never touches a distinctive line (N is at least the limit).

checker

Set Inspector on the 4.0 store (M.8d, issue #110; design 5 part_check, 4.4, 4.5, 2.4 invariant 10).

A session is a part_check row per copy (and one per figure line inside the copy): its order and answers are kept by token, and a token is a line's own id, which an in-place part-list switch keeps: p:<copy_part.id> for a required or spare part line and f:<copy_figure.id> for a whole figure line. Only live lines (retired_at IS NULL) are in a check. The set sequence is the required parts, then the whole figures, then an optional spare pass the user opts into at the end; a figure's own parts (its copy_part rows, copy_figure_id set) are checked in a separate session inside the copy.

Every answer is ONE event (part_check, <copy id>[:fig:<line id>], save) holding the old and new state of what it touched (the session row, the lines, the "Set is checked" flag, the copy's derived completeness) and the client's operation id, so a retried request returns the first answer's result. Undo is the registered undoer of that event (core.undo): it refuses when the touched rows changed since, restores them and writes event_undo. Check sessions restart at 4.0: the 3.3 checker:* settings were not migrated.

entity_id(copy_id, figure=None)

The event entity id of a session: the copy, or one figure line inside it.

sorter(mode, position)

Sort key of the rows of one kind for an order mode. Largest first is the physical size from the catalogue name and category (part_size.volume; the item's size_key is the sorted dimensions of the same name and cannot tell a plate from a brick), then the larger count, then one colour after another so a part's colours stay together.

normalise(s, kinds)

Read model of a stored session against the current sequence.

Tokens that are no longer part of the sequence (a line a switch retired, or spare rows before the spare question was answered "Yes") are dropped from the visible order without error, and the cursor keeps pointing at the same remaining step. A required line the session has not met (a switch to another part list added it, design 4.4) joins the end of the walk, so the check can still answer it and finish. Nothing stored is rewritten here; the next save persists the changed form and its receipt keeps the raw old value.

checked_flag(db, create=False)

The id of the status flag "Set is checked" (live), or None; with create it is made as a system flag and the creation is audited.

rows(db, copy_id, images=True, tokens=None, decorate=True, figure=None)

The sequence of a check: the required parts, the whole figures and the spare lines of a copy, all live.

With figure (a copy_figure id) the rows are exactly that figure's required (non-spare) component lines. decorate=False is the write path's light read (names and sizes for ordering, no identity or picture lookups); images=False leaves the pictures out of the decorated rows. Each row: token, kind (part, figure, spare), id, item_id, color_id, quantity, missing, damaged, checked (a figure line has no damaged or checked: its missing is the answer).

snapshot(db, copy_id, tokens=(), figure=None)

What an answer touches, exactly as a receipt keeps it and an Undo compares it.

recompute(db, copy_id)

Derived completeness after an answer (only a copy whose completeness is derived changes).

figure_view(db, copy_id, figure_id)

Check figure parts: one whole figure line's component rows inside this copy.

choices(db)

The chooser: every live copy with its counts and its set-level session. A fixed number of queries, however many copies.

saved_view(db, copy_id, tokens=(), figure=None)

The reply of a save: the changed rows only (the client keeps what it has, pictures included) and the progress counts.

mark_row(db, table, key, row, fields)

The checker's bounds and update for an answer (design 2.4 invariant 10: each count on its own, 0 to the quantity).

write(db, copy_id, payload, figure=None)

One ordered, idempotent answer and its receipt. figure is a copy_figure id (a figure check inside the copy).

mark(db, copy_id, edits)

Set counts on lines of a live copy: edits is [(token, fields)], a token p:<copy_part id> or f:<copy_figure id> (a figure line takes missing only) and fields among missing, damaged (each 0 to the line's quantity) and checked. One receipt; the derived completeness follows. A token that is no live line of the copy is NotFound. Returns the event id.

found_one(db, copy_id, token)

215: Found 1 outside a check session (the set page's missing-part card): the line's missing count drops by one, as Found 1 in the

Set Inspector does, and loose stock stays as it is. One part_check receipt, so Undo takes it back. Returns the event id.

set_checked(db, copy_id, wanted)

Set or clear the status flag "Set is checked" on a live copy (made when it does not exist yet and is wanted) and let the derived completeness follow. One receipt, also when nothing changes. Returns the event id.

mark_stock_parts(db, stock_id, edits)

Set counts on the part lines of a live loose minifigure: edits is [(item id, colour id, spare 0 or 1, fields)] with fields among missing, damaged (each 0 to the line's quantity) and checked. One collection receipt over stock_part. Returns the event id.

collection

Collection projections and guarded receipts on the 4.0 store.

The read side (M.8b, issue #110) is on the 4.0 collection: items are ids (core/items.py names them for a screen), copies and stock carry words (design 5.1), and a retired line (retired_at) is never a line of anything (design 4.4). The write side (M.8d) is save, new_lot, restore, photo, caption and want: each makes ONE receipt event and undo reverses it (see "writes" below).

colours(db)

Every colour the collection knows, by name.

priorities(db)

The whole source_priority setting (metadata, market price and picture orders), read once by a screen.

decorate(db, rows, loaded=None)

Identity (core.items), colour, picture and the unresolved mark for rows that carry item_id (and color_id).

Adds kind, item_type (the screen letter), number (the Rebrickable born number first, or a local BrickLink-born item's own BrickLink number), number_source, also (the preferred BrickLink number where it differs), number_text (what a screen writes), item_no (= number, '' for an unidentified item), unidentified, unresolved, image, color_name, rgb, ink. Name and year stay what the row has; a row without a name takes the item's. Loaded in a fixed number of queries (loaded is priorities(db) when the caller has it).

The query arguments of an item page for a row: the colour of a part.

properties(db, item_ids, kinds=None, loaded=None)

{item_id: {key: dict(value, unit, source, fetched_at, alternatives)}}: the best source per key by the metadata priority (a source order per kind word; kinds is {item id: kind} when the caller knows it, else it is read).

prices(db, item_id=None, colour=None, *, pairs=None, used_sold=False)

Current quotes (price_current over its observation, a voided observation never shows).

One item and colour -> a list; pairs (item id, colour id) rows -> {(item, colour): [quotes]}. used_sold keeps only the used, sold-guide, non-boxed quote: all a list row's price ever reads.

instances(db, kind=None, *, include_archived=False, item_id=None, stock_only=False)

Bulk physical rows (copies, then stock); API archive reads opt in, web defaults stay live-only.

kind is a kind word (None for all); item_id scopes the read to one item: same rows, same labels, same order; stock_only leaves the copies out (the API's stock list). Rows are decorated (decorate), copies are labelled Set N in creation order, flags are on the rows.

instance(db, entity, identifier)

One physical row (archived included) shaped exactly like instances(include_archived=True).

Keyed lookups only: no collection-wide scan, so a single read costs the same on any size of store.

copy_labels(db, ids)

{copy id: 'Set N'} for live copies among ids, numbered as instances(db, 'set') numbers them.

listing(db, type_letter, *, wishes=False)

The rows of a collection screen: type_letter S, M, P, I, O, G, B or C (wishes: every kind, type_letter is ignored).

A row is an item (and colour): its owned instances grouped, with counts in sets for parts and figures, the used-sold price, and the theme, category and year projected on the item (design 6.3).

part_colours(db, item_id)

Active colours of a part from stock, set lines and wishes, in one read.

Catalogue links of a set: BrickLink by its BrickLink number, Brickset by a confirmed Brickset link (a number of the set itself otherwise). An unidentified set, or one with no BrickLink number, has none.

part_gap(row, whole_missing=0)

Required replacements, with whole missing figures covering constituents.

retired_lines(db, copy_id)

The lines a switch retired, named for the page (design 4.4: kept, excluded from every count, ids still valid): decorated like any line, each with its marks.

part_list_sheet(db, copy_id, *, choose=None)

The Switch sheet's data (design 4.4): one option per list the copy could move to, each with what the switch would do, read from preview_switch (the lists the real receipt will hold). choose (a choice key) preselects an option; the default is the first that is not the copy's own list. A BrickLink list that is not fetched yet cannot be switched to in place, so it is shown unavailable. Read-only.

detail(db, item_id, *, colour=None, copy_id=None, stock_id=None, wish_id=None)

Everything the item page shows. colour is a colour id for a part (a part is always held in a colour).

instruction_files(db, item_ids)

{item id: [files]}: a booklet shares its files with the set of the same number, in both directions (concept: Boxes and instructions).

figure_detail(db, row)

Figure-wide display facts: its parts across every holder, the sets and the wishes it is in (retired lines excluded).

retail(db, item_id)

Retail price and availability (Brickset, #26) for the retail country, or None when the set has none of it.

exchange_rates(db)

{currency: dict(per_eur, as_of, source)} including EUR = 1 (the 4.0 exchange_rate table).

search(db, query, limit=20)

Local jump projection: what you own or want that a word of the query names, by name or by either number.

A word matches an item's name, its born (Rebrickable) number, any confirmed BrickLink number, the item id, or the id of a copy or stock line. Sets first, then the exact number, then the number.

columns(db, table)

The stored columns of a table (a generated column is derived, so it is neither kept in a receipt nor written back).

receipt(db, changes, field, entity='collection')

Write the one event of a request: changes is [(table, rows before, rows after)]. Returns the event id. entity is the event's kind: the workbench and the ledger write theirs under their own names and register the same undoer for them.

save(db, entity, ids, data, *, archive=False)

Edit or archive collection rows (copies, loose stock, wishes, lots), with their flags, under one receipt. Returns the event id.

data holds the column values to set (an empty string clears a column) and, for one row, flags (the flag ids the row should have among the live flags; links to archived flags are left alone). Stock that ends up with another live row's merge key merges into it (stock.plan_edit), and the receipt keeps both rows.

wish_collision(db, table, row)

Refuse, with a clear message, a wish write that would duplicate another live wish of the same list, item and colour.

merge_result(db, event_id=None, *, receipt=None)

The live stock rows a save merged other rows into, read from its committed receipt (empty when nothing merged).

undo(db, event_id, *, receipt=None)

Undo a collection receipt (or any event with a registered undoer) by its event id; a second Undo of the same event does nothing.

photo(db, root, entity, identifier, content, caption)

Keep an own photo of a copy or loose stock row: the file under data/images/own, the row in photo. Undo archives the row; the file stays (it is the user's).

new_lot(db, ids, data)

Assign existing loose stock to a new lot, without changing its quantities; rows that then share a merge key merge.

restore(db, table, identifier)

Restore an archived collection or location row under a receipt. A wish or stock row that would now duplicate a live one is refused.

want(db, item_id, colour=None)

Put an existing item (an unidentified figure, say) on the default wishlist under a receipt. A part needs its colour.

matching_headline(matches)

Count destination copies once, excluding loose-figure destinations.

matching_note(note)

Keep scanner provenance in the row title, leaving user notes visible.

defects(missing=0, damaged=0)

One unsigned defect label for server-rendered and materialized rows.

collection_prices

Display-only selection and history from recorded price observations (conditions are words, design 5.1).

priority(db)

The market-price source order per kind (setting.source_priority); a kind is a kind word, * is every other.

choose(rows, kind, priorities, condition=None, *, prefer_sold=False)

Source priority first; freshest successful observation within that source.

with_box(rows, kind, priorities)

A set's quote with box (slot 'boxed', any guide): what its price card shows as "With box" and what its max with box is held against (#131). One rule for both.

completeness

The shared derived-copy completeness rule; callers own writes and receipts.

derived_state(db, copy_id)

complete or incomplete for a copy whose completeness is derived, else None: the 4.0 rule is store4.copies's (live, non-spare, non-figure lines; retired lines are no lines).

connect

The app's one way to open the collection: the 4.0 store (M.8a, issue #110).

connect(path) keeps its name and signature so the ~40 call sites stay; it opens the 4.0 collection at path through store4.connect with catalogue.db beside it attached read-only. A 3.2/3.3 file is refused with the fix named, never opened.

OldStore

Bases: NotSetUp

The file at BW_DB_PATH is a 3.2/3.3 collection (or not a Brick Work file at all).

NoCatalogue

Bases: CatalogueNotLoaded

There is no catalogue.db beside the collection.

catalogue_path(path='brickwork.db')

The catalogue that belongs to the collection at path: catalogue.db in the same folder.

connect(path='brickwork.db', *, read_only=False, require_catalogue=True)

Open the 4.x collection at path (upgraded first when older, unless read_only) with its catalogue attached read-only.

Without the catalogue file require_catalogue decides: True (the default) refuses with the way to get one; False opens the collection alone (setup, and the first-run page that tells the user to get a catalogue).

upgrade(path)

Upgrade the collection at path in place when it is at an older 4.x version (app start; schema notes 4.1). A no-op when it is current; the refusals read as connect's.

contract

Location and deployment diagnostics for the OpenAPI contract (API v2; the v1 contract file is a reference only, M.11e, issue #113).

contract_v2_path()

The API v2 contract (M.11a, issue #113); the v2 blueprint serves it and checks its own routes against it, and the MCP adapter reads its tools from it (M.11e).

csv_safe

Moved to brickwork.store4.csv_safe (M.8a: store4 may not import core); this keeps the core callers working.

currency

Currency conversion for display and totals (#39). Stored amounts never change currency.

Rates are kept per 1 EUR (the ECB reference format), so the display currency can change without touching a rate: 1 EUR = rate × currency, EUR itself is 1. A rate is entered by hand or fetched from the ECB's keyless daily reference rates. convert() returns None when a rate is missing; callers then keep their per-currency behaviour. Converted figures are always shown marked (≈, with rate and date).

configured_currency(db)

The BrickLink currency preference of the collection (setting.bricklink_currency), else the documented default EUR.

rates(db)

{currency: dict(per_eur=Decimal, as_of, source)} including EUR = 1.

convert(amount, source, target, table)

amount in source as target, with the rate used; None without both rates.

set_rate(db, currency, per_eur)

Enter or clear (per_eur None) one rate by hand; one audited write.

fetch_ecb(db, http=None)

Replace the ECB-sourced rates from the daily reference feed. Hand-entered rates stay.

drainer

The embedded job drainer loop, shared by the web app and the API.

poll_delay(db)

None when nothing is queued, running, waiting or due for a retry; else seconds to the next look (at most 60).

pending(db)

Is there work for the drainer at start-up: an active job, or copies still pending on a finished one (jobs.has_leftovers; the process stopped before settling them). One drain settles what can be settled, then poll_delay lets the worker stop.

queue_startup_work(path)

At start (the embedded drainer's and brickwork worker's): a catalogue swapped in but not reconciled yet becomes one reconcile job (design 6.2), run by the drain like any other. Returns the job id, or None when nothing is due.

requeue_configured_keys(db)

139: a key set in the environment takes effect at a start: the steps that waited for one of the configured sources run again.

drain_once(path)

One pass of both runners. The fetch jobs go first: a BrickLink or Rebrickable list a pipeline job looks for may have landed. Then the copies a finished Fetch from Rebrickable asked for are made (add.settle_fetches); their pipeline job runs on the next pass.

loop(path, sleep, *, stop=None)

Drain due jobs, then sleep until the next deadline; return when nothing is pending.

spawn(target)

Seam for the embedded drainer's thread; tests replace it.

sleep(seconds)

Looked up at call time, so gevent's patching applies and tests can replace it.

enabled()

BW_EMBEDDED_WORKER=0 leaves draining to brickwork worker on headless installs.

start(app, path, fallback=None)

One drainer per app; a daemon thread is the standalone default.

The active flag is set and cleared under a lock (#68), and the drainer checks for due work once more before it clears the flag, so a job queued while it was stopping is not left waiting.

worker(*, path, stop=None, poll_seconds=None)

brickwork worker: drain the queue in this process until nothing is pending (the headless counterpart of the embedded drainer).

errors

Stable domain failures and their transport status, independent of Flask.

BrickWorkError

Bases: ValueError

A public failure with a stable code; detail must never contain credentials.

gaps

The gaps of copies as a BrickLink Wanted List (M.8d, issue #110; design 3.2, 4.4, 4.5).

A gap is what a copy lacks: a live set line's missing and damaged pieces (a retired line is no line; spares and a figure's own parts are not set gaps) and a live figure line's missing figures. The export names every gap by the item's BrickLink number, type and colour number (core.items.bricklink_key, which goes through store4.identity.resolve). An item with no confirmed BrickLink number cannot be named in a wanted list: it is skipped and listed, never exported under a made-up number and never a reason to stop the export. Its Planning route is planning.export (M.8g), which names a repair's remaining quantities through export_lines; the printable list is list_exports.

copy_gaps(db, copy_ids)

The gaps of live copies: [dict(copy_id, item_id, color_id, quantity, condition)], set lines first, then figures, in line order.

missing_lines(db, copy_ids)

The gaps combined per item, colour and condition, each named for BrickLink where it can be: (lines, skipped).

A line is dict(item_id, color_id, condition, quantity, name, color_name, bricklink=(type, number, colour number)); a skipped line is the same with bricklink=None and why (no BrickLink number or no BrickLink colour).

named_lines(db, total)

missing_lines for quantities already combined: total is {(item id, colour id, 'N'|'U'): quantity}.

export(db, copy_ids)

dict(xml, lines, skipped): BrickLink Wanted List XML of the named lines, the lines in it and the ones left out (and why).

export_lines(db, total, named=None)

The same for quantities already combined (total, see named_lines), e.g. what a repair still has to buy.

identify

Identify on the 4.0 store (M.8h, issue #110; design 5 identification, 3.2, 3.8): photo recognition mapped to item ids, and its history.

Recognition. Brickognize (core/sources/brickognize.py, budgeted and audited like every source) answers with BrickLink-style type letters and numbers. candidates turns each prediction into an item of the collection or catalogue, by the rules every number a person or a provider gives follows: the Rebrickable (born) number first, then the items a confirmed BrickLink link names (the pooled store4.identity.candidate_items: collection and catalogue together), then the catalogue's search index. A number that still names several items is listed apart as ambiguous, never guessed; a number that names none is listed apart as unknown. recognise is the one call the screens make (Part Spotter, What is this?, Add's photo mode): photo → choices and the unlisted.

History. store keeps one result in identification (candidates with their names and scores, a clean 256 px thumbnail under data/identifications/; the original photo stays in memory); archive hides it. Both write an identification:<id> event (create, archive) that undo reverses: created → archived, archived → restored, refused when the record changed since. The Part Spotter batch writes its own records inside its batch receipt (core/part_inventory.py) and is undone with the batch.

thumbnail(data)

A clean JPEG thumbnail (EXIF, ICC and other metadata dropped), or None if unreadable.

pick(db, kind, number)

(item id or None, why) for a number a provider gave: the born match, else the one item a confirmed BrickLink link names, else the one item the search index finds. Several candidates at the same step place nothing (why says so).

candidates(db, predictions, *, kinds=('part', 'minifigure'))

(choices, unresolved) for recognition predictions [dict(item_type 'P'|'M'|'S', item_no, score)].

A choice is the item decorated for a screen (collection.decorate) with score and what you already hold of it (owned_summary). A prediction of another kind than kinds is left out; one that names no item, several items, or an item still to check is listed in unresolved with its reason.

recognise(db, photo, kind=None, *, kinds=('part', 'minifigure'), http=None)

Ask Brickognize about a photo and map the answer: (choices, unresolved). kind (part, minifigure or None) picks the endpoint. The source counts the call against the daily budget and keeps the reply; a failure is a SourceError with no key in it.

store(db, root, photo, candidate_rows)

Record one result and its thumbnail in the caller's transaction; audited for Undo.

archive(db, identifier)

Archive a record (thumbnail kept); Undo restores it.

undo(db, event_id)

Undo a stored or archived record by its event id; a second Undo does nothing.

identity_admin

Identity admin on the 4.0 store (M.8i part 2, issue #110; design 3.2, 3.5, 3.6, 3.8): what needs a decision, and the decisions.

The Unresolved screen lists items, each with why it is there:

  • merge: a local item that may be an item the catalogue knows: it carries to_check Rebrickable links (the migration's candidates), or it is BrickLink-born and a confirmed BrickLink link on another item names the same number (the rule-4 tie, 8.3). Resolved by merge_item.
  • link: an item with a to_check link that is not such a candidate (a composition match of a figure, a same-number guess). confirm_link makes it the user's (manual), reject_link removes it.
  • status: an item_provider_status of to_check (no candidate, migration, kind differs). A kind difference is resolved by move_item_kind; the others by settle_status (the user looked: it stays unmatched) or by a link or merge.
  • unidentified: an item with no number from any provider that something references. Resolved by add_link (a typed BrickLink number) or a merge.

An item that nothing references and that has no candidate is not listed (a catalogue item with a to_check guess nobody owns, an undone Add's leftover). Every decision is one transaction and one event with a receipt, and has an Undo (core/undo).

count_unresolved(db)

How many items unresolved lists.

reason_counts(db)

{reason: items} over the list (an item with two reasons counts under both).

unresolved(db)

The list the Unresolved screen shows: one dict per item, ordered by kind and name.

item_id, kind, name, number_text, image, local, reasons, facets (what the screen filters by: the reasons, with status replaced by the status reasons no_candidate, kind_differs, migration), candidates (with name, number, picture and uses: what the user's copies hold of that candidate), links (the to_check links to confirm or reject), statuses, references (collection rows that name the item) and uses.

uses(db, item_id)

What the user's rows hold of an item: dict(copies, lines, stock, wishes) (copies that hold it, live part lines, loose stock rows, live wishes).

On a merge row's candidates this says which candidate the user's copies already use (the M.10 note: the migrated part lists name it).

find_target(db, loser, text)

The one item a typed number means for loser (same kind, not itself); a ValidationFailed says when it is none or several.

merge_item(db, loser, survivor)

Merge the local item loser into survivor (one transaction, one event, one receipt). Returns {merge_id, event_id, counts}.

move_item_kind(db, item_id, kind)

Change an item's kind with its rows (design 3.5). Returns {event_id, from, to, counts}.

change_item_variant(db, from_id, to_id)

Change an item's variant: everything owned moves from from_id to to_id (same kind), and BrickLink's number becomes to_id's (issue #151). One transaction, one event, one receipt. Returns {event_id, from, to, number, counts}.

A to_check link becomes the user's (manual). A Rebrickable candidate of a local item is not confirmed: that is a merge.

Remove a to_check link (Undo puts it back).

Link an item to a typed BrickLink number (manual), for every colour or (color_id, a Brick Work colour) for one. Refused when the item has that number in that scope already, or another item holds it (for one colour: holds it for that colour): that is a merge. (A number scoped to a colour may sit beside another item's unscoped link of it: the lookup tells them apart by the colour, design 3.2.)

set_preferred(db, link_id, color_id=None, item_id=None)

Make a link the preferred one of its item and provider: for every colour (color_id None; the link must hold for every colour), or for one colour (design 3.2: a new colour-scoped manual row, or the item's scoped row of that key made preferred). The previous preferred link of that scope loses the mark. item_id, when given, must be the link's item (the item page names its own). One logged change, Undo puts every row back.

settle_status(db, item_id, provider, reason)

The user looked and the item stays unmatched: a to_check status becomes unmatched.

merge_blocker(db, item_id)

Why an item cannot be merged away, in plain words, or None (design 3.6: a local item, or one the catalogue has retired).

The Links card of an item page: per provider the keys with status, colour scope, preferred and which one answers for colour, plus the provider status rows that have no key. editable is False for an item that is only in the catalogue (nothing to write to yet).

explain_refusal(db, text, loser=None, survivor=None)

A MergeRefused message in plain words for the merge screen (the refusals of design 3.6).

merge_preview(db, loser, survivor)

What merge_in would do, without doing it: runs the merge in a transaction and rolls it back (a write connection is needed).

dict(ok, refusal, rows, dropped, counts): rows = [(table words, rows changed)] biggest first; dropped = rows the survivor already has (kept as they are, the loser's go); refusal = plain-words reason, or None.

variant_blocker(db, item_id)

Why an item has no Change variant screen, in plain words, or None: a live catalogue figure or part of the collection that is not merged. (The rules of the change itself, such as a set, stay with the preview; a set, an item of yours or a retired one is merged or left alone here.)

variants(db, item_id, text, limit=12)

The items this one may change to, [dict(item_id, name, number_text, image, kind, year, how)]: live catalogue items of the same kind, never itself.

With no text, the suggestions: items that hold one of this item's BrickLink numbers (any colour scope; how = same_number) and items named by one of its links to check (to_check). With text, the number or name search of the merge page, narrowed to live catalogue items of the kind.

explain_variant_refusal(db, text, from_id, to_id)

A MergeRefused message of the change of a variant in plain words for its screen (the refusals of issue #151).

variant_preview(db, from_id, to_id)

What change_item_variant would do, without doing it (a write connection is needed, as for merge_preview).

dict(ok, refusal, busy, rows, photos, number, counts): rows = [(label, n)] for the rows that move, zeros included (loose stock, the lines of figures or parts in copies, wishes); photos = the photos that go with the stock; number = BrickLink's number that follows, or None.

unmerge_blocker(db, merge_id)

Why a merge cannot be taken back, in plain words, or None: the checks of unmerge_in, read-only.

merged_into(db, item_id)

The merges still standing into an item: [dict(merge_id, merged_id, name, number_text, at, via, blocker, counts)], newest first.

survivors(db, loser, text, limit=12)

Items the user may merge loser into: those a typed number finds, or (with no text) the candidates the Unresolved list names. [dict(item_id, name, number_text, image, kind, how)], same kind as the loser, never the loser.

importer

Import from BrickTracker (M.8j, issue #110): bricktracker plans and applies the import; bricktracker_upgrade and its vendored upstream migrations bring a copy of any upstream database to the newest schema it knows before it is read.

bricktracker

Import a BrickTracker database into the 4.0 collection: plan, then apply, undone as a whole (M.8j, issue #110; design 3.1, 3.5, 4.3, 5).

BrickTracker stores Rebrickable numbers, and Rebrickable numbers are the catalogue's born keys, so nothing here translates anything: a set number, a part number, a figure number and a colour number each name one item or colour, or none. Any upstream version is read: a copy of the user's app.db is upgraded with upstream's own migrations (bricktracker_upgrade), then read; the file itself is never written.

plan writes nothing to the collection. It keeps the upgraded copy under data/imports/<plan_id>/ and lists, per set, loose figure and part, lot and wish, what applying would do: add it, skip it as already imported, or ask about a duplicate (the same set already held, the same stock, lot name or wish). Whatever cannot be placed is listed with the reason (a set the catalogue does not know, a part line the copy's part list lacks) and never guessed. Storages, purchase places and flags are matched by name and reused.

apply re-reads the stored copy (deterministic) and writes the chosen rows in one transaction with one import:<plan> receipt event, like Add: an owned set becomes a copy through store4.copies.create_copy_in with design 4.3's preselection (its lines come from the part list, BrickTracker's missing / damaged / checked marks are put on the lines they name), every other owned or loose thing becomes stock of its current kind (a loose figure with its own part list, its marks on that list), a wish becomes a wish. undo reverses exactly that receipt (core.undo) and refuses over later edits. BrickTracker records no currency: purchase prices take the collection's BrickLink currency setting. Images and instruction files are not copied (Brick Work fetches pictures through its sources).

Lookup

Rebrickable numbers to the collection's ids: an item by (namespace, number), a colour by its Rebrickable number. Cached.

project(db, path)

The upstream file as the collection would take it: dict(copies, stock, lots, wishes, places, storages, flags, unplaced, kinds).

Pure reading: nothing is written. Each entry carries the resolved item id, or goes to unplaced with its reason.

plan(db, source, *, name=None)

Upgrade a copy of source, read it and compare it with the collection. Writes nothing to the collection.

apply(db, plan_id, decisions=None, *, queue=True)

Write the chosen rows in one transaction; returns the receipt event id and counts.

undo(db, event_id)

Reverse one import exactly (core.undo); refused over later edits.

bricktracker_upgrade

Upgrade a copy of a BrickTracker database to the newest upstream schema this importer knows.

The user's file is opened read-only and copied with SQLite's backup API (consistent even with a WAL); the vendored upstream migrations then run on the copy only, exactly as upstream applies them: the SQL file (rendered with the Python step's context where upstream has one), then PRAGMA user_version = N.

check(db)

The upstream schema version of an open database, or ValueError when it is not one this importer can read.

upgrade(db)

Apply the vendored migrations after the database's version, in order. Returns (from, to).

apply(db, start, stop=NEWEST)

Run migrations start+1..stop on db without checking it (an empty file gives a fresh upstream install).

upgraded_copy(source, target)

Copy source (never written) to target and upgrade the copy. Returns (from, to).

items

The app's identity facade over the 4.0 store (M.8a, issue #110; design 3, 6.3).

Every screen and writer that used to say (item_type, item_no) says an item id and asks this module for the rest:

  • label(db, item_id, color_id=None): what to show for an item (decision 3 of the #110 scope: the Rebrickable born number first, the preferred BrickLink number beside it where it differs; a local BrickLink-born item shows its BrickLink number, marked BrickLink).
  • find(db, kind, number): items for a number a person typed or pasted: born key, then the catalogue's search index, then BrickLink links.
  • colours_held(db, item_id): the colours a part is held in, for the per-colour source steps.
  • ensure(db, item_id): the item (and its links and colours) in the collection, projected from the catalogue when it is not yet.
  • bricklink_key(db, item_id, color_id): the BrickLink type, number and colour number, or Unidentified.
  • the kind vocabulary and the BrickLink type letter <-> kind mapping, which belong to the borders (imports, exports, BrickLink calls) only.

An item is read from the collection when it is projected there, else from the catalogue. Nothing here writes except ensure, which writes inside the caller's transaction (see store4.identity.project).

Unidentified

Bases: Unidentified

The item has no BrickLink number: a BrickLink call, list or export cannot name it.

Label dataclass

text property

One line for a screen: 3001, 3001 · BrickLink 3001a, BrickLink x1718c01 (BrickLink), or the name of an unidentified item.

kind_of_type(letter)

The kind of a BrickLink item type letter (S, P, M, G, B, C, I, O).

type_of_kind(kind)

The BrickLink item type letter of a kind.

label(db, item_id, color_id=None)

What to show for an item (and colour). Raises NotFound for an id neither file knows.

labels(db, pairs)

{(item_id, color_id): Label} for many pairs at once (list screens): the same answers as label, in a handful of queries (items, links, colours). A pair whose item neither file knows is left out.

identified_sql(alias='i')

SQL predicate on main.item aliased alias: the item has a number from some provider (a born key) or a confirmed BrickLink link (design 3.8: resolving an unidentified item is adding a link; born keys never change). What the Unresolved list, picture fetch and bulk refresh call "not unidentified".

(BrickLink type, number, colour number) of an item in a colour, or Unidentified when no confirmed link names it.

colours_held(db, item_id)

The colours a part is held or wanted in (copy lines, loose stock and loose figures' parts, wishes), ascending; [None] for a part held in none and for every item that is not a part. What a per-colour source step (pictures, prices) iterates.

ensure(db, item_id)

The item in the collection: projected from the catalogue (with its links and colours) when it is not there yet.

Writes inside the caller's transaction and does not commit; the caller's commit or rollback decides (store4.identity.project).

find(db, kind, number, *, limit=20)

The items a typed number may mean, best first: [dict(item_id, kind, name, number, matched_by)].

Order: the born key (the Rebrickable number, or a local item's BrickLink number), then the catalogue's search index over the born and every linked number, then confirmed BrickLink links. kind is a kind (or None for all). A merged item is replaced by the item it was merged into. Search takes both numbers: 3001 and the BrickLink 3001a find the same part when a link says so.

keys

Encrypted source-key store: one service for the API and the Catalogues & keys page.

Keys come from the environment first, then from source_key (Fernet, secret from BW_SECRET_KEY, see secrets.py). Only configured, origin and saved_at ever leave this module; values and ciphertext are never returned, logged, audited or put in an error.

fields(source)

Public input names: key for single-key sources, else the env name without its prefix.

field_map()

Input names per keyed source, for the Catalogues & keys form.

known(source)

A keyed source; raises the typed error a client should see otherwise.

stored_states(db)

Every live stored key's saved_at in one query, for lists.

state(db, source, names=None, saved=None)

Key status without decrypting anything: configured, origin (env|stored|null), saved_at.

credentials(db, source, names, *, label=None)

Resolve key values for a client: complete environment first, then the decrypted store.

known_user(source)

A per-user credential slot (rebrickable_user, brickset_user).

clean(source, payload)

Validate write-only input; messages never echo a submitted value.

save(db, source, payload, receipt=None)

Encrypt and store a key bundle (replace allowed). Returns the public key state.

receipt, when a list, gets the id of the audit event this call wrote.

archive(db, source, receipt=None)

Archive the stored key (it stops being used); Undo restores it.

undo(db, event_id, **options)

Reverse one save or archive receipt by its event id (core.undo.undo_event).

provider(path)

The credential provider for store4.keys (M.8a): the environment first, then the encrypted source_key table.

path is the collection file, or a callable returning it (looked up at call time, so the app's configuration applies). A source's credentials come from the environment only when all of its variables are set (as credentials does), else from the stored bundle; a missing, archived, undecryptable or unreadable entry is simply "not configured" (None), never an error text.

install(path)

Make provider(path) the credential source of the 4.0 fetchers. Idempotent; the app calls it once at start.

check(db, source, **options)

One budgeted call that proves the key works. The result carries no key material.

strip(snapshot)

Remove every stored key (live, archived, previous) from an in-memory export copy.

secure_delete zeroes the freed pages and VACUUM rewrites the file, so no ciphertext survives in the serialised bytes.

ledger

Storage on the 4.0 store (M.8e, issue #110): the counts, the page projection, and the writes with their history and Undo.

A write is one ledger receipt event (storage, place, move, archive): the tables it changed, before and after, as collection.receipt writes them. Undo is the collection's generic undoer (refused unless every row still equals what the request left) behind the three storage guards of 3.3 (undo_guard). The order receipts are core/orders.py's (M.8h): they use the same tracker.

visuals(db, rows)

Decorate rows that carry an item_id (identity, colour, picture) using a constant number of queries.

history(db, *, actions=None, limit=200)

The newest ledger saves, filtered in SQLite: [dict(id, at, receipt=dict(action), receipt_count, undone, undo_available)].

undo_guard(db, event, via, actor)

The storage guards, then the generic undo: a created location with contents or sub-locations stays; a moved lot that gained parts in its new storage stays; an item put back into a location that is no longer live stays.

undo(db, event_id)

Undo a ledger save by its event id; a second Undo of the same save does nothing.

undo_history(db, event_id)

Undo a ledger save from its history list; a save already undone says so.

storages(db)

Per-storage counts. part_lots is lots plus loose parts that are not inside a lot kept in the same storage.

Figures are minifigures; other is instructions, boxes, gear, books and catalogues. The last row is the virtual "Not stored".

storage_detail(db, identifier, rows=None)

rows are already-loaded storages(db), so a page does not run that query twice.

storage_view(db, identifier, rows=None)

The Storage page projection: storage_detail plus what the page groups by.

Adds the lot a part sits in (only a lot kept in this same storage) and the lots themselves, and splits the rows into sets, figures, part lots (lots, then loose parts) and other. A lot's size counts only the parts in its own storage, which is what a move carries.

storage_contents(db, identifier)

Live references to a storage in three queries, including a lot's own location: [(table, row)], tables copy, stock, lot.

storage_save(db, identifier, name, *, type=KEEP, parent_id=KEEP, note=KEEP)

Create or change a storage: type (STORAGE_TYPES or empty), parent_id (another live storage, never itself or one inside it) and note; fields left at KEEP stay as they are. Returns (storage id, event id).

place_kind(db, identifier, kind)

What kind of place a purchase location is; one receipt, Undo in History.

move(db, identifier, selections)

Move copies, loose stock and lots to a storage (None: Not stored). A lot carries its parts that sit in the lot's own storage.

storage_archive(db, identifier, *, move_to_unstored=False)

Archive a location and optionally move its live contents to Not stored in one receipt.

list_exports

Printable and CSV lists (#43) on the 4.0 store (M.8e, issue #110): missing parts, Part Runner results and the wishlist.

The same rows feed the CSV download and the printed page; colour names always travel with the colour id so a black-and-white printout still says which colour. Every row names its item as the screens do (core/items.py: the Rebrickable number first, the preferred BrickLink number beside it where it differs; an unidentified item by its name) and carries its picture (image).

missing(db, copy_ids)

Missing items of the chosen copies, combined per item, colour and condition, as the XML export does, except that items with no BrickLink number stay on the list (marked) instead of being left out: a printout is still useful while they wait to be matched.

list_records

Compact display-only Collection records; no database reads or writes.

Positional fields avoid repeating keys thousands of times. Filter defaults are omitted and restored by the shared browser decoder. Private instance metadata never leaves the server; instance ids remain available for bulk actions.

4.0 (M.8b): a row is an item id (item_id), conditions and statuses are words (design 5.1), and a screen writes the number as core.items says (Rebrickable number first, the BrickLink number beside it where it differs, a local BrickLink-born item marked): number, also (the BrickLink number where it differs) and source (bricklink for a local BrickLink-born item), put together by the browser; a search takes both numbers. An unidentified item has no number and shows its name.

listing

Shared collection filtering and stable keyset pagination, without database reads. Conditions and statuses are words (design 5.1); a search takes the Rebrickable number and the BrickLink number (also).

ordered(rows, sort, direction, group='')

rows in comparator's order, by keys: each row's values are computed once, not once per comparison.

Stable sorts from the least significant key to the most: the tie-breaks (number, then id, read only where numbers tie), the sort key (reversed where the comparator negates it), rows with a price first for used_price, then the group. Python's sort is stable, also with reverse, so the order is the comparator's.

manual_prices

Manual observations on the 4.0 store (M.8g, issue #110): a price a person records, and its Undo.

An entry names an item by id (core.items); the observation is written by sources.base.observation as source manual, so it becomes the current one of its key (item, colour, condition, guide, slot, region, currency) like any other. Undo voids the observation (voided_at) and, when it was the current one of its key, makes the latest remaining observation current again: history survives every reversal, and a voided observation is never shown (design 5, price_*).

record(db, entries)

Record 1 to 500 observations under one receipt; returns their ids.

undo(db, event_id)

Undo a manual-price receipt by its event id; a second Undo of the same event does nothing.

mcp

OpenAPI-driven MCP adapter: one tool call is one authenticated API v2 request (M.11e, issue #113).

Tools take numbers, never a Brick Work item id (decision 7): the contract marks an item-id path parameter x-mcp-item: true and the item schema x-mcp-item: key; the tool exposes provider, kind and no (key inside a body) for them, and the adapter resolves the number with GET /lookup before the call. Collection records (copy, stock line, wish, storage, ...) keep their ids, taken from list results.

Adapter

item_parameter(op)

The path parameter of op that is a Brick Work item id (marked x-mcp-item), or None.

item_properties(key=False)

What a tool takes in place of an item id: the provider, the kind and the number (key in a body, as the API names it).

without_ids(schema)

schema with every item key (x-mcp-item: key) reduced to provider, kind and key: a tool never names an item by id.

client

The two API transports. Neither calls domain services.

HTTPClient

base_url is the API root, for a default install http://127.0.0.1:3333/api/v2.

server

Official SDK protocol hosts for stdio and Streamable HTTP.

protocol(adapter)

The adapter supplier is request-local for HTTP and fixed for stdio.

http_app(flask_app, *, allow_write=False)

ASGI host: /mcp plus the same Flask application's WSGI routes.

media

Where pictures and PDFs live (#27): BW_MEDIA_PATH, or by default the folder that holds the database.

Stored paths stay relative ("data/images/...", "data/instructions/...", "data/thumbs/...", "data/identifications/..."), so moving the media means moving that data/ tree to the media folder; nothing in the database changes. The database, the BrickLink catalogue and import or sync plans stay beside the database. Every writer and every serving route resolves files through media_root() and keeps its is_relative_to check against it.

root_for(database_path)

The media root for a database file path.

meta

Local API metadata; no source transports or legacy stores.

order_exports

Order export on the 4.0 store (#44, M.8h): one row per saved order line, or one summary row per order, as CSV.

Totals are never summed across currencies. BrickLink's shipping, insurance and fees are not stored separately (only the order's grand total in purchase_order.cost), so the summary shows the grand total and the difference to the line totals as "Other costs" rather than inventing a split. The summary lists only orders with saved lines: an undone order and a draft never reviewed have none (#155).

A line is named the way BrickLink names it (type letter, number, colour number: items.label(...).bricklink), the colour by its Brick Work name. A saved line always has an item; a draft's lines are not exported.

order_missing

Order missing parts (#225): the copies with gaps, a choice among them, and the combined order list of that choice.

Read-only, on the same rows as Complete sets: planning.ranking gives every live copy with a gap, its lines already net of the loose stock the repair settings let it use, so the list here is the file planning.export writes for the same copies, line for line. The one thing added is where each combined line comes from: sources, the ticked copies and their quantities. Loose stock is taken from one pool over all copies in set-number order, as the export does, not afresh per choice. The XML itself is planning.export; nothing here writes it.

sell_flag(name)

Whether a flag's name says its copy is for sale or to be parted out.

candidates(db)

Every live copy with a gap, in set-number order: the ranking row plus parts and figures (the recorded gap), gain (the net gain of repairing it, or None where the ranking gives none), to_sell (the matching flag's name or None) and checking.

left_out(row, *, sell, gain)

Why a copy is left out of the default choice, or None: its sale flag (while sell) or no gain (while gain asks for it).

visible(rows, *, query='', theme='')

The rows the search and the theme leave on the screen.

ordered(rows, sort)

rows in the chosen order (ties by number, then copy).

combine(rows, chosen, quotes=None)

({(item id, colour id, 'N'|'U'): to buy}, {same key: [(copy id, to buy)]}) over the chosen copies only.

The grouping is planning.export's: the quantity still to buy of each line, per item, colour and condition.

order_list(db, rows, chosen)

The order list of the chosen copies: dict(lines, skipped, pieces).

A line is gaps.named_lines' (name, colour, condition, BrickLink key) with quantity, the screen's words for the item (number_text, image, colour rgb and ink) and sources: [dict(copy_id, label, quantity)], the largest first. skipped are the lines the file leaves out (no BrickLink number or colour), the same way. pieces is the sum of the lines in the file, a figure counting as many as its quantity.

line_key(line)

The line's address in the query: item, colour (0 for none) and condition, <item>.<colour>.<N|U>.

parse_choices(values)

{line key: 'bl'|'pab'} from the query's src values (<line key>.<b|p>); anything else is ignored.

element_ids(db, lines)

{(item id, colour id): Pick a Brick element id} for the lines: the catalogue's lowest element id of the part in that colour (a part and colour can have several; the lowest is always the same one).

split(db, order, *, prefer, chosen, config)

order (from order_list) with every line placed: source ('bl' or 'pab'), element_id (or None), key and price (the BrickLink quote's price as a Decimal, or None), then split: the totals per source.

A line goes where the query's chosen says; else to Pick a Brick when prefer is on and the line has an element id; else to BrickLink. A choice of Pick a Brick for a line with no element id is ignored. Pick a Brick has no price here (its prices are not looked up); the BrickLink total counts the lines with a quote and says how many have none. Shipping is the Repair settings' for the BrickLink part.

pick_a_brick_csv(order)

The Pick a Brick upload file of the lines placed on Pick a Brick: elementId,quantity, one line each, in the list's order.

orders

Orders on the 4.0 store (M.8h, issue #110; design 5 purchase_order, order_document, order_line, line_receipt, allocation, 3.2, 3.8).

An order is a BrickLink purchase; a document is one way it came in (the order XML, the store's e-mail, the BrickLink API), reviewed in phases (draft → reviewed → saved; undone when its import is taken back). A line keeps the provider's own words verbatim (raw_*) and, where it could be placed, the item and colour it stands for. Placing is by item id: a BrickLink type, number and colour number go through store4.identity.resolve_inverse(..., exhaustive=True) (the pooled answer: the collection's links and the catalogue's), so a key that names two items is never guessed. It becomes an unresolved line (state='by_hand', the candidates in suggestions_json) that the person matches in the draft. A colour name from an e-mail goes through the BrickLink colour names of color_link, an item name through exact names (the BrickLink item list when loaded, else the catalogue's); several hits stay for hand matching.

Receipts count what came out of the bag per saved line (line_receipt, every change logged in line_receipt_log). Allocation is explicit: a received piece goes to a copy's gap (the Workbench's gap semantics, core/workbench.py) or into loose stock, never by itself.

Every write is one event orders:<order id> / <action> whose receipt lists the tables it changed, before and after (the Workbench's Tracker, here Journal). Undo (core/undo.py) refuses unless every row still equals what the request left, puts the order rows back (an import is not erased: its document becomes undone, its lines and the raw record stay as provenance) and hands the copy and stock rows to the collection's generic undoer. A guard row keeps a copy's status as the allocation left it.

Journal(db)

The tracker of a request. Built on first use: workbench (the base class) imports add, which registers every undoer through undo, which imports this module, so the class cannot be made while this module loads.

copy_guard(db, copy_ids)

The status of copies as an allocation left it: the copy row and its status flags.

lines(db, order_id)

Every line of an order with its document's phase, what was received and what is allocated (live allocations), and its item's kind.

decorate_lines(db, rows)

Identity, colour and picture for lines that have an item; the provider's own words for the ones that have not (unplaced).

history(db, order_id, *, limit=200)

The newest saves of an order: [dict(id, at, receipt=dict(action), receipt_count, undone, undo_available)].

allocation_rows(db, order_id, line_rows)

The allocations of an order, newest first, each with its line, a target label and the event that undoes it (None when it is not the allocation's own save any more).

parse_xml(text)

BrickLink order or inventory XML → (rows, header): the provider's words only, nothing looked up.

parse_email(db, text)

A store's order e-mail → (rows, header). A line carries a BrickLink colour name and an item name, no number: the colour name is read through the colour links and the item name stays (name_query) for resolve_names. An instruction line names its set.

import_document(db, source, text='', *, choice='new', number=None, label=None, prepared=None)

Import one document as a draft. Returns dict(order_id, document_id, duplicate, event_id, placed, unplaced): how many lines were placed on an item and how many wait for hand matching. The same document again answers duplicate and writes nothing, unless it was undone: an undone document is not a duplicate, importing the same text again brings it back as a draft with its lines placed anew (#155).

fill_currencies(db)

Give a stored order the currency its raw documents name (order_document.raw_record_id -> source_record), when it has none and they agree: orders stored before a currency could stand without a total. One audited, undoable event per order; an order with no document, none that names a currency or two that differ is left as it is. Returns the ids filled. Safe to run again.

match(db, order_id, line_id, *, item_id=None, color_id=None, item_type=None, item_no=None, listed_only=False)

Place a draft line by hand: the item the person chose (an id from the catalogue search or a candidate), or a typed number that names exactly one item. A part needs its colour. listed_only (the API's BrickLink key that names no item): only a local item for a number the BrickLink item list names, never a number of another provider.

destinations(db, saved)

For each saved line, the copy gaps it could fill: the Workbench's gaps (a loose figure's own parts are not an order's concern), matched by item and colour, a booklet or box by the set's BrickLink number.

short_export(db, order_id)

The BrickLink wanted-list XML of what is still short: dict(xml, skipped). A line the BrickLink key cannot name (no confirmed number or colour) is left out and listed.

undo(db, event_id)

Undo an order save by its event id; a second Undo of the same save does nothing.

undo_history(db, event_id)

Undo from the order's history list; a save already undone says so.

ownership

What you already own of an identified part or figure (#45), across colours.

Photo identification gives the shape, not the colour, so the summary lists every colour you hold, the most held first, with where it is stored, plus the copies whose inventory needs that part (missing or damaged). Batched: a handful of queries for any number of candidates. Keyed by item id (4.0).

summary(db, item_ids)

{item id: dict(total, colours, needed_by, needed_count)} for the identified parts and figures item_ids (4.0: items are ids).

Loose stock counts parts and figures held on their own (live, quantity above zero); needed_by lists the live copies whose live, non-spare lines of that item are missing or damaged (a retired line is no line, design 4.4).

text(info)

'You have 14 in 3 colours: Black 8 (Bin A3), Red 4 (Bin C1)' or 'Not in your collection'.

part_groups

Display groups for part lists. Categories always come from item.category_name.

order(groups, mode=DEFAULT)

Order only the groups present; an unused Other section is never inserted.

part_inventory

Part Spotter on the 4.0 store (M.8e, issue #110): photo proposals and atomic inventory batches.

A batch is one part_inventory:<batch> receipt event holding the stock changes stock.add_loose made (rows before and after), the lot it created, the photos it kept and the identification records it wrote; Undo (core/undo.py) takes the stock back, archives the lot, the photos and the records, and writes event_undo. It is refused when the stock changed since (undo the later save first). The recognition call itself (Brickognize, /workbench/part-to-inventory/detect) is Identify's (M.8h, core/identify.py): candidates here is its mapping of predictions to items.

candidates(db, predictions)

(choices, unresolved) for recognition predictions of parts and minifigures: core/identify.py maps each number to an item (never guessing between several) and adds what you already hold of it.

keep_photo(db, root, item_id, color_id, content, *, stock_id=None, copy_id=None)

An own photo of the piece or set: the file under data/images/own, the row in photo (on the stock row or the copy). Returns the photo id. Add's photo mode keeps its photo the same way.

save(db, pieces, *, batch, condition='used', lot_id=None, new_lot=None, keep_photos=True)

Add up to 6 identified pieces to loose stock in one receipt. A batch id saved twice returns the first receipt (a retry).

undo(db, event_id)

Undo a batch receipt by its event id; a second Undo does nothing.

part_size

How big a part is, from its catalogue name and category, in brick volumes (a 1 x 1 brick is 1).

Set Inspector orders "Largest first" by this, so the pieces that are easiest to find in a pile and take the most room come first. The first size in the name gives width x depth x height in studs and bricks ("Brick 2 x 4" is 8, "Brick 1 x 2 x 3" is 6); plates, tiles and baseplates without a height are a third of a brick ("Plate 2 x 4" is 2.67, "Baseplate 32 x 32" is 341). A name without a size falls back to its family. It is an estimate for ordering, never stored or shown.

partout

Part-out value (#38): what a copy's present parts and figures are worth one by one.

Sum over the copy's required parts (spares excluded, ADR 0011; figure components counted through their figure) of the used quote per part and colour, plus each present minifigure as a whole (usual BrickLink practice), using the same usable-quote rule as Complete sets (fresh, in the BrickLink currency and region). Coverage says how many lines had a price; unpriced lines are listed so a price refresh can be queued. Batched: three queries for any number of copies.

A total is given only when at least COVERAGE of the lines have a price: summing the few priced lines of a set whose other parts have none yet showed a meaninglessly small value (four of 54 lines gave 0.18).

values(db, copy_ids)

{copy id: dict(value, currency, lines, priced, enough, unpriced, unpriced_count, unpriced_items)} for live copies.

Lines are the copy's live set lines (a retired line is no line, spares and a figure's own parts excluded) and its live figure lines, each as (item id, colour id): the quantity present is the quantity less the missing. unpriced names up to 50 lines (dict(item_id, color_id)); unpriced_items is the item ids (up to 200) a price refresh could fetch, an unidentified item left out.

compare(copy, partout, config=None)

Complete, as-is and parted-out side by side for one copy record carrying complete_quote/current_value.

pictures

Pictures and their source order on the 4.0 store (M.8f, issue #110; design 6.8, 3.2).

Three sources, in the order of the setting source_priority.pictures (per kind, * for the rest; Settings edits it, the migration translates 3.3's):

  • rebrickable_dump: the catalogue's picture_url for (item, colour), a plain download. The dump carries one for a part in a colour some inventory uses, and a set or figure (no colour) is served by the item's own dump img_url (#122); a part in another colour has none (the order then falls through).
  • bricklink: img.bricklink.com by the resolved BrickLink key and colour (items.bricklink_key), a plain download, no key.
  • rebrickable_api: GET /parts/{num}/colors/{id}/ (sets and figures: their own record), needs a key and is budgeted.

Defaults: parts rebrickable_dump, bricklink, rebrickable_api; everything else bricklink, rebrickable_dump.

The pipeline's pictures step runs one handler per source in that order (handler(source)); each serves the colours of the item that still have no picture and raises StepFailed when some remain, so the next source gets a go. A colour is served by a picture row of the item with that colour or none. picture_known answers what the Add review asks before confirming (design 6.8, last paragraph).

The bulk refresh (Settings, Tools & Admin) is state, selection and queue: one pictures job per held item that lacks a picture (scope missing) or every held item (scope all).

NoPicture

Bases: Exception

This source has no picture for the item and colour (the reason is the message): the order moves on.

order(db, kind)

The picture sources to try for an item of kind, in order: the stored setting (kind, else *), else design 6.8's default. A source this module does not know is ignored, so an old value never breaks a job.

served(db, item_id, color_id)

A picture of the item for this colour exists: one of that colour, or one with no colour (it serves every colour).

dump_url(db, item_id, color_id)

The catalogue's picture URL for (item, colour); a part with no colour takes the "no colour" row. Raises NoPicture.

img.bricklink.com for the item's resolved BrickLink key and colour. Raises NoPicture without a confirmed BrickLink number.

api_target(db, item_id, color_id)

(path, key in the answer) for the Rebrickable API picture of the item, or NoPicture: only a Rebrickable-born item has a number there, and a colour is asked by its Rebrickable colour number (a local, BrickLink-born colour has none).

can_serve(db, item_id, color_id, source)

Could source serve this item and colour right now? (a URL it can name; for the API also a key, since it is a budgeted call)

picture_known(db, item_id, color_id=None)

Is a picture of this item in this colour stored, or could one source of the order fetch it? The Add review warns "no picture known for this colour" when not (design 6.8): no picture_url, no resolvable BrickLink key and no Rebrickable key.

fetch(db, job, source, *, http=None, image_http=None, root=None)

Serve the item's colours that have no picture from source. Returns the number of pictures stored; raises StepDone (unidentified: nothing can picture an item that has no number), StepWait (budget) or StepFailed (some colours remain).

preview(db, item_id, *, http=None, image_http=None, root=None)

217: the picture of an item being reviewed on Add, fetched now so the review can show it (Ulf, via wiki-agent, 08.11.: "Fetch and

store when selected"). Only items without colours (sets, minifigures and the other kinds; a part's picture depends on the colour chosen later). Returns the picture id, or None when no source has one. A stored picture is a cache, not collection data; the item is projected into the collection (items.ensure) because a picture belongs to an item there; a projected item nobody holds shows in no list. The pictures step's sources in their order, without the Rebrickable API (its daily budget is the jobs'); a source that fails is passed over, so the review never waits.

handler(source, *, http=None, image_http=None, root=None)

The pipeline handler (db, job, step) of one picture source; http and image_http are the test seams for the API and the image downloads.

handlers(**options)

{('pictures', source): handler} for the three sources.

state(db)

Coverage from database rows only (no file is opened): per kind how many held (item, colour) pairs have a picture, which sources they came from, and how many are unidentified (no source can name them).

selection(db, scope, kinds)

The items a refresh would queue: one entry per item (not per colour) of the chosen kinds, missing those with a colour that has no picture, all every held item. Unidentified items are left out (no source can name them) and counted separately.

matrix(db)

What the refresh dialog adds up (bw-pictures.js): per kind and scope how many items a refresh would queue, how many unidentified ones it would leave out (unresolved), and unknown_colour, which 4.0 has none of. The dialog also asks for the kind ?.

queue_missing(db, item_ids, *, via=None, actor=None)

Queue a pictures job for each of item_ids that is identified, has a colour held without a picture (the missing rule of selection) and has no live pictures job. For the paths that add items without Add's own job (#183): call it after the write commits. One batch per call (#127), so interactive work goes first. Returns the job ids.

Only a step that has not started yet counts as busy: a running one read its colours when it began, so a colour added meanwhile gets a job of its own (Codex review). Queueing is best effort: the write it follows has committed, so a failure here (a locked database, say) is logged and never raised. The caller's receipt stays true, and Refresh pictures finds what is still missing.

queue_for_copies(db, copy_ids, *, via=None, actor=None)

217: after copies of a set are written, queue the pictures their parts and minifigures still lack (the set's own picture is its

Add job's step). The parts are the copies' live lines; a copy that later switches to its BrickLink list by itself is queued again by the drainer, and one whose offered list is accepted by hand gets the rest from Refresh pictures. Best effort, like queue_missing: call it after the write (inside a caller's transaction it is part of that transaction).

queue(db, scope, kinds, expected_count=None, via='app', actor=None)

Queue a pictures job for each selected item that has no live pictures job. Returns (job ids, selected count); with expected_count the call refuses when the estimate on the page no longer matches, so a confirmation is never a surprise.

pipeline

The import pipeline on the 4.0 store (M.8c, issue #110; design 5.2, 6.5, 3.8): one pipeline job per Add, six steps.

A job is a row of job (source import, kind item, item_id the item) with one job_step per step name; the order and the step names come from the setting import_order:<kind> (the migration carries it, kind words), default STEPS. The job survives a restart and is run by core.drainer beside store4.jobs (which runs only its own two fetch kinds and ignores this one).

What a step does until the source handlers return (M.8f):

  • identity, inventory, matching finish locally from the catalogue and the collection; no network. A step that finds nothing to work with ends done with the code unidentified (the item has no number from any provider, or no BrickLink number for matching) or not_found (an identified set or figure with no part list); a finding is not a failure, so the job finishes.
  • pictures, metadata and market_price need a source handler. Handlers are registered with register(step, source, handler) (M.8f does that). A step with no handler for any of its sources ends done with error = SKIPPED, count 0 and no source, so the job finishes and a screen can say the step was skipped; it is not failed and queue_retry leaves it alone. A handler registered later runs for new jobs; steps already skipped stay skipped (a pictures refresh is queued explicitly).

A handler is handler(db, job, step); it returns normally (step done), raises StepDone(code, message) (done with a finding, such as unidentified for an item with no number the source knows), StepWait(retry_at, why) (waiting for budget) or StepFailed(code, message) (failed). The job is done when no step failed or waits, waiting_for_budget with the earliest retry_at while one waits, else failed; queue_retry puts failed steps back.

StepWait

Bases: Exception

Raised by a handler: the step waits for a budget or a rate pause until retry_at (an ISO stamp).

StepNeedsKey

Bases: Exception

Raised by a handler: the source has no key (#139). Not a failure: the step ends done with source_not_configured and its reason, and storing the key puts it back in the queue (requeue_for_key). A key the source refused is a failure (StepFailed).

StepDone

Bases: Exception

Raised by a handler that found nothing to work with: the step ends done with the finding (unidentified, not_found).

StepFailed

Bases: Exception

Raised by a handler: the step failed with one of the job_step error codes.

Pipeline

queue_steps(item_id, steps=None, *, via, actor=None, refresh=False, batch=None)

Queue a job for the item and return its id. Writes inside the caller's open write transaction (Add's). refresh asks the steps to fetch again what they already have (a picture refresh of every held item). batch (#127) names the bulk action the job belongs to: the drain runs interactive jobs ahead of batch jobs, and Jobs shows a batch as one row.

queue_retry(job, step=None)

Put the failed steps (or the named failed step) back to queued, and the job with them.

drain(limit=100)

Run the pipeline jobs that may run now: queued ones, claims that died, and waiting or failed ones whose retry is due.

127: the next job is chosen one at a time, interactive jobs (an Add's, a single Update) before batch jobs (a bulk refresh), oldest

first within each. Before a batch job, a runnable fetch job (an Add's part list) ends the pass, so the drainer runs it next; a bulk refresh of thousands of items never keeps a new set waiting for more than the job in hand.

register(step, source, handler)

Run handler(db, job, step) for step when source is the one being tried (M.8f).

yield_worker(db)

Never switch greenlets while this connection holds a transaction.

cooperative(db, rows)

Cooperate after each unit unless a transaction is still open.

owner_pid_alive(pid, boot_id=None, start_ticks=None, *, path=None, job=None)

A same-pid row is live only when this process holds its job claim; another pid only when it is the process that claimed: same kernel boot id and same start ticks, NULL identity counting as dead where /proc answers (#167, store4.jobs.pid_alive). Where /proc cannot answer (macOS) the pid alone decides.

validate_priority(value)

A source_priority value: some of the six steps, each {kind or '*': [source ids]}.

order_for(db, kind)

The steps to run for an item of kind: import_order:<kind>, else every step in STEPS order.

sources_for(db, step, kind)

The source ids a step tries, in order: the local steps use the catalogue; the others the kind's list of source_priority (the setting when it names the step, else DEFAULT_PRIORITY).

claim_batchmates(db, step, source, *, exclude, limit, accept=lambda item_id: True)

Claim up to limit other jobs whose NEXT pending step is step on source (the earlier steps all done or failed), so a handler can ask for them in one call. Not running, not cancelled, no live owner. Each claim is the worker's own (see Pipeline._claim): a job_owner row and state='running' written in one transaction after re-checking the job, so a concurrent worker finds it owned and skips it. A waiting job counts whether or not its retry is due: the caller's own run just proved the budget allows the call. Returns the claims for finish_batchmate / release_batchmate; accept(item_id) lets the caller leave a job out.

finish_batchmate(db, claim, step, source, code=None, message=None)

The claimed job's step ends done (with a finding code, as a handler's StepDone) and the job goes back to the queue.

release_batchmates(db, claims)

Release every claim in claims (best effort: one that cannot be released does not stop the others; it is raised after).

release_batchmate(db, claim)

The call failed: the job is exactly as it was before the claim.

fetch_waiting(db)

A fetch job (store4's kinds: an Add's Rebrickable set or BrickLink part list) may run now: a batch job lets it go first.

requeue_for_key(db, source)

139: the pipeline steps that ended waiting for source's key go back in the queue, with their jobs. Inside the caller's open

write transaction (the key's save); returns how many steps.

new_batch(name)

A batch id for one bulk action (#127): its name and a random tail, such as pictures-3f9a1c2b.

batches(db, limit=20)

The newest bulk actions, newest first: each batch's steps, its counts by outcome and its failures (item, reason) for the Jobs row.

cancel_batch(db, batch, *, via='app', actor=None)

Stop what is left of a batch: its jobs not started yet, or waiting for budget, end as failed with Cancelled and no retry. A job running right now finishes. Returns how many were cancelled. One event for the batch.

drain(db, limit=100)

Run the pipeline jobs that may run now; the results of the jobs this call ran.

queue_new_items(db, item_ids)

Queue the normal import for items that are new to this store and have no job yet (Add's rule, for imports and sync).

planning

Planning on the 4.0 store (M.8f the Prices screen, M.8g the rest; issue #110; design 5 copy.pinned, wish, price_*, 3.8).

Complete sets ranks the live copies that have gaps by what a repair gains per unit spent (ranking, from the Workbench's gaps, the cached BrickLink quotes and the repair settings in setting); the Wishlist screen reads the live wishes (wishes); Prices reads the priced items (priced_items). Everything is by item id (core/items): a copy is a set item, a gap line an (item, colour). A retired line is no line (design 4.4). Saves are audited events: a pin changes copy.pinned, the repair settings change setting rows, and Undo (undo, registered with core/undo) puts them back unless a later save touched the same thing. The BrickLink XML export of selected copies names every line through core/gaps (items.bricklink_key); a line with no confirmed BrickLink number is skipped and listed, never exported under a made-up number.

ratio_cap(ratios)

Linear-interpolated 90th percentile, rounded up to a whole multiple.

ranking_sort_key(copy, group='ranked', *, include_pins=True)

One ordering rule for planning, its public list and value cursors.

compute(copies, config, now)

Pure projection. Inputs are copy records, complete quote and missing lines.

Lines already carry the shared Workbench gap and stock projection. Decimal values retain precision; no currency conversion.

configuration(db)

The repair and price settings from setting, with the documented defaults for what is not stored.

priced_items(db, kind=None)

Lean Prices projection (M.8f): one row per priced (item, colour), no observation lists.

Four bounded reads, none of them per row. The row's stale flag is true when any current quote is past its source's freshness; the display price is chosen from each (condition, source, guide)'s freshest successful observation with Collection's own selection rules. The full observation list of one identity is price_details, loaded on demand. kind is a screen letter (S, P, M, ...): only that kind's items. Voided observations never show.

price_records(rows)

Compact positional Prices rows (see PRICE_FIELDS): identity, kind, owned/wanted, the display price as text with its age and stale flag, and the item id the page links and refreshes by. Refresh is available to every row that is not unresolved.

price_details(db, item_id, colour=None)

One identity's observation list for the Prices Details disclosure, display text only.

display_prices(db, rows)

One bulk observation read for the identities of rows (items with item_id, color_id); Collection's condition/source selection.

Every row gets price_records (its observations, newest first, a voided one never), condition_prices and display_price: the price at the wish's condition when it names one, else the best overall.

max_limit(row, price)

The max a quote is held against, with its currency: a boxed quote (slot 'boxed') against the wish's max with box when it has one (#131), else, like every other quote, against its max.

max_comparison(row, rates=None, price=None, *, conversion='max_conversion')

Compare a quote on a wish (by default the price shown, display_price) with the max it is held against (max_limit).

State is 'under' (price at or below the max), 'over', 'differs' (both currencies known and different and no rate for them) or None (no max, no price or a max without a recorded currency). margin = max minus price, only for a real comparison; a converted price is kept on the row under conversion.

compare_maxima(row, rates=None)

A wish's two verdicts: the price shown against its max (max_state, max_margin) and, for a set with a quote with box, that quote against the max with box, or the max when no max with box is set (max_boxed_state, max_boxed_margin, #131).

wishes(db)

The live wishes of every active list, as Collection lists them, with the price shown, the comparison with the max, how much is filled and the retail line of a set. Filled lines drop to the bottom (#35); the order within each part stays as listed.

ranking(db)

Complete sets: every live copy with a gap, grouped and ranked (compute), decorated for the screen: rows is the whole list in screen order (ranked, free repairs, outside break-even, needs evidence) and settings the configuration used.

save(db, action, data)

pin (id, pinned: a boolean) sets copy.pinned; settings (values: all five repair settings) writes them to setting. One audited event each; returns its id.

undo(db, event_id)

Undo a pin or a settings save by its event id; a second Undo of the same event does nothing.

export(db, ids)

The BrickLink Wanted List of the chosen copies' repair lines: dict(xml, lines, skipped) as gaps.export, but the quantity of a line is what is still to buy (the gap less the loose stock the repair settings let it use). A copy with no gap is not on the list.

plans

Applied markers for import and sync plans, kept in the collection's plan table (#64; 4.0 design 5, M.8j).

A plan's rows live in a JSON file for 24 hours; whether it was applied is the database's to say. The marker is a plan row inserted in the apply transaction, so it commits or rolls back with the rows it stands for, and a second apply of the same plan finds it (the id is the primary key, so a second mark fails too). The plan file still records the result for its page, but never decides whether a plan applies.

applied(db, kind, plan_id)

The stored result of an applied plan ({event_id, counts, at}), or None.

mark(db, kind, plan_id, result, *, source=None, created_at=None)

Record the plan as applied (result: event_id, counts, at); a second mark for the same plan fails.

price_settings

Price preferences in the setting table of the 4.0 store (M.8g); no Flask dependency.

pull_list

Pull a detected set out of a lot (#202): the pull list of a candidate set for one lot, the found counts kept between visits, and Make it a copy (ADR 0029).

The list. Its lines are the set's own parts (the latest Rebrickable list, sub-sets expanded, spares left out as in ADR 0011) and its whole minifigures, as Build Scout reads them (build_scout.scout_for), so the pieces on the page are the pieces make_copy_in can move. Per line: need (what the set has), held (what the lot can give: workbench.available, damaged rows give nothing), ticked (found in the bulk and not filed yet) and found = min(need, held) + ticked, so inventory minus the lot is need - held and to find is need - found.

Progress is the found counts that are not in the lot yet, one setting row pull:<lot id>:<set item id> holding {"found": {"<item>:<colour or 0>": n}}. Saving it is not an event (a tick is a note, not a change of the collection); only Make it a copy is. The reader clamps every count to what the line still lacks, so a lot that gained the part meanwhile never counts it twice. No table, so no schema step (ADR 0029).

Make it a copy files the found counts in the lot as stock (stock.add_loose without its own audit event, "found in the bulk", used), then build_scout.plan_takes over the lot's rows gives min(have, need) per line and found_copy.make_copy_in moves exactly those into a new copy: one transaction, one event (copy / make_from_stock), the rest of the set recorded as missing on the copy, the lot keeps the leftovers. The progress row is deleted in the same transaction. Undo is Build Scout's (found_copy.undo_make_copy_in): the stock rows go back as the event left them and the copy is archived. The pieces ticked in the bulk were filed in the lot, so after an Undo the lot holds them and they count as found (held); the ticks are not restored, which would count them twice.

stored(db, lot_id, set_item_id)

The saved found counts as {"item:colour": n} (empty when none, or unreadable).

lot_pieces(db, lot_id)

Every piece the lot holds, whatever its condition (parts and whole figures).

state(db, lot_id, set_item_id)

The pull list as data: dict(lot, set, lines, totals, lot_pieces, nested, refusal).

lines are dict(key, item_id, color_id, kind, need, held, ticked, found, category_id, category) (a whole figure has kind figure and no colour; the caller orders them, the names live in the item tables). totals count pieces: need, held, found, to_find, lines, lines_to_find, figures_need, figures_found. refusal is the sentence Make it a copy would be refused with, or None. Raises errors.NotFound (no such lot or set) and ValueError (no catalogue, or a set with no part list).

moves(info)

The three numbers of the confirm: dict(moves, missing, stays) of a state: found pieces leaving the lot, the rest recorded as missing, what stays.

set_found(db, lot_id, set_item_id, item_id, color_id, found)

Save the found count of one line (an absolute count, held included): 0 to need; a count below held leaves the pieces the lot already has as they are. Idempotent, writes no event. Returns the line as state shows it. ValueError: the line is not on the list or the count is out of range.

plan_takes(info, rows)

[(stock id, amount)] so that every line of the list gets min(have, need) from the lot's rows, the largest usable row first, then the lowest id (the order of build_scout.plan_takes, which reads the parts index and so cannot plan a set of whole figures only).

make_copy(db, lot_id, set_item_id)

Make it a copy: the found pieces of the pull list move out of the lot into a new copy of the set, one transaction, one undoable event. Returns dict(event_id, copy_id, pieces, figures, missing_parts, missing_figures, completeness, stays). Refused (ValueError, nothing written): a set made of other sets, nothing found yet, no part list; errors.NotFound for a lot or set that is not there.

retail

Retail price, availability and savings from the Brickset data (#26), on the 4.0 store (M.8f).

Brickset gives, per country, the LEGO retail price (a price_observation(guide='retail', source='brickset')) and the first and last dates a set was available at LEGO (item_property). The retail country is the retail_country setting (DE by default). Status: Retired once the last date has passed, Retiring within six months of it, otherwise Available. Savings compare a copy's price paid only when it is in the retail currency, or when both rates exist (#39); anything else is "not comparable". Batched: two queries for any number of sets.

lookup(db, item_ids)

{item id: dict(price, currency, country, first, last, status)} for the sets that have any of it.

saving(paid, paid_currency, info, rates=None)

'20 % under retail' style comparison, or None when it cannot be compared honestly.

39: another currency is compared when both rates exist; the result then says converted.

spent_vs_retail(db)

Statistics (5c): price paid against retail per currency, per purchase year and per theme.

display_currency(db)

The currency totals are shown in: display_currency, else the BrickLink currency, else EUR.

secrets

Shared credential discovery for redaction and export checks.

secret_values(values=None)

Return secret values of at least eight characters, longest first.

Ignore variable prefixes. Default to the live environment; callers may also supply app configuration.

key_file()

secret.key beside the collection (the data folder that holds brickwork.db).

encrypt(value)

Encrypt a source credential or JSON bundle; the caller stores only ciphertext.

decrypt(ciphertext)

Fail closed with fixed text when the secret changed or ciphertext is invalid.

entry_available()

True when a secret key (BW_SECRET_KEY or secret.key) can encrypt; never reveals why a bad value is bad.

fingerprint(value)

Keyed digest of a JSON-able secret payload, for idempotent replay without storing a plain hash.

set_scoring

Sets versus a pile of parts (#199): one read-only scoring engine over the catalogue's inventories.

Given a parts multiset {(item id, colour id): quantity} (a figure is (item id, None)), the engine ranks the catalogue's sets by how much of the pile each set explains. Features built on it (set detection in a lot, Buildable sets) call score; nothing here writes.

Three layers, so the scale test and the benchmark can use the middle one without a database:

  • the loader (load_rows) reads the attached catalogue file into plain rows;
  • build_index(rows) turns rows into inverted lists: per (part, colour), the sets whose inventory holds it with the quantity, kept as two array('I') per key in set order. A query touches only the sets that share a part with the input;
  • rank / page score an input against an index.

Semantics (ADR 0026):

  • A candidate is a set item with a live latest inventory (is_latest, not retired) that has at least one line. Spare lines are left out (ADR 0011). A figure counts twice over: its own inventory's parts are lines of the set (quantity times the figure's quantity in the set) and the figure itself is a whole line (figure, None). They are separate lines of the set, so a loose figure and a loose pile of its parts each give their own evidence, and a set's line and quantity totals count both. A sub-set (inventory_set) is expanded into its parent with its quantity multiplied, recursively, with a cycle guard.
  • Rarity weight of a line: w = ln(N / df), N the number of candidate sets, df the number of candidate sets that hold the line. A line every set holds weighs 0; a line in two sets of 25,000 weighs about 9.4.
  • Per set: evidence = sum over matched lines of w * min(have, need) / need; coverage = evidence / sum of w over all the set's lines. The ranking is by evidence, ties by coverage, then by set id.
  • Stickers (#201): a line flagged Sticker is a hint that the print on the part comes from a sticker sheet, and the catalogue does not say which part a sticker goes on. An unconfirmed sticker raises, for each set that holds the line and a sticker sheet (a part of the category named Stickers), that line's evidence by HINT_FACTOR - 1 (a quarter): the line then counts 1.25 times its own weight, never more than 1.5. A confirmed sticker (the person chose the set; sticker:<set item id> joins the line's flags) is near-certain evidence for that set: the line's evidence for it is CONFIRMED_FACTOR times ln N, the largest weight any line can carry (one set holds it), instead of its own share; the other sets get nothing extra from that line. The adjustment is applied after the cached ranking (apply_stickers), so the cache key stays the parts.
  • Swaps (the same part in another colour, an alternate mould by part_relationship types A and M, in either direction) are listed per returned set, from what the exact match left over, and never enter any number above.

Rows

What the index is built from; plain Python values, so a test or the benchmark can make them without a catalogue.

Index

Inverted lists over the candidate sets of one catalogue build.

build_key(db)

What names the attached catalogue build: its id space, its newest build, the file's stamp and the collection's activations. A rebuild, a new download or a swapped file changes it.

load_rows(db)

The catalogue's set and figure inventories (latest, live, spares left out) as Rows. db has the file attached as catalogue.

build_index(rows)

Index from Rows. Memory is two small arrays per line, not a tuple per posting.

index_for(db)

The index of the attached catalogue, built once per catalogue build and process (a new build replaces it).

clean(parts)

{line key: quantity} from {(item id, colour id or None): quantity}; a quantity below 1 is no line.

rank(index, lines)

(ranked rows, matched line count): one (position, evidence, covered quantity, covered lines, complete lines, distinctive found) per set that shares a line with the input, best first.

marks_of(lines, flags)

{line key: {flag name, ...}} for the flags that sit on a line of the (cleaned) input (a sticker confirmation, sticker:<id>, is not a flag a person sees: stickers_of reads it).

stickers_of(lines, flags)

{line key: frozenset of the set item ids the sticker is confirmed for} for the lines flagged Sticker (the empty set: not chosen yet).

sticker_extra(index, position, share, confirmed)

What a sticker-flagged line adds to a set's evidence on top of its own share there (can be negative: a confirmed sticker replaces the share).

apply_stickers(index, ranked, lines, stickers)

The ranked rows with the sticker evidence added and sorted again (the cached ranking itself is never changed).

ranking(db, lines)

(index, (ranked rows, matched line count), index built now, ranking from the cache) for cleaned input lines: the whole ranking, not a page, so a caller that filters the sets itself (candidate sets, #200) can page after it. Cached per catalogue build and input.

score(db, parts, flags=None, *, limit=20, offset=0, detail=10)

The ranked sets for a parts multiset, dict(catalogue, input, total, limit, offset, cached, results).

parts is {(item id, colour id or None): quantity}; flags optionally {(item id, colour id or None): {flag name, ...}}, passed through to each set's matched lines and summed per flag. Results are cached per input and catalogue build (the ranking; the page's detail is rebuilt, it is cheap). Read-only.

from_lot(db, lot_id)

(parts, flags) of the live loose stock in a lot (its flags are the stock rows' tags). ValueError when there is no such lot.

from_storage(db, storage_id, *, below=True)

(parts, flags) of the live loose stock kept in a storage, and in the storages inside it unless below is false.

from_loose(db)

(parts, flags) of all live loose parts and figures.

from_stock(db, stock_ids)

(parts, flags) of a saved selection of stock rows (ids of live rows; a missing or archived one is ValueError).

Size-aware part search (#41): "1x2", "2x1", "1 x 2", "1*2" and "1×2" find the same parts.

A size key is the dimensions of the first size in a name, sorted ascending and joined by "x" ("Plate 2 x 1" and "Plate 1 x 2" → "1x2"; "Brick 1 x 2 x 3" → "1x2x3"; "Tile 1 1/2 x 2" → "1.5x2"). A query may hold a full size (any order, x, ×, * or a space between two or three bare numbers) or a partial one ("x2", "2x": any part with that dimension); the other words match as before. JavaScript mirrors this in static/web/toni/size-search.js (tested against the same table).

parse(query)

(full key or None, partial dimension or None, the remaining words).

source_ops

Source administration on the 4.0 store (M.8f, issue #110): the source list, enable or disable, and the pipeline's handlers.

A source is a row of source (seeded; enabled is the switch) and a client class in core.sources. register_handlers hands the pipeline its three steps' handlers (pictures, metadata, market_price); the app calls it once at start. The 3.3 operations that had no 4.0 premise are gone: the BrickLink colour table and item names come with the catalogue builds (catalogue import bricklink, M.8i), not from a source call.

register_handlers(**options)

Make the pipeline run the source steps: pictures (three sources), Brickset metadata, BrickLink prices. options are the test seams (http for the API calls, image_http for downloads, root for the picture cache). Idempotent.

view(db, source)

What the Catalogues & keys list shows of a source: its switch, key state and the day's budget.

sources

Offline catalogues and API clients for brickwork.db.

base

Source calls on the 4.0 store (M.8f, issue #110; design 3.2, 5.2, 6.8): writes on item ids, budgets on source_budget.

A source is a client of one provider (bricklink, rebrickable, brickset, brickognize). Everything a call needs is in the collection file: the day's counter, the pause a provider asked for (blocked_until) and the time of the last call (last_attempt_at) are the row of source_budget for (source, day); the raw reply is a source_record under the pipeline job that asked; what a reply says about an item is written on item ids (property_value, observation). store4.jobs keeps the same rows for the Rebrickable set fetch and the BrickLink part list, so one daily budget covers both runners.

A call is Source.request: refuse when the source is disabled or has no key, count the call against the budget (or raise WaitingForBudget with the time it may run again), make the HTTP request outside any write transaction, keep the reply, normalise it in one savepoint. A reply that the source refuses because of the key is KeyRejected; every other failure is a SourceError whose text holds no key material.

ColourCodeError

Bases: ValueError

A fixed, public validation message; never includes source payloads.

SourceError

Bases: SourceError

A safe, public source failure.

KeyRejected

Bases: SourceError

The source itself refused the key (an authentication error), not a network or server fault.

SourceNotConfigured

Bases: SourceError

The source has no usable key; the detail never contains key material.

NotFoundAtSource

Bases: SourceError

The provider has no such item (HTTP 404, or an empty answer to a lookup).

InvalidReply

Bases: SourceError

The provider answered, but the answer cannot be used; another try gives the same.

WaitingForBudget

Bases: SourceError

retry_after()

Whole seconds until the budget allows another call, at least one.

Reply

Bases: str

Reply text that remembers the HTTP status of an error answer.

Source

configured()

True when the environment or an encrypted stored key can serve this source.

credential_values()

Key values in the order of credentials: environment first, then the store.

ready()

Typed refusal before any request is built, reserved or charged.

reserve(*, charged=True)

Count one call, or raise WaitingForBudget when the day's budget is used or the source asked for a pause.

The write lock covers the budget, the pause, the spacing and the count together, so two processes never spend the same call or send together. When the spacing has not passed, the lock is let go, the wait is slept, and the check starts again.

block(retry_seconds=None)

The source said no more for now: for retry_seconds, else until the next UTC midnight. Returns the retry stamp.

request(kind, key, request, normalize, *, charged=True, job_id=None)

One budgeted call; returns what normalize(parsed reply) returns (it writes in one savepoint, with the reply kept).

key_refused(reply)

True when an error reply (already parsed) means the key itself was refused.

audit_origin(via, actor=None)

Keep app-initiated source writes audited as app, including workers.

stamp(moment=None)

A UTC instant as the text the 4.0 columns hold (2026-10-27T08:30:00.123Z).

parse(text)

The instant a stored stamp names (the 4.0 Z form or an offset form), always timezone-aware.

redact(value, secrets=None)

Mask secret values; callers redacting many strings pass one secret_values() snapshot.

transport(request)

Transport seam for tests; return raw text, including HTTP error replies (with their status).

write(db, sql, params=())

Execute a mutation and append its audit event in the caller's transaction (the table is the event's entity).

item(db, item_id)

The item in the collection (projected from the catalogue when it is not there yet); returns its id.

Writes inside the caller's transaction. A source never creates an item: an id neither file knows is NotFound.

property_value(db, source, item_id, key, value, unit=None)

Keep what source says about item_id under key (one row per item, key and source; the newest reply wins).

observation(db, source, item_id, *, guide, price, currency, color_id=None, condition=None, slot=None, region=None, qty=None, sold=None, minimum=None, maximum=None, provider_key=None, observed_at=None, job_id=None, error=None)

Record one price observation and make it the current one of its key (item, colour, condition, guide, slot, region, currency).

condition is new/used (BrickLink's N/U are accepted); provider_key is the provider's own key that was priced (BrickLink: P:3023/1, type, number and colour number). The observation stays when a newer one replaces it as current.

budget_used(db, source, day)

The calls counted against source on a UTC day.

record_change(db, entity, old, new)

Retain before/after snapshots for source-owned display data.

BrickLink Store API on the 4.0 store (M.8f, issue #110; design 3.2, 3.8, user journey 2): the price guide, keyed by item id.

A price is asked for an item (and colour) through the one resolution function (items.bricklink_key: the BrickLink type, number and colour number of the item's confirmed link). An item without one is Unidentified: the step ends done with the code unidentified (design 3.8, Q21) and nothing is called. The observation keeps the key it priced as provider_key (P:3023/1, S:6081-1), and a reply that proves an inferred link right upgrades it to matched/bricklink_api.

The part lists (/subsets) are store4.jobs' (bricklink_subsets); BrickLink's colour table comes with the catalogue builds; this client keeps the price guide, the key check and the order endpoints (list_orders, order_items).

Bases: Source

key_refused(reply)

A credential error: not an IP, signature-request, clock or nonce problem.

check_key()

Cheapest authenticated read; proves the four values without changing data.

price(item_id, color_id=None, condition='used', *, job_id=None)

The BrickLink price guide of an item (and colour) in one condition: the sold guide, then the stock guide when nothing sold.

Writes one price_observation per call made (the stock one only when no sale was recorded). items.Unidentified when the item has no confirmed BrickLink number; nothing is called then.

checkpointed(job_id, operation)

The recorded result of a guide call this pipeline job already made, else None.

An answer for this BrickLink key proves a rule-made (inferred) link right: it becomes matched/bricklink_api (user journey 2). manual and matched links are never touched (fetched.add_item_link).

provider_key(letter, number, colour_key=None)

The BrickLink key an observation priced, in the form the migration wrote: P:3023/1, S:6081-1, M:air003.

watch(db, item_id, color_id, condition)

Note that the item's price in this condition is kept fresh: price_watch (item, colour, condition) with the time of the last fetch. origin says why it is watched: owned when a copy or loose stock holds the item, else wanted.

market_price(db, job, step, *, http=None)

The market_price step: each colour the item is held in, new and used. Resumes after a budget wait without repeating a call that already landed (job_call keeps one row per finished key and condition). StepDone('unidentified') for an item with no BrickLink key.

brickognize

Explicit photo recognition through the budgeted, audited source interface.

brickset

Brickset API v3 on the 4.0 store (M.8f, issue #110; user journey 2): set metadata and country retail prices, never market values.

What Brickset says about a set is item_property(source='brickset') and, for the retail country, one price_observation(guide='retail'). A set Brickset does not know gets item_provider_status(brickset, unmatched, no_match) (design 3.2): looked, nothing found.

The Brickset number of an item is, in order: a confirmed brickset link (item_link), the Rebrickable number of a Rebrickable-born set (6081-1 is Brickset's form too), else the BrickLink number of a BrickLink-born set (6081 is read as 6081-1).

Brickset

Bases: Source

number_of(item_id)

Brickset's number for a set item, or None when the item has nothing to ask with.

refresh(item_ids, *, job_id=None)

Ask Brickset about sets (up to 500 per call). Returns the item ids Brickset knew; the others get unmatched/no_match.

metadata(db, job, step, *, http=None)

The metadata step: properties and retail price. StepDone('unidentified') for a set with no number to ask with, StepDone('not_found') when Brickset does not know it (a finding, not a failure: asking again gives the same).

194: Brickset charges per getSets call and one call takes up to BATCH numbers, so the step also takes the other jobs whose next

step is this one (pipeline.claim_batchmates, the worker's own claim), asks for all in one call and ends each one's step with its own result. If the call fails they are released as they were and only this job gets the failure.

images

Local picture cache on the 4.0 store (M.8f, issue #110; design 6.8). No credentials, redirects or remote image hot-links.

A picture is a file under data/images/<source>/<kind>/<sha256>.<ext> and a row of picture (item, colour or none, source, kind). Only the three picture sources of design 6.8 write here: rebrickable_dump, bricklink, rebrickable_api. The download is a plain GET of a public image URL on a host that belongs to the source; it is not an API call and counts against no budget.

picture_kind(source, kind)

Only Rebrickable's API part pictures are renders; every other picture is a photo (design 6.8, picture.kind).

download(db, root, source, item_id, kind, url, *, color_id=None, image_kind=None, http=None)

Fetch one picture from a public image URL and store it for the item (and colour). Returns the stored local_path.

The URL must be https on a host of the source; a redirect, a login in the URL or another port is refused.

store(db, root, source, item_id, kind, data, *, color_id=None, image_kind='photo', url=None, fetched_at=None)

Verify the bytes, copy them atomically into the cache, then replace the picture row of (item, colour, source, kind).

The row is written in the caller's transaction (a savepoint when one is open) with one picture:stored event; the file is written first, so a failed write never leaves a row without its file.

rebrickable

Rebrickable API v3 on the 4.0 store (M.8f, issue #110; design 3.2, 6.8): picture URLs and external-id evidence, keyed by item id.

The catalogue and the set part lists come from the dump and from store4.rebrickable_api (the rebrickable_set job); this client asks for what neither has: the picture of a part in a colour (/parts/{num}/colors/{id}/) or of a set or figure, and a part's record, whose external_ids say which BrickLink number it is. Evidence about a BrickLink number is written the way the collection records links: exactly one BrickLink id gives a matched/external_ids link; none or several give item_provider_status(bricklink, unmatched, no_match) and no link. A refresh never replaces a manual link and never downgrades a confirmed one.

Rebrickable

Bases: Source

picture_url(path, key, *, item_id=None, job_id=None)

The picture URL in the record at path (part_img_url or set_img_url). A part's own record (/parts/{num}/) also tells which BrickLink number it is: with item_id that evidence is recorded (record_match).

part(item_id, *, job_id=None)

The record of a Rebrickable-born part; its BrickLink evidence is recorded. Returns the record.

record_match(item_id, record)

Write what a part record says about the item's BrickLink number (call inside the request's savepoint).

stickers

Stickers on loose parts (#201): the Sticker flag, the sets a sticker may belong to, and the confirmed choice (ADR 0028).

A person ticks "Has a sticker" when filing a part (part_inventory.save): the stock row gets the tag flag named Sticker. The catalogue has sticker sheets (parts of the category named Stickers) but does not say which part a sticker goes on, so candidates is the fallback: the live catalogue sets whose latest inventory holds the part in that colour and a sticker sheet. The person then confirms one (confirm, "This one"), keeps the flag as a hint (nothing to store) or says it is no sticker (clear, which removes the flag).

Storage. A confirmation is an event, not a row: stock_sticker:<stock id>, field confirm (new_value holds the set, the sheet and the stock line) or clear ("Not a sticker"). The line's current state is its newest event of those two that was not undone, so Undo of a later choice shows the earlier one again. A confirmation counts only while the stock row holds the Sticker flag. The scoring engine reads it from the same events (set_scoring._stock_rows). No table, so no schema step (ADR 0028).

flag_id(db)

The id of the live tag flag named Sticker (any case), or None.

ensure_flag(db)

(flag id, created): the Sticker flag, made as a visible tag when the collection has none yet.

line(db, lot_id, stock_id)

The live loose part stock_id in lot lot_id as a dict (name, number, colour); errors.NotFound when the lot has no such part line.

decision(db, stock_id)

The confirmed sticker of a stock line as dict(set_item_id, sheet_item_id, set_number, set_name, event_id), or None.

states(db, stock_ids)

{stock id: dict(flagged, confirmed)} for the stock lines: flagged is whether the Sticker flag is on it, confirmed its decision (only while flagged).

candidates(db, lot_id, stock_id, *, limit=3, offset=0)

The sets a sticker on this part line may belong to: dict(stock, total, limit, offset, results).

Each result is a live catalogue set (not retired) whose latest, live inventory holds the part in the line's colour (spare lines left out, ADR 0011) and at least one sticker sheet (a part of the category Stickers, by name): set_item_id, set_number, set_name, year, sheet_item_id, sheet_number, sheet_name and sheet_picture (the catalogue's picture address, or None). A set with several sheets is one row, its sheet with the lowest number first. Sets with a sheet picture come first, then the earlier year, then the number.

confirm(db, lot_id, stock_id, set_item_id, sheet_item_id)

Record "This one": the sticker on this part line is the sheet sheet_item_id of set set_item_id, one event (undoable). The pair must be one of the line's candidates; the line must hold the Sticker flag. Choosing what is already chosen writes nothing. Returns dict(event_id, changed).

clear(db, lot_id, stock_id)

Record "Not a sticker": the Sticker flag leaves the line and a confirmation, if there is one, stops counting; one event (undoable). A line without the flag writes nothing. Returns dict(event_id, changed).

stock

Loose stock on the 4.0 store (M.8c, issue #110; design 2.4 invariant 9, 3.5, 4.5): add, merge and undo of an add.

The merge key of a live stock row is (item, colour, condition, lot, storage) with NULL equal to NULL, the unique index stock_merge_key (a loose minifigure never merges: it carries its own part list). Callers own the transaction and the write lock; add_loose raises when none is open. Editing and merging on edit (plan_edit, apply_edit) are the collection-writes piece (M.8d); the ledger is core/ledger.py (M.8e).

part_digest(db, stock_id)

The part lines of a loose figure as text (every column that a check, a Take or Give or a mark can change), for an Undo to compare.

dependents(db, stock_id)

Something made after the row was added that points at it or at its part lines: a Take or Give (into it, or out of it), an allocation.

add_loose(db, *, item_id, color_id=None, quantity, condition, lot_id=None, storage_id=None, note=None, via=None, actor=None, record=True)

Add quantity of an item to loose stock: onto the live row with the same merge key, else a new row. A part needs a colour and any other kind has none. Returns the change {id, created, quantity, before, after} (rows as dicts), which is what undo_added needs and what the Add receipt keeps. record=False writes no audit event: the caller's own event is the receipt (the pull list's Make it a copy).

undo_added(db, change, *, via=None, actor=None)

Reverse one add_loose change: archive the row it created, or take its quantity (and the state it merged in) back off the row it increased. Refused when the row differs from what the add left (any later change: edit, merge, take, a later add onto it), when a loose figure's part lines differ from what the add left, or when a Take / Give or an allocation points at the row or its lines.

plan_edit(db, rows, edits)

Where an edit of live stock rows lands: [(row id, target id or None)], in the order the rows are given.

A row whose merge key, once edits are applied, equals the key of another live row merges into it (invariant 9: one live row per key). The other row is a stock row outside the selection, or a selected row that kept or took that key earlier in the order. A loose minifigure never merges (it carries its own part list); a selected row whose key does not change targets itself. One query finds the outside rows however many rows are edited.

merge_state(before, new, quantity)

The state of a pile a second pile was added onto: counts added (None only when neither has one), capped by the new quantity; checked stays only when both were checked (the rule of a stock merge on edit).

apply_edit(db, rows, edits, plan, *, stamp=None, via=None, actor=None)

Write an edit as plan_edit planned it, inside the caller's transaction, in a fixed number of statements however many rows.

A row that merges is archived with the edits applied (so its merge key names the row it went into) and its quantity, missing and damaged counts are added to the target (checked stays only when both were checked); every other row takes the edits. Each row a merge touches also gets its own merge event with the row before and after.

sync

Collection sync with Rebrickable and Brickset (#10) on the 4.0 store (M.8j, issue #110; design 3.5, 5.2): owned sets and wanted sets, pull and send.

Pull imports what the site has and Brick Work lacks, row by row, in one undoable transaction; it never deletes here. Send writes counts there after a review; removals only when ticked; Brick Work's Undo cannot reverse a remote change. One copy here is one physical unit; the sites hold quantities. Condition, completeness, box, instructions and storage never leave Brick Work.

Every row speaks an item id. A remote set number is a Rebrickable number (Brickset's number-variant is the same numbering): it is translated to an item by its born key, else by one confirmed BrickLink link, and an item is placed by its current kind (design 3.5): a set becomes a copy (design 4.3's preselection, as Add), every other kind becomes stock of that kind. Sending names an item by its born key or through items.bricklink_key; an item with no number at all is left out and listed as not on the site.

State per source lives in sync_state (link, baseline, last pull, last send; the migration carries only the link, so the first sync after the cutover shows a full comparison and _changed_both has nothing to compare against). Plans are JSON files under data/sync/ and expire after 24 hours; whether a plan was applied is the plan table's (core/plans.py).

connect(db, source, username, password, *, http=None)

Exchange username + password at the site for its token/hash and store only that, encrypted.

The password is used for one request and dropped; it never reaches the database, an event, a log line, an error or a reply.

paste(db, source, value)

Store a token/hash the user generated on the site (write-only).

Which remote lists are used. Rebrickable: owned (list ids, empty = all but the wishlist list), wishlist (one list id or null), send_to (the owned list that receives sends). Both sources: wish_list (the Brick Work wishlist that pulls and sends wishes, default 'default').

translate(db, remote_numbers)

Remote set number -> the item id it names, or None. A number is a Rebrickable number (Brickset's number-variant is the same numbering): the item whose born key it is (a local BrickLink-born item counts by its BrickLink number), else the one item a confirmed BrickLink link names; a number that names several items, or none, or only a part or figure, is None (the plan lists it as not matched).

reverse(db, item_ids)

Item id -> the set number a site knows it by: the Rebrickable number of a Rebrickable-born item, else the BrickLink number (items.bricklink_key; a local BrickLink-born item is its own). An item with no number at all is None: it is left out and listed.

plan_pull(db, source, *, http=None)

What the site has that Brick Work lacks. Writes nothing to the collection.

apply_pull(db, plan_id, ticked=None, *, via=None, actor=None)

Place one copy per missing unit of a set (design 4.3's preselection; flagged, condition and completeness unset), loose stock of the other kinds, and the missing wishes, in one transaction with one receipt. Never deletes or lowers anything here.

undo(db, event_id)

Reverse one pull exactly (core.undo); refused when anything it wrote was changed since.

plan_send(db, source, *, http=None)

The remote calls a send would make: one row per set whose count differs. Removals unticked.

apply_send(db, plan_id, ticked=None, *, http=None)

Perform the ticked remote calls in order. Each row records sent, unchanged, failed, paused (budget) or key_rejected. A budget stop pauses the rest; a rejected key or token stops the rest. Re-applying the same plan retries only rows that were not sent. Nothing is retried silently.

adapters

Remote collection adapters for Rebrickable and Brickset (#10): owned sets and wanted sets only.

Every call goes through the source's budgeted, recorded transport, except the one-time connect, which must not store anything it sends or receives. Job payloads and errors never carry the user token, the userHash or a password: paths are recorded as templates.

ConnectRefused

Bases: ValidationFailed

The site refused the username or password. The message never names either.

RebrickableUser

Bases: Rebrickable

User endpoints under /api/v3/users/ (set lists and their sets).

connect(username, password)

Exchange username and password for a user token. Nothing is recorded.

fetch(link)

Remote counts: owned = the linked owned lists (all but the wishlist list when unset).

send(row, link, state)

Set the count of one set on the target list: POST (new), PUT (count) or DELETE (0).

BricksetUser

Bases: Brickset

Brickset collection methods: login, checkUserHash, getSets (owned/wanted), setCollection.

set_ids(numbers)

Brickset setIDs for set numbers, one charged getSets per 500 numbers.

throttle

Failed-login throttling per client (#48): after N failures, a delay that doubles up to a cap.

In-process and per worker: a restart forgets the counters, and several processes count separately. That bounds guessing to a few attempts per delay without a shared store. Nothing submitted is kept.

Throttle

wait(client)

Whole seconds this client must wait before trying again; 0 when it may try now.

thumbnails

On-demand picture thumbnails (320 and 640 px wide), cached under data/thumbs.

A thumbnail is generated on first request from the stored original and kept beside it; the file name carries the image id, width and the original's mtime and size, so a refreshed picture never serves a stale thumbnail. Originals that cannot or need not be shrunk (animated, unreadable, too many pixels, already small) are served as they are.

thumbnail_path(root, image_id, width, original)

Cache path for this original at this width.

get(root, image_id, width, original)

Return (path, mimetype) of the file to serve; mimetype None means the original.

lookup(root, image_id, width, original)

Request path: the cached thumbnail path (and mimetype), else (original, None) and a background job.

Never decodes or resizes an image itself.

pregenerate(database)

Maintenance: build both thumbnails for every stored picture; returns the count made.

tokens

Scoped bearer credentials. Secrets are generated once; audit contains metadata only.

undo(db, event_id, operation=None)

Undo a token change by its event id (core.undo): (response, undo event id, replayed). A second Undo of the same event answers that it was undone already and writes nothing. operation (an API Idempotency-Key) is checked for shape only; replays are M.11's.

transaction

Composable core transactions, including rollback of newly installed files.

schema(db, text)

Execute additive DDL without sqlite3.executescript's implicit COMMIT.

undo

Undo on the 4.0 store: the dispatcher and the two SQL predicates (M.8a, issue #110; design 2.4 invariant 11, 5).

4.0's event log starts fresh and is append-only; "undone" is a row in event_undo(event_id, undo_event_id) that the undoing operation writes in the same transaction as its own event. There is no 3.3 undo JSON to read: a receipt is an event whose (entity_kind, field) has a registered undoer, and the undoer reads the receipt in the event's new_value.

Pieces register the undoers of what they write (register), at import time, as they are ported; M.8a registers the one store4 already owns (a part list switch). undone_sql / undoable_sql are the predicates event pages and histories embed.

register(entity_kind, field, handler, *, admin=False)

Make events (entity_kind, field) undoable by handler. Registering the same pair twice is a programming error.

undone_sql(alias='e')

Predicate: the event has been undone (a row in event_undo). Uses the primary key of event_undo.

undoable_sql(alias='e')

Predicate: the event has a registered undoer. It does not say whether it was undone already: combine with undone_sql.

undo_event(db, event_id, *, via='app', actor=None, identity=None)

Undo one event by its registered undoer, in one transaction (a savepoint inside the caller's).

Returns dict(undone_event_id, already_undone, undo_event_id). Unknown event -> NotFound; no undoer -> NotUndoable; an undoer that refuses (the rows changed since) -> UndoBlocked with its message. identity (an API token's) is checked for the admin scope only when the undoer needs it; the web passes none.

valuation

Value and gain per copy, the Statistics figures and the insurance report (#57), on the 4.0 store.

A copy's price paid is its purchase_price; a null or zero price is "no price paid". Its worth is the copy's own current_value (source override, "Your value") or else the current used or new sold average of the set for the copy's condition and completeness: new for a new or sealed copy, used for any other, with the discount for a copy with a gap that the set detail applies (incomplete_discount_pct; a missing or damaged line, not the completeness label). The quote is the one the Prices card shows (the unvoided observations, then collection_prices.choose by source priority, sold first and freshness) and is stale by the same rule (decorate_prices: older than the source's max_age_days). A stale quote still counts, marked.

Everything is in the display currency with the rate and its date when a conversion happened (#39). A missing rate makes that figure None with a reason, never a guess, and never two currencies in one figure. Gain is worth minus price paid, in data a negative number for a loss, and the percent of the price paid as a whole number. The totals count only the copies that have both a price paid and a worth, so the gain compares like with like; the others are counted apart.

Batched: a fixed number of queries for any number of copies (labels, copies, prices, gaps, rates, settings and, for the report, photos).

copies(db, copy_ids=None)

One row per live copy (only those in copy_ids when given: the set detail asks for its own copy), in set-number order (numbers compared as numbers, then the copy's creation), as a list of dicts:

id, item_id, number (the set number, e.g. 7286-1), name, year, theme, condition, completeness, purchase_date, currency (the display currency), paid_original ({amount, currency} as recorded, or None), paid (a money body, below, or None) and paid_reason (None, no_price_paid or no_rate), worth (a money body or None) and worth_reason (None, no_quote or no_rate), worth_source (override, used, new or None), worth_date (the quote's day YYYY-MM-DD; None for an override), worth_stale (bool; never for an override), worth_sales (the number of sales behind the quote, or None), worth_price_source (the price source, e.g. bricklink, or None) and worth_discount_pct (the incomplete-set discount applied to the quote, or None), gain ({amount, percent, currency}, worth minus paid, negative for a loss, or None when either is None; percent is None when the price paid is 0.00 after conversion).

A money body is {amount: Decimal (2 places), currency: display currency, converted: bool, rate: Decimal or None, rate_as_of: day or None, original: {amount, currency} as recorded, or None unless converted}.

paid_vs_value(db, rows=None)

The Statistics figures (5c), in the display currency, as strings for money. rows are copies(db) when the caller has them.

currency; copies (live copies); comparable_copies (price paid and a worth: the only copies in the totals; the frames say "Charts need more data" when it is under 3, enough says so); paid, worth, gain (negative for a loss), percent (of the price paid), all over the comparable copies and None when there are none; without_price_paid, without_quote and without_rate (copies counted apart: no price paid, no quote at all, a figure that needs a missing rate); worth_without_price_paid (the worth of the copies with a worth and no price paid, never in the totals); converted and rates_as_of (any figure converted, and the oldest rate date used); by_theme and by_year: lists of {label, paid, worth, gain, percent, copies} over the comparable copies, themes by price paid (largest first), years ascending with Not recorded last.

valuation(db)

{'copies': [copy rows as JSON bodies], 'statistics': paid_vs_value}, both from one read of the copies.

photos(db, copy_ids)

{copy id: [photo ids]} of the copies' own live photos in creation order (never the set's cover), one query.

report(db)

The insurance report: {'header': {...}, 'rows': [...]}. Rows are every live copy in set-number order, as copies(db) (money as strings, see valuation) plus photo_ids (the copy's own photos).

The header: generated (today, YYYY-MM-DD), currency, copies (all live copies), paid and worth (over the copies with a price paid and a worth, the same figures as paid_vs_value), gain, percent, comparable_copies, the counts without_price_paid, without_quote and without_rate (for the footnote), converted and rates_as_of.

report_csv(db, data=None)

The report as CSV text, one row per live copy (the same rows and order as report), made safe for a spreadsheet (csv_safe). Money is in the display currency; a figure that could not be given is empty and the Remarks column says why. data is report(db) when the caller has it.

wish_progress

Wishlist progress (#35): how much of each wished line you already have.

Derived, not booked: parts count loose stock of the same part and colour in a usable condition (new or used; parts inside set copies are needed there and never count); sets count live copies; minifigures and other items count loose stock. A manual value per line (qty_filled_manual, also set from BrickLink's QTYFILLED on import) overrides the derived one. Three queries for any number of lines.

annotate(db, rows)

Add qty_filled_manual, qty_filled, qty_filled_source and filled to wish rows (4.0: item_id, color_id, words).

wishlists

Named wishlists, their receipts and the portable exports on the 4.0 store (M.8g, issue #110; design 5 wishlist / wish, 5.1).

A wishlist is a wishlist row; a wish is a wish row with a uuid id, a kind word, an item id and, for a part, a colour id. A list holds one live wish per item and colour (wish_live_key). Every change is one collection receipt event (collection.receipt), reversed by collection.undo. The exports name an item for BrickLink through core.items (the preferred confirmed link, else a BrickLink-born number) and for Rebrickable by its Rebrickable number: a row that cannot be named is left out and listed in the summary, never exported under a made-up number and never a reason to stop the export.

export(db, identifier='all', *, format='bricklink', kind='parts')

dict(content, summary, filename, mimetype): the live wishes of one list (or all active lists) as a BrickLink wanted list (XML) or a Rebrickable CSV of parts or sets. summary says how many rows were exported, which were left out and why, and which limits were dropped.

workbench

The Workbench on the 4.0 store (M.8e, issue #110; design 5 stock_transfer, 4.4, 3.8): gaps, the Matchmaker, the Part Runner, Take and Give.

Gaps. What a copy lacks: a live set line's missing and damaged pieces, a live figure line's missing figures, a missing booklet or box (a set copy without the status flag "Instructions present" / "Box present"), and a loose figure's missing parts. A retired line is no line, a spare is not a gap, a figure's own parts are not set gaps (design 4.4, 4.5).

Matching is by item id. A gap names an item (and a colour for a part); loose stock matches when it is the same item in the same colour. A booklet or box is the exception: a set's booklet is the item whose BrickLink number is the set's BrickLink number, so a gap and a stock row meet on ('instructions', <that number>). Nothing is blocked because a link is still "to check" (the Rebrickable id is exact); the mark stays on the row (unresolved) for the screen. Retired lines are never matched.

Take and Give move loose stock into a gap and lower the gap in one transaction under one workbench receipt event (tables before and after, undone by core/undo.py through the collection's generic undoer). Each move also writes a stock_transfer row naming the rows involved and that event; Undo keeps those rows (they are provenance, like the event itself).

A receipt's last entry may be context (what a history row says); it is not a table.

Tracker

Collects the rows a request changes, before and after, and writes the one receipt event (collection.receipt's tables format).

touch a row (or link set) before changing it; created the rows the request inserts; write then snapshots every touched row again, keeps those that really changed and writes the event. A modified row and a created row of one table are separate entries (the undoer archives created rows and puts modified rows back).

snapshot(table, ids)

The rows of table with these keys; a subclass adds tables (the orders' journal does).

flag(db, name)

The id of the live status flag called name, or None unless there is exactly one.

defective_figures(db)

Ids of the loose figures with a live part line marked missing or damaged: they are not offered.

available(db, row, *, defective=None)

How many of a loose stock row can be given: none when damaged, none for a figure whose own parts are defective.

part_gap(row, whole_missing=0)

Required replacements, with whole missing figures covering constituents.

identity(row)

What a gap or a stock row is matched by: key when the row carries one, else item and colour.

stock_key(row, numbers)

The matching key of a stock row (numbers is _booklet_numbers of the booklet and box rows).

gaps(db, only=None)

Every gap, decorated for a screen; only=(copy_ids, stock_ids) keeps just those copies' and loose figures' gaps.

A gap is dict(target, target_id, copy_id, need, item_id, color_id, key, copy, ...) plus the decoration of collection.decorate (name, item_type, item_no, number_text, color_name, rgb, ink, image, unresolved, ...). target is part (target_id = the copy_part id), figure (the copy_figure id), instructions / box (the copy id) or stock-part (the stock_part id of a loose figure, whose id is copy_id). Parts first, then figures, booklets and boxes, then loose figures' parts.

relevant(db, item_id, colour, kind, *, copy_ids=(), stock_ids=())

Copies and loose figures whose gaps can reach the matches of one item: those holding a gap on it, and the named ones.

A superset is fine: gaps(only=...) returns every gap of these and the caller filters. A copy is relevant when it holds a gap on the item's identity or is named; its other gaps matter because completes counts all of a copy's gaps. A booklet or box can fill any set copy, so all copies are relevant to it.

completion_context(db, all_gaps)

Group required gaps once and load copy-level ranking facts once: ({copy: Counter(key)}, {copies with "Set is checked"}).

matches(db, *, all_gaps=None)

Loose stock that could fill gaps: one row per (gap, stock row), completing matches first, then the larger amount.

group_matches(db, rows, all_gaps)

Candidate coverage per copy; stock is capped within each copy, not reserved.

picker(db, selected, ranking=None, *, all_gaps=None)

The missing parts of the chosen copies, one row per part and colour, in the picking order.

picker_keys(rows)

Availability is based on the selected snapshot, never guessed dimensions.

reduce_gap(db, tracker, gap, amount, *, update_completeness=True)

Lower a gap by amount (a part's missing first, then its damaged; a figure's missing and its parts'; a flag set).

requested_gap(db, source, copy_id, target, target_id)

Read only the requested copy (or loose figure) and line under the write lock; None when it is not a gap source can fill.

move_stock(db, tracker, source, match, amount)

One source row into one gap: the stock arithmetic behind every transfer and take. Returns the stock_transfer row to write.

transfer(db, source_id, copy_id, target, target_id, amount)

Give (or Take) amount of one stock row into one gap. Raises StaleMatch when the stock or the gap changed.

take_for_gap(db, copy_id, item_id, color_id, quantity_wanted)

Take N of one missing part (or whole minifigure) from loose stock into a set copy, all or nothing.

Sources follow the Matchmaker's order per gap (largest usable amount first, then stock id); one transaction and one receipt, so one Undo reverses every move.

found(db, item_id, color_id, destination, assignments, amount)

The picker's "Found it": lower the chosen copies' gaps of a part, add it to loose stock, or only record the find.

booklet_item(db, number)

(item id, error) of the instruction booklet a number names: the full number (4819-1), or a bare set number with exactly one booklet (4819). A booklet is an item of kind instructions: one the collection already holds, else one the BrickLink item list (catalogue bricklink_item, type I, when loaded) names, which is created as a local BrickLink-born item. A set's own number does not name its booklet. The item id is ('new', number, name) for a booklet to create.

booklets(db, text, amount)

Add physical booklets to loose stock: one booklet number per line, an optional ,quantity.

undo(db, event_id, *, receipt=None)

Undo a workbench receipt; a second Undo of the same receipt does nothing.

history(db, limit=50)

The newest workbench saves that are not undone: [dict(id, at, action, label, quantity, undo_available)].