# BidBrain project status, part 14: v3.1, due in dates and Auc status (26 August 2026) Moved unchanged from CLAUDE.md on 2026-09-18. The standing rules stay in CLAUDE.md, which lists every part of this log. - v3.1 AND v3.1.1 RELEASED, THE FIRST RELEASES CUT FROM STEVEN'S OWN COPY, 2026-08-26 (Steven: "how do we push everything we have done to git to make a proper release"). 29 unpushed local commits (everything from the card display preference and Carwow login automation through the v3.0 merge, the Motorway and DealerKit login automation, the Integrations redesign and the hide connections work) pushed to origin/main as a clean fast forward (origin had nothing new, checked), tagged v3.1 and released via gh with notes pulled from a new CHANGELOG.md section, the established discipline. Steven's own gh auth (bigdougie2025) already had real push access, checked via the API before assuming. The README's roadmap parity block retitled "Path to v3.2", the same carry over pattern v3.0 itself set (the parity work was earmarked for v3.1, this release shipped as the logins and Integrations work instead, the goal moves unchanged). - CI THEN TURNED RED ON THE v3.1 PUSH, AND THE FAILURE PREDATED TODAY: Mark's own v3.0.1 push had already failed identically, checked in the Actions log before assuming today's work broke it. Root cause: render_purchases()/render_settings() read theme colours fresh from the database at render time (since the v3.0 era theme work), and on a checkout whose database has never been initialised (CI's bare runner, or a REAL fresh install opening the purchases page before its first run, which made this worth a patch release rather than a shrug) db.get_settings raised "no such table: settings" instead of reading as nothing saved yet. Fixed in get_settings itself (a never initialised database reads as {}, any other OperationalError still raises), proven by replicating CI's own exact steps locally (a clean clone, dealer_config from the example, no database anywhere: failed the same way as CI before the fix, 527 passed 0 failed after), and shipped as v3.1.1 the same hour. Local suite 540 passed throughout (the 13 extra tests need this machine's real data). - RETAIL EST. CAP, PLUS A MANUAL PER ROW OVERRIDE, 2026-08-26 (Mark: "a setting in for the retail est to never go above £14995, GU68XPP use as example we are suggesting £16495 however due to the new rule (which need to be able to be changed from the setting) this would now be £14995, also add the ability to edit it from the row if we want to price it manually lower"). Two related but distinct figures: pricing.DEFAULT_SETTINGS gained retail_price_cap (None, editable from a new Retail est. cap card on the settings page, validated in serve._validate_settings the same way as every other optional numeric setting); purchases gained a new nullable column retail_estimate_override, set from the purchases page row itself via a pencil button next to the Retail est. figure (editRetail/ saveRetail in render.py, POSTing to a new /api/set-retail-estimate, db.set_retail_estimate_override). Both are applied in ONE place, db.list_purchases, never in record_purchases and never separately in the DealerKit push path or the bias bid comparison, so every reader of a purchase row (the page, the Retail Price push, the Margin and vs Bias bid figures) automatically agrees: an override always wins, then the cap if one is set and the stored figure exceeds it, then the plain formula figure. The real, uncapped figure the formula computed is kept under retail_estimate_raw, never overwritten, so lowering or clearing the cap later still recovers it. A small badge on the row (C for capped, M for manually set) and a line in the existing Cazana 115%/Margin popover ("Before the cap £X" or "Manually set, Yes, Formula figure £X") say plainly when the shown figure is not the plain formula output. Verified live end to end: GU68XPP's own real stored figure is £16,495 (confirmed via the database directly before any change), saving a £14,995 cap through the real settings page brought it down to exactly £14,995 on the real purchases page (badge titled "Capped by the settings retail price cap"), typing £12,495 into the row's own edit field and saving replaced it live with no page reload (badge "Manually set", popover showing the real £16,495 formula figure alongside), and Reset correctly reverted to the £14,995 capped figure, with the database's own retail_estimate_ override column confirmed cleared back to None afterward. 12 new tests (no cap leaves the figure alone, a binding cap lowers it and flags capped, a non binding cap leaves it alone and unflagged, an override wins over the cap even when lower than it, an overridden row is never flagged capped, clearing the override reverts to the cap not the raw figure). 548 tests pass. - CARWOW'S OWN DUE IN DATE, wired up the same way as Motorway's, 2026-08-26 (Mark: "navigate to the won, then go in to the listing and theres a card with collection confrimed and collection date wire this up the same way as MW"). Investigated live first, not guessed: a won car's own plain listing page carries a real expandable "Collection confirmed" card (an `expandable-content` checkbox toggle, `data-content-closed="Collection confirmed for 28th Aug"`) with a Collection date row inside it, confirmed against three real purchases at different real sale stages (collection_ arranged and even quote_accepted, an earlier stage than Motorway's own equivalent), all three already carrying a real scheduled date. Unlike Motorway, Carwow only ever shows the one "Collection date" label, never a separate past tense "Collected" one, so there is no confirmed-vs-scheduled choice to make here, just the one figure, written as an ordinal date ("28th Aug 2026", "1st Sep 2026") rather than Motorway's DD/MM/YYYY. - New carwow.read_collection_date(page, listing_id), same shape and same "own dedicated page.goto, own docstring" convention as motorway.read_collection_date, navigating to the same plain listing page carwow.read_purchase_gallery already visits (a second, separate navigation, matching how Motorway's own breakdown and collection date reads are already two separate page visits, not an optimisation this codebase makes for either platform). Returns "YYYY-MM-DD" or None, never guessed, when the card is not on the page at all. - Wired into BOTH real call sites that read a Carwow purchase's own detail, not just one: the bulk daily/on demand read_all() Carwow block in purchases_run.py, and the separate single car refresh_ purchase() used by --check-purchase (a real, distinct code path, a first pass wiring it into read_all() alone would have left Check Carwow on the purchases page itself still blank). collection_ date is already in db._PURCHASE_REFRESHABLE_FIELDS (added for Motorway the day before), so no schema or fill-once change was needed, a later reschedule already overwrites cleanly for either platform. - Verified live end to end, not just the regex: read_collection_ date run directly against three real listings (14210244, 14226049, 14181900) returned 2026-08-28, 2026-09-01, 2026-08-28, matching the real page text exactly; ran python3 purchases_run.py --check- purchase GU68XPP for real, confirmed collection_date landed as 2026-08-28 in the database directly, then confirmed the real live purchases page's own Due in column reads 2026-08-28 for that row. 3 new tests (the real card's own ordinal format, a single digit ordinal day, no card on the page at all never guessed). 551 tests pass. - DUE IN DATE PUSHED TO DEALERKIT, 2026-08-26 (Mark: "lets now wire up the due in date in BB to DK"). A prior session (see the "TAX DATE" and "GENUINELY BLOCKED" entries earlier this file, 2026-08-23) concluded DealerKit's own delivery_on field had no editable UI path anywhere on the vehicle record, checked at the time under Description, Additional Details and this same Miscellaneous tab. That conclusion was wrong, or the field is newer: Mark found it live himself, navigating the real UI ("on the car listing page go to edit then Miscellaneous... the due in / in-stock date it there"), a real "Due In / In-stock On" date picker under Location & Delivery Details, confirmed to write straight to delivery_on. - Needed a fresh DealerKit session first, the saved one had died (`data/dealerkit_state.json` returning the sign in page); the new keychain based `auto_login_dealerkit.py` had no credential stored under this mechanism, so a manual relogin (`python3 login.py dealerkit manual`, the same standing flow) was used instead, Mark confirming "logged in" in chat, session saved fresh (landed on the real dashboard). - New `bidbrain.dealerkit_write.set_delivery_date(page, vehicle_id, date_iso)`, built the same way as every other DealerKit date write in this project (never a raw API PATCH): opens the field via `_click_ui_select_field` (the same `.ui-datepicker__label- text` component the Add to Funding modal's own date fields already use, confirmed live to also cover this one) and picks the day via `_pick_calendar_date`; the Miscellaneous panel's own "SAVE" button is one of 24 elements sharing that exact text on the page (23 hidden duplicates from other Edit Vehicle tiles rendered off screen, checked live, the same shape already solved for Pricing's own "Save Changes"), so every match is tried with a real `.click()` until one genuinely succeeds, verified via a fresh API read, retrying once, raising loudly rather than reporting a false success. - A real timezone subtlety found and handled: DealerKit stores delivery_on as UTC midnight of the LOCAL (UK) day, confirmed live by setting it to "2026-08-28" and reading back "2026-08-27T23:00 :00+00:00" (BST is UTC+1). A plain string slice comparison would have been off by a day for half the year, so `_delivery_date_ matches(stored_iso, date_iso)` adds a fixed 4 hour buffer before taking the date, correct whether the UK is on GMT or BST without needing to know the exact DST boundary. `set_delivery_date` is a genuine no op, returns False without touching anything, when the stored date already matches. - Wired into `dealerkit_write.push_purchase` as a fifth pushable item, "due_in_date" (alongside price/photos/documents/retail), included by default. Deliberately NOT given Retail Price's own "never overwrite once DealerKit has one" guard: a due in date is a plain fact BidBrain reads fresh off the platform, not Mark's own market judgement, so a re push always re syncs to whatever BidBrain's own collection_date currently says, self correcting a stale or wrong DealerKit date on a reschedule. `check_dealerkit_ record` gained a `due_in_date` field (presence of delivery_on only, matching every other row's own shape); the purchases page gained a new "Due in date" popover row with a real push checkbox (`dkBox`, no `blockOnDk`), separate from the existing "Due in" row (which is about the vehicle's own life_cycle_status stage, not this date, a genuinely different fact, kept distinct rather than conflated), and counted into the row's own master status icon alongside every other real pushable item. New `purchases. dk_due_in_date_pushed_at` column, `db._DK_ITEM_COLUMNS` gained the matching entry, `serve.py`'s and `purchases_run.py`'s own `--items`/items validation sets widened to the fifth value throughout, `purchases_run.py --push-dealerkit REG`'s own printed summary gained a matching due in date reporting line. - Verified live end to end, not just the unit tests: GU68XPP (336) written from None to a real date and read back correctly via the API; FY70UCF (339) proven to correctly no op on a second call once already correct; a genuinely fresh purchase, BT16TYV (337, real collection_date 2026-09-01), pushed through the ACTUAL CLI (`python3 purchases_run.py --push-dealerkit BT16TYV --items due_in_date`, not a raw script), printed "Due in date set on DealerKit from BidBrain's own figures: 2026-09-01.", confirmed via the API (`delivery_on` "2026-08-31T23:00:00+00:00", exactly local midnight on 2026-09-01 BST) and the database's own `dk_due_in_date_pushed_at` flag; the real live purchases page's own DealerKit popover then showed a genuine green "Due in date" row with `data-ondk="1"` and the correct "already on DealerKit, tick to push it again" tooltip, and the row's own master icon correctly still read "some" (several other genuinely missing items on this particular purchase, not a false green). 5 new tests (`_delivery_date_matches` across a BST shifted value, a GMT/winter value, a genuine mismatch, nothing stored yet, and an unparsable value, none of them a crash). 556 tests pass. - PUSHED THE MISSING DUE IN DATES FOR REAL, 2026-08-26 (Mark: "check all the carwow listings for collection dates" then "can you make sure all due in dates for the due in cars on DK are in should be 6 missing"). Not a code change, a real live pass using the feature just built: pulled every real DealerKit vehicle at life_cycle_status 5 ("Due In") via the stocklist API, 9 of them, 3 already correct from earlier the same session, 6 missing, matching Mark's own count exactly. Of the 6, 4 had a real collection_date on BidBrain's own side (EY16UJJ, KF66GJZ, AY66WFW, OY19KHD) and were pushed straight away via `dealerkit_write.set_delivery_date`, one continuous session, all confirmed via a fresh API read afterward. The other 2 (PK68AXZ, BU16WZB) genuinely have no date to give: checked live against Motorway's own delivery timeline for both rather than assumed, PK68AXZ's own collection was rescheduled on 24/08 ("Reason: Seller issue") with no new date set yet, BU16WZB's is still "reviewing Seller documents and getting their collection availability", not even allocated to a transport partner. Both held back rather than guessed, golden rule 4; they will fill in on their own the next time a Motorway payment breakdown read finds a real date. Noticed in passing, not a bug: AY66WFW moved from Due In to In Stock (a real, independent status change on DealerKit's own side) between the first stocklist read and the verification pass moments later, its own delivery_on write landed correctly either way. No test or code change, this entry records the real state of the account after a genuine live action. - AUC STATUS BECAME A 3 STAGE PROGRESS TRACKER, 2026-08-26 (Mark: "under the auc status change this to 3 icons for biding bid, Documents, delivery"). Asked which of two shapes Mark wanted before building, a single dropdown style AskUserQuestion rather than guess at a UI redesign: whether Bid should always show done (every row here is already a won purchase, so it is the fixed starting point of the tracker) or only show done once the platform's own status confirms the win is locked in. Confirmed: always done, "Progress tracker". - The old single traffic light dot plus plain status text (built earlier the same day) is replaced with 3 small icon stages, Bid, Documents, Delivery, reusing the exact same tick/ring circle language the DealerKit popover rows already use elsewhere on this page (dkTickSvg/dkRingSvg), so a stage reads the same way wherever it appears on this card. Derived purely from the real status word already on file per platform, no new data capture: Motorway's own real vocabulary (Awaiting confirmation, Reviewing documents, Checks complete, Delivery requested, Delivered, Complete) genuinely distinguishes Documents done from Delivery done (Checks complete lights Documents but not Delivery); Carwow's own real status enum (quote_accepted, collection_ arranged, collected) does not carry that same finer distinction, both Documents and Delivery light together once past quote_ accepted, an honest reflection of what the platform's own status value can actually tell apart, not a fabricated one. The real status word itself is never lost, still shown as a plain hover tooltip on the group ("Collection arranged", "Checks complete"), and the existing "Estimated breakdown" popover (Motorway's own payment breakdown not yet confirmed) is untouched, still wraps the same headline when applicable. - Verified live against the real running purchases page, not just the unit tests: read the actual DOM for a genuine example of all four real status states across both platforms currently on file, matching the intended design exactly: Motorway "Reviewing documents" (Bid won, Documents not yet, Delivery not yet), Motorway "Checks complete" (Bid won, Documents done, Delivery not yet), Carwow "collection_arranged" and "collected" (both: Bid won, Documents done, Delivery done). 556 tests pass (this is render only, no gate, pricing or push logic touched, no dedicated new tests, matching this page's own established pattern for a pure display change). - AUC STATUS ICONS GIVEN THEIR OWN GLYPHS, WORDING AND A THIRD COLOUR, 2026-08-26, straight after the tracker above (Mark: "change the bid won to icon £... deliver to a document icon & Tool tip to 'Documents Checked'... delivery to a car icon and tool tip to 'Delivery arranged' the documents could 'checking' & delivery could be 'Delivey needs arranging' which would be Orange"). The generic tick/ ring pair (borrowed from the DealerKit popover rows) is replaced with a per stage glyph inside the same filled circle badge shape: a plain "£" for Bid, a small document/lines glyph for Documents, a simple car silhouette for Delivery, each drawn white on the badge's own fill colour. A genuine third colour, not just green or dim grey: ORANGE (#d39e00, reusing the exact real hex the old traffic light dot already used for "still waiting on something", not a new shade) for a stage that is KNOWN but not yet reached (Documents "Checking", Delivery "Delivery needs arranging"), green (var(--icon), matching every other confirmed mark on this page) once reached (Bid "Bid Accepted", Documents "Documents Checked", Delivery "Delivery arranged"), the same dim grey only when the status itself was never read at all. Icon size bumped from 16px to 20px the same session (Mark: "make the icons slightly bigger"), checked live via `svg. getAttribute('width')` reading 20 on the real page. Verified live against the same three real rows as the tracker itself: BU16WZB (Reviewing documents) now reads "Bid Accepted" green, "Checking" orange, "Delivery needs arranging" orange; KF66GJZ (Checks complete) reads green, "Documents Checked" green, "Delivery needs arranging" orange; GU68XPP (collection_arranged) reads all three green ("Delivery arranged"), each checked directly against the real circle's own `fill` attribute, not just eyeballed. 556 tests pass (render only, no new tests, same reasoning as the tracker itself). - DELIVERY STAGE FIXED TO TRUST collection_date OVER THE COARSE STATUS WORD, 2026-08-26, straight after (Mark: "i dont think the MW delivery is wired up correctly for the Auc status as KF66GJZ & AY66WFW for examples have delivery dates arranged but are orange"). Real bug, not a display quirk: both cars genuinely had a real collection_date on file (pushed to DealerKit as their own Due In date earlier the same session), but Motorway's own main purchases list status still read "Checks complete", a word that never matched the Delivery stage's own regex ("deliver" or exactly "complete"), since that list and Motorway's own delivery timeline (where collection_date actually comes from, motorway.read_collection_date) are two separate pages that do not update in lock step. auDeliveryDone now checks r.collection_date FIRST, unconditionally true whenever BidBrain has one on file regardless of platform or the status word, only falling back to the old status based guess when no collection_date is known at all. Verified live: KF66GJZ and AY66WFW (both with a real due in date already showing on the page) flipped straight to green "Delivery arranged"; BU16WZB and PK68AXZ (both still genuinely without one, see the earlier "6 missing" entry) correctly stayed orange "Delivery needs arranging", proving this is not a blanket always-green change. 556 tests pass (render only, no dedicated new tests, same reasoning as the tracker itself). - A DUE IN VIEW ON THE PURCHASES PAGE, 2026-08-26 (Mark: "add a view like on the live auctions for gap fillers for all the still active cars that are due in, use the due in date for this and show todays & onwards dates", extended the same day: "and delivery dates not confirmed, add these", confirmed via AskUserQuestion after the message cut off mid sentence: include them in the same view rather than a separate one, grouped at the end). - A new toggle, independent of the platform tabs (works whichever of All/Motorway/Carwow is selected), filters to every "still active" car (stillDueIn(r), the real status word, not yet Delivered/Complete on Motorway or collected on Carwow) that either has a real due in date on or after today (never a guessed or past one) or has no date confirmed at all yet. Dated rows sort soonest first; undated ones sort to the end with a clear orange "Not confirmed" label in the Due in column instead of a bare dash, only inside this view, the plain unfiltered table still shows a plain dash as before. - Moved the same day (Mark: "move the due in button to next to stats") from the platform tabs row into the section header next to Stats, restyled to match its pill button shape rather than a tab. - Also that day, Mark: "remove and cars bought before june". Investigated live first rather than guessed at scope: 57 of 107 real purchase records are dated before 2026-06-01, over half the table. Asked which he meant, hide from the page or genuinely delete for good, given how large and how permanent the second option would be; confirmed display only. The Bought date filter now starts pre filled to 1 June 2026 on page load (set on #datefrom before the first render(), the real records and every DealerKit push tied to them are completely untouched), dropping the default visible count from 107 to 50; Reset or clearing the date box still shows everything. A genuine, not fully resolved interaction found while verifying live: with the June cutoff active, the Due in view itself also narrows (24 to 9), since the bought date filter runs before it in render(); judged reasonable rather than a bug, the cars that drop out are exactly the old, long since bought, still-unconfirmed ones, arguably stale records rather than genuinely still inbound cars, and Reset still reaches them if ever needed. - Verified live end to end throughout: the real toggle correctly showed 7 dated plus 17 undated cars (24 total) before the June cutoff existed, sorted and labelled correctly; combined with the Motorway tab it correctly narrowed to just that platform's own 2 due in cars; the button's own new position confirmed via `nextElementSibling` directly against the real DOM; the June default confirmed to read "2026-06-01" on load, narrowing the real page from 107 to 50, and Reset confirmed to restore all 107 with the date box cleared. 556 tests pass throughout (render only, no gate or pricing logic touched, no dedicated new tests).