# BidBrain project status, part 2: Interactive cockpit, private link and learning loop (June to July 2026)

Moved unchanged from CLAUDE.md on 2026-09-18. The standing rules stay in CLAUDE.md, which lists every part of this log.

- Interactive cockpit and private link, BUILT and proven:
  - serve.py is now an interactive local server (stdlib http.server, ThreadingHTTPServer, 127.0.0.1:8765). GET serves the page and /api/hidden JSON. POST handles /api/hide, /api/unhide, /api/bid, writing to the local database. The page must be opened at the served address (localhost or the private link), never as a file, or the toggles cannot save.
  - New db tables: hidden_cars (reg unique, reason, make, model, name, hidden_at, active, reviewed) and bids (reg+sale_date unique, make, model, name, max_bid, reserve, created_at, active). db functions: hide_car, unhide_car, hidden_regs, hidden_notes, mark_notes_reviewed, set_bid, bids_for. connect now uses timeout=5 for the threaded server.
  - Cockpit page (render.py render_page): each car has a Bid star (Steven's own shortlist, the app never bids) and a Hide, would not buy button with a reason box. A control bar gives search (reg, make, model), platform filter, sort (cheapest reserve, biggest headroom, platform), and Show only my bids. A seen before badge shows on repeat cars. JS uses fetch() to the api. Hidden cars page is hidden.html (render_hidden), live from /api/hidden, with unhide.
  - daily_run: run() and render_only() both suppress hidden regs (db.hidden_regs) and pre tick the bid stars (db.bids_for) via the shared _write_pages, which writes cockpit.html and hidden.html. New command python3 daily_run.py --hidden-notes lists new hide reasons for Claude to mine into rules in pricing.py, then db.mark_notes_reviewed.
  - Private web address: Cloudflare quick tunnel. cloudflared installed at ~/.local/bin/cloudflared (downloaded binary, no Homebrew). Start with: python3 serve.py (background), then ~/.local/bin/cloudflared tunnel --url http://localhost:8765. It prints an unguessable https://<random>.trycloudflare.com link that proxies GET and POST to the local server, so the toggles work from a phone. The quick tunnel link changes on restart and has no password by design. Swap to a stable named tunnel later. Proven: bid and hide POSTs persist to the database both on localhost and through the tunnel link.
  - Rule learning loop: hiding a car suppresses that exact reg automatically. Recurring reasons become hard rules by Claude reviewing --hidden-notes and editing the rule lists, the same way this session turned Steven's pasted reasons into rules.
  - Cockpit controls redesign (Steven's feedback): the bid control is a star button on the photo, top right, sitting with the grade circle and platform badge (gold when on). Hide is a quiet "Hide, not for me" link under the bid button that opens a small reason panel. The old full width buttons were removed. starbtn and hide-row in render.py.
  - One hide control everywhere (Steven found two confusing): there is now a SINGLE hide mechanism used on every page. A quiet "Would not buy this" link opens a reason field, and one "Hide it" button saves the reg and reason to the hidden_cars table via POST /api/hide. It lives in render.py as the shared HIDE_BLOCK (markup) and HIDE_JS (behaviour) constants, reused by both the valued cockpit card (_shortlist_card) and the pre valuation review card (_review_card). The trigger says "Would not buy this" and only the commit button says "Hide it", so there are not two competing hide actions. The old review page checkbox plus copy and paste textarea flow (toggleCar, localStorage, the paste box) is GONE, replaced by this same DB saving flow. Do not reintroduce a second hide mechanism. Note cockpit.html is written by two paths: render_page (the real valued run) and render_review (daily_run.py review, the pre valuation cull); both now share the one hide control.
  - Automatic daily schedule, BUILT via launchd. LaunchAgents in ~/Library/LaunchAgents: com.bidbrain.daily runs python3 daily_run.py at 17:10 every day (after 5pm so both platforms show next day stock) and com.bidbrain.serve keeps serve.py up (KeepAlive). Logs in data/logs. Load or reload with launchctl load -w. This is why the project had to move off the Desktop. The old com.bidbrain.tunnel agent (cloudflared quick tunnel) is RETIRED, its plist archived as com.bidbrain.tunnel.plist.disabled, replaced by Tailscale Funnel below.
  - Browser leak fixed, was causing hung runs (2026-06-09). browser.open_reader_context, when a saved session state exists (Glass's, Cazana, Carwow), did playwright.chromium.launch then b.new_context, but callers only ever call ctx.close(), which closes the context and leaks the whole Chrome. Over a full run that is one leaked browser per valuation; a manual run piled up to 379 chrome-headless-shell processes and hung (main thread stuck in select, 1.8s CPU in 51 min). Fix: open_reader_context now wraps ctx.close so it also closes the launched browser (monkey patched the context's close to call the original then b.close). Proven: 5 open/close cycles stayed flat and returned to 0 chrome processes. The persistent profile branch (Motorway) was never affected since ctx.close on a persistent context closes its browser anyway.
  - Run progress bar, DONE (2026-06-09). daily_run writes data/run_progress.json via _progress(phase, message, done, total) at each step (phase is reading, valuing, writing, done, error; ts is a wall clock time.time()). __main__ wraps run() so a crash writes phase error then re raises. serve.py /api/run-status now also returns the progress object plus the server clock now. The cockpit shows a thin progress bar as a full width row inside the sticky control bar (render.py .runbar/.runtrack/.runfill, renderProgress in JS): green fill with done/total while valuing, an indeterminate sweep while reading (total 0), amber plus "no update for Ns" if the server clock shows the progress ts is over 90s stale (so a future hang is visible not a silent spinner), red with the message on phase error (polling stops, message stays up). The page polls run-status every 3s while a run is active and reloads when phase is done. Works for the 5.10pm scheduled run too since it reads the same file, not just manual runs.
  - Manual Run now button, DONE (2026-06-09). The cockpit control bar has a green "Run now" button (render.py, .runbtn). It POSTs to serve.py /api/run, which starts a full daily run as a detached background subprocess (sys.executable daily_run.py, cwd project root, output to data/logs/manual_run.log), the same as the 17:10 scheduled job. serve.py guards against overlapping manual runs with an in memory _run_lock and _run_proc, and exposes GET /api/run-status ({running, last_run mtime of data/last_run.json}). The page confirms before starting, then polls run-status every 8s and reloads when the run finishes. Caveat: the in memory guard only blocks manual vs manual overlap, it does not know about the 17:10 launchd run, so avoid pressing Run now right at 17:10. Also serve.py now sends no-cache headers (Cache-Control no-store, Pragma, Expires) on every response via an end_headers override, so phones always get the fresh page after a run or a design change.
  - PERMANENT private web address, DONE (2026-06-09), replaces the Cloudflare quick tunnel. The link never changes: https://mac-studio.tailc25a81.ts.net/cockpit.html . Set up with Tailscale Funnel (free, no domain, no password by design, unguessable but public, which matches the old behaviour). Tailscale app installed on the Mac Studio, signed in as stevendouglas@, machine name mac-studio, tailnet tailc25a81.ts.net. Funnel enabled once via the account link, then turned on with: /Applications/Tailscale.app/Contents/MacOS/Tailscale funnel --bg 8765 (proxies https root to the local serve.py on 127.0.0.1:8765, GET and POST both work so bid and hide save from a phone). The --bg config persists in the Tailscale daemon across reboots, so no launchd agent is needed for it; the Tailscale app must stay running and signed in. Check it with: /Applications/Tailscale.app/Contents/MacOS/Tailscale funnel status . Turn off with: tailscale funnel --https=443 off . cloudflared is no longer used (binary still at ~/.local/bin/cloudflared if ever needed). Proven: the link returned HTTP 200 and served the cockpit on first hit.
  - Smarter cockpit, Part A DONE (2026-06-09). All annotations, never gates, brain untouched (34 pricing tests pass). (1) Reserve drop flag: sightings now stores reserve and cap_clean (db.py SCHEMA plus an idempotent _migrate in init_db that ALTER TABLEs old databases; record_sightings writes them; db.reserve_history(reg, sale_date) returns prior reserves newest first). daily_run._reserve_drop_flag compares today's reserve to the last seen one and appends "Reserve dropped £X since last seen (was £Y)." in the flag loop next to repeat flags. Only shows once two days of runs have recorded a reserve, since old sighting rows have NULL reserve. (2) Top summary tiles: daily_run._summary(shortlist, held) -> {consider, fill_gap, single_valuation, held}, computed inside _write_pages so run() and --render-only agree; render._summary_block draws the tiles under the control bar; render_page takes summary=None. (3) Body type chip from car.body_type (shown only when present, Motorway has it, Carwow does not) plus data-bodytype. (4) Profit line in the hero: "Max bid leaves £3,000 for prep, fees, VAT and profit." (imports FLAT_SPREAD). (5) New sort "Best opportunity": gap stock cars first then biggest headroom; reserve stays the default sort per the brief. Until stock data exists (Part B), fill_gap reads 0 and Best opportunity degrades to a headroom sort, both correct. Verified live on a side port preview.
  - Platform grouping with tabs, DONE (2026-06-09). The shortlist is no longer mixed: render._platform_sections splits it into per platform sections (Motorway, Carwow, Other if any), each a <section class="platset" data-plat=...> with a heading and its own .grid and a live count badge (.setcount). The control bar has an All / Motorway / Carwow tab group (.tabs/.tab, setTab in JS) replacing the old platform dropdown; All shows every section, a platform tab shows just that one. applyView now loops .platset sections, filters by search and View shortlist within each, sorts within each (sortCards: cheapest reserve or best opportunity), updates each count, and hides empty or non active sections. The redundant "Platform" sort option was removed (tabs do that now); "Biggest headroom" was already removed; sorts are Cheapest reserve (default) and Best opportunity. Held and rejected sections are not platform split and always show.
  - Learning from shortlists, quiet capture DONE (2026-06-09), no cockpit change yet (Steven chose "just learn quietly for now", signal = make and model). Every star now captures richer detail: the bids table gained year, mileage, grade, fuel, body_type, cap_clean, platform (db.py SCHEMA + _migrate ALTERs old databases); render adds matching data-* attributes to each card and cardData sends them; serve.py _bid passes a details dict to db.set_bid which stores them. db.preference_profile() aggregates all active bids into by_make, by_model (the signal), plus price/mileage/year bands and platform split. View it with python3 daily_run.py --habits (CLI only, not on the page). Older bids keep make/model but NULL on the new fields; new stars capture everything. Proven: a star POST stored all fields and --habits shows the real profile (27 stars, top makes Ford then Vauxhall, top model Ford Fiesta, median reserve £5,950). To surface later: a "looks like your kind of car" badge or a "your taste" sort, both can read preference_profile.
  - Stock awareness (smarter cockpit Part B), DONE (2026-06-10), annotation only, never a gate, 42 pricing tests pass. Steven's dealer system is myclickdealer.co.uk (the public clickdealer.co.uk is marketing only; the first login attempt opened that by mistake). One time login captured via login.py clickdealer manual, session saved to data/clickdealer_state.json, recon snapshot in data/inspect/clickdealer_tab0.html. The in stock list is stock_list.php with status=ins (URL in browser.SITES clickdealer stock_url; the reader appends today's date_day/month/year params). It is a classic server rendered table: each vehicle row has the VRM and a combined "MAKE MODEL" cell two cells later. bidbrain/readers/clickdealer.py: parse_stock(html) -> (counts, pairs) finds rows by VRM regex; read_in_stock(playwright) opens the report and fails loudly (RuntimeError, proven) if the Make / Model header or rows are missing, with a hint to re run login. Model matching across naming styles: clickdealer.model_key(make, model) = first token of normalised make + first token of normalised model, hyphens treated as spaces, used on BOTH sides, so MERCEDES-BENZ A-CLASS and Mercedes A Class both key to "mercedes a", BMW 1 SERIES and 1 Series 118i Sport to "bmw 1" (proven on the real list: 60 vehicles, 40 keys). db.record_stock_snapshot(pairs, date) replaces same date rows in stock_snapshots; db.latest_stock_counts() rebuilds {key: count} from the newest snapshot, {} if none. daily_run.run() reads stock FIRST (progress phase "stock", in ACTIVE_PHASES) and snapshots; a failed read never stops the run (falls back to the last good snapshot). The stock level is recomputed in _write_pages from latest_stock_counts (sets car._stock_level/_stock_count) so a full run and --render-only always agree. daily_run._stock_flag thresholds: gap 0, ok 1 to 3, too_many 4+ (Steven's choice). render: a small badge next to the plate (fills a gap green / N in stock grey / N in stock, plenty amber) plus data-stock, which lights up the existing "Fill a gap" summary tile and the gaps first "Best opportunity" sort. Proven live: 60 in stock, 54 of 98 shortlist cars fill a gap, badges match the real stock (CLA 1, Ateca 1, Clubman 1), gap cars float to the top under Best opportunity.
  - login.py flashing fixed (2026-06-10): the watcher loop screenshotted the visible window every 2s which made it flash; the live screenshot is now taken every 20s (URL polling stays at 2s).
  - Phase 3 learning loop, BUILT (2026-06-10), annotation only, brain untouched, 51 tests pass. The monthly run (monthly_run.py) reads three Clickdealer reports per month: sold_vehicles.php (start_day/month/year params; per car: stock no, reg, separate Make and Model, fuel, mileage, sale and purchase price, purchased/sold dates, Days in stock straight off the report), sales_margins.php (start_date/end_date as DD/MM/YYYY; per car SIV which is the prep total, and Margin which is net sales minus purchase minus SIV, verified arithmetically on a real car), joined by stock number, plus each car's supplier (MOTORWAY, CAR WOW, City Auction Group etc.) read from the selected option of the v[supplier] dropdown on vehicle_details.php?id=N (about 41 extra page reads a month, Steven approved). Parsers are VRM anchored with fixed offsets, built against captured fixtures in data/inspect (clickdealer_sold_may.html etc.) and covered by tests. Limitation, flagged to Steven: Clickdealer does not expose the prep split by type per car, only per car totals (SIV) plus a business wide monthly split (siv_expenses.php), so per model figures use prep totals.
  - Phase 3 data flow: sales_history table (one row per sold car, UNIQUE(stock_number, sold_date), re run safe upsert; source holds the supplier) -> db.rebuild_learned_models() rolls it up per model_key into learned_models (n_sold, avg_days_to_sell, avg_prep_total, avg_margin, spread_held = sold count with positive margin after prep, by_platform JSON). The empty pre Phase 3 learned_models/sales_history tables are dropped and rebuilt by _migrate (guarded: refuses if they ever hold rows). Commands: python3 monthly_run.py (previous month), --month YYYY-MM, --backfill N. Seeded with a 12 month backfill (Steven's choice). Scheduled via launchd com.bidbrain.monthly on the 10th at 07:30 (the brief says the previous month's report is read on the 10th); a failed month prints FAILED and exits non zero but other months still ingest.
  - Phase 3 corrections, Steven 2026-06-10, both signed off:
    (1) MARGIN BASIS. The headline margin everywhere (sales_history.margin, learned averages, card notes) is TotalMargin minus prep: everything earned including finance commissions, with the SIV paid for. Clickdealer's TotalMargin column does NOT deduct prep (it is CarMargin plus AdditionalProfit; CarMargin is net sales minus purchase, no prep). Worked Golf example: TotalMargin 4747, SIV 1128, headline margin 3619. The raw total_margin is stored alongside; the old metal after prep figure (margins report Margin column) is no longer stored but derivable. parse_sold now reads TotalMargin (VRM anchor +14).
    (2) PER CAR PREP SPLIT EXISTS. The SIV button on a car's vehicle_details page leads to siv.php?id=N (plain GET), which itemises that car's expenses with date, supplier, TYPE (Buyers Fee, Bodywork, Repairs, Shipping, Accessories, Valet...), description and amount, plus Total SIV, the additional profit block, purchase price and margins. clickdealer.parse_siv(html) sums it per type; verified the split sums exactly to the page's Total SIV. The monthly walk now reads two pages per sold car (vehicle_details for supplier, siv.php for the split), prep_split stored per car as JSON in sales_history and averaged per type into learned_models.prep_split. The earlier note that per car splits were not exposed was wrong, it only looked at static HTML and missed the SIV button.
  - Valuation wrong car protection, DONE (2026-06-12, Steven's catch: a private plate now on a different car returned that car's price, a Kia Sportage "retailing" at 24,149 against CAP 7,250). EVERY Glass's and Cazana lookup now passes expect=(make, model) and the result page must describe that car or the price is refused (glass.match_tokens, shared by cazana via details_match; the model token used is the first with 3+ characters so "1 Series" matches on "series" not "1"). The private plate fallback already verified, now the primary lookups do too. Backstop flag, not a gate, in pricing.assess: governing value over 2.5x CAP Clean adds "Valuation looks high against CAP. Check it is the right car before bidding."
  - Valuation sources now fail INDEPENDENTLY (2026-06-12). The old valuing loop wrapped both lookups in one try, so when Glass's RAISED (dead login shows "did not show the valuation box") the exception skipped Cazana too and every car went held: the 2026-06-12 17:10 run produced 0 shortlisted, 87 held, an empty page. daily_run's valuing loop now has per platform guards (_try_glass/_try_cazana) plus a circuit breaker: after a platform errors once it is skipped for the rest of the run (no 15s timeout per car), printed loudly once; both dead stops the loop loudly. Proven same day: rerun with Glass's still dead priced all 87 on Cazana alone, 80 shortlisted.
  - Session fragility seen 2026-06-12: the Glass's session died within a day of a fresh login (despite Don't ask for 30 days, possibly not ticked or Glass's invalidates automated sessions); myclickdealer moved to Keycloak SSO (id.clickdealer.co.uk) and its saved session also expired, the stock read fell back to the last snapshot as designed. Both need a re login via login.py when convenient; the run survives either being dead.
  - ON DEMAND GLASS'S, the chosen resolution to the single session problem (Steven 2026-06-15), supersedes "everyone stay off Glass's". The scheduled run (and Run now) now prices every car on CAZANA ONLY, so the page is always full and never depends on a live Glass's session: daily_run.run() takes use_glass=False by default (sets glass_dead = not use_glass up front so Glass's is never attempted inline). Glass's is added ON DEMAND: a new amber "Run Glass's checks" button on the cockpit (render.py, .glassbtn beside Run now) POSTs serve.py /api/run-glass, which spawns python3 daily_run.py --glass as its own process (separate _glass_proc, blocked while a daily run or another Glass's pass is in flight). daily_run.glass_pass() loads the last run cache, opens a HEADED Glass's login window on the Studio via _glass_login (browser.open_context headless=False, polls every 2s for #plateNumberInput which is the logged in signal glass.lookup uses, up to 300s, refreshing run_progress each loop so the cockpit shows a live countdown not a stale warning, then saves glass_state.json), then runs glass.lookup for every shortlist+held car (private plate fallback included), re assesses with both valuations now present, rebuilds the daily run flags (repeat, reserve drop, valued on original plate, history), re splits, re saves the cache and rewrites the pages. A car held only for want of any valuation moves onto the shortlist once Glass's reads. This fits the single shared Glass's login perfectly: Glass's is only ever touched while Steven is sitting there freshly logged in, so nothing evicts it and there is no session to keep alive between runs. Progress phase "glass-login" added to render ACTIVE_PHASES. _session_health_notices repurposed: a missing Glass's value is now NORMAL (an info banner "These cars are priced on Cazana. Click Run Glass's checks to add Glass's and re price." plus a per card info note "Priced on Cazana, run Glass's checks to add it", not an amber fault); a missing Cazana value across the whole run is still a real "Cazana looks down, re login" fault. Note: serve.py runs under launchd com.bidbrain.serve (a LaunchAgent in the GUI session) so the headed login window from the --glass subprocess should appear on the Studio screen; confirm on first live use. 98 pricing tests pass; brain (pricing.py) untouched, this is all run orchestration and presentation.
  - Glass's pass now runs as ONE continuous session, the fix for it dying partway (2026-06-16). The first on demand pass died after about 28 cars ("did not show the valuation box") because it opened a fresh headless browser per car from glass_state.json; Glass's reads that churn of new sessions as a logout and kills it. Now glass.lookup is split: valuate_on_page(page, reg, mileage, expect) runs one valuation on an already open page, and lookup() just wraps it with its own context. glass_pass()/_glass_login keep the single HEADED logged in window open and run the whole list on that one page (one session, like a human doing many valuations), then close it. valuate_on_page lets a session dead RuntimeError propagate (circuit breaker stops the pass) but returns None on any other per car glitch so one bad car does not end the pass. The loop also skips cars that already have a Glass's value, so if the session ever does drop, clicking Run Glass's checks again continues from where it left off (already valued cars are saved to the cache each pass). The "run Glass's checks to add it" wording Steven queried was correct: those cars were the ones never reached after the session died, not a labelling bug.
  - "Run Glass's checks" vs "Glass's had no match" now durable (2026-06-16, Steven saw a Golf say "run Glass's checks to add it" while a Tiguan showed Glass's only, both had been through the same completed pass). Root cause: the no match state was a card flag (GLASS_NOMATCH_FLAG) that any re assessment (recheck_details, a later pass) wiped, reverting the wording to the generic "run Glass's checks". Fixed with a real Car field car.glass_checked (saved in the cache, survives re assessment). glass_pass sets it True for every car it reaches (the one that kills a session and those after stay False). render._insight_rows now branches the "Glass's valuation could not be read" flag on car.glass_checked: checked and no value -> "Glass's had no match, priced on Cazana"; unchecked -> "Priced on Cazana, run Glass's checks to add it". _session_health_notices counts nomatch/pending from glass_checked too. GLASS_NOMATCH_FLAG removed. The current cache was backfilled glass_checked=True (the last pass had completed over all cars), so the 15 Cazana only cars now correctly read "had no match". Reminder: Glass's and Cazana each match different cars, so a car can legitimately have one and not the other.
  - Silent detail read failure caught and made loud (2026-06-16, Steven spotted L321 LSY, a Mercedes A owned under 6 months, showing when it should be hard rejected). Root cause: the cache had keeper_start, selling_vrm and mech_issue EMPTY on all 117 Motorway cars, so the under 6 months owner rule, the known issue rule and the private plate fallback never ran. The Motorway detail pass (motorway.gate_details) had silently failed for the whole run (a dead session returns the car unchanged, no exception), and the run shipped the list anyway. gate_details itself is fine (tested live: read keeperStartDate 04/05/2026 and assess correctly rejected). Fixes: (1) run() now FAILS LOUDLY after the detail pass, if fewer than half of 5+ candidates returned a keeper date it raises rather than ship a half checked list (golden rule 5). (2) New python3 daily_run.py --recheck-details (daily_run.recheck_details) re reads the detail pages for the cars already on the last run and re assesses, keeping the existing valuations so a Glass's pass is not lost; used to clean a list in place. First run dropped 18 cars: 3 under 6 months (incl L321 LSY at 1 month), 3 Alfas, 2 Mazda 2.2 diesels (learned bans re applying on re assess), and 9 for warning lights. NOTE the warning lights rule is broad: it rejected tyre pressure and service reminder lights, which may be too aggressive, flagged to Steven to refine (treat trivial lights as a flag not a rejection).
  - Shortlist stars now sync live on page load (2026-07-05, Steven: "my latest shortlist seems wiped"). Root cause: the page only pre ticked bid stars at RENDER time (db.bids_for in _write_pages), so cars starred AFTER the last run/render showed un ticked on a reload, looking wiped though the bids table was intact. Fixed: serve.py GET /api/bids?sale_date=X returns db.bids_for(sale_date); the cockpit JS fetches it on load and lights the matching .starbtn (reg normalised uppercase, spaces stripped), then applyView so the Shortlisted count is right. A reload now always reflects the current stars. No data was ever lost, only the display. render.py, serve.py, 121 tests pass.
  - Clickdealer stock: health banner plus an on demand "Run Clickdealer stock" button (2026-07-03, Steven, after the Clickdealer SSO login had silently expired and the gap flags were stale from 15 June). (1) Banner: daily_run._stock_health_notice reads db.latest_stock_date (newest stock_snapshots date); if it is more than 2 days old (or none) it returns a banner "Stock counts are from DD Mon. The Clickdealer login may have expired ... Click Run Clickdealer stock to refresh them", added to the render notices in _write_pages. (2) Button: mirrors the Glass's plumbing. render .clickbtn "Run Clickdealer stock" (grey, leftmost of the three run buttons), serve.py /api/run-clickdealer spawns python3 daily_run.py --clickdealer as its own process (_clickdealer_proc, blocked while any run is in flight, killed by Stop), progress phase clickdealer-login in ACTIVE_PHASES, setRunning now takes which in {run,glass,click}. daily_run.clickdealer_pass() opens a HEADED Clickdealer window on the Studio via browser.open_context + shared _login_win_args (extracted from _glass_login, reads BIDBRAIN_GLASS_WIN), navigates to the stock report, polls up to 300s for the "Make / Model" header (once back on myclickdealer and off the id.clickdealer SSO host it navigates to the stock url, never bouncing him mid login), saves clickdealer_state.json, parses stock, snapshots for today, and rebuilds the cockpit from the last run cache so the gap flags refresh with no re valuing. Reminder: like Glass's and Cazana, the Clickdealer SSO session expires; the run falls back to the last snapshot and the banner now surfaces it. 121 tests pass.
  - Valuation circuit breaker made tolerant, and empty runs can no longer overwrite a good list (2026-06-23, after the 17:10 run priced a load on Cazana then ONE bad lookup tripped the old one-strike breaker, held everything, and shipped a 0 priced page over the good list). Two fixes in daily_run.run(): (1) _try_glass/_try_cazana now count CONSECUTIVE failures and only declare a source dead after DEAD_AFTER=4 in a row, a clean call (even a no match None) resets the counter, so a single transient glitch no longer kills the source for the whole run. (2) After valuing, if there were cars to value but NONE got priced, run() raises loudly and does NOT save the cache or rewrite pages, so the previous good list survives rather than being replaced by an all held empty page. Cazana (Percayso) session had genuinely expired by the evening (worked midday, dead by 17:10), fixed by python3 login.py cazana then a re run, which restored 72 shortlisted. Reminder: Cazana sessions expire like Glass's, the _session_health_notices banner now says so clearly; re login and Run now.
  - Motorway stock page redesign broke the read (2026-06-17, Steven saw "Run failed: waiting for [data-testid=vehicleCardLink]"). Motorway dropped the vehicleCardLink testid in a page redesign (the list container is now data-testid vehicle-list-v2 and the cars are plain a[href^="/vehicles/"] links). motorway.read_export and read_live now wait on a[href^="/vehicles/"] instead. The daily run failed loudly and kept the previous good cache (fail loud working), so no bad data shipped. Proven: read_export returns 190 cars again. If Motorway changes the list markup again, re probe with a quick stock page load and update the wait selector.
  - Walked away cars never shown again (2026-06-16, Steven: never want to see a car we bought then the sale was cancelled, do not send a driver to one we walked away from). New walked_away table (reg unique, platform, name, status, noted_at) plus db.record_walked_away, walked_away_regs, walked_away_list and db.suppressed_regs (= hidden_regs union walked_away_regs). daily_run run() and _write_pages now suppress db.suppressed_regs everywhere they suppressed hidden_regs, so a cancelled car is dropped before valuing and from the page for good. Capture is in purchases_run: a new _is_cancelled(row) routes any purchase whose status contains void/cancel/withdrawn to db.record_walked_away and OUT of the bought list and accounts email. Carwow: parse_won no longer skips data-listing-state="voided", it returns the card tagged voided (captured even with no price), routed to walked_away by _is_cancelled. Motorway has its OWN Cancelled list (Steven sent a screenshot), captured live 2026-06-16: Purchases > Cancelled at https://pro.motorway.co.uk/account/purchases/cancelled?dateRangeType=ALL&order=DESC&sortBy=dateBid&page=N , 25 rows a page, about 450 total, newest first, each role="row" carrying the plate, name and a Cancelled status pill (no /vehicles/ href on these rows, so the complete page selector finds none, parse by role=row text instead). motorway.read_cancelled(playwright, max_pages) reads it (page one for the daily top up, more for backfill) and fails loudly if page one does not render. purchases_run.read_all now also reads it each day (cancelled_pages=1); python3 purchases_run.py --backfill-cancelled reads the whole history (cancelled_pages=20). Backfill run 2026-06-16 captured 448 cancelled cars (446 Motorway + 2 Carwow), all suppressed, the live page dropped two cars. The keyword check left in motorway.read_purchases (cancel/void/withdrawn) is now just a harmless backstop, the dedicated list is the real source. Manual hide remains the catch all. Audit with python3 daily_run.py --walked-away. Identity is the registration (a no plate car cannot be tracked). 111 tests pass.
  - Summary tiles replaced by an interactive filter row (2026-06-16, Steven, mocked first then signed off). The four static dash tiles are gone; render._summary_block now draws a slim chip row (.fbar/.fchip): All, Gap fillers, Auto only, Held back, and Shortlisted (star icon, pushed right on its own with margin-left:auto). Each chip filters the list to that slice, one at a time (setCat in JS sets currentCat, applyView honours it). catOk maps gap->data-stock=gap, auto->data-auto=1 (new attr on the card from _gearbox_label), short->.starbtn.on; held is handled at section level (heldMode hides the .platset sections and shows #heldsec, the held back section now carries that id). Counts are recomputed in JS per active platform tab (ignoring search), so they stay honest switching Motorway/Carwow. The old "View shortlist" checkbox is REMOVED, the Shortlisted chip replaces it (toggleBid now just calls applyView). daily_run._summary(shortlist, held, bidset) returns consider/fill_gap/auto/held/shortlisted for the initial paint. Held back cards gained a "View on Motorway/Carwow" button (.viewbtn, _plain_card, opens car.listing_url in a new tab) so a held car can be eyeballed on the platform; held/rejected slim cards now carry data-platform and data-search for filtering. 111 tests pass.
  - Platform logos on the View button, source pill removed, yellow reg plate with copy (2026-06-16, Steven). The "View on Motorway/Carwow" CTA now shows the platform WORDMARK in white instead of the word. White SVG logos were captured from the public sites (motorway.co.uk, carwow.co.uk header SVGs), recoloured white, and saved as assets/mw_logo_white.svg / cw_logo_white.svg plus URL encoded data URIs assets/mw_logo_white.datauri / cw_logo_white.datauri (render._logo loads them into MW_LOGO_WHITE / CW_LOGO_WHITE). Motorway's wordmark only fills 34 of its 48 tall canvas (measured via getBBox) so it is cropped to viewBox "0 7 167 34" in the data URI, and rendered at .platlogo.motorway 93x19 vs .platlogo.carwow 110x14 so the two read the same visual size. The CTA is now a flex row (View on + logo + the same up right arrow); plain text "View on platform" fallback when source is unknown. Since the button names the platform, the photo source pill (.hf-src) was REMOVED from the bottom left idstack, leaving just the reg. The reg is now a YELLOW UK number plate (render._reg_plate, .hf-reg yellow bg, black border, Arial Narrow bold) with a small copy button (.hf-reg-copy, _COPY_SVG) that copies the plain reg via navigator.clipboard (copyReg in render_page JS, works on https/localhost, flashes "Copied X"). No plate cars show a grey NO PLATE plate, no copy. 118 tests pass. Logos are used only to label the link to that platform.
  - Plate font and copy cursor (2026-06-16, Steven). The reg plate uses the Staatliches font (Google Fonts, free for commercial use, added to the existing Figtree link) at its regular weight (a bold stroke was tried and reverted, Steven found it too heavy). The real Charles Wright plate font was ruled out, it is free for personal use only and needs a paid commercial licence. render .hf-reg-num. Condition grade chip is colour coded again (lost in the hf redesign): render .hf-chip-grade with g1 green, g2 yellow, g3 amber, g4/g5 red, class set from the grade digit. The copy button shows the pointer (hand) cursor like the View link (.hf-reg-copy cursor:pointer plus .hf-reg-copy svg pointer-events:none so the hover lands on the button).
  - Shortlist card CTA reworded (2026-06-16, Steven). The hf card button "Open listing to bid" now reads "View on Motorway" / "View on Carwow" (from car.source) with the same up right arrow as the held back .viewbtn (a.hf-cta::after content \\2197), so the two match. The disabled "No listing link" state is unchanged.
  - Header slimmed (2026-06-16, Steven). The dark topbar (BidBrain cockpit title plus the date and "star the cars" blurb) was removed from render_page. The title is now a stylised "BidBrain" wordmark (.controls .brand, Bid in ink and Brain in the green accent) sitting top left inline with the All/Motorway/Carwow tabs in the sticky control bar. The search box was shrunk from flex grow to a fixed 190px. Other render functions (render_review, render_hidden) keep their own topbar.
  - Engine chip on each card (2026-06-16, Steven). The platforms jam litres, trim and fuel into one engine string (for example "1.6 CDTi Exclusiv 5dr 4WD Diesel"), so render._engine_label(car) pulls the first litres token plus the fuel field into a clean "2.0 Diesel" chip (litres alone or fuel alone if only one reads, nothing if neither). It sits in the spec chip row between mileage and owners, slightly emphasised (.hf-chip-engine) since the engine matters for the bans. The messy trim middle is dropped, it is already in the title.
  - Automatic / semi auto highlight (2026-06-16, Steven, "make sure it stands out"). render._gearbox_label(car) returns "Automatic" or "Semi auto" for those gearboxes and None for manual or unknown (CVT never reaches a card, it is banned). Shown as a bold solid blue pill (.hf-chip-auto) in the spec row right after the engine chip. Only auto and semi auto carry the pill, so they stand out against the manual cars which carry none. Transmission is well populated on both platforms (Motorway and Carwow), so this is reliable.
  - Taste learning now SURFACED on the cockpit (2026-06-16, Steven: "learn what I like" from the cars he stars). The quiet capture (db.preference_profile, the bids table) was already running; now it shows. daily_run._taste_match(car, profile) scores each car against his starred history (make he stars often weighted most, a matching model adds more, plus fitting his usual price, mileage and year bands) and returns a match flag plus a score. _write_pages calls db.preference_profile() and sets car._taste_match and car._taste_score on every shown car, so run() and --render-only agree and it updates live as he stars more (preference_profile reads active bids each render). render.py shows a purple "Your kind of car" badge (.hf-gap.taste) on matched cards and a new sort "Best match for you" (data-taste, ranks by taste score then room). The default sort stays Best opportunity. Match threshold: a make starred 4+ times sitting in his usual price and mileage; on the current 51 star profile that badges about half the list, loosen or tighten in _taste_match if Steven wants it rarer. Annotation only, brain untouched, 104 tests pass.
  - Hide notes now LEARN into rules automatically (2026-06-16, Steven: "the whole point is for bidbrain to learn"). Hiding a car already suppresses that exact reg; now the reasons also become permanent rules. A scheduled agent "bidbrain-hide-note-learning" (Claude app, daily 18:00, after the 17:10 run) reads python3 daily_run.py --hidden-notes, turns each clear generalizable reason into a rule in pricing.py (BANNED_MAKES / BANNED_MODELS / BANNED_ENGINES / VALUE_CAPS only, never the pricing maths or gate thresholds), adds a test, runs test_pricing.py and keeps changes only if green, marks every note reviewed, rebuilds the page, and logs what it did here. Subjective one offs (for example "spec too basic") stay plain hides, no rule. Caveat: scheduled agents run only while the Claude app is open, so the learning fires next time the app is open if it was shut at 18:00, the daily car run itself is unaffected (separate launchd job). First batch done by hand this session from 5 notes: added Alfa Romeo to BANNED_MAKES, added Mazda 2.2 Skyactiv-D diesel to BANNED_ENGINES, and fixed banned_engine to read the fuel field as well as the engine text so a "1.0 Zetec" petrol Fiesta is caught by the Ford 1.0 EcoBoost ban (it slipped because the engine string did not contain the word petrol). 104 tests pass.
  - Hide note learning run, 2026-06-16: from 6 new notes, added Smart to BANNED_MAKES (never buy a Smart car) and Fiat 500X to BANNED_MODELS (the existing Fiat 500 ban did not catch the 500x token). Skipped as not generalizable: the Hyundai i10 ("only buy automatic versions" is conditional, not a clean ban), the Mercedes A ("owned under 6 months" is already a hard rule in assess, the car only showed because that run's Motorway detail read had failed). Already covered, no change: the Mazda CX-5 2.2 diesel (already in BANNED_ENGINES) and the Alfa Romeo (already in BANNED_MAKES). All six notes marked reviewed. 121 tests pass.
  - Hide note learning run, 2026-08-26: from 1 new note, added "master" to VAN_MODELS (a 2019 Renault Master MLL35TW, a tipper bodied large commercial, hidden as "its a pickup truck", so it belongs with the vans rather than as a new class of its own). Three tests added: the Master is caught by class_exclusion and rejected by assess, and a Renault Clio still passes, proving this is a model ban not a Renault ban. Note marked reviewed, cockpit rebuilt, 543 tests pass. Spotted while in this list and NOT changed, flagged for a human: VAN_MODELS carries "traffic" but the real Renault model is spelled Trafic, and since the van check is a substring match, "traffic" never matches a genuine Trafic. A Motorway Trafic is still caught by its body type, a Carwow one may not be. Adding "trafic" alongside it would close that, left for Mark or Steven to confirm rather than widened off my own bat.
  - Hide note learning run, 2026-09-02: from 2 new notes, added "isuzu" to BANNED_MAKES (a 2020 Isuzu D-MAX Utah, hidden as "i would never buy an isuzu", a plain make ban) and "movano" to VAN_MODELS (a 2016 Vauxhall Movano R3500 L3H1 CDTI DRW, hidden as "this is not a car, its a commercial vehicle", so it belongs with the vans alongside the Renault Master). Five tests added: Isuzu caught by banned_make and rejected by assess, the Movano caught by class_exclusion and rejected by assess, and a Vauxhall Corsa still passing, proving the Movano ban is a model ban not a Vauxhall ban. Both notes marked reviewed, cockpit rebuilt, 1099 tests pass.
  - Glass's pass now tops up missing Motorway photos, DONE (2026-06-16). Motorway photos are scraped per car (not in the CSV) and the daily scrape sometimes only gets part of the list, leaving the page mostly photo less. glass_pass() now calls _fetch_motorway_photos(p, cars) at the end of its browser session (only fetches cars with no photo yet, a no op when the daily run already got them), so an interactive Glass's pass self heals the photos. Backfilling a run without re valuing is still python3 daily_run.py --photos-now. A photo fetch failing never affects Glass's values or pricing.
  - Stop button, DONE (2026-06-16). A red Stop appears in the run progress bar while any run or Glass's pass is active (render.py .stopbtn, stopRun() posts serve.py /api/stop-run, which os.killpg's the run/glass process group, safe for data since both only rewrite the cache and pages at the very end, then writes a "stopped" progress so the bar clears). Added after Steven hit Run now instead of Run Glass's checks by mistake. The two run buttons now have a 22px gap and a divider (render.py .controls #runbtn::before) so Run now is not fat fingered instead of Glass's.
  - Glass's session death ROOT CAUSE FOUND (2026-06-15), and it is NOT a cookie problem. A storage_state "save-back" (writing ctx.storage_state(path=sp) on close so rotated cookies persist) was tried and REVERTED the same day: it could not help and was harmful. Cookie inspection disproved the staleness theory, the Glass's auth cookie was valid for another 23 hours yet the live session was already dead ("did not show the valuation box"), so Glass's ends the session SERVER side while the cookie still holds. No cookie mechanism (save-back, persistent profile) can keep a session the server has killed, and the save-back actually WIPED the Glass's token by saving the logged out state while probing a dead session. browser.open_reader_context is therefore back to leak-fix-only (close the launched browser when the context closes, no save-back). The real cause: Glass's account is a SINGLE shared login, one active session at a time (Steven confirmed his plan is single login only, and he or staff occasionally log in during the day). Every human Glass's login silently evicted BidBrain's session and vice versa. RESOLUTION chosen by Steven 2026-06-15: he and all staff will STOP logging into Glass's, leaving BidBrain the sole user, so the single session is never evicted and should hold the full 30 days (the "Don't ask for 30 days" tick). The _glass_login_notice countdown banner warns 5 days before the 30 day expiry, and _session_health_notices shows immediately if Glass's drops, so a future death is visible at once. If Glass's still dies with nobody else logging in, the next theory is a Glass's server idle/absolute TTL or fingerprint binding, NOT cookies, do not re-try save-back. Glass's needs a fresh login (login.py glass, tick Don't ask for 30 days) because the diagnostic probe wiped the saved token. Cazana, Carwow, Clickdealer were never the problem and stay on plain storage_state snapshots.
  - Session health banners, DONE (2026-06-15). daily_run._session_health_notices reads the just-written run: if priced cars exist but a source read nothing across the whole run, that source is down and a banner says so ("Glass's was not read in the last run, priced on Cazana alone, re login when convenient"); if nothing priced and cars held, both are down. render_page now takes notices (a list) as well as the single notice (the Glass's 30 day countdown), drawing each as a .notice banner. So a dead login shows on the cockpit the moment Steven opens it, not days later via held cars.
  - Glass's login countdown, DONE (2026-06-10, Steven asked for 5 days warning). Glass's sessions last 30 days (the "Don't ask for 30 days" tick at login). daily_run._glass_login_notice reads the age of data/glass_state.json (its mtime is refreshed each login capture): inside 5 days of expiry it returns a countdown line, past 30 days a likely expired line, missing file a no saved login line, otherwise None. _write_pages passes it to render_page(notice=...) which draws a quiet amber banner (.notice) above the summary tiles. Renewing the login clears it automatically since the state file mtime updates. Other platforms have no fixed expiry so no countdown, the run fails loudly when one dies. Even with Glass's expired, cars still price on Cazana alone under the single valuation rule.
  - Prep figure correction, Steven 2026-06-10: the reported prep (cards and learned averages) is ACTUAL prep only. clickdealer.NON_PREP_TYPES excludes Buyers Fee, Shipping, Delivery, Collection and the VAT reclaim/repayment accounting lines; actual_prep(split) computes it per car from the stored split. The margin is unchanged (still TotalMargin minus the FULL SIV, fees and all, since they are real money out). Example: Mercedes A average prep fell from £1,123 (with fees) to £597 actual.
  - Phase 3 card notes: _write_pages annotates each shown car (same place as the stock badge so run and --render-only agree) with its model's track record via daily_run._history_flag and db.learned_for(), appended as a flag line: "Sold 6 of these before. Average 23 days to sell, prep £1,323, margin £1,616 after prep." House style edge cases: a negative average margin reads "lost £200 after prep" (no dash), under 1 day reads "sold same day". No line when the model has never been sold. Wording is Steven's to refine.
  - Auction card REDESIGNED from a design handoff (2026-06-11, spec in data/inspect/handoff/design_handoff_auction_card/README.md). The shortlist card in render.py is now the "hf" card: Figtree font (Google Fonts link in the page head), hf design tokens scoped on .hf-card (--hf-ink #182420, --hf-accent #1f8a5b etc., 16px radius, page bg #f3f4f0), 240px photo header with overlaid source pill, watch star and reg plate (pretty reg via _pretty_reg), title row with optional Fills a gap / Seen before badges, spec chips plus a bordered CAP chip (a Grade chip was added beyond the locked spec, flagged to Steven), the PRICE GAUGE (reserve 0% to retail 100%, dot at max bid, computed in _hf_gauge) with TIGHT MODE when midPct < 26 (reserve label and room pill drop below the track, copy flips to "£X over reserve", reserve dot hidden) plus an under reserve rose pill variant the design did not cover, single line insight rows mapped from assessment flags by _insight_rows (warn/good/info, copy shortened with middle dots, ellipsis truncation, the capped wording uses a comma not the design's dash per golden rule 7), collapsible valuation detail (hidden attribute + toggleVals JS), CTA anchor + quiet verdict (Would buy when max bid >= reserve). The hide trigger was relabelled "Hide this car" so it does not clash with the verdict wording. All JS hooks preserved: classes card and starbtn, the data-* attributes, toggleBid, hide flow, tabs/search/sort. Old card CSS (photo/plat/grade/hero/statcols/chips/valn/bid/starbtn/repeat/stock) removed; slim held/rejected cards keep their styles; grid is repeat(auto-fill, minmax(min(360px,100%),1fr)) gap 20 align-items start. Verified against screenshots/cards.png on desktop and 390px mobile, including the real Skoda Kamiq SO70RVJ everything-on card. Caveat learned: preview_screenshot only captures at scroll zero, so bring a card to the top with the search filter before screenshotting.
  - Preview tooling rule (2026-06-11): NEVER put serve.py / port 8765 in .claude/launch.json. That server is production, owned by launchd (com.bidbrain.serve, KeepAlive) with the Tailscale Funnel link wired to 8765; the preview tool trying to manage it causes port conflicts and risks taking the phone link down. The only launch.json entry is "static-check", a read only python3 -m http.server on port 8771 over the project folder, used to eyeball cockpit.html. POST actions (star, hide, run) do not save through the static preview, that is expected; verify those against the real server.
  - Card fixes after Steven's review (2026-06-11): (1) WIDE MODE added to the gauge, the design only covered the left collision; when midPct > 60 the max bid label would overlap the retail hero, so it drops below the track (left) with the room pill beside it, mirroring tight mode (top row is then reserve plus retail only). Gauge modes: tight < 26, normal 26 to 60, wide > 60. (2) Equal card heights: .grid align-items stretch (was start, deviates from the handoff by Steven's instruction), .hf-card and .hf-body are flex columns, and a .hf-foot wrapper (valuation detail + CTA row + hidebox) carries margin-top auto so CTAs line up across a row; h3 reserves two lines (min-height 2.5em). (3) The verdict footnote ("Would buy this") was REMOVED as irrelevant; "Hide this car" sits in its place right of the CTA (.hf-ctarow .hidelink). (4) Steven noticed the in stock count badges vanished: the handoff only specified Fills a gap and Seen before, and the redesign dropped the ok/too_many stock badges with it. Restored 2026-06-12 as photo badges in the same bottom right stack: "N in stock" (white, .hf-gap.ok) and "N in stock · plenty" (amber, .hf-gap.many). Lesson: when applying a design handoff, carry over data the design did not know about rather than dropping it. (5) The room pill is ALWAYS right aligned and (6) the RESERVE is ALWAYS bottom left (both Steven's overrides of the handoff): every gauge has the same bottom row, reserve left + room pill right (.hf-gauge-bottom, the .hf-room normal mode row is gone). The top row is the max bid label at its dot plus the retail hero; tight (<26) left anchors the max bid label and hides the 0% dot, wide (>60) moves the max bid label into the middle of the bottom row so it cannot collide with the retail hero. Room copy: "+£X room" normally, "£X over reserve" in tight, "£X under reserve" rose pill when max bid is below reserve. The hf card no longer uses the shared HIDE_BLOCK (its hidebox is inlined in .hf-foot, same JS); HIDE_BLOCK remains for the review page.
  - Daily purchases capture and accounts email, BUILT (2026-06-12), 93 tests pass. purchases_run.py runs at 15:50 daily (launchd com.bidbrain.purchases; sales close 15:30, buys populate about 15 minutes later). It reads what was actually bought: Motorway /account/purchases/complete (motorway.read_purchases, datagrid rows a[role=row] with reg, name, status, price, relative date won) and Carwow /dealers/listings/filtered/won (carwow.read_won/parse_won, cards with Sold for price, total incl fees and VAT, CAP, Ended date; voided = cancelled, skipped). Both URLs captured from Steven's own navigation in login.py manual sessions, never guessed. Purchases go to the purchases table (UNIQUE platform+vehicle_ref; record_purchases returns only NEW rows; first capture was baselined so history is never emailed). New buys are emailed to accounts@reallyeasycarcredit.co.uk with the price paid and the achievable retail (the governing value, joined by reg from the assessments table last 4 days plus the run cache including selling plates; unmatched cars say price it by hand). Email goes through the Mail app on the Studio signed in as sales@ (osascript, no passwords stored; Steven approved this standing daily email; tested to AppleScript compile level, first real send is the first live run). No buys, no email (Steven's choice); a platform read failure prints ATTENTION and exits non zero. Commands: --dry-run prints the email, --baseline seeds without emailing.
  - Motorway session note (2026-06-12): probing URLs logged the Motorway session out (or it expired coincidentally); do not probe guessed URLs on logged in platforms, capture real URLs from Steven's navigation instead. Motorway stays on the persistent profile (a motorway_state.json from a login capture was set aside as .unused so open_reader_context keeps using the proven profile path).