# BidBrain project status, part 12: Steven's reconnected copy, login automation and the nav redesign (25 to 26 August 2026) Moved unchanged from CLAUDE.md on 2026-09-18. The standing rules stay in CLAUDE.md, which lists every part of this log. - PLATFORM ENABLED/DISABLED IS NOW A REAL, PERSISTENT SETTING, 2026-08-25 (Steven: "if i turn them off in the run setting will that keep as my default run? or will i have to change the settings every time... I want to change the settings once and then the autoruns only use the settings I have chosen"). Checked before answering rather than assumed: the Run modal's per platform checkboxes (built 2026-08-20) were ALWAYS a one off, single click selection, never persisted, and the scheduled 5:10pm job calls run() with platforms=None (no --platforms flag at all in the plist), which always meant every one of the 5 platforms, every day, whatever a manual run had last done. There was no way to turn a platform off for good, only remember to uncheck it by hand each time, exactly the gap Steven suspected. - New daily_run.PLATFORM_ENABLED_DEFAULTS ({key: True for every PLATFORM_KEYS entry}), same shape and philosophy as the existing INTEGRATION_DEFAULTS (Glass's, DealerKit stock/purchases, LE Capital funding), but deliberately kept OUT of pricing.DEFAULT_SETTINGS: whether a platform is READ AT ALL is an orchestration concern, not a buying rule, and pricing.py stays pure. A new pure, unit tested _select_platforms(requested, enabled_map) is the one place the real selection is computed: starts from requested (None or empty means every platform, exactly what the scheduled job's own call always looks like) then narrows to only the platforms switched on in Settings. Critically this narrowing happens AFTER, as a floor, not instead of, the requested set, so a Run modal pick that still explicitly lists a disabled platform cannot bring it back either, matching INTEGRATION_DEFAULTS' own stated guarantee ("a disabled integration can never run by some path the toggle forgot about") extended to platforms for the first time. run()'s own selected = ... line now reads through this, unknown platform keys still validated and rejected loudly first as before, only real, known keys ever get silently narrowed by the enabled floor. - UI: each platform's own settings tab (Motorway, Carwow, Auction4Cars, Dealer Auction, DealerWay) gained a "Read this platform" card above its existing Hard gate card, one checkbox, "Include in daily and manual runs", wired into the same paintGate()/syncGate() round trip the gate fields already use so it saves through the existing "Save settings" button, no new save flow needed. serve.py's _merged_settings gained a platform_enabled key (PLATFORM_ENABLED_DEFAULTS merged with whatever is saved, the same per key merge pattern _merged_settings already uses for "platforms" and "integrations", chosen specifically because a wholesale replace instead of a merge is the exact bug class that bit "platforms" when DealerWay was added, see the entry above this one) and a new _validate_platform_enabled (mirrors _validate_integrations: every known key required, an unknown key rejected outright, not silently accepted). - A DISABLED PLATFORM NO LONGER TRIGGERS A FALSE "PLEASE LOG IN" ALARM EITHER: _platform_health_notices now skips any platform switched off in Settings, since its last recorded read may well have failed (plausibly why it was turned off in the first place) and nagging to relogin to something deliberately not run would be a real false alarm, not a genuine one. Verified live: with Auction4Cars and Dealer Auction genuinely never logged in on this Mac (found during this same review, both recorded real read failures), disabling DealerWay produced zero mention of it in the notices while the other two's real, honest alarms stayed exactly as they were. - Verified live end to end through the real running server, not just the unit tests: saved a real change through /api/settings turning DealerWay off, confirmed it round tripped; then, in a genuinely FRESH Python process (the same shape as how the scheduled job actually starts each day, so no stale in process cache could be hiding the real answer), confirmed _select_platforms(None, ...) (the scheduled job's own call shape) excluded DealerWay, AND separately confirmed a simulated Run modal call that still explicitly listed all 5 platforms, dealerway included, ALSO excluded it, proving the floor holds against both paths asked about. Reset back to every platform enabled afterward, this review was proving the mechanism, not deciding Steven's real settings for him. 13 new tests (PLATFORM_ENABLED_DEFAULTS' own shape, 5 real _select_platforms scenarios including the "still explicitly requested but disabled" case, the validator's round trip and 4 rejection cases, a settings table round trip proving one platform's own save never touches another's or any other setting). 521 tests pass; 4 pre-existing hand built settings payloads in the price ceiling tests needed platform_enabled added to them too (a new required top level key, the same friction "platforms" itself already causes any hand built payload, not a real regression). - CARWOW'S DEAD SESSION WAS READING AS "0 AUCTION CARS", A REAL FAULT LOOKING LIKE A QUIET DAY, 2026-08-25 (Steven ran Carwow only, it "finished", nothing showed; investigated live rather than guessed). The manual log read plainly: "Carwow: read 0 auction cars... Carwow done: 0 shortlisted, 0 held", no error anywhere, indistinguishable from a genuinely empty filtered list. Checked live: the saved session (last logged in the day before) had died, the real stock page redirected straight to a login form, confirmed by loading it fresh and reading page.url and the page's own visible "Sign in... to continue to your carwow dealer account" text. - ROOT CAUSE: carwow.read_all_pages waits for [data-listing-id] on each page load; when that times out it just break's, on the reasoning that the page genuinely has no more listings. A dead session redirected to a login page ALSO never shows [data-listing-id], so the exact same timeout fires for both cases, and the code could not tell a real empty page from a dead login. This is the same class of "silent wrong answer" bug already fixed once for Cazana and Glass's (both raise a clear RuntimeError instead), Carwow's own reader had never been given the same treatment. - FIX, and a real gap caught building it: new carwow._looks_logged_out(url), checked when the wait times out; if true, raises "Carwow did not show the stock list, the login may have expired. Run: python3 login.py carwow, then try again." instead of silently breaking. First version only checked the HOST (assuming a dead session always redirects to auth.carwow.co.uk, the one real redirect actually observed live), but browser.SITES["carwow"]["login_url"] is itself dealers.carwow.co.uk/dealers/login, the SAME host as the real stock list, just a different path, a case the host only check would have completely missed, reintroducing the identical bug for a different redirect shape. Fixed to also check for "/dealers/login" in the URL on the real host. No wiring needed in daily_run.py: _read_carwow already wraps the whole read in try/except and falls back to the last known good Carwow cars on ANY exception (built 2026-08-21 for exactly this "do not wipe a platform to zero on a failed read" reason), records a real site health failure (db.record_site_health), which the settings page's bell and red dot mechanism already surfaces, so this fix slots straight into machinery that already existed rather than needing anything new built around it. - Verified live, not just against synthetic test strings: replayed the EXACT real redirect URL captured during today's incident (a real auth.carwow.co.uk/u/login?state=... string) through _looks_logged_out and confirmed it is caught; confirmed the real healthy stock URL is not flagged; then, after a fresh real login, confirmed a live read still returns real cars with no false positive. 3 new tests (the real stock URL, the auth.carwow.co.uk redirect, and specifically Carwow's own same host /dealers/login path, the exact shape the first draft of this fix would have missed). 524 tests pass. - CARWOW LOGIN AUTOMATED TOO, THE SAME WAY AS CAZANA, 2026-08-25 (Steven, after the dead-session incident above: "do number 1", having weighed the one real risk honestly first, that Carwow's login runs through Auth0, which has configurable captcha/bot detection Claude could not confirm was active or not without actually trying a real automated login). Checked Carwow's real login page live before building anything: a plain #username/#password form with a real type=submit "Continue" button, no cookie banner, no visible 2FA, genuinely simpler in shape than the Auth0 bundle's own captcha config options first suggested. - setup_cazana_credential.py generalised into setup_credential.py (cazana or carwow), the exact same hidden getpass flow, one script instead of a near duplicate per platform; the old Cazana only version deleted outright, nothing else in the repo called it directly (auto_login_cazana.py always read the keychain itself via `security`, never imported the setup script). - New auto_login_carwow.py, mirroring auto_login_cazana.py's shape closely: reads the keychain entry for dealers.carwow.co.uk, fills #username/#password, clicks the real Continue button, then (since Auth0's own redirect chain has no single fixed "logged in" element to wait on the way Cazana's search box gives one) navigates straight to the real stock URL and reuses carwow._looks_logged_out (built earlier the same day for the reader's own dead session fix) as the one true signal of success, saving the session only once that check passes. A captcha challenge, a wrong password, or any other block all land on the same still-logged-out path and raise loudly with whatever error text the page itself shows, never guessed, never a false "logged in" reported. - THE REAL TEST: no captcha or bot check fired. Ran for real, headless, with Steven's own freshly stored credential: "Logged in to Carwow. Session saved.", then proven with a genuine live stock read straight after (10 real cars) that the saved session actually works, not just that the script printed success. Also proven through the real scheduled path itself, not just run directly: a new launchd job com.bidbrain.carwow-login (17:00 daily, 10 minutes before the 17:10 run, the same slot and reasoning as com.bidbrain.cazana-login) kicked off via launchctl and produced the identical clean success. - Carwow's own session lifetime looks meaningfully shorter than Cazana's 30 day one (this was the SECOND dead Carwow login this same session, having needed a fresh manual login once already near the start), so a daily refresh, same cadence as Cazana, is the sensible default rather than trying to measure a real expiry window first. 524 tests pass (this is live browser automation, verified live end to end exactly as described, not unit tested, matching every other auto_login_*.py in this project). - NAV BAR REDESIGNED FOR CLARITY, ONE SHARED IMPLEMENTATION ACROSS ALL FOUR PAGES, 2026-08-25/26 (Steven, on his own reconnected copy: "i dont like the way the icons at the top right hand corner of bidbrain are handled, they arent very clear what they are... they all seem to change from screen to screen too, all very messy"). Investigated live before proposing anything: the icon order genuinely differed page to page, the Didn't win icon (three plain concentric circles) read ambiguously as a person or avatar at 18px, the shortlist star showed a bare corner badge with no label on three of the four pages while the cockpit's own showed a live count and text, and underneath all of it the same SVG, CSS and JS was hand copy pasted separately into render_purchases, render_lost_bids, render_settings and render_page rather than built once, which is the actual root cause of pages drifting apart from each other over time. Design agreed through several rounds with Steven before any code was touched (his explicit rule: "dont do any work until i give the ok"). Landed on: Shortlisted, Alerts and Cockpit (home) always visible in the same order on every page, each with a real text label, never icon only; a "More" dropdown holding Purchases, Didn't win, then Settings last below a divider, same order everywhere. Steven's own reasoning, kept directly: Alerts needs to stay visible for its badge, Purchases and Didn't win are occasional lookups suited to a dropdown, Settings belongs at the bottom of that list, and Home deserves the same always visible treatment as Shortlisted and Alerts since it is a high frequency "get me back" action, not an occasional one. Built as one shared set of Python functions in render.py rather than four separate implementations: _MORE_PAGES (the dropdown's own fixed order, Settings always last), _more_dropdown(current) (builds the dropdown, skips the current page's own entry, never a self link, the divider omitted when Settings itself was skipped or nothing else survived to divide from), _nav_home(current) (the Cockpit icon, empty string on the cockpit itself), _nav_star_link() and _bell_skeleton() (the Shortlisted and Alerts markup for the three pages with no run data of their own at render time), plus shared _NAV_CSS, _NAV_JS_MORE and _NAV_JS_FETCH constants. The ambiguous three circle icon was replaced with a plain crossed circle (_CROSS_SVG), the same "no, missed" shape used everywhere from a strikethrough to a plain X, unambiguous even at this size. Deliberately did NOT force the three client fetch pages (purchases, lost bids, settings) to become server rendered like the cockpit's own bell: render.py cannot import daily_run.py (daily_run.py already imports render.py, so that direction would be circular), and the data fetch split is invisible to the end user, so only the presentation layer (CSS, JS, HTML structure) was unified, the underlying mechanism (server rendered on the cockpit, GET /api/notices fetched live on the other three) was left exactly as it was. Wiring used a placeholder sentinel plus .replace() approach for the three pages, whose own return values are plain triple quoted strings, not f-strings (matching render_settings' own existing {{PLACEHOLDER}} convention already in the file): __NAV_ITEMS__, /*__NAV_CSS__*/ and //__NAV_JS__ are substituted in after the string literal, avoiding the risk of converting a large existing template to an f-string and mis-escaping literal braces throughout its embedded CSS and JS. render_page (the cockpit) already is an f-string, so its own nav row and CSS interpolate the same shared functions and _NAV_CSS/_NAV_JS_MORE constants directly. The cockpit's own Shortlisted button (a genuinely different in page filter toggle with a live count, not a navigation link) was left exactly as it was, per the design, it already showed a label. Verified live end to end on the real running server (cockpit.html is a static file written by daily_run.py, not rendered per request like the other three, so python3 daily_run.py --render-only was needed to pick up the change there; the other three render fresh on every request from serve.py so needed only a launchd restart): all four pages now show Shortlisted, Alerts, Cockpit (hidden on the cockpit itself) then More in the same order; the More dropdown correctly self excludes whichever page it is on and omits the divider when Settings itself was the one excluded; the bell and the new More dropdown open and close independently and both close on an outside click; the Cockpit link navigates correctly from every other page. 524 tests pass (this round is render only markup restructuring, no gate or pricing logic touched). Not yet pushed to the shared repo, a local commit on Steven's own copy pending his OK, since renaming the "Didn't win" icon and unifying the shared nav code could benefit Mark's copy too, but this was built and verified against Steven's own live data this session, not yet offered to Mark as a PR. - MOTORWAY LOGIN AUTOMATED TOO, THE SAME WAY AS CAZANA AND CARWOW, 2026-08-26 (Steven, right after a real dead Motorway session blocked a purchases sync: "is there any way to automate it like we have the cazana/carwow ones?"). Checked Motorway's real login page live first, same discipline as the other two: a plain #username/#password form with a "Keep me signed in on this device" checkbox and a type=submit button, no CAPTCHA visible. `setup_credential.py` gained a third site, "motorway" (server pro.motorway.co.uk), no other change needed, it was already generic. New `auto_login_motorway.py`, closely modelled on `auto_login_carwow.py`: reads the keychain credential, fills #username/#password, ticks #remember, clicks the real submit button, then verifies success by landing on a real dealer page (https://pro.motorway.co.uk/vehicles) and checking the URL is not still on the login/signin path, mirroring login.py's own real time auto detect signal for this exact site (host correct, "login" and "signin" both absent from the path). A wrong password or an unexpected block (a CAPTCHA, a changed page) lands on the same still-logged-out check and fails loudly, never guessed. ONE REAL DIFFERENCE from Cazana and Carwow, worth remembering: Motorway has no saved `motorway_state.json`, its login lives in the SHARED PERSISTENT BROWSER PROFILE (`data/browser_profile`, `bidbrain.browser.open_context`) that every other Motorway read and write already relies on (`open_reader_context` falls back to it when no state file exists). So `auto_login_motorway.py` logs in through that SAME profile rather than a fresh throwaway context, and there is nothing separate to save afterward. This also means it must NEVER run at the same time as anything else touching Motorway, only one process can hold that profile at once (see `open_context`'s own docstring). Checked Steven's own real `launchctl list` before picking a time: his Mac runs the purchases sync at 15:50 and the single combined daily run at 17:10 (no separate Motorway-only slot the way Mark's own Mac has), so the new `com.bidbrain.motorway-login` job was scheduled at 06:00, well clear of both, rather than alongside Cazana/Carwow's own 17:00 slot which would have risked a genuine profile clash with whatever runs at 17:10. Verified: the "no saved credential yet" path fails with a clear, actionable message (confirmed live, no credential stored yet at build time); the new plist passes `plutil -lint`, loads cleanly, and `launchctl print` confirms the real registered 06:00 calendar interval. Not yet proven with a real login (Steven still needs to run `setup_credential.py motorway` himself, the credential step no session can do on his behalf), so the first real firing (or a manual `python3 auto_login_motorway.py --headed` once the credential is saved) will be the first true end to end proof, same as Carwow's own build. - MOTORWAY LOGIN AUTOMATION: A REAL BUG FOUND AND FIXED THE SAME DAY, 2026-08-26 (right after Steven stored the credential and ran it for real). auto_login_motorway.py's first version always expected the real login form after navigating to LOGIN_URL, but the persistent profile turned out to still be genuinely logged in (Steven's own earlier session from 2026-08-25 had not actually died, purchases_run.py's own read failure the day before must have been something else, a page hiccup rather than a dead session): /signin silently redirected straight to the real dealer homepage ("Live sale", "Bids & offers", "Purchases", "Dashboard", a user initials chip all visible, no login form anywhere), so waiting on #username timed out and the script reported FAILED for a session that was actually fine. Confirmed live, both that the fresh login form Steven would see if genuinely logged out is real (checked earlier the same day in an isolated browser tab) and that the persistent profile's own redirect behaves differently once already logged in, proving the two states genuinely differ, not a fluke. Fixed: after navigating to LOGIN_URL, the script now checks the landing URL with the same _looks_logged_out() signal used everywhere else here; already logged in prints "Already logged in to Motorway. Nothing to do." and returns immediately, only actually falling through to the fill in and submit flow when Motorway genuinely shows the form. Verified live: the fixed script correctly recognised the real, already logged in profile this time. Still honestly unproven: the real fill in, submit, and verify path itself (mechanically identical to the already proven auto_login_carwow. py, and matches the real page inspected live, but Motorway shares one persistent profile with every other automated Motorway read and write, with no safe throwaway copy to force a genuine logout against without risking the one real working session), so its first true end to end proof will be whenever this profile's own session next actually expires, whether that is the 06:00 scheduled job or sooner.