Technician tool v0.3

Imported from the Google Drive folder 004-technician_tool/v0.3.

docs/firmware-diagnostics-proposals.md

@@ -0,0 +1,90 @@
+# Firmware diagnostics improvement proposals
+
+> **Status: draft proposal, AI-generated (Claude) for review by the firmware team.**
+> Nothing here is implemented. The firmware source was only read for analysis;
+> no firmware was changed. Every item links to an observation from bench
+> testing box `A4:CB:8F:C9:46:F8` (firmware v1.0, build `45fa9d5b9`) or from the
+> sample logs, so the value of each change can be judged.
+
+## Why
+
+Troubleshooting currently depends on reverse-engineering free-text log lines and
+on a status LED whose states overlap. Observed problems:
+
+| # | Observation | Effect for technicians |
+|---|---|---|
+| O1 | A new, uncommissioned box shows **green** for about 60 s, then purple | Looks healthy during installation |
+| O2 | Three different states share **white** (SD error, factory reset, setup portal) | Needs serial or guesswork |
+| O3 | **Blue** means "RTC did not answer", but the RTC is just the first I2C device | A loose I2C cable is misreported as an RTC fault |
+| O4 | Every build reports `App version: 1.0` | Builds are told apart only by ELF SHA |
+| O5 | Restarts after 150 s in an error state happen **without any log line** | A self-restart looks like a crash |
+| O6 | With the RTC not running and no Internet, sensor tasks **loop forever** on `RTC is uninitialized` | An offline box records nothing, and it isn't obvious why |
+| O7 | The OTA token is printed in plain text (`OTA token loaded from NVS: %s`, `received token: %s`) | Secrets end up in technician logs |
+| O8 | Native USB loses output while no host is reading; a stale FIFO fragment is glued to the next line | The early boot after a power cycle is lost, and lines get corrupted |
+| O9 | The microphone reports "shutdown state" (GPIO3) with no hint of the cause | A technician can't tell a switch from a fault |
+| O10 | The deployed build logs some messages under different tags than the source (e.g. SD mount under `INFLUX_BUFFER`) | Parsers must ignore tags |
+
+## Serial output
+
+Priority: **P1** = large gain for little effort, **P2** = valuable, **P3** = nice to have.
+
+### P1
+
+1. **One structured boot line** with a fixed prefix, printed right after `app_init`:
+   ```
+   KOTA: boot fw=1.0.15 build=45fa9d5b hw=2 mac=A4CB8FC946F8 reset=USB_UART_CHIP_RESET mode=matter commissioned=0 last_restart=status_watchdog
+   ```
+   The tool can then identify firmware, hardware and mode from a single line (fixes O4 and O10). Fill `PROJECT_VER` from `FIRMWARE_VERSION` (currently `15` in `pinout.hpp`) so every build has a unique version.
+2. **Log every status change** inside `Status::set_status()`, with a reason argument:
+   ```
+   KOTA: status OK -> CONNECTION_ERROR reason=influx_flush_failed
+   ```
+   This is the exact LED colour. Today the tool has to infer it from about 15 different messages.
+3. **Log the reason before every `esp_restart()`** and store it (RTC memory or NVS) so it can be printed at the next boot as `last_restart=`. Values: `status_watchdog_150s`, `out_of_resources_30s`, `button_factory_reset`, `setup_portal_toggle`, `ota_done`, `init_failed:<step>` (fixes O5).
+4. **Stop printing secrets**: print only the first and last 4 characters of the OTA token, or `token=present` (fixes O7).
+5. **Warn once and continue when the time is unset**, instead of the `while (now == 946684800)` loops in `device_status.cpp` and the sensors. Record data with a `time_valid=false` field or buffer it with uptime, then correct it after SNTP (fixes O6). This is a behaviour change, but the biggest data-loss risk found.
+
+### P2
+
+6. **I2C bus scan at boot**, before any sensor init:
+   ```
+   KOTA: i2c SDA=1 SCL=1 0x39=AS7341 0x68=DS1307 0x6B=SEN66(missing) 0x76=BME688
+   ```
+   Reading the SDA/SCL levels first detects a stuck-low bus (shorted or broken cable). The scan then separates "whole cable" from "one sensor" (fixes O3).
+7. **RTC detail**: at init, print the DS1307 clock-halt (CH) bit and the raw time (`KOTA: rtc running=0 time=2000-01-01 (coin cell?)`). After SNTP sets the time, re-read it 2 s later to verify it's ticking.
+8. **SD card detail**: card type, size, FAT type, free space, which `ENV.TXT` keys are present (names only, never values) and the buffer size in lines. Example: `KOTA: sd type=SDHC size=7.4GB fs=FAT32 free=7.3GB env_keys=INFLUX_URL,INFLUX_TOKEN buffer_lines=42`.
+9. **Periodic health line** every 60 s. `DeviceStatus` already computes almost everything:
+   ```
+   KOTA: health up=312s status=CONNECTION_ERROR wifi=disconnected rssi=0 time_valid=0 sd=ok buffer=42 heap=81234 psram=6.1M last_upload=never ota=unregistered
+   ```
+10. **Matter commissioning status at boot**: `KOTA: matter fabrics=0 window=open` plus the QR payload and manual code, so the tool can show the QR code on screen when the sticker is missing.
+11. **Microphone enable reason**: `KOTA: mic enabled=0 (GPIO3=1, expected 0)` once at boot, instead of a warning every 10 s (fixes O9).
+12. **Stable error IDs** in messages, e.g. `[E-SD-01] Failed to initialize the card`, documented in one table shared by the firmware, the technician tool and the service docs.
+
+### P3
+
+13. **Console commands** over USB (the firmware already runs a `console` task): `diag` (full health dump), `i2cscan`, `sdinfo`, `ledtest` (cycles all colours so the technician can confirm the LED hardware), `version`, `bootlog`. The tool could then run checks without a reboot.
+14. **Boot-log ring buffer** in RAM, dumped with `bootlog` or on first host connect. This recovers the early boot lost on native USB after a power-switch cycle (fixes O8).
+15. **Button events**: log the press start and release with duration (`KOTA: button press 1.2s -> setup_portal`). This allows a safe button test and shows accidental factory resets.
+
+## RGB LED
+
+1. **Separate "not connected yet" from "OK"** (fixes O1): **cyan breathing** while the box is uncommissioned / advertising over BLE, fast cyan while a phone is connected, and a green flash when commissioning completes.
+2. **Split the white states** with patterns (fixes O2):
+   - SD card / `ENV.TXT` error: white, fast blink (2 Hz)
+   - factory reset in progress: white, solid
+   - Wi-Fi setup portal: white, slow breathing
+3. **Blink codes for red and blue** (fixes O3): N short pulses followed by a pause identify the part.
+   - Red: 1 = BME688, 2 = SEN66, 3 = AS7341, 4 = microphone, 5 = flicker, 6 = self-check failed
+   - Blue: 1 = RTC not responding, 2 = I2C bus stuck low
+4. **Split purple**: solid = no network/DNS; blinking = Internet works but the server rejected the box (InfluxDB 401/404, OTA 401). These need different fixes (network vs SD card settings).
+5. **Show the cause before a self-restart**: flash the error colour 3 times before `esp_restart()`. After boot, show the previous boot's error colour for 3 s (from RTC memory), so a reboot loop has a visible cause.
+6. **Update progress**: yellow blinking while downloading (faster as progress grows); orange stays solid for the self-check.
+7. **Brightness and distinguishability**: colours are set at 10 % (`255/10`). Yellow and orange, and white and cyan, are hard to tell apart in daylight. Prefer fewer colours plus patterns, and raise brightness for error states.
+8. **Heartbeat LED (GPIO6)**: keep 1 Hz for "running", add a double-blink when offline, so the heartbeat alone gives a rough state even when the RGB LED is faulty.
+
+## What changes in the technician tool
+
+- The structured `KOTA:` lines (items 1, 2, 3, 9) would replace most regex rules in `rules.toml` and make the LED model exact rather than inferred.
+- The console commands (item 13) would let guided checks test hardware directly (I2C scan, LED test) instead of asking the technician to power-cycle.
+- The new LED patterns would add answers to the Quick check (e.g. "white, blinking fast") and remove the follow-up questions currently needed to separate the white states.