Fault Recovery and Configuration Preservation
This page was reviewed against the 2026-10-04 development code and existing logs. The OTA self-check 22 items, the disable lifecycle, and the RTC/SD/UART recovery-related code have local host pass records. The r7 host suite still fails overall, and a targeted r11 pass cannot replace a unified candidate or field acceptance; nothing has been deployed based on this. Field behavior is governed by the actually deployed version. The current software has no standalone rescue mode that covers all boot failures. Configuration backup, network hotspot, and OTA rollback solve different problems.
Judge What Happened First
| Symptom | Current expected behavior | What you cannot conclude from it |
|---|---|---|
| One config file missing or unparseable | Normal storage load may try the previous-generation backup | All fields, references, and wiring are correct; the main file is fixed |
| Storage mount failure | Normal boot path stops; no auto-format | The web page will definitely still open |
| Config migration shows pending restart | The prepared managed config is verified and applied at next boot | Already in effect; peripherals already working |
| Migration conflict/invalid/recovery blocked | The corresponding candidate is rejected; state and recovery material are kept | You can directly delete the staging directory or re-import |
| STA temporarily offline | Retries per config and runtime state; AP fallback may occur when allowed | Every failure creates a hotspot, or the old IP never changes |
| Insufficient memory | Some optional features may be paused and recovery attempted | Admin pages always stay reachable; all resources are released |
| SD or bus failure | Errors and diagnostics returned; some devices support explicit recovery actions | Status queries auto-fix hardware, or the recovery button has no side effects |
| OTA failure | The upload session may be terminated; whether rollback works depends on the actual firmware, bootloader, partitions, and old image | A failed upgrade always switches back to the old version |
Repeated restarts change field state and may trigger config rollback or cleanup. Keep the first-failure evidence first, then decide the specific recovery action; do not make format/full-erase/factory-reset the default first step.
What to Preserve Before Maintenance
- Verify the device MAC/chip, board revision, current port or IP, and firmware and web versions. A device name or old IP alone cannot prove you found the right device.
- Save the original network, auth/users, MQTT and platform bindings, peripherals, rules, screen layouts, and related files; record the coverage and summary of this backup. Parts that cannot be obtained are explicitly marked missing.
- Keep raw backups containing credentials private. Public issue reports contain only desensitized information and summaries; "export reusable plan" filters out some information and cannot replace the original recovery backup.
- Verify wiring, power, and loads. Restarting, re-enabling peripherals, or importing enabled rules can change outputs; pumps, valves, motors, and other loads must have their safety conditions confirmed by field personnel.
- Define the operations allowed this time: read-only troubleshooting, config changes, bus recovery, restart, or deployment. Unconfirmed actions stay unexecuted; one troubleshooting authorization does not replace all later write authorizations.
Configuration Errors and Power Loss
When the main config cannot be parsed, the program may load the previous-generation .bak and try to restore the main file. A loadable backup only means that generation's data is readable; it does not mean all business configs are valid, nor does it guarantee the main slot recovers. Keep the relevant logs, and after an authorized fix, verify the files, runtime state, and the next boot.
When both the main file and the backup are unavailable, modules behave differently. Network still tries partial NVS backups and may fall back to defaults when nothing is usable; there is no guarantee that "any config automatically returns to the last fully working state".
Temp files and backup commits reduce write-interruption risk but cannot guarantee lossless power-off in every case. Some non-critical configs may first enter a deferred-write cache; storage hardware faults, insufficient space, incomplete writes, and inconsistency between multiple files still need investigation. "Page save succeeded", "committed to disk", and "applied" must be verified separately.
Per-file import and managed cross-file migration are different entry points. With the former, earlier files may already be saved when a later file fails; the latter uses a reviewed session and candidate summary and applies a limited set of configs at boot. Neither is an automatic rollback of the whole Flash or firmware.
How Managed Migration Confirms Recovery
Use the migration state and confirmation flow in Device Configuration. After preparation succeeds, still verify the post-restart state, target identity, and actual config; when a request times out or the final response is lost, query first — do not replay the whole package directly.
When the UI explicitly offers cleanable or isolatable staging, first verify that session/revision and the preserved material, then perform the corresponding confirmation. Isolation applies only to staging forms the program recognizes; it does not mean arbitrary broken config can be safely isolated. A transaction in application or blocked recovery cannot be replaced with ordinary cleanup; when boot is already blocked, the web may not offer this operation and a specific offline maintenance plan is needed.
This migration entry point preserves device identity and account bindings; network and protocol are read-only dependencies, and user/auth files are not writable. Ordinary import and direct file writes do not automatically have the same guarantees.
Special Boundaries of Network Recovery
With STA credentials present and auto-fallback not allowed by config, the device's retry path keeps attempting STA recovery and may not create a hotspot. Confirm the hotspot name, address, and availability from actual config and field evidence; do not guess from old documentation.
The source has a separate last-resort AP path: only when that AP also fails to start does the network-init failure counter increment; reaching the threshold triggers cleanup of the network-specific NVS backup and a restart. This does not delete the network main file and .bak in LittleFS, and it is not a full device factory reset. When NVS writes fail, the actual persisted count may differ from the log's intent. Valid JSON may still be preferred at the next boot, so this count cannot be assumed to have repaired a bad network config.
Before changing the network, keep the original network config and a recoverable admin entry; after a disconnect, first confirm the current MAC/IP and routing state — do not trial-and-error with repeated SSID changes, config wipes, or frequent restarts.
OTA and Boot Failures
Upload OTA has stalled-session cleanup: the related check entry records a failure and clears the session state after 60 seconds without a new chunk. The upgrade partition may already contain partial bytes — this is not proof of "never written" and cannot replace verification of the current boot image.
The development candidate has connected the NEW/PENDING_VERIFY pending states with a 90-second application-run window; repeated boot checks do not refresh the window. Expiry confirmation runs at most 3 times at 5-second intervals; SDK failure exhaustion keeps the unconfirmed state and does not auto-restart. On critical initialization failure, save diagnostics first, then try to mark the new version invalid; SDK failure does not auto-restart, and a restart is only explicit after success. These 22 scenarios passed locally in the r7 host log, but the host suite still fails overall and cannot be treated as deployed devices or accepted rollback behavior.
The 90 seconds is only an application-run window, not a full hardware self-check. Actual rollback also depends on the bootloader, an available upgrade partition, and a valid old image; 4MB single-app layouts cannot be assumed to be supported based on experience with other board types. Do not claim rescue capability just by changing one config item or copying an old image.
If a normal boot fails during storage or migration recovery, network and web may not be established yet. In that case, keep the evidence and make a maintenance plan consistent with that device, its partitions, and the config backup. Any reflash must define the target and what is preserved first; never full-erase first and back up later.
SD and Bus Recovery
SD status queries do not auto-remount. An explicit recovery action may unmount, release, and re-acquire SPI resources and actually accesses the media; confirm the card, power supply, owning peripheral, and shared bus before operating. After a failure, keep the original error and counts; do not click recovery repeatedly to create new disturbances.
I2C reinitialization only covers the software bus entry; it does not guarantee releasing electrical pull-downs or fixing all shared devices. busy, unknown, or pending-cleanup means completion is not confirmed; do not assert outputs have stopped or resources released just because the config is disabled. Verify other buses and outputs per their own device documentation.
RTC, UART, and Diagnostic Sampling
RTC access, 35 start/stop, 36 calendar read, and 37 explicit copy of the system local time are implemented; the calendar, the quality of the system source, and power-loss retention are different pieces of evidence. After a time source fails, a new valid sync record is required — refreshing state does not restore the old record; the copy action does not actively request NTP, and a failed write may still have changed the RTC. Actual time calibration and power-loss retention remain NOT_RUN; see DS1302 RTC.
The generic UART has controller leases and receive generations; every real reinitialization establishes a new receive stream; when disabled, rebuilt, or the source cannot be confirmed, related rules stop the remainder, and already sent data is not recalled. Local initialization/send success is not peer success or a recovered fault. Real send/receive, hardware overflow, and long-term recovery remain NOT_RUN; see UART Example.
System info/diagnostic bundles have per-pool memory snapshots, and resource state can show busy, retiring, and other unfinished states; a developer single-task stack snapshot also has a restricted entry. These endpoints do not auto-perform feature toggles or fault-recovery experiments; overlapping memory pools cannot be summed, missing values are not zero, and task-stack sampling may affect scheduling. The six groups of feature state, resource release/recovery, and peaks still require real-device measurement and are currently NOT_RUN; see System Health Monitor for endpoints and permissions.
What Counts as Recovery Complete
The 2026-10-04 development candidate also fixed a missing manager destructor link in the C6 config that disables OTA: a disabled config still creates no OTA service, and the resource exit order on enable is preserved. The real disable-translation-unit link regression passed locally in the r7 host; the target build and actual boot must still be confirmed with the same-candidate evidence; this item does not mean device rollback/rescue capability has been accepted or field firmware updated.
Configuration recovery should verify saved content, target identity, management network, auth, MQTT, layouts, and related files, then validate business behavior. Deployment must have write and independent verification evidence for this specific artifact; old reports cannot be reused. API readback, physical actions, fault recovery, and long-term stability are recorded separately.
Keep the original failure and recovery process. Short-term normality after recovery does not erase the earlier interruption and cannot be counted as 24-hour or 7-day stability passing. The actual devices, fault injection, physical recovery, memory measurement, and long-term stability items covered by this page remain NOT_RUN.
Related: System Health Monitor, Network Configuration, Current Improvement Acceptance Status.