Instability in sensing preview and cannot start training (single ESP32-S3-DevKitC-1)

[Sann32]

Hi everyone,

I’m trying to get RuView running with a single ESP32-S3-DevKitC-1 (8MB flash). I followed the instructions from the v0.5.0-esp32 release and the main README. The firmware is flashed correctly, provisioned with my WiFi credentials and the server IP, and the sensing server (Rust version) receives data from the ESP32.

However, I’m facing two main issues:

Sensing preview is very unstable
In the Observatory UI, the “Sensing” section (the mini preview) constantly flickers/jitters. The detected persons appear and disappear, and the skeleton is not stable. Sometimes it shows two persons even though I only have one ESP32 and I am alone in the room.

Cannot start training
I tried to record a dataset for training, but the Rust server does not have a --record flag (my binary was built from the latest main). I attempted to use the Python scripts to capture data, but I’m not sure how to structure the dataset for the --pretrain or --train commands. When I run
./sensing-server.exe --pretrain --dataset recordings/my_session
I get errors about missing files or unsupported format. I suspect the data I captured is not in the expected format (MM-Fi / Wi-Pose).

My setup:

Hardware: ESP32-S3-DevKitC-1 (one unit, no mmWave attached)

Firmware: v0.5.0-esp32 (8MB flash)

Server: Rust sensing-server built from main (latest commit)

OS: Windows 10

Server command: sensing-server.exe --source esp32 --udp-port 5005 --http-port 3000 --ws-port 3001

UI accessed at http://localhost:3000/ui/index.html

What I’ve tried:

Re-flashing the ESP32 several times, verifying provision parameters.

Checking the UDP packets with Wireshark — they seem to be coming in regularly.

Looking for a --record flag in the Rust server (not present) and trying the Python capture scripts.

What I’d like to know:

How can I stabilize the sensing output with a single ESP32? Is the instability normal for a single‑node setup, or does it indicate a configuration problem?

What is the correct workflow to record CSI data and then run self‑supervised pretraining (or supervised training) using the Rust server? Should I use a different branch or a specific tool?

If the Rust server doesn’t have --record, how do I capture a dataset that the training pipeline accepts?

I’ve attached a screenshot showing the UI with two persons detected (though only I am present) (But there's a sister in the next room.). Any guidance would be greatly appreciated. Thanks in advance!

[ruvnet]

Hi @Sann32 — thanks for the detailed write-up and screenshots. Let me address each of your questions.

1. Instability with a single ESP32

Single-node instability is expected behavior, not a configuration problem. With one ESP32, the system can only see CSI from a single viewpoint, and WiFi multipath reflections (signals bouncing off walls, furniture) are indistinguishable from a second person. Your sister in the next room is also likely contributing — WiFi CSI penetrates walls, so her movement creates real signal perturbations that the algorithm counts as a second person.

Recent fixes that should help:

• PR fix(sensing-server): person count hysteresis to stop flickering #295 — Added hysteresis to person count estimation. The count now requires consistent signal changes over a smoothing window before updating, which reduces the rapid flickering you're seeing.
• PR fix(sensing-server): detect ESP32 offline after 5s timeout #300 — ESP32 offline detection with a 5-second timeout, preventing stale data artifacts.

Make sure you're running a recent build of the sensing server (post-March 24).

To reduce phantom detections with 1 node:

• Ask your sister to stay still or leave the room during calibration
• Add --filter-mac YOUR_ROUTER_MAC when provisioning to isolate CSI from your specific router
• Add a second ESP32 at a different position/height for better spatial discrimination

2. Recording CSI data

The Rust sensing server does not have a --record CLI flag. Recording is done via the REST API while the server is running:

# Start recording (note: "recording" not "record")
curl -X POST "http://localhost:3000/api/v1/recording/start?label=empty"

# Wait ~60 seconds, then stop
curl -X POST "http://localhost:3000/api/v1/recording/stop"

# Verify recordings
curl http://localhost:3000/api/v1/recording/list

Recordings are saved as .jsonl files in data/recordings/ relative to where the server is running.

3. Training workflow

There are two separate training paths:

A. Adaptive training (room calibration — recommended for your case)

This trains a lightweight classifier on your room's CSI characteristics:

# Record 3 sessions: empty room, sitting still, walking
curl -X POST "http://localhost:3000/api/v1/recording/start?label=empty"
# wait 60s
curl -X POST "http://localhost:3000/api/v1/recording/stop"

curl -X POST "http://localhost:3000/api/v1/recording/start?label=still"
# wait 60s
curl -X POST "http://localhost:3000/api/v1/recording/stop"

curl -X POST "http://localhost:3000/api/v1/recording/start?label=walking"
# wait 60s
curl -X POST "http://localhost:3000/api/v1/recording/stop"

# Train the adaptive model
curl -X POST http://localhost:3000/api/v1/adaptive/train

# Check status
curl http://localhost:3000/api/v1/adaptive/status

After adaptive training, the system blends the trained model's predictions (70%) with raw heuristics (30%), which significantly reduces noise for your specific room.

B. Neural network training (research-grade — not needed for basic use)

The --pretrain and --train flags expect MM-Fi or Wi-Pose formatted datasets (see ADR-015), not raw recordings from the server. This is a GPU training pipeline for the full DensePose neural network and is not needed for presence/motion detection.

# This requires a properly formatted dataset — not raw recordings
./sensing-server --pretrain --dataset /path/to/mmfi-dataset --dataset-type mmfi

For your use case (stabilizing detection with 1 ESP32), adaptive training (option A) is the right path.

Summary

Issue	Cause	Fix
Flickering persons	Single-node multipath + sister next room	PR #295 hysteresis + adaptive training
2 persons detected	WiFi penetrates walls	Filter MAC + calibrate with empty room
--record not found	Recording is via REST API, not CLI	Use /api/v1/recording/start
--pretrain format errors	Expects MM-Fi/Wi-Pose datasets	Use adaptive training instead

Let me know how it goes after adaptive training.

[CouldntSee]

This is interesting approach; I'll try to replicate what he did and see what I can do aside from the solution already been provided. Thanks, op, for the solution.

[ruvnet]

Update: Fixes Shipped in v0.5.1+

Several improvements since your report should help:

1. Instability / Ghost Detections

Root cause identified and fixed (PR #326): All nodes were sharing a single classification pipeline, so multipath reflections from a single ESP32 could register as multiple people. The server now uses per-node state tracking — each ESP32 gets its own independent frame history, smoothing buffers, and person count. This significantly reduces ghost detections.

Additionally, PR #325 fixed the edge vitals path so the UI now renders detections from the ESP32's on-device processing (Tier 2+), which is more stable than server-side classification for single-node setups.

2. Single ESP32 Limitations

With a single ESP32-S3, you should expect:

• Presence detection: Reliable (works well with one node)
• Person count: 1 person is accurate; 2-3 people is possible but less reliable with one viewpoint
• Skeleton: Approximate — based on CSI features, not camera. Movements will animate the skeleton, but positions are estimated.
• Flickering: Some frame-to-frame variation is normal. The hysteresis smoothing (PR fix(sensing-server): person count hysteresis to stop flickering #295) reduces this but doesn't eliminate it entirely.

3. Training

The training workflow is:

1. Record data via the UI or REST API:

   curl -X POST http://localhost:3030/api/recording/start?label=present_moving
   # walk around for 60s
   curl -X POST http://localhost:3030/api/recording/stop

   Repeat for each class: absent, present_still, present_moving, active

2. Train the adaptive classifier:

   curl -X POST http://localhost:3030/api/train

3. The model is saved to data/adaptive_model.json and auto-loaded on restart.

How to Upgrade

git pull origin main
cd rust-port/wifi-densepose-rs
cargo build --release -p wifi-densepose-sensing-server

Also reflash your ESP32 with v0.5.1-esp32 for the watchdog fix.