Most smart home setups rely on static triggers: motion detectors flipping lights, wall thermostats reacting to dry air temperatures, and rigid schedules executing whether you're wide awake or dead tired.
Using AntiGravity as my autonomous coding collaborator, I spent the last few weeks moving past simple if-this-then-that automation. Instead, I built a local-first, low-latency, biological-feedback comfort suite—anchored by Home Assistant, custom Python daemons, and an offline Electron dashboard dubbed Project Emily.
Here is how the entire system fits together.
The Architecture: Local, Sovereign, and Unshackled
The environment runs locally without external cloud dependencies:
- Central Nervous System: Home Assistant managing hardware state and services via a local REST/WebSocket gateway (
gateway.py).
- Hardware Fleet: A Hitachi split-system air conditioner (via Sensibo), TP-Link Tapo L530 lighting, a hubless Philips Hue Bluetooth bulb in the kitchen, and a Powerpal optical meter tracking the main switchboard.
- Network & Radio Extension: An ESP32 running an ESPHome Bluetooth Proxy with active connection watchdogs, bridging BLE commands to the kitchen without packet drops.
- Sensory Daemons: Custom Python engines running locally on Ubuntu, backed by a lightweight SQLite database (
lan_state.db).
+----------------------------------------+
| Project Emily Companion |
| (Electron Desktop Canvas) |
+-------------------+--------------------+
|
v
+------------------------+ +--------------------+ +-----------------------+
| MediaPipe Vision | | Homelab Gateway | | Environmental Sensors |
| FaceMesh / Posture / |--->| (gateway.py) |<---| - External Temp/RH |
| PERCLOS Eye Tracking | | | | - Powerpal Grid Watts |
+------------------------+ +---------+----------+ +-----------------------+
|
v
+-----------------------+
| Home Assistant |
| (Local REST / States) |
+-----------+-----------+
|
+------------------------+------------------------+
| |
v v
+--------------------------+ +--------------------------+
| Sensibo / Hitachi | | ESPHome ESP32 |
| Apparent Temp HVAC | | Bluetooth Proxy |
| (±0.5°C Subtle Steps) | | (Kitchen Hue Bulb) |
+--------------------------+ +--------------------------+
1. True Bioclimatic Comfort (Beyond Simple Thermostats)
Standard thermostats read dry-bulb temperature and call it a day. But humans feel heat based on enthalpy—the combined effect of temperature, relative humidity, and wind chill.
AntiGravity and I implemented the Australian Bureau of Meteorology (BOM) apparent temperature model directly into lan_mcp.py:
AT = T + 0.33 \times e - 0.70 \times ws - 4.00
Using an external sensor feeding live dry-bulb temperature and humidity, the system calculates the indoor comfort deficit. If dry air makes 21.0°C feel like a chilly 18.5°C, the engine automatically adjusts the Hitachi split system by gentle, imperceptible \pm0.5^\circ\text{C} increments—preserving a genuine physiological baseline rather than a theoretical number.
2. Somatic State Tracking: Real-Time FaceMesh & PERCLOS
Rather than relying on manual adjustments, the environment listens directly to physical fatigue and tension cues:
- Corrugator & Masseter Tension: Using MediaPipe FaceMesh, the daemon monitors inner-eyebrow furrowing and jaw clenching. It starts with a 10-second neutral calibration at boot, creating a personalized baseline that adjusts for camera angle and head pose.
- Drowsiness via PERCLOS: By tracking the Eye Aspect Ratio (EAR) across rolling 60-second windows, the system measures the percentage of eye closure. If PERCLOS crosses 15%, indicating severe eye fatigue or impending microsleep, the system responds with a gentle, non-jarring 2-second restorative amber pulse (
2200K) across the Tapo bulbs before returning to focus mode.
3. Elevated Flow: Deep Blue & Cool White
Environmental transitions are designed to support mental states without being disruptive:
When the vision daemon detects sustained positive valence—relaxed brows, relaxed jaw, and naturally upturned lip corners measured against the inter-pupillary baseline—the system initiates an Elevated Flow state.
Over a gradual 10-minute (600-second) circadian fade, the room shifts into my preferred aesthetic:
- Primary Ambient (
light.lounge): Fades to a sharp, clear 5500K Cool White at 70% brightness.
- Accent Perimeter (
light.window): Melts into a rich, deep electric blue (rgb: [0, 30, 255]) at 45% brightness.
- Kitchen (
light.kitchen): Bridges via the ESP32 proxy to a clean 5000K task hue.
The automation enforces a strict 1-hour (3600-second) mode hold and requires 30 minutes of sustained relaxation before releasing. This eliminates rapid, jarring lighting shifts while working.
4. Zero-Friction Telemetry with Powerpal
Earlier iterations used power thresholds to lock out HVAC setpoints whenever total household draw exceeded 2,200W. Over time, that interventionist logic introduced unnecessary friction.
Working through AntiGravity, we stripped out the artificial restrictions, lockout states, and desktop lock scripts. Instead:
- Powerpal operates strictly as a metrics-only observer.
- Real-time grid draw, HVAC compressor states, and apparent temperatures log directly to
lan_state.db.
- The custom Project Emily Electron dashboard renders continuous time-series curves on an HTML5 canvas—displaying live wattage and apparent temperature alongside system telemetry.
What I Learned Building with AntiGravity
- Local-first autonomy demands stability: Fast API loops fail when devices drop offline. Running ESPHome with active gateway ping watchdogs and fallback reboots made Bluetooth smart bulbs reliable without needing proprietary hubs.
- Slow down the feedback loop: Quick, aggressive automations draw attention to themselves. Extending transitions to 10-minute fades and 1-hour holds allowed the space to adapt to body states naturally in the background.
- Co-engineering with an autonomous partner: Developing with AntiGravity made complex implementations straightforward—from matrix math for head-pose correction down to raw SQLite time-series schemas.
The smart home shouldn't be another control dashboard to manage. Built properly, it acts as a subtle sensory sanctuary that adapts to you in silence.
Comments