EMBEDDED SYSTEMS · CONTROLS

Self-Balancing Robot with Computer Vision

Two-wheeled inverted-pendulum robot that balances, follows a color target, and stops for obstacles.

Context: EECS 373: Introduction to Embedded Systems · University of Michigan
Role: Team of 4 · PID control, sensor integration, bare-metal engineering, firmware
When: Winter 2026

C
STM32L4R5ZI-P
ARM Cortex-M4
I2C
SPI
PWM
PID

Constraint

An inverted pendulum has no steady-state balance: the robot must correct itself continuously while also steering, tracking, and avoiding collisions.

Decision

Wrote a closed-loop PID controller from scratch on an ARM Cortex-M4 MCU at ~100 Hz and layered four prioritized control states on top, keeping the robot balanced through every mode change using hardware interrupts and standard serial protocols.

Evidence

Robot balances on its own, follows Pixy2 color targets, accepts PS2 manual override, and hard-stops from time-of-flight sensors; diagnostics stream over LPUART.

Front view of the robot with labeled components — Front: Pixy2 camera, VL53L0X ToF, Nucleo L4R5ZI-P, BNO055 IMU, HG-37 motors

Side view of the robot showing the stacked chassis and kickstand supports — Side: stacked laser-cut chassis, L298N H-bridge, PS2 receiver, kickstand supports

01 // Problem & constraints

Keep a two-wheeled robot upright on an STM32 Nucleo-L4R5ZI-P while adding vision tracking, manual control, and collision safety, without any of these breaking the balance loop.

02 // Architecture

▪BNO055 IMU and VL53L0X time-of-flight sensor on I2C; Pixy2 camera on SPI.
▪PS2 wireless controller read over a custom SPI-like protocol.
▪PWM-driven HG-37 12 V brushed DC motors through an L298N H-bridge, powered by a 12 V LiPo.
▪LPUART telemetry for live diagnostics and remote PID tuning.

03 // Hierarchical control states

▪1 · Emergency stop: VL53L0X / HC-SR04 force a stop to prevent collisions.
▪2 · Manual override: the pilot steers with the PS2 controller while PID keeps balance.
▪3 · Autonomous track: Pixy2 target position becomes a setpoint offset.
▪4 · Stationary: hold vertical equilibrium by default.
▪The PID balance loop runs continuously regardless of the active state.

04 // Mechanical design

Laser-cut acrylic and wood platforms with 3D-printed structural parts, laid out for symmetry and careful placement of heavy components (battery low, electronics stacked above) to suit the inverted-pendulum dynamics.

05 // Next steps

▪Add a dedicated battery-management IC with cell-level voltage monitoring and low-battery cutoff.