COOP GitHub
ShellHacks 2026

A camera that looks ahead.

COOP is an AI tracking camera on a Raspberry Pi 5 that spots people and cars, predicts where they're heading, and turns on stepper motors to keep them in frame, streamed live to your browser.

View the code Devpost page
Placeholder · demo video (16:9)
Accelerators
0
all inference on the Pi 5 CPU
Field of view
63° × 49°
OV5647, 3.6 mm lens
Aim lead
150 ms
covers pipeline latency
Watchdog
2 s
motors stop if the Pi goes quiet

01 · Pipeline

How it works

Every frame runs the same loop. Vision stays on the Pi; precise step timing lives on a microcontroller.

  1. 1

    Camera

    OV5647 through Picamera2, 640×480 BGR frames.

    Pi 5
  2. 2

    YOLO

    YOLO11n at 320 px finds people and vehicles; ByteTrack keeps a stable ID on each.

    Pi 5 · CPU
  3. 3

    Kalman

    Pixels become world angles; a constant-velocity filter estimates speed and aims 150 ms ahead.

    Pi 5
  4. 4

    Arduino

    Target positions over USB serial (T pan tilt); AccelStepper ramps each move.

    Uno · 115200 baud
  5. 5

    Steppers

    NEMA 17 motors on TMC2209 drivers turn the camera onto the target.

    12 V

Pick a target

Stay on the current track ID while it's visible. Otherwise choose by class priority (people first), then by size. An operator can override from the dashboard by clicking any box.

Think in angles

When the camera turns, the whole scene slides across the frame. Tracking the world angle (gimbal angle + in-frame offset) means the camera's own motion never looks like target motion.

Watch it live

A Flask server streams the annotated MJPEG feed and a JSON status API to a dashboard any phone can open, with Auto / Manual / Stop control and an event log.

02 · The math

From a pixel to a prediction

Two small pieces of math do most of the work.

Pixel → world angle

A pinhole model turns the target's pixel column x in a frame of width w into an angle off the optical axis, then adds the gimbal's own angle. Tilt is the same with y, h and the vertical FOV.

With FOVh = 63°, the frame edge maps to exactly ±31.5°. Using atan (not a linear scale) keeps off-center targets accurate.

Constant-velocity Kalman filter

State is angle and angular velocity on both axes. Each frame predicts forward by the frame time Δt, then corrects with the new measurement.

Predict 𝐱←𝐅𝐱 𝐏←𝐅𝐏𝐅T+𝐐
Gain 𝐊=𝐏𝐇T (𝐇𝐏𝐇T+𝐑)−1
Update 𝐱←𝐱+𝐊(𝐳−𝐇𝐱) 𝐏←(𝐈−𝐊𝐇)𝐏
Aim θaim=θ+ θ˙·tlead

H picks out the two angles we measure; q = 200 (deg/s²)² lets the target change speed, R = 0.5 deg² models detection jitter, and tlead = 0.15 s.

03 · Hardware

One 12 V supply, two brains

The Pi 5 handles vision. An Arduino Uno generates the step pulses, because Linux can't time them reliably.

Placeholder · build photo (3:2)
  • Raspberry Pi 5, 8 GBvision + control, active cooler
  • OV5647 camera5 MP, 3.6 mm, 75° diagonal
  • Arduino UnoAccelStepper, USB serial to the Pi
  • TMC2209 ×2STEP/DIR, 8 microsteps
  • NEMA 17 ×2pan + optional tilt
  • 12 V ≥5 A supplymotors direct, Pi via 5.1 V buck
                ┌─► buck 12V→5.1V ─USB-C─► Pi 5 ─USB─► Uno ─STEP/DIR─► TMC2209 ×2 ─► NEMA 17 ×2
12 V supply ────┤                           │                                ▲
                └───────────────────────────┼──────────── VM (motor power) ──┘
                                            └─ camera ribbon (22-pin)
Placeholder · finished unit
Placeholder · dashboard tracking
Placeholder · mount / enclosure

04 · Team

Two people, built in parallel

Everything runs on a laptop with a webcam and simulated motors, so software and hardware never blocked each other.

David Hernandez

Software

Vision, tracking, prediction, the control API and the dashboard.

Teammate name

Design & hardware

Enclosure, mount, steppers and wiring.

See it for yourself

Open source, MIT licensed. Runs on a laptop with --source webcam --no-motors.

GitHub Devpost