Real-time operations UI for autonomous racing (IAC ground station)
Telemetry and control UI for Caltech's Indy Autonomous Challenge program: compress sensor fusion and vehicle state into something an operator can trust while the car is moving faster than most people drive on a highway.
partners: Caltech · BeyondAI · series: Indy Autonomous Challenge (IAC) · window: 2023–2024 · domain: mission-critical ground station · speeds: 200+ mph · transport: UDP from vehicle · targets: Linux, macOS, Windows
My role
I led product design for the ground-station experience end to end: framing what operators needed to see and do in the loop, turning research-side sensor fusion into actionable screens, and keeping the interface modular enough for Caltech students and race engineers to evolve it. The work sat in an R&D partnership between Caltech and BeyondAI across 2023–2024, with the same north star as the program writ large: support competition in autonomous racing at speeds above 200 mph without burying the humans who still own go/no-go judgment.
This was explicitly a mission-critical, real-time problem: not a post-hoc analytics dashboard, but a live console where delayed or ambiguous telemetry has a physical cost.
Program context
The Indy Autonomous Challenge (IAC) pushes full-size open-wheel cars on oval and road courses with autonomy stacks that must fuse perception, planning, and control under broadcast-grade scrutiny. The ground station is where that stack meets humans: race engineers and operators who need to interpret state, intervene when policy allows, and recover cleanly when something drifts.
On the research side, the team was iterating on sensor fusion and autonomy behaviors. On the product side, the job was to translate those capabilities into actionable telemetry visualization: charts and indicators grounded in what the vehicle and pipeline actually emit, not decorative dashboards. The partnership outcome we cared about was practical support for Caltech's competition effort in that environment, not a one-off demo that could not survive contact with the next student cohort.
Real-time operations context
At 200+ mph, "slow UI" is not a polish issue; it is a decision-quality issue. Operators are not browsing; they are maintaining situational awareness, catching faults, and choosing interventions on a compressed timeline. The interface has to bias toward critical metrics first, keep updates smooth enough that the eye can trust motion and deltas, and avoid dumping every intermediate signal into view just because the backend can produce it.
The recurring failure mode in telemetry-heavy domains is data overload: everything is "important," so nothing is. The design challenge was to stay highly intuitive and modular while still exposing the depth race engineers need when something goes wrong, including smooth real-time updates across the stack.
Requirements and platform
The charter was concrete: a ground-station interface for real-time monitoring and control of an autonomous race car—map views, essential vehicle data, and interaction paths that let operators influence behavior when the mission allows. Incoming data arrived over UDP from the vehicle; the UI had to ingest that stream and surface the metrics that matter in race conditions.
Non-negotiables from the program side included:
- Real-time data visualization end to end, not batch-refresh storytelling.
- Cross-platform compatibility across Linux, macOS, and Windows so pits and lab benches could run the same tooling.
- A system modifiable by undergraduate and graduate students without a rewrite every season: modular layouts, sensible defaults, room to extend.
- Handling UDP vehicle data and rendering the essential metrics the operators defined as safety-critical or race-critical.
Project goals spelled out the operator story: a clear, easy-to-understand interface for quick decisions under pressure; modular, adaptable structure for future updates; seamless operation across OS targets; smooth real-time visualization; and interactive controls such as engine start, speed input, and trajectory selection where the stack exposed those affordances.
Ground station surfaces
The shipped layout clustered work into a few persistent surfaces so operators could build muscle memory: situational awareness on maps, numeric and mode state beside controls, time-series evidence for debugging motion and localization, and a dedicated safety readout that did not compete with exploratory charts.
Local map
A local map showed the team's race car and nearby vehicles within roughly a 100-meter radius: enough context for close-proximity racing without pulling attention to whole-track abstraction when the immediate tactical picture mattered more.
Full-track map
A full-track map displayed the entire circuit and all car positions, giving race engineers the global picture—overtakes, gaps, and pack behavior—that local framing intentionally trims away.
Vehicle planner and telemetry
The vehicle planner and telemetry band concentrated high-salience state: speed, RPM, battery, race mode, system status, and planning status, alongside engine controls and speed input for the interactive paths the stack supported.
Data plots
Real-time plots backed diagnosis: control heading errors, lateral errors, steering, GPS, RTK diff age, throttle, and brake. The intent was operator-grade signal, not chart junk: each series had to earn its pixels against cognitive load.
Safety monitor
The safety monitor surfaced system statuses and alerts such as soft stop and emergency stop, with color-coded indicators so stop conditions read faster than prose ever would.
Research, prototyping, iteration
Initial research surveyed telemetry interfaces in autonomous vehicles and motorsports, with emphasis on where teams drown operators in channels and where they starve them of the one series that explains a failure.
Findings that drove IA and visual hierarchy:
- Operators need immediate access to critical metrics: speed, position, and system health—not three clicks deep.
- Smooth real-time updates matter as much as layout; stutter reads as untrustworthy data.
- Customization and flexibility in how data displays are not luxuries when student teams rotate and experiments change week to week.
Design objectives stayed tight: prioritize critical data (speed, RPM, race mode) for visibility; keep the system modular and adaptive for rapid iteration; use simple, intuitive layouts so non-specialists could still run sessions safely.
Execution moved through wireframes and high-fidelity prototypes in Figma and PyQt for early testing, with input from students, race engineers, and UI/UX collaborators. Iteration focused on clarity and real-time interaction: fewer novel widgets, more predictable patterns, faster read paths.
Operator testing and feedback
Sessions validated the direction rather than reinventing it: operators responded well to prioritization of critical data and the smooth real-time feel of the readouts. Parameter adjustments landed as intuitive for the controls in scope. The modular structure got explicit praise as something students and engineers could extend without breaking the core story of the console.
What shipped
The deliverable was an operator-centric, mission-critical UI that translated R&D sensor fusion work into actionable telemetry visualization, aligned with Caltech's IAC competition needs above 200 mph. It balanced dense engineering truth with a surface calm enough to use under stress: key metrics surfaced first, maps split local vs global context, plots carried the diagnostic load, and safety state stayed visually loud.
- Ground-station experience covering maps, planner/telemetry, live plots, and safety monitoring in one coherent layout.
- UDP-driven real-time visualization and control hooks the program defined as in-scope (engine start, speed input, trajectory selection).
- Cross-platform targeting and a modular Figma system intended to survive student handoffs and continued iteration.
- A deliberate bridge between advanced research prototypes and operational requirements in a safety-critical, real-time setting—not a static mockup divorced from the car.
The design stays open to refinement as hardware, rules, and autonomy stacks shift; the constraint we optimized for was a foundation that could absorb those changes without throwing away operator habits.
Closing
Racing autonomy makes the abstract concrete: if the ground station lies, people stop trusting it mid-session. If it tells the truth but buries it under noise, they miss the one signal that mattered. The PM/design job here was to keep fidelity to the stack while respecting human bandwidth—so Caltech could compete with tooling that felt as serious as the speeds on the telemetry strip.