TRI-SYNC: Building an Open-Source Digital Twin for Autonomous Connected Vehicles

Digital twins are often presented as complex industrial systems requiring expensive sensors, proprietary platforms, and large infrastructure. TRISYNC explores a different approach: building a small, accessible, open-source digital twin that can observe, simulate, control, and eventually coordinate physical autonomous vehicles.

In this project, we build a real-time digital representation of multiple small robotic cars. An OpenCV-based camera acts as the perception layer, detecting and tracking the physical vehicles and translating their movements into the digital environment. The digital twin maintains a synchronized virtual representation of the vehicles, their positions, movements, and interactions.

The interesting part begins when the digital and physical worlds are connected.

We can control the virtual vehicles and use the same system to influence the physical cars. Multiple vehicles can communicate with each other, allowing us to explore autonomous behavior, coordination, collision avoidance, and distributed decision-making.

The overall architecture looks like:

Physical Vehicles

↓

OpenCV Camera / Perception

↓

Real-Time State Estimation

↓

Digital Twin

↓

Autonomous Decision Layer

↓

Vehicle-to-Vehicle Communication

↓

Physical Vehicle Control

↓

Updated Physical State

↓

Digital Twin

TRISYNC is intentionally built around open technologies and open models wherever possible. The goal is not to build another closed autonomous-driving platform, but to demonstrate how students and FOSS communities can build their own cyber-physical systems using accessible hardware and open-source software.

During the talk, we will explain the architecture, demonstrate real-time tracking and synchronization, show how the virtual vehicles can be controlled, and discuss how the system can evolve from remotely controlled vehicles into autonomous connected agents.

We will also discuss the engineering challenges we encountered, including perception errors, synchronization latency, coordinate mapping, communication between vehicles, and maintaining consistency between the physical and virtual worlds.

The project will be presented as an open-source experiment that attendees can reproduce, extend, and use as a starting point for their own robotics, digital-twin, IoT, and autonomous-systems projects.

Understand the fundamental architecture of a real-time digital twin.

1. Learn how OpenCV can be used as a low-cost perception layer for physical systems.
2. See how a physical robotic vehicle and its digital representation can remain synchronized.
3. Understand how digital twins can be connected to autonomous decision-making.
4. Learn how multiple vehicles can communicate and coordinate with each other.
5. Explore how open-source software and accessible hardware can be used to build cyber-physical systems.
6. Understand practical challenges such as latency, tracking errors, coordinate transformation, and synchronization.
7. Leave with an architecture that can be reproduced and extended for robotics, IoT, autonomous systems, and smart manufacturing.