01 Background

The FIRST Robotics Competition (FRC) is an international robotics competition for high schoolers. Every year, thousands of students around the world spend 6 weeks building a 120-lb robot to play a 3 vs. 3 game that changes every year. The students are encouraged to use 3D printing, metalworking techniques, CAD, and Java programming to design and build their robots.

Although I'd been on a robotics team in high school, we competed in the smaller FIRST Tech Challenge (FTC) competition. When I moved to Pittsburgh recently, I wanted to pass on the robotics knowledge I had gained from WPI and my career in robotics and sought out a local FRC team. I recently became a mentor for Team 3260, named the Sarah Heinz Advanced Robotics Program (SHARP) based out of the Sarah Heinz House in the North Side neighborhood of Pittsburgh.

I took a keen interest in understanding the programming team's major challenges. I learned quickly that their biggest hurdle every year was that they had a hard time writing any code until the hardware and electronics teams were finished building the robot. This had led to flaky software in the past and no time to tackle more complex logic, such as safety modes to shut off nonessential electronics when the robot started browning out due to high current draw. As a software engineer, I took issue with this, and resolved to solve the problem to help the students have an easier time with the software component of the robot.

Having worked in the industry for a few years, I happened to know that there was a fairly common solution to this problem—working with a simulator! A simulated model of the robot, although it wouldn't be a perfect reflection of reality, would allow the software team to develop and test quickly with simulated encoders, limit switches, and ultrasonic sensors without having to wait until the physical robot was done. This technique is common, for example, at Sikorsky where flying the actual aircraft often was prohibitively expensive, and at Uber where there are so many more software developers than actual cars. I set to work, and built something resembling a videogame that helped springboard the programming team this year. Check out the result on GitHub.


02 Design

To start, I wrote out some basic requirements and quickly settled on an architecture. I decided on three separate modules all communicating over fast UDP using JSON messages. The first would mimic an Xbox controller (so the students wouldn't always be forced to share a real one), the second would run their software (which I called the "core"), and the third would be the actual simulation of the robot (which I called the "vehicle" because "robot" is a bit ambiguous).

The simulator architecture.

The Python joystick, Java core, and C++ sim all talk over UDP.

The simulator architecture.

Rather than try to dig into the FRC-provided libraries and tie myself to Java, I decided to write the simulator in time- and space-efficient C++. I used OpenSceneGraph for the 3D visualization, which is a lightweight wrapper around OpenGL, and Box2D, an open-source C++ physics engine most famous for its use in Angry Birds.

To model the inputs to the robot, I wrote a virtual Xbox controller so the students wouldn't have to share the physical controllers and could code even when working remotely without any controllers around. I decided to write it in Python to get it working quickly. The visualization is done with PyQt, and I used Inkscape to quickly trace out the shape of the controller and joysticks as Git-friendly SVG images. The virtual controller can be controlled by mouse, but is much easier to use with the WASD and arrow keys. A physical Xbox controller can also be plugged into the computer and used instead by passing physical instead of virtual as a command line argument to the application.

Finally, the programmers' code runs in a third module in the middle, which is a simple Java framework which aims to provide sensor and joystick inputs in the same way as WPI Lib, the Java framework published by FRC. By separating these three components, it becomes very easy to kill the "core," make a small adjustment, and relaunch it to test with the joystick and sim still running.

The full simulator pipeline in action.

A simulated robot drives around the field using a virtual Xbox controller

The full simulator pipeline in action.


03 Results

The simulator is still a work in progress, although so far it's been successful in getting the students curious about programming and learning to develop faster. We also recently used the simulator for driver tryouts, and it promises to be helpful for driving practice if nothing else. Feature development has been a bit slow, so modules tend to become available in the simulator at about the same time as they become available on the real robot so far. I'm hopeful that in future years, the simulator will progress faster because it will already have the physics library and other infrastructure ready to go.

The sim being used for driver tryouts.

A student drives the simulated robot on a large screen as other students cheer her on

The sim being used for driver tryouts.


04 Future Work

For now, the robot simulation itself is still a bit overly simplistic. For example, a push forward on the controller results in an instantaneous velocity in the simulation. I'm working on modeling the actual forces involved, such as the inertia of the robot, slippage at the wheels, and reduced lateral friction at the omni wheels, but I've put that aside for now to focus on other, more crucial game-playing features first.

I'm also working on adding multiplayer functionality. This would allow other teammates to play defense and simulate a full 3 vs. 3 match inside the simulator. One way to do this might be to add a fourth application which simply receives the world state from the sim and simply renders it. This would allow each player to run their own joystick, comms, and view while all communicating to a single simulator over the network. To improve the visuals, I think it'd be a lot of fun to learn to use something more powerful like Unreal Engine, but that's probably best left for the off-season.

All in all, building this simulator has been a great experience. It's helped Team 3260 so far, and it's been a valuable learning experience for me. I hope that by open-sourcing it, other teams can extend it quickly and discover more uses than I can alone. Most of all, I hope it inspires more FIRST students to get excited about programming and helps evolve the game for years to come.