Controlling a FIRST Tech Challenge Robot from Anywhere on the Planet
by Team RevAmped Robotics
After the FTC season was cancelled in March due to the COVID-19 pandemic, we realized that it would be a long time before a normal FTC season could be carried out. In addition, we realized that our team wouldn’t be able to meet in person for the duration of the pandemic, and that other teams would face the same challenge. To compound this issue even further, FIRST and ORTOP’s goal as an organization is to reach out to students and inspire them to pursue STEM activities through robotics. The vast majority of this effort is done through in-person activities, whether it be at competitions, outreaches, or fairs and other events. With the onset of the pandemic, most, if not all of these pathways cannot be conducted in an in-person fashion. Thus, it became clear that a large portion of the FTC program had to be shifted online, and as a team competing in FTC, we asked ourselves how we could best adapt to this. As mentioned above, we generalized this transition into three main factors: team meetings, testing, and outreaches.
With these three aspects solidified, we explored ways to adapt to each. A commonality of all three was the involvement of the robot itself. In team testing and meetings, the ability to access the robot is paramount to an effective session and a better season. In addition, many outreaches are centered around demoing the robot to the public. This is because showing the robot gives a hands on experience that shows the possibilities of STEM and FTC, one of the main goals of FIRST. Because of this, we knew incorporating the robot as the center of our transition was critical. In addition, we knew that we would need to access the robot to use it. Without the option of in person maintenance, we would need to be able to control the root remotely. This was the basic premise of our remote robot project, which is the center of our team’s adaption to the pandemic.
Goals
- Develop a way to access and use the robot remotely
- Make it accessible to as many people as possible
- Exercise skills used in FIRST
The top-level goal became a way to control the robot remotely, ideally from anywhere in the world. Obviously, this presented several different challenges to solve. This led to the project breakdown into 3 separate parts: Data transport from a remote user (browser/front-end) to a back-end server that can read data sent from the user. Server connection to the Robot Controller in order to control the robot. Near live stream viewing to the browser for the remote user. Part 1 became clearer as we decided to use our team website as a GUI for the robot. We planned out a simple interface with a controller API and a live streaming video in order for the user to interact with the robot. In order to read the controller inputs, we ran an HTML file on a Flask server, allowing us to record the gamepad inputs on the browser.
Once this was done we needed a way to use these inputs to control the robot. Taking inspiration from a similar project by Team 11089 Innov8rz, we wrote a program that sent UDP packets to the robot controller from the Flask server through wifi. This was possible because the Raspberry Pi that hosted the server was connected to the RC via wifi-direct. From that point, a program on the RC listened for the packets and controlled the robot based on the data.
The third aspect of the project was a way to reliably live stream to the browser. Most live stream services have upwards of 5 to 10 second delays. This is simply unfeasible, and we needed a near live streaming relay. In addition, we needed to be able to live stream to the browser, and not to select devices. This made streaming programs like Parsec unavailable to use. After many tests, including using a Pixy cam and a server hosted by the Raspberry Pi, we decided to use a Zoom meeting plugin on the website, which allowed near live streaming, albeit with a compromise in quality.
While this project started out as a way to challenge our programmers, it quickly became something much larger. We were able to build a pathway that allowed us to remotely control our robot from anywhere on the planet, with little latency and manual gamepad compatibility. Not only did this become extremely useful for our team through TeleOp practice and other testing, it also became a viable way to conduct online outreaches in a hands on manner. Even more promising, this type of model could show potential as a way to run remote FTC seasons, and with some more work, it could very possibly be scalable.
This leads to our next steps: while we have created a functional way to control our robot remotely, there are several issues to tackle before this could be viably used by teams across FTC. First, a smooth queuing system would need to be implemented on the website, ensuring only one person was controlling the robot at a time. Second, hardware durability testing would need to be done in order to make sure the Pi server could reliably run and serve as the middleman between the browser and the robot. Third, further streaming research could help in increasing stream quality and further reducing delay. Finally, even more robot features such as autonomous testing could be implemented for even more remote testing efficiency.
Check out this video for a short live demo of our project!
