First draft of TFOD entry page, updated for Freight Frenzy. Not linked yet, still private.
64
Using-TensorFlow-in-Freight-Frenzy.md
Normal file
64
Using-TensorFlow-in-Freight-Frenzy.md
Normal file
@ -0,0 +1,64 @@
|
|||||||
|
### What is TensorFlow?
|
||||||
|
FTC teams can use [TensorFlow Lite](https://www.tensorflow.org/lite/), a lightweight version of Google's [TensorFlow](https://www.tensorflow.org/) machine learning technology that is designed to run on mobile devices such as an Android smartphone. A _trained TensorFlow model_ was developed to recognize game elements for the 2021-2022 Freight Frenzy challenge.
|
||||||
|
|
||||||
|
_<p align="center">[[https://raw.githubusercontent.com/wiki/WestsideRobotics/FTC-training/Images/010-TFOD-Cube-Duck-crop-2.png]]<br/>
|
||||||
|
This season's TFOD model can recognize Freight elements._
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
TensorFlow Object Detection (TFOD) has been integrated into the FTC control system software, to identify and track these game pieces during a match. The FTC software (SDK version 7.0) contains TFOD Sample Op Modes that can recognize the Freight elements Duck, Box (or Cube), and Cargo (or Ball).
|
||||||
|
|
||||||
|
Also, FTC teams can soon use a new tool to train their own TFOD models, to recognize their custom Team Shipping Element (TSE) and/or to improve recognition of Freight elements. This training could take into account certain conditions of distance, angle, lighting and background.
|
||||||
|
|
||||||
|
This new tool, FTC Machine Learning Toolchain, was [announced 10/7/2021](http://firsttechchallenge.blogspot.com/2021/10/new-machine-learning-tool-beta-testing.html) for upcoming beta testing by interested FTC teams.
|
||||||
|
|
||||||
|
### How Might a Team Use TensorFlow in Freight Frenzy?
|
||||||
|
|
||||||
|
For this season's challenge, during the pre-Match stage a single die is rolled and the field is randomized.
|
||||||
|
|
||||||
|
|
||||||
|
_<p align="center">[[https://raw.githubusercontent.com/wiki/WestsideRobotics/FTC-training/Images/020-TFOD-Barcode.png]]<br/>
|
||||||
|
Randomization_
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
At the beginning of the match's Autonomous period, a robot can use TensorFlow to "look" at the **Barcode** area and determine whether the Duck or optional Team Shipping Element (TSE) is in position 1, 2 or 3. This indicates the preferred scoring level on the **Alliance Shipping Hub**. A bonus is available for using the TSE instead of a Duck.
|
||||||
|
|
||||||
|
_<p align="center">[[https://raw.githubusercontent.com/wiki/WestsideRobotics/FTC-training/Images/030-TFOD-levels.png]]<br/>
|
||||||
|
Alliance Shipping Hub_
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
### Important Note on Phone Compatibility
|
||||||
|
|
||||||
|
TensorFlow Lite runs on Android 6.0 (Marshmallow) or higher, a requirement met by all currently allowed FTC devices. If you are a Blocks programmer using an older/disallowed Android device that is not running Marshmallow or higher, TFOD Blocks will automatically be missing from the Blocks toolbox or design palette.
|
||||||
|
|
||||||
|
### Sample Op Modes
|
||||||
|
|
||||||
|
The FTC software (SDK version 7.0 and higher) contains sample Blocks and Java op modes that demonstrate TensorFlow **recognition** of Freight elements Duck, Box (cube) and Cargo (ball). The sample op modes also show **where** in the camera's field of view a detected object is located.
|
||||||
|
|
||||||
|
Click on the following links to learn more about these sample Op Modes.
|
||||||
|
|
||||||
|
* [Blocks TensorFlow Object Detection Example](https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/Blocks-Sample-Op-Mode-for-TensorFlow-Object-Detection)
|
||||||
|
* [Java TensorFlow Object Detection Example](https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/Java-Sample-Op-Mode-for-TensorFlow-Object-Detection)
|
||||||
|
|
||||||
|
### Using a Custom Inference Model
|
||||||
|
|
||||||
|
Teams have the option of using a custom inference model with the FIRST Tech Challenge software. As in the past, some teams may want to use the [TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) to create an enhanced model of the Freight elements or TSE, or to create a custom model to detect other entirely different objects. Other teams might also want to use an available pre-trained model to build a robot that can detect common everyday objects (for demo or outreach purposes, for example).
|
||||||
|
|
||||||
|
The FTC software includes sample op modes (Blocks and Java versions) that demonstrate how to use a **custom inference model**:
|
||||||
|
|
||||||
|
* [Using a Custom TensorFlow Model with Blocks](https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/Using-a-Custom-TensorFlow-Model-with-Blocks)
|
||||||
|
* [Using a Custom TensorFlow Model with Java](https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/Using-a-Custom-TensorFlow-Model-with-Java)
|
||||||
|
|
||||||
|
These tutorials use examples from a previous FTC season (Skystone), but the process remains generally valid for Freight Frenzy.
|
||||||
|
|
||||||
|
As noted above, soon FTC teams will have a streamlined tool for training their own TFOD models. Watch for announcements regarding the FTC Machine Learning Toolchain, already scheduled for beta testing.
|
||||||
|
|
||||||
|
### Detecting Everyday Objects
|
||||||
|
|
||||||
|
You can use a pretrained TensorFlow Lite model to detect **everyday objects**, such as a clock, person, computer mouse, or cell phone. The following advanced tutorial shows how you can use a free, pretrained model to recognize numerous everyday objects.
|
||||||
|
|
||||||
|
* [Using a TensorFlow Pretrained Model to Detect Everyday Objects](https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/Using-a-TensorFlow-Pretrained-Model-to-Detect-Everyday-Objects)
|
||||||
|
|
||||||
|
_<p align="center">[[https://raw.githubusercontent.com/wiki/FIRST-Tech-Challenge/FtcRobotController/images/Using-a-TensorFlow-Pretrained-Model-to-Detect-Everyday-Objects/tfliteDemo.png]]<br/>TensorFlow can recognize everyday objects._
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
_<p align="right">updated 10/21/21</p>_
|
Reference in New Issue
Block a user