SCDS/FtcRobotControllerSDK
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated Java Sample TensorFlow Object Detection Op Mode (markdown)

FTC Engineering
2020-10-27 11:23:17 -04:00
parent a3ae62490a
commit 3258b39b55
1 changed files with 11 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
Java-Sample-TensorFlow-Object-Detection-Op-Mode.md

@ -33,6 +33,7 @@ Since TensorFlow will receive image data from Vuforia, the op mode attempts to c
// The TFObjectDetector uses the camera frames from the VuforiaLocalizer, so we create that
// first.
initVuforia();
initTfod();
```
You can initialize both the Vuforia and the TensorFlow libraries in the same op mode. This is useful, for example, if you would like to use the TensorFlow library to determine the ring stack and then use the Vuforia library to help the robot autonomously navigate on the game field to navigate to the appropriate target zone from its starting position.
@ -88,17 +89,19 @@ Also note that the by default, the minimum detection confidence level is set to
tfodParameters.minResultConfidence = 0.8f;
```
Let's take a look at the initial blocks in the op mode. The first block in the op mode (excluding the comment blocks) initializes the Vuforia library on the Android Robot Controller. This is needed because the TensorFlow Lite library will receive image data from the Vuforia library. Also, in the screenshot below, the Vuforia system will use an externally connected webcam named "Webcam 1" (which should match the camera name in your robot's configuration file).
After the TensorFlow Object Detector is created it loads the TensorFlow model data
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/blocksInit.png]]<br/>Initialize the Vuforia and TensorFlow libraries.<p>
```
tfod.loadModelFromAsset(TFOD_MODEL_ASSET, LABEL_FIRST_ELEMENT, LABEL_SECOND_ELEMENT);
```
In the screenshot shown above, the sample op mode disables the camera monitoring window on the Robot Controller. If you are using a REV Robotics Control Hub as your Robot Controller (which lacks a touch screen) you normally want to disable the camera monitoring window and use the Camera Stream function instead to view the output of the webcam. If you are using an Android phone as your Robot Controller, however, you can enable the camera monitoring window so you can see the camera output on the Robot Controller's touch screen.
The arguments TFOD_MODEL_ASSET, LABEL_FIRST_ELEMENT, LABEL_SECOND_ELEMENT are defined earlier in the op mode and are season specific.
You can initialize both the Vuforia and the TensorFlow libraries in the same op mode. This is useful, for example, if you would like to use the TensorFlow library to determine the ring stack and then use the Vuforia library to help the robot autonomously navigate on the game field to navigate to the appropriate target zone from its starting position.
Note that in this example the ObjectTracker parameter is set to true for this block, so an _object tracker_ will be used, in addition to the TensorFlow interpreter, to keep track of the locations of detected objects. The object tracker _interpolates_ object recognitions so that results are smoother than they would be if the system were to solely rely on the TensorFlow interpreter.
Also note that the Minimum Confidence level is set to 70%. This means that the TensorFlow library needs to have a confidence level of 70% or higher in order to consider an object as being detected in its field of view. You can adjust this parameter to a higher value if you would like the system to be more selective in identifying an object.
```
private static final String TFOD_MODEL_ASSET = "UltimateGoal.tflite";
private static final String LABEL_FIRST_ELEMENT = "Quad";
private static final String LABEL_SECOND_ELEMENT = "Single";
```
If a camera monitor window is enabled for the TensorFlow library, then the confidence level for a detected target will be displayed near the bounding box of the identified object (when the object tracker is enabled). For example, a value of "0.92" indicates a 92% confidence that the object has been identified correctly.