Updated Java Sample TensorFlow Object Detection Op Mode (markdown)

FTC Engineering
2020-10-27 11:32:48 -04:00
parent 3258b39b55
commit 533471756a

@ -116,14 +116,38 @@ In the landscape image above, the approximate coordinate values for the Left, To
### Activating TensorFlow ### Activating TensorFlow
In this example, the Op Mode activates the TensorFlow object detector before waiting for the start command from the Driver Station. This is done so that the user can access the "Camera Stream" preview from the Driver Station menu while it waits for the start command. Also note that in this example, the op mode does not activate the Vuforia tracking feature, it only activates TensorFlow object detection. If you want to incorporate Vuforia image detection and tracking you will also need to activate (and later deactivate when you are done) the Vuforia tracking feature. In this example, the Op Mode activates the TensorFlow object detector before waiting for the start command from the Driver Station. This is done so that the user can access the "Camera Stream" preview from the Driver Station menu while it waits for the start command. Also note that in this example, the op mode does not activate the Vuforia tracking feature, it only activates TensorFlow object detection. If you want to incorporate Vuforia image detection and tracking you will also need to activate (and later deactivate when you are done) the Vuforia tracking feature.
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/blocksTensorFlowActivate.png]]<br/>Activate TensorFlow.<p> ```
/**
* Activate TensorFlow Object Detection before we wait for the start command.
* Do it here so that the Camera Stream window will have the TensorFlow annotations visible.
**/
if (tfod != null) {
tfod.activate();
// The TensorFlow software will scale the input images from the camera to a lower resolution.
// This can result in lower detection accuracy at longer distances (> 55cm or 22").
// If your target is at distance greater than 50 cm (20") you can adjust the magnification value
// to artificially zoom in to the center of image. For best results, the "aspectRatio" argument
// should be set to the value of the images used to create the TensorFlow Object Detection model
// (typically 16/9).
tfod.setZoom(2.5, 16.0/9.0);
}
```
### Setting the Zoom Factor ### Setting the Zoom Factor
When TensorFlow receives an image from the robot's camera, the library downgrades the resolution of the image (presumably to achieve a higher detection rate). As a result, if a target is at a distance of around 24" (61cm) or more, the detection accuracy of the system tends to diminish. This degradation can occur, even if you have a very accurate inference model. When TensorFlow receives an image from the robot's camera, the library downgrades the resolution of the image (presumably to achieve a higher detection rate). As a result, if a target is at a distance of around 24" (61cm) or more, the detection accuracy of the system tends to diminish. This degradation can occur, even if you have a very accurate inference model.
You can specify a zoom factor in your op mode to offset the effect of this automatic scaling by the TensorFlow library. If you specify a zoom factor, the image will be cropped by this factor and this artificially magnified image will be passed to the TensorFlow library. The net result is that the robot is able to detect and track an object at a significantly larger distance. The webcams and built-in Android cameras that are typically used by FTC teans have high enough resolution to allow TensorFlow to "see" an artificially magnified target clearly. You can specify a zoom factor in your op mode to offset the effect of this automatic scaling by the TensorFlow library. If you specify a zoom factor, the image will be cropped by this factor and this artificially magnified image will be passed to the TensorFlow library. The net result is that the robot is able to detect and track an object at a significantly larger distance. The webcams and built-in Android cameras that are typically used by FTC teans have high enough resolution to allow TensorFlow to "see" an artificially magnified target clearly.
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/setZoom.png]]<br/>Set Zoom Factor.<p> ```
// The TensorFlow software will scale the input images from the camera to a lower resolution.
// This can result in lower detection accuracy at longer distances (> 55cm or 22").
// If your target is at distance greater than 50 cm (20") you can adjust the magnification value
// to artificially zoom in to the center of image. For best results, the "aspectRatio" argument
// should be set to the value of the images used to create the TensorFlow Object Detection model
// (typically 16/9).
tfod.setZoom(2.5, 16.0/9.0);
```
If a zoom factor has been set, then the Camera Stream preview on the Driver Station will show the cropped area that makes up the artificially magnified image. If a zoom factor has been set, then the Camera Stream preview on the Driver Station will show the cropped area that makes up the artificially magnified image.
@ -132,7 +156,31 @@ If a zoom factor has been set, then the Camera Stream preview on the Driver Stat
### Iterating and Processing List of Recognized Objects ### Iterating and Processing List of Recognized Objects
The op mode will then iterate until a Stop command is received. At the beginning of each iteration, the op mode will check with the object detector to see how many objects it recognizes in its field of view. In the screenshot below, the variable "recognitions" is set to a list of objects that were recognized using the TensorFlow technology. The op mode will then iterate until a Stop command is received. At the beginning of each iteration, the op mode will check with the object detector to see how many objects it recognizes in its field of view. In the screenshot below, the variable "recognitions" is set to a list of objects that were recognized using the TensorFlow technology.
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/whileLoop.png]]<br/>The op mode gets a list of recognized objects with each iteration of the while loop.<p> ```
if (opModeIsActive()) {
while (opModeIsActive()) {
if (tfod != null) {
// getUpdatedRecognitions() will return null if no new information is available since
// the last time that call was made.
List<Recognition> updatedRecognitions = tfod.getUpdatedRecognitions();
if (updatedRecognitions != null) {
telemetry.addData("# Object Detected", updatedRecognitions.size());
// step through the list of recognitions and display boundary info.
int i = 0;
for (Recognition recognition : updatedRecognitions) {
telemetry.addData(String.format("label (%d)", i), recognition.getLabel());
telemetry.addData(String.format(" left,top (%d)", i), "%.03f , %.03f",
recognition.getLeft(), recognition.getTop());
telemetry.addData(String.format(" right,bottom (%d)", i), "%.03f , %.03f",
recognition.getRight(), recognition.getBottom());
}
telemetry.update();
}
}
}
}
```
If the list is empty (i.e., if no objects were detected) the op mode sends a telemetry message to the Driver Station indicating that no items were detected. If the list is empty (i.e., if no objects were detected) the op mode sends a telemetry message to the Driver Station indicating that no items were detected.