SCDS/FtcRobotControllerSDK
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated Blocks Sample TensorFlow Object Detection Op Mode (markdown)

FTC Engineering
2020-10-23 21:18:59 -04:00
parent 004e1620b7
commit 42ab6450ea
1 changed files with 15 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
Blocks-Sample-TensorFlow-Object-Detection-Op-Mode.md

@ -35,11 +35,25 @@ The origin of the coordinate system is in the upper left-hand corner of the imag
In the landscape image above, the approximate coordinate values for the Left, Top, Right, and Bottom boundaries are 455, 191, 808, and 547 respectively (pixel coordinates). The width and height for the landscape image above is 1280 and 720 respectively. In the landscape image above, the approximate coordinate values for the Left, Top, Right, and Bottom boundaries are 455, 191, 808, and 547 respectively (pixel coordinates). The width and height for the landscape image above is 1280 and 720 respectively.
### Iterating and Processing List of Recognized Objects ### Activating TensorFlow
In this example, the Op Mode activates the TensorFlow object detector before waiting for the start command from the Driver Station. This is done so that the user can access the "Camera Stream" preview from the Driver Station menu while it waits for the start command. Also note that in this example, the Op Mode does not activate the Vuforia tracking feature, it only activates TensorFlow object detection. If you want to incorporate Vuforia image detection and tracking you will also need to activate (and later deactivate when you are done) the Vuforia tracking feature. In this example, the Op Mode activates the TensorFlow object detector before waiting for the start command from the Driver Station. This is done so that the user can access the "Camera Stream" preview from the Driver Station menu while it waits for the start command. Also note that in this example, the Op Mode does not activate the Vuforia tracking feature, it only activates TensorFlow object detection. If you want to incorporate Vuforia image detection and tracking you will also need to activate (and later deactivate when you are done) the Vuforia tracking feature.
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/blocksTensorFlowActivate.png]]<br/>Activate TensorFlow.<p> <p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/blocksTensorFlowActivate.png]]<br/>Activate TensorFlow.<p>
### Setting the Zoom Factor
When TensorFlow receives an image from the robot's camera, the library downgrades the resolution of the image (presumably to achieve a higher detection rate). As a result, if a target is at a distance of around 24" (61cm) or more, the detection accuracy of the system tends to diminish, in spite of the accuracy of your inference model. You can specify a zoom factor in your op mode to offset the effect of this automatic scaling by the TensorFlow library.
If you specify a zoom factor, the image will be cropped by this factor and the artificially magnified image will be passed to the TensorFlow library. The net result is that the robot should be able to detect and track an object at a significantly larger distance.
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/setZoom.png]]<br/>Set Zoom Factor.<p>
If a zoom factor has been set, then the Camera Stream preview on the Driver Station will show the cropped area that makes up the artificially magnified image.
<p align="center">[[/images/Blocks-Sample-TensorFlow-Object-Detection-Op-Mode/magnifiedArea.png]]<br/>Camera Stream Preview Indicating Magnified Area.<p>
### Iterating and Processing List of Recognized Objects
The Op Mode will then check with the object detector to see how many objects it recognizes in its field of view. In the screenshot below, the variable "recognitions" is set to a list of objects that were recognized using the TensorFlow technology. The Op Mode adds a telemetry message to indicate on the Driver Station how many objects were detected in the camera's field of view. The Op Mode will then check with the object detector to see how many objects it recognizes in its field of view. In the screenshot below, the variable "recognitions" is set to a list of objects that were recognized using the TensorFlow technology. The Op Mode adds a telemetry message to indicate on the Driver Station how many objects were detected in the camera's field of view.
<p align="center"><img src="https://github.com/FIRST-Tech-Challenge/WikiSupport/blob/master/tensorflow/images/blocksGetRecognitions.png" width="500"><br/>Get a list of recognized objects from TensorFlow.<p> <p align="center"><img src="https://github.com/FIRST-Tech-Challenge/WikiSupport/blob/master/tensorflow/images/blocksGetRecognitions.png" width="500"><br/>Get a list of recognized objects from TensorFlow.<p>