SCDS/FtcRobotControllerSDK
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated Using a TensorFlow Pretrained Model to Detect Everyday Objects (markdown)

Liz Looney
2022-09-23 13:09:30 -07:00
parent c2e3c0a3ff
commit 1e8896b68d
1 changed files with 71 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files

138
Using-a-TensorFlow-Pretrained-Model-to-Detect-Everyday-Objects.md

@ -54,6 +54,16 @@ Now the files are where we want them to be for this example.
### Modifying a Sample Op Mode
Use the OnBot Java editor to create a new op mode that is called "TFODEverydayObjects" and select "ConceptTensorFlowObjectDetection" as the sample op mode that will serve as the template for your new op mode. Note that a copy of the full op mode that was used for this example (except for the Vuforia key) is included at the end of this tutorial.
#### Add import Statements
Your op mode will need the following additional import statements:
```
import com.qualcomm.robotcore.util.RobotLog;
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
```
#### Modify the Name and Enable the Op Mode
Modify the annotations to change the name to avoid a "collision" with any other op modes on your robot controller that are based on the same sample op mode. Also comment at the @Disabled annotation to enable this op mode.
@ -63,14 +73,6 @@ Modify the annotations to change the name to avoid a "collision" with any other
public class TFODEverydayObjects extends LinearOpMode {
```
#### Specify Your Vuforia License Key
Before you can run your op mode, you must first make sure you have a valid Vuforia developer license key to initialize the Vuforia software. You can obtain a key for free from [https://developer.vuforia.com/license-manager](https://developer.vuforia.com/license-manager). Once you obtain your key, replace the VUFORIA_KEY static String with the actual license key so the Vuforia software will be able to initialize properly.
```
private static final String VUFORIA_KEY =
" -- YOUR NEW VUFORIA KEY GOES HERE --- ";
```
#### Specify the Paths to the Model and to the Label Map
Modify the sample op mode so that you specify the paths to your model file ("detect.tflite") and to your label map file ("labelmap.txt").
@ -81,14 +83,12 @@ public class TFODEverydayObjects extends LinearOpMode {
private String[] labels;
```
#### Add import Statements
Your op mode will need the following additional import statements:
#### Specify Your Vuforia License Key
Before you can run your op mode, you must first make sure you have a valid Vuforia developer license key to initialize the Vuforia software. You can obtain a key for free from [https://developer.vuforia.com/license-manager](https://developer.vuforia.com/license-manager). Once you obtain your key, replace the VUFORIA_KEY static String with the actual license key so the Vuforia software will be able to initialize properly.
```
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
private static final String VUFORIA_KEY =
" -- YOUR NEW VUFORIA KEY GOES HERE --- ";
```
#### Create readLabels() and getStringArray() Methods
@ -136,8 +136,8 @@ Create a method called readLabels() that will be used to read the label map file
RobotLog.vv("readLabels()", "No labels read!");
}
}
```
Important note: The readLabels() method actually skips the first line of the "labelmap.txt" file. If you review Google's [example TensorFlow Object Detection Android app](https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android) carefully you will notice that the app actually extracts the label map as metadata from the .tflite file. If you build and run the app, you will see that when the the app extracts the labels from the .tflite file's metadata, the first label is "person". In order to ensure that your labels are in sync with the known objects of the sample .tflite model, the readLabels() method skips the first line of the label map file and starts with the second label ("person"). I suspect that the first line of the label map file might be reserved for future use (or it might be an error in the file).
You will also need to define the getStringArray() method which the readLabels() method uses to convert an ArrayList to a String array.
@ -162,22 +162,7 @@ You will also need to define the getStringArray() method which the readLabels()
}
```
#### Adjust the Zoom Factor
When I tested this example op mode, I disabled the digital zoom factor because for my testing, the phone was relatively close to the target objects. If you would like to test the op mode using small targets at larger distances, you can use a zoom factor greater than one to magnify the target object and increase the detection reliability.
```
// Uncomment the following line if you want to adjust the magnification and/or the aspect ratio of the input images.
//tfod.setZoom(2.5, 1.78);
```
#### Adjust the the Minimum Confidence Level
You can set the minimum result confidence level to a relatively lower value so that TensorFlow will identify a greater number of objects when you test your op mode. I tested my op mode with a value of 0.6.
```
tfodParameters.minResultConfidence = 0.6f;
```
#### Call the readLabels() Method
#### Modify the runOpMode() Method to Call the readLabels() Method
Call the readLabels() method to read the label map and generate the labels list. This list will be needed when TensorFlow attempts to load the custom model file.
```
@ -191,9 +176,30 @@ Call the readLabels() method to read the label map and generate the labels list.
initTfod();
```
#### Load Model from a File
Modify the initTfod() method to load the inference model from a file (rather than as an app asset). Include the array of labels that you generated from the label map when you load the custom model file.
#### Modify the initTfod() Method
##### Adjust the the Minimum Confidence Level
You can set the minimum result confidence level to a relatively lower value so that TensorFlow will identify a greater number of objects when you test your op mode. I tested my op mode with a value of 0.6.
```
tfodParameters.minResultConfidence = 0.6f;
```
##### Set isModelTensorFlow2 to false because this model is a TensorFlow 1 model.
```
tfodParameters.isModelTensorFlow2 = false;
```
##### Load Model from a File
Modify the initTfod() method to load the inference model from a file (rather than as an app asset). Include the array of labels that you generated from the label map when you load the custom model file.
```
if (labels != null) {
tfod.loadModelFromFile(TFOD_MODEL_FILE, labels);
}
```
The initTFod() method should now look like this:
```
/**
* Initialize the TensorFlow Object Detection engine.
@ -202,9 +208,11 @@ Modify the initTfod() method to load the inference model from a file (rather tha
int tfodMonitorViewId = hardwareMap.appContext.getResources().getIdentifier(
"tfodMonitorViewId", "id", hardwareMap.appContext.getPackageName());
TFObjectDetector.Parameters tfodParameters = new TFObjectDetector.Parameters(tfodMonitorViewId);
tfodParameters.minResultConfidence = 0.6;
tfodParameters.minResultConfidence = 0.6f;
tfodParameters.isModelTensorFlow2 = false;
tfodParameters.inputSize = 300;
tfod = ClassFactory.getInstance().createTFObjectDetector(tfodParameters, vuforia);
if(labels != null) {
if (labels != null) {
tfod.loadModelFromFile(TFOD_MODEL_FILE, labels);
}
}
@ -261,9 +269,12 @@ The example op mode (except for the Vuforia license key) is included below for r
package org.firstinspires.ftc.teamcode;
import com.qualcomm.robotcore.eventloop.opmode.Disabled;
import com.qualcomm.robotcore.util.RobotLog;
import com.qualcomm.robotcore.eventloop.opmode.LinearOpMode;
import com.qualcomm.robotcore.eventloop.opmode.TeleOp;
import com.qualcomm.robotcore.util.RobotLog;
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
import org.firstinspires.ftc.robotcore.external.ClassFactory;
import org.firstinspires.ftc.robotcore.external.navigation.VuforiaLocalizer;
@ -271,17 +282,12 @@ import org.firstinspires.ftc.robotcore.external.navigation.VuforiaLocalizer.Came
import org.firstinspires.ftc.robotcore.external.tfod.TFObjectDetector;
import org.firstinspires.ftc.robotcore.external.tfod.Recognition;
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
/**
* This 2020-2021 OpMode illustrates the basics of using the TensorFlow Object Detection API to
* determine the position of the Ultimate Goal game elements.
* This 2022-2023 OpMode illustrates the basics of using the TensorFlow Object Detection API to
* determine which image is being presented to the robot.
*
* Use Android Studio to Copy this Class, and Paste it into your team's code folder with a new name.
* Remove or comment out the @Disabled line to add this opmode to the Driver Station OpMode list.
* Remove or comment out the @Disabled line to add this OpMode to the Driver Station OpMode list.
*
* IMPORTANT: In order to use this OpMode, you need to obtain your own Vuforia license key as
* is explained below.
@ -339,13 +345,11 @@ public class TFODEverydayObjects extends LinearOpMode {
// The TensorFlow software will scale the input images from the camera to a lower resolution.
// This can result in lower detection accuracy at longer distances (> 55cm or 22").
// If your target is at distance greater than 50 cm (20") you can adjust the magnification value
// If your target is at distance greater than 50 cm (20") you can increase the magnification value
// to artificially zoom in to the center of image. For best results, the "aspectRatio" argument
// should be set to the value of the images used to create the TensorFlow Object Detection model
// (typically 1.78 or 16/9).
// Uncomment the following line if you want to adjust the magnification and/or the aspect ratio of the input images.
//tfod.setZoom(2.5, 1.78);
// (typically 16/9).
tfod.setZoom(1.0, 16.0/9.0);
}
/** Wait for the game to begin */
@ -360,26 +364,26 @@ public class TFODEverydayObjects extends LinearOpMode {
// the last time that call was made.
List<Recognition> updatedRecognitions = tfod.getUpdatedRecognitions();
if (updatedRecognitions != null) {
telemetry.addData("# Object Detected", updatedRecognitions.size());
telemetry.addData("# Objects Detected", updatedRecognitions.size());
// step through the list of recognitions and display boundary info.
int i = 0;
for (Recognition recognition : updatedRecognitions) {
telemetry.addData(String.format("label (%d)", i), recognition.getLabel());
telemetry.addData(String.format(" left,top (%d)", i), "%.03f , %.03f",
recognition.getLeft(), recognition.getTop());
telemetry.addData(String.format(" right,bottom (%d)", i), "%.03f , %.03f",
recognition.getRight(), recognition.getBottom());
}
telemetry.update();
// step through the list of recognitions and display image position/size information for each one
// Note: "Image number" refers to the randomized image orientation/number
for (Recognition recognition : updatedRecognitions) {
double col = (recognition.getLeft() + recognition.getRight()) / 2 ;
double row = (recognition.getTop() + recognition.getBottom()) / 2 ;
double width = Math.abs(recognition.getRight() - recognition.getLeft()) ;
double height = Math.abs(recognition.getTop() - recognition.getBottom()) ;
telemetry.addData(""," ");
telemetry.addData("Image", "%s (%.0f %% Conf.)", recognition.getLabel(), recognition.getConfidence() * 100 );
telemetry.addData("- Position (Row/Col)","%.0f / %.0f", row, col);
telemetry.addData("- Size (Width/Height)","%.0f / %.0f", width, height);
}
telemetry.update();
}
}
}
}
if (tfod != null) {
tfod.shutdown();
}
}
/**
@ -396,8 +400,6 @@ public class TFODEverydayObjects extends LinearOpMode {
// Instantiate the Vuforia engine
vuforia = ClassFactory.getInstance().createVuforia(parameters);
// Loading trackables is not necessary for the TensorFlow Object Detection engine.
}
/**
@ -408,8 +410,10 @@ public class TFODEverydayObjects extends LinearOpMode {
"tfodMonitorViewId", "id", hardwareMap.appContext.getPackageName());
TFObjectDetector.Parameters tfodParameters = new TFObjectDetector.Parameters(tfodMonitorViewId);
tfodParameters.minResultConfidence = 0.6f;
tfodParameters.isModelTensorFlow2 = false;
tfodParameters.inputSize = 300;
tfod = ClassFactory.getInstance().createTFObjectDetector(tfodParameters, vuforia);
if(labels != null) {
if (labels != null) {
tfod.loadModelFromFile(TFOD_MODEL_FILE, labels);
}
}