9-26-23 restructure all. First run the OpMode "as is" to detect the default/built-in model. Then move on to upload and specify a TFOD model file (here the white Pixel again).

2023-09-26 21:23:34 -07:00
parent 4b6810020c
commit eb9549b6f9
1 changed files with 172 additions and 126 deletions
--- a/TFOD-Sample-OpMode-with-Java-Builder.md
+++ b/TFOD-Sample-OpMode-with-Java-Builder.md
@ -2,6 +2,8 @@
 This text does not appear at the published wiki.
 v02 9-26-23 restructure based on first running the OpMode "as is" to detect the default white Pixel.  Then move on to uploading and specifying a TFOD model file.  In early testing, the OpMode would not run "as is", but this was due to a missing assets folder with its default .tflite model file.
 v01 9-25-23 posted first draft of tutorial at FTC wiki.
 Per request 9-22-23 from Danny Diaz, this new article is based on:
@ -30,7 +32,7 @@ For the 2023-2024 game CENTERSTAGE, the official game element is a hexagonal whi
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/010-TFOD-recognition.png]]</p>
-For extra points, FTC teams may instead use their own custom TFOD models of game elements, called **Team Props** in CENTERSTAGE.  That option is covered in this tutorial, along with showing how to use the default model.  Custom TFOD models are created by teams using the same [**FTC Machine Learning process**](https://ftc-docs.firstinspires.org/en/latest/ftc_ml/index.html).
+For extra points, FTC teams may instead use their own custom TFOD models of game elements, called **Team Props** in CENTERSTAGE.  That option is covered in this tutorial, after showing how to use the default model.  Custom TFOD models are created by teams using the same [**FTC Machine Learning process**](https://ftc-docs.firstinspires.org/en/latest/ftc_ml/index.html).
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/020-team-props.png]]</p>
@ -38,56 +40,6 @@ This tutorial shows **OnBot Java** screens.  Users of **Android Studio** can fol
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Downloading the Model
 The Robot Controller allows you to load a trained inference model in the form of a TensorFlow Lite (`.tflite`) file.
 Here we use the standard FTC `.tflite` file from CENTERSTAGE (2023-2024), available on GitHub at the following link:
 [CENTERSTAGE TFLite File](https://github.com/FIRST-Tech-Challenge/WikiSupport/blob/master/tensorflow/CenterStage.tflite)
 <!-- 
 <i>For competition, teams can use the FTC [**Machine Learning toolchain**](https://ftc-docs.firstinspires.org/en/latest/ftc_ml/index.html) to train their own custom models of Team Props.  This uses the same process as the default model described here; simply specify your custom model filename and labels.</i>
 <i>Very advanced teams could use [Google's TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) to create their own custom inference model.</i>
 -->
 Click the "Download Raw File" button to download the `CenterStage.tflite` file from GitHub to your local device (e.g. laptop).  See the green arrow.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/030-Centerstage-public-repo.png]]</p>
 Now the default TFOD model is stored on your laptop.
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Uploading to the Robot Controller
 Next, you need to upload the TFOD model to the Robot Controller.  Connect your laptop to your Robot Controller's wireless network, open the Chrome browser, and navigate to the FTC "Manage" page:
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/040-Manage-page.png]]</p>
 Scroll down and click on "Manage TensorFlow Lite Models".
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/050-Manage-TFLite-Models.png]]</p>
 Now click the "Upload Models" button.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/060-Upload-Models.png]]</p>
 Click "Choose Files", and use the dialog box to find and select the downloaded `CenterStage.tflite` file.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/070-Choose-Files.png]]</p>
 Now the file will upload to the Robot Controller.  The file will appear in the list of TensorFlow models available for use in OpModes.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/080-Centerstage-tflite.png]]</p>
 <i>**Android Studio** users should instead store the TFOD model in the project **assets** folder.  Look for `FtcRobotController -> assets`.  Left-click on the assets folder, choose `Open In` and a file explorer, then copy/paste your `.tflite` file.</i>
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Creating the OpMode
 At the FTC **OnBot Java** browser interface, click on the large black **plus-sign icon** "Add File", to open the New File dialog box.
@ -110,82 +62,10 @@ By default, the Sample OpMode assumes you are using a webcam, configured as "Web
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Basic OpMode Settings
 This Sample OpMode is **almost** ready to use, for detecting the default/built-in model (white Pixel for CENTERSTAGE).
 First, change the filename here:
 ```java
 private static final String TFOD_MODEL_FILE = "/sdcard/FIRST/tflitemodels/myCustomModel.tflite";
 ```
 to this:
 ```java
 private static final String TFOD_MODEL_FILE = "/sdcard/FIRST/tflitemodels/CenterStage.tflite";
 ```
 Later, you can change this filename back to the actual name of your custom TFOD model.  Here we are using the default (white Pixel) model just downloaded.
 =========
 **Android Studio** users should instead verify or store the TFOD model in the project **assets** folder, and use:
 ```java
 private static final String TFOD_MODEL_ASSET = "CenterStage.tflite";
 ```
 OR (for example)
 ```java
 private static final String TFOD_MODEL_ASSET = "MyModelStoredAsAsset.tflite";
 ```
 =========
 This line **does not** need to be changed:
 ```java
 // Define the labels recognized in the model for TFOD (must be in training order!)
 private static final String[] LABELS = {
   "Pixel",
 };
 ```
 ... because "Pixel" is the correct and only TFOD Label in that model file.
 Later, you might have custom Labels like "myRedProp" and "myBlueProp" (for CENTERSTAGE).  The list should be in alphabetical order and contain the labels in the dataset(s) used to make the TFOD model.
 ==========
 Next, scroll down to the Java method `initTfod()`.
 Here is the Java **Builder pattern**, used to specify various settings for the TFOD Processor.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/140-Builder-settings.png]]</p>
 The **yellow ovals** indicate its distinctive features: create the Processor with the `new Builder()` object, and close/finalize with the `.build()` method.
 <i>This is the streamlined version of the Builder pattern.  Notice all the `.set` methods are "chained" to form a single Java expression, ending with a semicolon after `.build()`.</i>
 Uncomment two Builder lines, circled above in green:
 ```java
 .setModelFileName(TFOD_MODEL_FILE)
 .setModelLabels(LABELS)
 ```
 These Builder settings tell the TFOD Processor which model and labels to use for evaluating camera frames.
 <i>**Android Studio** users should instead uncomment the line `.setModelAssetName(TFOD_MODEL_ASSET)`.</i>
 <i>**That's it!**</i> You are ready to test this Sample OpMode.
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Preliminary Testing
 This Sample OpMode is **ready to use**, for detecting the default/built-in model (white Pixel for CENTERSTAGE).
 Click the "Build Everything" button (wrench icon at lower right), and wait for confirmation "BUILD SUCCESSFUL".
 If Build is prevented by some other OpMode having errors/issues, they must be fixed before your new OpMode can run.  For a quick fix, you could right-click on that filename and choose "Disable/Comment".  This "comments out" all lines of code, effectively removing that file from the Build.  That file can be re-activated later with "Enable/Uncomment".
@ -220,11 +100,177 @@ If you don't have a physical Pixel on hand, try pointing the camera at this imag
 For a larger view, right-click the image to open in a new browser tab.
 <i>**Congratulations!**</i>  At this point the Sample OpMode and your camera are working properly.  Ready for a custom model?
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Downloading the Model
 Now we describe how to load a trained inference model in the form of a TensorFlow Lite (`.tflite`) file.
 Instead of an **actual custom model**, here we use the standard FTC model of the white Pixel from CENTERSTAGE (2023-2024).  Later, your team will follow this **same process** with your custom TFOD model, specifying its filename and labels (objects to recognize).
 The standard `.tflite` file (white Pixel) is available on GitHub at the following link:
 [CENTERSTAGE TFLite File](https://github.com/FIRST-Tech-Challenge/WikiSupport/blob/master/tensorflow/CenterStage.tflite)
 <!-- 
 <i>For competition, teams can use the FTC [**Machine Learning toolchain**](https://ftc-docs.firstinspires.org/en/latest/ftc_ml/index.html) to train their own custom models of Team Props.  Then use the process described here; simply specify your custom model filename and labels.</i>
 <i>Very advanced teams could use [Google's TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) to create their own custom inference model.</i>
 -->
 Click the "Download Raw File" button to download the `CenterStage.tflite` file from GitHub to your local device (e.g. laptop).  See the green arrow.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/030-Centerstage-public-repo.png]]</p>
 Now the TFOD model is stored on your laptop.
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Uploading to the Robot Controller
 Next, upload the TFOD model to the Robot Controller.  Connect your laptop to your Robot Controller's wireless network, open the Chrome browser, and navigate to the FTC "Manage" page:
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/040-Manage-page.png]]</p>
 Scroll down and click on "Manage TensorFlow Lite Models".
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/050-Manage-TFLite-Models.png]]</p>
 Now click the "Upload Models" button.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/060-Upload-Models.png]]</p>
 Click "Choose Files", and use the dialog box to find and select the downloaded `CenterStage.tflite` file.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/070-Choose-Files.png]]</p>
 Now the file will upload to the Robot Controller.  The file will appear in the list of TensorFlow models available for use in OpModes.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/080-Centerstage-tflite.png]]</p>
 <i>**Android Studio** users should instead store the TFOD model in the project **assets** folder.  Look for `FtcRobotController -> assets`. (Its full/hidden path name is FtcRobotController\src\main\assets.)  Right-click on `assets`, choose `Open In` and `Explorer`, then copy/paste your `.tflite` file into that assets folder.</i>
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Basic OpMode Settings
 This Sample OpMode can now be modified, to detect the uploaded TFOD model.
 Again, this tutorial uploaded the standard TFOD model (white Pixel for CENTERSTAGE), just to demonstrate the process.  Use the same steps for your custom TFOD model.
 First, change the filename here:
 ```java
 private static final String TFOD_MODEL_FILE = "/sdcard/FIRST/tflitemodels/myCustomModel.tflite";
 ```
 to this:
 ```java
 private static final String TFOD_MODEL_FILE = "/sdcard/FIRST/tflitemodels/CenterStage.tflite";
 ```
 Later, you can change this filename back to the actual name of your custom TFOD model.  Here we are using the default (white Pixel) model just downloaded.
 =========
 **Android Studio** users should instead verify or store the TFOD model in the project **assets** folder as noted above, and use:
 ```java
 private static final String TFOD_MODEL_ASSET = "CenterStage.tflite";
 ```
 OR (for a custom model)
 ```java
 private static final String TFOD_MODEL_ASSET = "MyModelStoredAsAsset.tflite";
 ```
 =========
 For this example, the following line **does not** need to be changed:
 ```java
 // Define the labels recognized in the model for TFOD (must be in training order!)
 private static final String[] LABELS = {
   "Pixel",
 };
 ```
 ... because "Pixel" is the correct and only TFOD Label in the standard model file.
 Later, you might have custom Labels like "myRedProp" and "myBlueProp" (for CENTERSTAGE).  The list should be in alphabetical order and contain the labels in the dataset(s) used to make the TFOD model.
 ==========
 Next, scroll down to the Java method `initTfod()`.
 Here is the Java **Builder pattern**, used to specify various settings for the TFOD Processor.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/140-Builder-settings.png]]</p>
 The **yellow ovals** indicate its distinctive features: **create** the Processor object with `new Builder()`, and **close/finalize** with the `.build()` method.
 <i>This is the streamlined version of the Builder pattern.  Notice all the `.set` methods are "chained" to form a single Java expression, ending with a semicolon after `.build()`.</i>
 Uncomment two Builder lines, circled above in green:
 ```java
 .setModelFileName(TFOD_MODEL_FILE)
 .setModelLabels(LABELS)
 ```
 These Builder settings tell the TFOD Processor which model and labels to use for evaluating camera frames.
 <i>**Android Studio** users should instead uncomment the line `.setModelAssetName(TFOD_MODEL_ASSET)`.</i>
 <i>**That's it!**</i> You are ready to test this Sample OpMode again, this time using a "custom" (uploaded) TFOD model.
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Testing with Custom Model
 In OnBot Java, click the "Build Everything" button (wrench icon at lower right), and wait for confirmation "BUILD SUCCESSFUL".
 Now run your updated OpMode from the Driver Station.  The OpMode should recognize objects within the camera's view, based on the trained TFOD model.
 Test the **Camera Stream** preview during the INIT phase.
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/200-Sample-DS-Camera-Stream.png]]</p>
 Tap to refresh the image.  Expand or revert the preview size as needed.  Close the preview, with 3-dots and Camera Stream again.
 After the DS START button is touched, the OpMode displays Telemetry for any recognized object(s):
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/210-Sample-DS-Telemetry.png]]</p>
 The above Telemetry shows the Label name, and TFOD recognition confidence level.  It also gives the **center location** and **size** (in pixels) of the Bounding Box, which is the colored rectangle surrounding the recognized object.
 Also test the RC's video **LiveView**, using HDMI or [**`scrcpy`**](https://github.com/Genymobile/scrcpy):
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/240-Sample-RC-LiveView.png]]</p>
 For a large view of this standard model, right-click the image to open in a new browser tab:
 <p align="center">[[https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/images/TFOD-Sample-OpMode-with-Java-Builder/300-Sample-Pixel.png]]</p>
 When your team creates, uploads and specifies a custom model containing **red and blue Team Props**, the OpMode will recognize and process those -- instead of the standard model shown here.
 <p align="right"><i>[[Return to Top|https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/TFOD-Sample-OpMode-with-Java-Builder]]</i><p>
 ## Program Logic and Initialization
-During the INIT stage (before DS START is touched), this OpMode calls a **method to initialize** the TFOD Processor and the FTC VisionPortal.  After DS START is touched, the OpMode runs a continuous loop, calling a **method to display telemetry** about any TFOD recognitions.  The OpMode also contains optional features to remind teams about **CPU resource management**, useful in vision processing.
+How does this simple OpMode work?
 - During the INIT stage (before DS START is touched), this OpMode calls a **method to initialize** the TFOD Processor and the FTC VisionPortal.
 - After DS START is touched, the OpMode runs a continuous loop, calling a **method to display telemetry** about any TFOD recognitions.
 - The OpMode also contains optional features to remind teams about **CPU resource management**, useful in vision processing.
 You've already seen the first part of the method `initTfod()` which uses a streamlined, or "chained", sequence of Builder commands to create the TFOD Processor.