where to find the sample input and the expected output data for the v3-small_224_1.0_uint8.tflite model?

[ashalli]

hi, if someone knows where to find the example input and expected output data for the v3-small_224_1.0_uint8.tflite model, could you please share the information? thanks!

[geekyNads]

Hi @ashalli, I found a likely source for the sample/test data.

The exact mobilenet_v3-small_224_1.0_uint8.tflite model was added to Google's Android MLTS model repository here:

https://chromium.googlesource.com/aosp/platform/test/mlts/models/+/73313692ee90e6d768cd086fbb7c94298ecaf1ac

That commit includes the exact mobilenet_v3-small_224_1.0_uint8.tflite model as well as the mobilenet_topk_aosp.json test configuration. The configuration points to:

• image_classification — input dataset
• image_classification/ground_truth_labels.txt — expected/ground-truth labels
• image_classification/labels.txt — class labels
• Inception — preprocessing
• Input shape: [1, 224, 224, 3]

The commit also notes that the expectedTop1 values were obtained using TFLite CPU.

So if you're looking specifically for the original sample inputs and expected classification results, the Android MLTS image_classification test assets appear to be the relevant place to look.

I'm interested in digging further into this and can try to track down the actual image_classification input files and the exact expected output for the uint8 model if they're still publicly available.

[UdaySharmaGitHub]

Hi @ashalli,

I found the test assets and configuration for the exact v3-small_224_1.0_uint8.tflite model.

The model was added to Google's Android MLTS (Machine Learning Test Suite) model repository in this commit:

https://android.googlesource.com/platform/test/mlts/models/+/73313692ee90e6d768cd086fbb7c94298ecaf1ac

The repository contains the exact:

mobilenet_v3-small_224_1.0_uint8.tflite

under:

assets/mobilenet_v3-small_224_1.0_uint8.tflite

It also contains the associated image-classification test assets under:

assets/image_classification/

The dataset includes the input JPEGs, ground_truth_labels.txt, labels.txt, and the dataset README. The README says the dataset is based on the Open Images Dataset V4 test set and contains roughly 1,400 images.

Model test configuration

The corresponding MLTS configuration is:

assets/models_list/mobilenet_topk_aosp.json

For mobilenet_v3-small_224_1.0_uint8, the configuration specifies:

Input shape:      [1, 224, 224, 3]
Input type:       uint8
Input scale:      0.007812
Input zero point: 128
Preprocessor:     Inception
Dataset:          image_classification
Ground truth:     image_classification/ground_truth_labels.txt
Labels:           image_classification/labels.txt

The expected Top-1 accuracy is 0.56510067, and the commit notes that the expectedTop1 values were obtained using TFLite CPU.

So, if you're looking for the original test setup, these are the relevant pieces:

• Model: mobilenet_v3-small_224_1.0_uint8.tflite
• Input images: assets/image_classification/image-*.jpeg
• Ground-truth outputs: assets/image_classification/ground_truth_labels.txt
• Class labels: assets/image_classification/labels.txt
• Preprocessing: Inception
• Expected aggregate Top-1: 0.56510067
• Quantization: scale 0.007812, zero-point 128
• Input tensor: [1, 224, 224, 3]

The complete image_classification directory is available here:

https://android.googlesource.com/platform/test/mlts/models/+/73313692ee90e6d768cd086fbb7c94298ecaf1ac/assets/image_classification/

And the exact model is here:

https://android.googlesource.com/platform/test/mlts/models/+/73313692ee90e6d768cd086fbb7c94298ecaf1ac/assets/mobilenet_v3-small_224_1.0_uint8.tflite

One small clarification: dataSize: 1 in the MLTS configuration refers to the per-inference input size for the uint8 model; the associated test dataset itself contains many images.

So if your goal is to reproduce the original MLTS/TFLite result, I would use the image_classification images + ground_truth_labels.txt together with the preprocessing and quantization parameters above, rather than looking for a separate input/output tensor file.

GitHub: https://github.com/UdaySharmaGithub
If this answer was helpful, feel free to check out my profile for other projects and contributions.
Hope this helps!