ESP-TFLite-Micro workshop - Assignment 3: Convert and embed the model

Recap from Assignment 2

#

In the previous assignment, you trained a 1D convolutional neural network with four classes:

The model accepts a float32 tensor with shape (1, 200, 3): one sample containing 200 time steps from the BMI270 x, y, and z gyroscope axes. Its output is a float32 tensor with shape (1, 4), containing one score for each class.

Assignment 2 saved the trained model here:

gesture_recognition/
└── training/
    └── simple_1dcnn_model.h5

What you will do

#

In this assignment, you will:

1. Convert the Keras H5 model to TensorFlow Lite.
2. Inspect the converted model’s size, tensor shapes, types, and operators.
3. Optionally compare Keras and TensorFlow Lite predictions.
4. Convert the .tflite file to a C++ byte array.
5. Replace the model embedded in the ESP-IDF application.

Assignment steps

#

The workshop code repository and the training script bundle provide the convert_model.py and embed_model.py files used in this assignment. The code listings below explain their implementation and provide a manual fallback.

Step 1: Convert the model

#

Remain in the gesture_recognition/training directory with the Python environment from Assignment 2 activated.

Open the supplied convert_model.py. If you are following the exercise without the downloadable assets, create it with this code:

 1from pathlib import Path
 2
 3import tensorflow as tf
 4from tensorflow import keras
 5
 6
 7KERAS_MODEL_PATH = Path("simple_1dcnn_model.h5")
 8TFLITE_MODEL_PATH = Path("model.tflite")
 9
10model = keras.models.load_model(KERAS_MODEL_PATH)
11converter = tf.lite.TFLiteConverter.from_keras_model(model)
12
13converter.optimizations = [tf.lite.Optimize.DEFAULT]
14converter.target_spec.supported_types = [tf.float32]
15converter.inference_input_type = tf.float32
16converter.inference_output_type = tf.float32
17
18tflite_model = converter.convert()
19TFLITE_MODEL_PATH.write_bytes(tflite_model)
20
21print(f"Saved {TFLITE_MODEL_PATH}")
22print(f"Model size: {len(tflite_model)} bytes")

Run the conversion:

python convert_model.py

The input and output remain float32, matching the gesture application’s current preprocessing and inference code. This is not full-integer quantization.

Step 2: Inspect the TensorFlow Lite model

#

The supplied convert_model.py already includes the following checks. If you created the script manually in Step 1, append them now:

 1interpreter = tf.lite.Interpreter(model_content=tflite_model)
 2interpreter.allocate_tensors()
 3
 4input_details = interpreter.get_input_details()
 5output_details = interpreter.get_output_details()
 6
 7print("Input:", input_details[0]["shape"], input_details[0]["dtype"])
 8print("Output:", output_details[0]["shape"], output_details[0]["dtype"])
 9
10assert input_details[0]["shape"].tolist() == [1, 200, 3]
11assert input_details[0]["dtype"] == tf.float32.as_numpy_dtype
12assert output_details[0]["shape"].tolist() == [1, 4]
13assert output_details[0]["dtype"] == tf.float32.as_numpy_dtype
14
15print("Operators:")
16for operation in interpreter._get_ops_details():
17    print(f"- {operation['op_name']}")

Run the script again:

python convert_model.py

Expected tensor information:

Input: [  1 200   3] <class 'numpy.float32'>
Output: [1 4] <class 'numpy.float32'>

The operator list determines which kernels must be registered in the TensorFlow Lite Micro resolver. The exact list depends on the TensorFlow version and converter output, so use the generated list rather than assuming it matches another model.

Step 3 (optional): Compare predictions

#

Before embedding the model, compare both formats with a known sample. Add these imports:

import numpy as np
import pandas as pd

Then append:

 1sample = (
 2    pd.read_csv("./shuttle_dataset/new_gesture.csv", header=None)
 3    .iloc[0]
 4    .to_numpy(dtype=np.float32)
 5    .reshape(1, 200, 3)
 6)
 7
 8keras_output = model.predict(sample, verbose=0)
 9
10interpreter.set_tensor(input_details[0]["index"], sample)
11interpreter.invoke()
12tflite_output = interpreter.get_tensor(output_details[0]["index"])
13
14print("Keras output:", keras_output)
15print("TFLite output:", tflite_output)
16print("Keras class:", int(np.argmax(keras_output)))
17print("TFLite class:", int(np.argmax(tflite_output)))
18
19np.testing.assert_allclose(keras_output, tflite_output, rtol=1e-4, atol=1e-5)

The output values may differ slightly because conversion can change graph implementation details, but both models should normally select the same class.

Step 4 (optional): Explore full-integer quantization

#

Full-integer quantization can reduce model size and may improve inference efficiency, but it requires a representative dataset:

 1import numpy as np
 2import pandas as pd
 3
 4
 5representative_data = np.vstack([
 6    pd.read_csv("./shuttle_dataset/o.csv", header=None).values,
 7    pd.read_csv("./shuttle_dataset/v.csv", header=None).values,
 8    pd.read_csv("./shuttle_dataset/unknown.csv", header=None).values,
 9    pd.read_csv("./shuttle_dataset/new_gesture.csv", header=None).values,
10]).reshape(-1, 200, 3).astype(np.float32)
11
12
13def representative_dataset():
14    for sample in representative_data[:100]:
15        yield [sample.reshape(1, 200, 3).astype(np.float32)]
16
17
18converter = tf.lite.TFLiteConverter.from_keras_model(model)
19converter.optimizations = [tf.lite.Optimize.DEFAULT]
20converter.representative_dataset = representative_dataset
21converter.target_spec.supported_ops = [
22    tf.lite.OpsSet.TFLITE_BUILTINS_INT8,
23]
24converter.inference_input_type = tf.int8
25converter.inference_output_type = tf.int8

This workshop deploys the float32 model. An int8 model requires additional input quantization and output dequantization in the ESP-IDF application, so do not replace model.tflite with the quantized version unless you also update the inference code.

Step 5: Convert the model to a C++ array

#

Use the supplied embed_model.py to convert the TensorFlow Lite FlatBuffer into an aligned, constant C++ array:

 1from pathlib import Path
 2
 3
 4model_data = Path("model.tflite").read_bytes()
 5rows = [
 6  ", ".join(f"0x{value:02x}" for value in model_data[offset:offset + 12])
 7  for offset in range(0, len(model_data), 12)
 8]
 9formatted_data = ",\n    ".join(rows)
10
11Path("model.cpp").write_text(
12  '#include "model.h"\n\n'
13  "alignas(16) const unsigned char model_tflite[] = {\n"
14  f"    {formatted_data}\n"
15  "};\n\n"
16  "const unsigned int model_tflite_len = sizeof(model_tflite);\n",
17  encoding="utf-8",
18)
19
20print("Created model.cpp")

python embed_model.py

xxd -i model.tflite > model.cpp

1unsigned char model_tflite[] = {
2  // Model bytes...
3};
4unsigned int model_tflite_len = /* model size */;

1#include "model.h"
2
3alignas(16) const unsigned char model_tflite[] = {
4  // Keep all bytes generated by xxd here.
5};
6
7const unsigned int model_tflite_len = sizeof(model_tflite);

Alignment gives TensorFlow Lite Micro a suitably aligned FlatBuffer address. The symbol names must remain model_tflite and model_tflite_len because the application refers to them by those names. The example already contains main/include/model.h with the required declarations, so you do not need to create or replace that file.

Step 6: Add the model to the ESP-IDF project

#

From the training directory, replace the model files in the gesture recognition application:

cp model.cpp ../main/model.cpp

Graphical alternative: In Finder, File Explorer, or the VS Code Explorer:

1. Copy training/model.cpp to main/model.cpp, replacing the existing file.
2. Confirm the replacement when prompted.

The resulting project layout should include:

 1gesture_recognition/
 2├── main/
 3│   ├── app_model.cpp
 4│   ├── model.cpp
 5│   ├── CMakeLists.txt
 6│   └── include/
 7│       └── model.h
 8└── training/
 9    ├── simple_1dcnn_model.h5
10    ├── model.tflite
11    └── model.cpp

The example’s main/CMakeLists.txt registers the whole source directory:

1idf_component_register(
2    SRC_DIRS "."
3    INCLUDE_DIRS "." "include"
4)

Because model.cpp is inside main/, it is compiled automatically. Assignment 4 updates the application labels and deploys the four-class model.

Acceptance criteria

#

• The converted .tflite model loads in a host-side interpreter.
• Its input tensor is float32 with shape (1, 200, 3).
• Its output tensor is float32 with shape (1, 4).
• model.cpp defines the constant symbols declared by the example’s existing model.h.
• The model byte array and length are available to the inference component.

Next step

#

Assignment 4: Deploy and test on-device inference

Back to workshop home