ESP-TFLite-Micro workshop

Welcome to the ESP-TFLite-Micro workshop!

About this workshop

#

This hands-on workshop covers the complete workflow for running machine learning directly on a microcontroller, with no network connection or cloud service involved. You will repeatedly move a development board by hand to record a gesture of your choice, turn those recordings into a labeled dataset, and train a small neural network on your computer to tell your gesture apart from other movements. You will then compress the model so that it fits in the limited memory of the microcontroller, embed it in the firmware, and measure how reliably the board recognizes the gesture as you perform it.

This workflow is implemented in the ESP-SensairShuttle gesture recognition example, which the workshop uses as its starting point. Specifically, you will collect motion data from the BMI270 IMU on the ESP-SensairShuttle, train a lightweight gesture classification model, convert it to TensorFlow Lite format, and run inference on the ESP32-C5 with ESP-TFLite-Micro.

By the end of the workshop, you will be able to:

• Capture and label time-series IMU data
• Prepare a dataset for gesture classification
• Train and evaluate a compact 1D convolutional neural network with Keras
• Convert the trained Keras model to TensorFlow Lite
• Embed a model in an ESP-IDF application
• Run and validate on-device inference with ESP-TFLite-Micro

Prerequisites

#

Required hardware:

• Computer running Linux, Windows, or macOS
• ESP-SensairShuttle
• USB cable that supports power and data

Required software:

• ESP-IDF v6.1
• Python 3.12 with TensorFlow, pandas, NumPy, matplotlib, and scikit-learn
• Git
• A serial terminal

Agenda

#

• Machine learning on microcontrollers: Learn how TensorFlow Lite Micro, ESP-TFLite-Micro, and ESP-NN run machine-learning models on Espressif SoCs.
• Hardware: The ESP32-C5 and ESP-SensairShuttle: Get familiar with the processor, development board, and connections used in the workshop.
• Bosch Sensortec sensors: Explore the BMI270, BMM350, and BME690 sensors supported by the board.
• Install ESP-IDF: Install and activate ESP-IDF v6.1 with the ESP-IDF Installation Manager.
• Download and flash the gesture recognition example: Build the reference application for ESP32-C5, flash it, and verify BMI270 inference.
• Assignment 1: Collect and prepare gesture data: Capture BMI270 gyroscope samples and prepare the training dataset.
• Assignment 2: Train and evaluate the model: Build a compact 1D CNN and evaluate its classification accuracy.
• Assignment 3: Convert and embed the model: Convert the trained model to TensorFlow Lite and then to a C++ array.
• Assignment 4: Deploy and test on-device inference: Integrate ESP-TFLite-Micro, run gesture inference, and validate the result on the board.

Time requirements

#

Workshop resources

#

• Workshop gesture recognition project: Maintained ESP-IDF firmware, pretrained model, and training workspace used throughout the workshop.
• Training scripts: Downloadable scripts for data capture, validation, model training, conversion, and embedding.
• Upstream ESP-SensairShuttle gesture recognition example: Original example on which the workshop project is based.
• Gesture Recognition Based on TFLite: Additional background on the end-to-end workflow.

Feedback

#

If you have any feedback about the workshop, feel free to start a new discussion on GitHub.

Next step

#

Introduction: Machine learning on microcontrollers