↓ Skip to main content
Developer Portal
Developer Portal

ESP-GMF v1.0: General Multimedia Framework, First Official Release

·4 mins·
ESP-GMF Multimedia Audio Video AI Release
Author
Espressif author
Jason Mao
Embedded Multimedia Engineer at Espressif
Table of Contents
ESP-GMF v1.0 is the first official release of Espressif’s General Multimedia Framework, a lightweight, unified framework for building audio, video, and AI multimedia applications on Espressif chips. This post introduces the release, walks through what is new for each component area, and lists the full v1.0 component set.

Introduction
#

We are releasing ESP-GMF (General Multimedia Framework) v1.0, the first official release of Espressif’s unified software platform for developing multimedia applications.

In the past, multimedia applications were often built on separate frameworks, each with its own development model, interface design, and runtime mechanism. This limited code reuse and made it harder to extend functionality across products and the wider ecosystem.

ESP-GMF was created to change that. It brings Espressif’s previously scattered multimedia capabilities under a single software architecture, with a unified data-flow model, component system, and development framework — covering use cases from playback and recording to AI voice, video calling, Bluetooth audio, and rendering, all built on the same framework. Compared with our earlier audio framework, ESP-ADF, ESP-GMF offers a more modular architecture, broader hardware and media support, and better code reuse across audio, video, and AI applications.

Alongside ESP-GMF, we have also completed the ESP Multimedia Core — a unified library that consolidates foundational multimedia building components such as media protocols, audio/video codecs, and audio/image processing algorithms. It gives ESP-GMF, and the products built on it, a stable, long-term foundation, and we will keep expanding it together with the framework and the wider ecosystem.

Overview
#

ESP-GMF v1.0 is the first official, API-stable release of the framework. It consolidates the earlier development versions, promotes every official component to a 1.0.x baseline, and introduces several new advanced modules — all of which are summarized in the ESP-GMF components summary below. Key updates by area:

  • Promoted all components to 1.0.x — public APIs are now considered stable.
  • Added 4 new advanced/atomic modules: esp_player, esp_asrc, esp_video_render, and gmf_fft.
  • New elements: aud_muxer, aud_howl, aud_asrc (gmf_audio) and ai_vad, ai_ns, ai_doa (gmf_ai_audio).
  • gmf_core adds an esp_gmf_data_queue + data-bus factory for variable-sized block queues, stronger memory-alignment handling for data_bus/payload, and HOWL/DOA/MUXER capability definitions.
  • gmf_audio adds muxer/howl/ASRC elements, decoder input-PTS support, and bumps esp_audio_codec to v2.5 (G722 encode/decode, OGG decode).
  • gmf_video / esp_video_render add hardware-blend and multi-region overlay, PPA software color-convert, manual compose, and broader display-backend support.
  • esp_capture / esp_bt_audio add ESP32-S31 v4l2 + MJPEG→RGB decode, LE Audio (BLE) and Auracast PAST support, and adopt the shared esp_gmf_data_queue.
  • Board management migration: esp_board_manager is now the standalone espressif/esp-board-manager component, and examples use esp-bmgr-assist (idf.py bmgr) instead of per-example prebuild / idf_ext.py scripts.

ESP-GMF components summary
#

All official components in this repo:

ComponentCategoryVersionHighlights
esp_playerNew Modulev1.0.1New embedded multimedia player: demux + decode + audio/video render with seek; decoder/IO stability fixes
esp_audio_simple_playerEnhancementsv1.0.0Breaking esp_asp_prev_func_t signature change, weak-typed URL playback detection, dynamic URI→IO mapping, esp_audio_simple_player_get_pool, bit-depth conversion Kconfig
esp_captureEnhancementsv1.0.0Shared esp_gmf_data_queue adoption, esp-sr v2.4, ESP32-S31 v4l2 + MJPEG→RGB decode, multi-region overlay & share_overlay, restart-bypass fix
esp_bt_audioEnhancementsv1.0.0LE Audio (BLE) support, Auracast PAST + clock-sync check, Classic/LE UI in the bt_audio example, standalone board-manager
esp_audio_renderEnhancementsv1.0.0ESP32-S31 example support, payload-clear memory-access fix, standalone board-manager example migration
esp_video_renderNew Modulev1.0.0New video + UI composition module: multi-backend display, dual stream, overlay/widget system, and manual compose
esp_asrcNew Modulev1.0.1New audio sample-rate converter (sample rate / bit depth / channel count) with HW–SW cooperative architecture; ESP32-P4 & S31 support
esp_board_managerExternal (standalone repo)0.5.15Migrated out of esp-gmf into its own repository; examples integrate it via esp-bmgr-assist (idf.py bmgr)
gmf_loaderEnhancementsv1.0.0Loader setup for muxer/howl/asrc and standalone AI elements (VAD/NS/DOA), G722/OGG and HOWL effect configuration, standalone board-manager
gmf_app_utilsEnhancementsv1.0.0Adopted the standalone esp-board-manager component (>=0.5)
gmf_examplesEnhancementsv1.0.0New pipeline_record_audio_muxer & pipeline_howl examples; migrated examples to standalone esp-board-manager + esp-bmgr-assist (idf.py bmgr)
gmf_audioEnhancementsv1.0.0New aud_muxer/aud_howl/aud_asrc elements, decoder input-PTS support, esp_audio_codec v2.5 (G722/OGG), finer-grained mutex protection
gmf_videoEnhancementsv1.0.0vid_overlay hardware blend & multi-region, ESP32-S31 v4l2, PPA software color-convert, plus alignment/bypass and IDF v5.x/6.x build fixes
gmf_ai_audioEnhancementsv1.0.0New ai_vad/ai_ns/ai_doa elements, esp-sr v2.4.4, ESP32-S31 support, board-manager-based example configuration
gmf_ioEnhancementsv1.0.0Default HTTPS certificate-bundle support for HTTP IO and IDF v6.0+ build fixes
gmf_miscEnhancementsv1.0.0No functional delta in v1.0; version aligned with the v1.0 ecosystem
gmf_fftNew Modulev1.0.0New fixed-point Q15 real FFT/IFFT with PIE acceleration (S3/P4/S31) and a scalar fallback covering all ESP32 series
gmf_coreEnhancementsv1.0.0Variable-sized block data queue + data-bus factory, stronger data_bus/payload alignment (breaking esp_gmf_fifo_set_align), HOWL/DOA/MUXER caps, IO seek/reload & resource-leak fixes, FourCC additions/corrections

Where to go next
#

If you build something on top of ESP-GMF, we would love to see it — share it on the esp32.com forum.

Related

Developing a Zephyr IoT app with AI

·15 mins
How-To Zephyr Iot AI LLM
Still on improving our AI-based, embedded software workflow, this article applies the same AI-agent discipline to Zephyr. Using the same ESP Dualkey kit from M5Stack, specs, integration rules, plan→execute→commit→test, reusable modules, journals, and Git - without redefining the product.

Introduction to ESP FOURCC

·5 mins
Multimedia ESP-GMF
This article explores ESP FOURCC standard based on FOURCC (Four Character Code), a standardized 32-bit identifier used in multimedia for format consistency, reducing conversion overhead and debugging complexity. It highlights Espressif implementation in ESP-GMF, where esp_fourcc.h defines FOURCC values for video, audio, container, image, and pixel formats, optimizing efficiency in embedded systems.

AES67 audio-over-IP on the ESP32-P4

·10 mins
AES67 RAVENNA Audio PTP RTP ESP-IDF ESP32-P4 Networking
AES67/RAVENNA is the audio transport behind a lot of broadcast and live-sound infrastructure, and it normally runs on dedicated silicon or Linux boxes. This article is about getting a working, PTP-synchronized AES67 endpoint onto an ESP32-P4 — how the clock sync, the low-latency receive path, and the I2S playout fit together, what the measured latency is, and where the edges are.