What TensorFlow Lite for Microcontrollers is
TensorFlow Lite for Microcontrollers (TFLM) is a version of TensorFlow’s inference runtime designed for microcontrollers with tens or hundreds of kilobytes of RAM, no operating system, no file system and no dynamic memory allocation. You train a model on your computer, convert it to a compact format, compile it into your firmware, and the microcontroller runs inference (making predictions) on sensor data locally.
Typical TFLM projects include:
- Keyword spotting: detecting a wake word like “yes” or “no” from a microphone
- Gesture recognition from an accelerometer
- Anomaly detection on vibration or current sensors
- Simple person detection from a low-resolution camera
Running models on the device avoids sending raw data to the cloud, which saves power and bandwidth, cuts latency and keeps sensitive data local. This style of work is often called TinyML. For larger camera-based workloads that need more compute, see embedded AI and computer vision at the edge.
Compared with the TensorFlow Lite runtime for phones, TFLM:
- Supports a subset of operations, chosen for small models
- Uses a fixed memory region you provide (the tensor arena) instead of
malloc - Has a very small core. The project’s documentation has described the core runtime as fitting in about 16 KB on an Arm Cortex-M3, though your total footprint depends on the operations and model you include
- Is usually used with int8 quantized models
A note on naming: in 2024 Google renamed TensorFlow Lite to LiteRT, and some documentation now calls TFLM “LiteRT for Microcontrollers.” The code still lives in the tflite-micro GitHub repository and still uses the tflite namespace, so most tutorials and APIs carry over. Expect to see both names.
What you need
Hardware options
TFLM runs on many 32-bit microcontrollers. Popular starting points:
- ESP32 boards (especially ESP32-S3, which has vector instructions that Espressif’s optimized kernels can use)
- Arm Cortex-M boards, such as Arduino Nano 33 BLE Sense-class boards, STM32 Nucleo or Discovery boards, and Raspberry Pi Pico (RP2040) or Pico 2 boards
- Boards with an onboard microphone, IMU or camera, if you want to try the speech, gesture or person detection examples
A board with at least a few hundred kilobytes of RAM and a megabyte or more of flash gives you room to experiment. The smallest examples fit in much less.
Software
- Python with TensorFlow installed, for training and converting models (Google Colab works well if you don’t want to set it up locally)
- Your board’s toolchain: ESP-IDF, Arduino IDE, PlatformIO, STM32CubeIDE or a GCC Arm toolchain with CMake or Make
- The TFLM library, either from the
tflite-microrepository or packaged for your platform (for example Espressif’sesp-tflite-microcomponent)
The TFLM workflow at a glance
- Collect data from the sensor you’ll use in the field.
- Train a small model in TensorFlow/Keras.
- Convert it to the
.tfliteFlatBuffer format, usually with int8 quantization. - Embed the model in your firmware as a C byte array.
- Run inference with the TFLM
MicroInterpreter: copy sensor data into the input tensor, callInvoke(), read the output tensor. - Measure and iterate: check accuracy on the device, memory use and inference time.
The rest of this guide follows that workflow, using the classic “hello world” model that learns to approximate a sine wave. It’s a toy, but it covers every step without needing a sensor.
Step 1: Train a tiny model
import numpy as np
import tensorflow as tf
# Generate training data: x in [0, 2*pi], y = sin(x) plus a little noise
x = np.random.uniform(0, 2 * np.pi, 2000).astype(np.float32)
y = np.sin(x) + 0.1 * np.random.randn(*x.shape).astype(np.float32)
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(1,)),
tf.keras.layers.Dense(16, activation="relu"),
tf.keras.layers.Dense(16, activation="relu"),
tf.keras.layers.Dense(1),
])
model.compile(optimizer="adam", loss="mse", metrics=["mae"])
model.fit(x, y, epochs=200, batch_size=64, validation_split=0.2, verbose=0)
This model has only a few hundred parameters, so it’s small even before quantization. In real projects, keep the model small from the start. A model that hits 99% accuracy on your laptop but needs 2 MB of RAM won’t run on most microcontrollers.
Step 2: Convert and quantize to int8
Quantization stores weights and activations as 8-bit integers instead of 32-bit floats. That cuts model size to roughly a quarter, and on many microcontrollers it makes inference much faster, especially with optimized kernels. The converter needs a representative dataset, a sample of realistic inputs, to choose the quantization ranges:
def representative_dataset():
for value in x[:500]:
yield [np.array([[value]], dtype=np.float32)]
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
tflite_model = converter.convert()
with open("sine_model.tflite", "wb") as f:
f.write(tflite_model)
print(f"Model size: {len(tflite_model)} bytes")
Before going near hardware, check the quantized model’s accuracy on your computer with the Python tf.lite.Interpreter. If it’s much worse than the float model, the representative dataset may not cover the real input range.
The converter API has changed over the years and may print deprecation notices as Google moves tooling under the LiteRT name. If a snippet doesn’t work with your TensorFlow version, check the current conversion docs.
Step 3: Turn the model into a C array
Microcontrollers usually have no file system, so the model is compiled into flash as a byte array. On Linux or macOS:
xxd -i sine_model.tflite > sine_model_data.cc
Edit the generated file so the array is const (it stays in flash instead of being copied to RAM) and aligned:
// sine_model_data.cc
#include "sine_model_data.h"
alignas(16) const unsigned char g_sine_model[] = {
0x1c, 0x00, 0x00, 0x00, 0x54, 0x46, 0x4c, 0x33, /* ... */
};
const unsigned int g_sine_model_len = 2488; // use the length xxd generated
// sine_model_data.h
#pragma once
extern const unsigned char g_sine_model[];
extern const unsigned int g_sine_model_len;
The bytes and length above are illustrative. Use the values xxd produced for your model.
Step 4: Run inference on the microcontroller
Here’s a minimal, platform-neutral inference routine using the current TFLM API:
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "tensorflow/lite/micro/micro_log.h"
#include "tensorflow/lite/micro/system_setup.h"
#include "tensorflow/lite/schema/schema_generated.h"
#include "sine_model_data.h"
namespace {
const tflite::Model* model = nullptr;
tflite::MicroInterpreter* interpreter = nullptr;
TfLiteTensor* input = nullptr;
TfLiteTensor* output = nullptr;
constexpr int kTensorArenaSize = 4 * 1024; // start generous, trim later
alignas(16) uint8_t tensor_arena[kTensorArenaSize];
} // namespace
bool setup_model() {
tflite::InitializeTarget();
model = tflite::GetModel(g_sine_model);
if (model->version() != TFLITE_SCHEMA_VERSION) {
MicroPrintf("Model schema %d not supported", model->version());
return false;
}
// Register only the operations this model uses
static tflite::MicroMutableOpResolver<1> resolver;
resolver.AddFullyConnected();
static tflite::MicroInterpreter static_interpreter(
model, resolver, tensor_arena, kTensorArenaSize);
interpreter = &static_interpreter;
if (interpreter->AllocateTensors() != kTfLiteOk) {
MicroPrintf("AllocateTensors() failed");
return false;
}
input = interpreter->input(0);
output = interpreter->output(0);
MicroPrintf("Arena used: %d bytes", (int)interpreter->arena_used_bytes());
return true;
}
float predict(float x) {
// Quantize the float input to int8
int8_t x_q = (int8_t)(x / input->params.scale + input->params.zero_point);
input->data.int8[0] = x_q;
if (interpreter->Invoke() != kTfLiteOk) {
MicroPrintf("Invoke failed");
return 0.0f;
}
// Dequantize the int8 output back to float
int8_t y_q = output->data.int8[0];
return (y_q - output->params.zero_point) * output->params.scale;
}
What each part does:
GetModel()maps the byte array as a model without copying it.MicroMutableOpResolver<N>registers only the operations your model needs, which keeps code size down.Nis the number of operation types you add. The sine model uses fully connected layers (the ReLU activations are fused into them), so one is enough. A convolutional model might needAddConv2D(),AddDepthwiseConv2D(),AddMaxPool2D(),AddReshape(),AddSoftmax()and so on.- The tensor arena is a static buffer that holds input, output and intermediate tensors. TFLM plans all memory up front, so there’s no heap fragmentation at run time.
AllocateTensors()lays out the tensors in the arena. It fails if the arena is too small.Invoke()runs the model once.
If you follow older tutorials, you’ll see ErrorReporter objects passed to the interpreter and an AllOpsResolver. Both have been removed or discouraged in current TFLM, which uses MicroPrintf for logging and expects you to register ops explicitly. When an older example won’t compile, check the matching example in the tflite-micro repository.
Working with quantized inputs and outputs
With inference_input_type = tf.int8, the model expects int8 values, not floats. Each quantized tensor has a scale and a zero point:
- Quantize:
q = round(x / scale) + zero_point, clamped to −128 to 127 - Dequantize:
x = (q - zero_point) * scale
The code above truncates for brevity. In real code, round and clamp the value. For multi-value inputs such as audio features or image pixels, apply the same formula to every element.
The single most common TinyML bug is preprocessing that differs between training and the device: a different scaling, axis order, sample rate or feature calculation. If the model works in Python but gives nonsense on the board, compare the exact input tensor bytes on both sides.
Sizing the tensor arena
There’s no formula you can rely on to compute the arena size in advance, so measure it:
- Start with a generous arena (as much RAM as you can spare).
- After
AllocateTensors()succeeds, printinterpreter->arena_used_bytes(). - Set
kTensorArenaSizeto that value plus a safety margin, and confirm it still works.
Your model’s weights live in flash, but activations need RAM in the arena. Large convolutional layers on high-resolution inputs are what usually blow the RAM budget, so reducing input resolution or channel count is often the most effective fix.
Getting it onto real hardware
ESP32
Espressif maintains esp-tflite-micro, which packages TFLM as an ESP-IDF component and integrates Espressif’s ESP-NN optimized kernels. You can add it to an ESP-IDF project through the component manager:
idf.py add-dependency "espressif/esp-tflite-micro"
The repository includes ESP32 ports of the hello world, micro speech and person detection examples, which are the easiest place to start. To send results to a dashboard or server, combine them with MQTT on the ESP32, for example publishing only when a gesture or anomaly is detected.
Arduino boards
For several years an official Arduino TensorFlow Lite library supported boards like the Nano 33 BLE Sense. Library availability and maintenance have shifted over time, and community-maintained forks exist. Check the Library Manager and recent forum threads for the version that currently works with your board and core. If you hit compatibility problems, using the vendor’s native SDK or PlatformIO with a TFLM build is often more predictable.
Other Cortex-M boards and optimized kernels
On Arm Cortex-M, TFLM can use CMSIS-NN optimized kernels, which can speed up int8 inference considerably compared with the portable reference kernels. Some newer microcontrollers include an Arm Ethos-U NPU, which TFLM supports when you compile the model with Arm’s Vela tool. Chip vendors also offer their own model-deployment tools (for example ST’s edge AI tools for STM32). These can be easier to use than raw TFLM on their own chips, at the cost of tying you to that vendor. The trends in embedded systems article covers the move toward NPUs in microcontrollers.
You can also build and test the TFLM examples on your desktop before touching hardware. The repository’s README and each example’s README list the current build commands.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| “Didn’t find op for builtin opcode …” | Operation not registered in the resolver | Add the missing Add...() call and increase the resolver’s template count |
AllocateTensors() fails |
Tensor arena too small | Enlarge the arena, then trim using arena_used_bytes() |
| Schema version mismatch | Model converted with an incompatible TensorFlow version | Update TFLM or convert with a compatible TensorFlow version |
| Predictions are nonsense | Input not quantized, or preprocessing differs from training | Apply scale and zero point; compare input tensors with Python |
| Hard fault at startup | Unaligned model or arena, or stack overflow | Use alignas(16); make large objects static; increase stack size |
| Inference too slow | Reference kernels, float model, or model too large | Use int8 plus CMSIS-NN or ESP-NN; shrink the model |
| Model doesn’t fit in flash | Too many parameters | Reduce layer widths, use depthwise convolutions, check quantization was applied |
Where to go next
- Run the official examples: hello world, micro speech (keyword spotting) and person detection. Get one running on your board unmodified before building your own.
- Build a gesture classifier from accelerometer data. It’s a small dataset, a small model and a satisfying result.
- Try vibration anomaly detection on a fan or motor, a realistic industrial use case.
- Look at Edge Impulse or similar platforms if you want help with data collection, feature extraction and deployment. They can export libraries that use TFLM-compatible runtimes.
- Plan for updates. Models improve as you collect more field data. Design your device so models can be replaced with over-the-air firmware updates, either as part of the firmware image or in a separate flash partition.
FAQ
Can I train models on the microcontroller with TFLM? No. TFLM is an inference runtime. You train on a computer or in the cloud, then deploy the trained model. On-device learning on microcontrollers is an active research area but isn’t what TFLM is built for.
Do I have to use int8 quantization? TFLM can run float32 models, and that’s useful for debugging. But int8 models are about a quarter of the size and are usually much faster on microcontrollers, especially with optimized kernels. Most deployed TFLM models are int8.
Can I use PyTorch models? Google’s LiteRT tooling includes options for converting PyTorch models, and there are third-party routes as well. Check that the resulting model uses only operations TFLM supports and that it quantizes cleanly.
How much RAM do I need?
It depends entirely on the model. The sine example needs only a few kilobytes of arena, while keyword spotting and small vision models commonly need tens to a few hundred kilobytes. Measure with arena_used_bytes() rather than guessing.
Is TFLM still maintained after the LiteRT rename?
The tflite-micro repository has continued to receive updates. Check its recent commit history and release notes before starting a long-lived product.