Google AI Edge LiteRT Samples
This repository contains official and community contributed sample applications, model recipes, agent skills and utilities for LiteRT (formerly known as TensorFlow Lite), Google's open source, high-performance on-device machine learning framework and LiteRT-LM, a specialized orchestration layer for running LLMs with LiteRT, unlocking maximum performance and efficiency.
Note Please access the interactive web page with a collections of demos there at: https://google-ai-edge.github.io/litert-samples/
The samples demonstrate different API paradigms (LiteRT CompiledModel API and legacy Interpreter API, Tensor API, LiteRT-LM) and provide end-to-end model conversion and deployment pipelines.
🔥 What's New
- 🐱 Streaming TTS (KittenTTS nano): Added a tiny (15M-param, 32 MB) streaming text-to-speech Android sample — dynamic-length LiteRT graphs, sentence-level streaming playback, live TTFA/RTF metrics (
samples/litert/text_to_speech_streaming/). - 🎙️ Speech Recognition (ASR): Added end-to-end Automatic Speech Recognition sample using the CompiledModel API.
- 📸 PhotoTalk Sample App: Added multimodal sample app combining LiteRT vision processing with LiteRT-LM audio/text generation (
samples/litert/phototalk_sample_app/). - 🗣️ Qwen3-TTS & Qwen3 ASR: Added model recipes, conversion scripts, and Tensor API implementations for Qwen3-TTS and Qwen3 ASR.
- 🎨 Bonsai Image 4B: Added text-to-image diffusion model sample with Python inference and conversion tools (
models/bonsai/bonsai_image_4b/). - 🤖 Agent Skills & Utilities: Added four lifecycle agent skills (
skills/: conversion, quantization, on-device verification, app scaffolding), a GPU conversion toolkit (utilities/litert_gpu_toolkit/), and shared Kotlin helpers (utilities/common/).
📂 Repository Structure
1. samples/ — Application Samples
All runnable sample applications and interactive playgrounds are organized under samples/:
samples/litert/: Standard samples using the LiteRT CompiledModel API. Designed for modern hardware acceleration (GPU/NPU) and asynchronous execution.- Samples: Speech Recognition, PhotoTalk, Text-to-Speech, Image Generation (text-to-image), Image Segmentation, Image Classification, Digit Classification, Qualcomm NPU acceleration (Gemma, MobileNet, Fast VLM), Google TPU sample app.
samples/litert_model_zoo/: One Android app that runs 29 vision and audio models through the LiteRT CompiledModel API, each downloaded on demand from Hugging Face.samples/litert_interpreter/: Legacy samples using the Interpreter API.- Samples: Broad compatibility examples for Android, iOS, and Python (Image Classification, Object Detection, Image Segmentation, Audio Classification).
samples/litert_lm/: High-level Engine samples for Large Language Models (LLM/SLM).samples/end_to_end/: Complete full-system pipelines (e.g. ImageNet model conversion, preprocessing, and classification).samples/tensor_api_playground/: Interactive Web/WASM playground demonstrating LiteRT Tensor API capabilities directly in the browser (Gemma 3, Image Segmentation, Mandelbrot, Game of Life).
2. models/ — Model Recipes & Export Pipelines
Contains standalone model conversion scripts, export recipes, and model-specific utilities. Many are working in process. Start with the model conversion cookbook: the step-by-step instructions for taking a Hugging Face checkpoint to a verified .litertlm bundle or .tflite graph, with the recipes as worked examples.
3. utilities/ — Shared Tools & Helper Scripts
utilities/common/: Shared Kotlin helpers for Android samples (camera pipeline, audio capture, CompiledModel runner, image/tensor and math helpers).utilities/litert_gpu_toolkit/: Pre-conversion patches that rewrite common PyTorch patterns into forms the LiteRT GPU delegate accepts, plus a post-conversion checker.
4. skills/ — Agent Automation & Skills
Custom AI agent skills that carry a model through the LiteRT deployment lifecycle, in order — see skills/README.md for the full index:
litert-conversion-workflow/: Hugging Face LLM / VLM checkpoint → verified.litertlmbundle for LiteRT-LM.gpu-clean-conversion/: PyTorch / Hugging Face model → GPU-resident LiteRT model.accuracy-safe-quantization/: Quantize (fp16 / int8 / int4) without losing accuracy.on-device-verification/: Prove the converted model on the actual device.compiled-model-app-scaffolding/: Build an Android app around the verified model.litert-compiled-model-migration/: Android TFLite → LiteRT CompiledModel V2 with NPU JIT acceleration and zero-copy buffers.
🛠️ Getting Started
Prerequisites
- Android: Android Studio (latest stable version).
- iOS: Xcode (latest version).
- Python: Python 3.9+ and
pip install ai-edge-litert. - Web / WASM: Modern browser with WebGPU / WebAssembly support.
Running a Sample
For Samples Using Compiled Model API
- Navigate to
samples/litert/<sample_name>. - Ensure you have a device with a supported NPU/GPU (e.g., modern Pixel, Samsung, or Qualcomm/MediaTek devices).
- Follow the specific setup instructions in the sample's
README.md.
For Web / Tensor API Playground
Visit "Interactive Web" tab, or open
samples/tensor_api_playground/index.html or run index.html at the repository root
via a local HTTP server.
📚 Documentation
- LiteRT Overview: ai.google.dev/edge/litert
- LiteRT-LM Overview: ai.google.dev/edge/litert-lm
- CompiledModel API Guide: LiteRT for Android
- Model Conversion: Convert models to LiteRT
- LiteRT CLI: CLI for end-to-end journey
🤝 Contributing
Contributions are welcome!
- Read CONTRIBUTING.md.
- Fork the repo and create a branch.
- Submit a Pull Request.
📄 License
Apache License 2.0. See LICENSE for details.
Disclaimer: This is a sample repository maintained by Google. It is provided "as is" without warranty of any kind.