🥽 AI Smart Glasses

[!IMPORTANT]

🚧 More documentation is being added 🚧

An open-source Linux-based smart glasses platform for medical, industrial, educational, and consumer applications

中文 | English

📋 Project Overview

This is a Linux-based open-source smart glasses project in early development stage (45% documentation completeness).

The open-source scope is primarily the system software and application stack. Development is not limited to our integrated hardware kit: any RV1106B-based development board can be used as a starting point, with peripheral adaptation based on your own hardware design.

Developer Community | Contact: iam5tilllearning@foxmail.com

WeChat Group	Discord Community
Scan to join WeChat group	Join Discord

🧩 Hardware Options

You can develop this project in two ways:

• Use your own RV1106B-based development board to work on the open-source system and software stack. This option is cheaper, but bring-up can involve many issues, and you may spend considerable time on system compatibility, drivers, and related problems. If your goal is to quickly develop AI glasses interaction applications, we recommend buying our carefully tuned AI glasses device.
• Use our deeply integrated AI glasses kit if you want a faster bring-up path with pre-wired peripherals, preloaded firmware, and less hardware integration work.

Optional: Get the Official AI Glasses Kit — Recommended for faster bring-up, not required for development

Backup purchase link — Use this if the primary purchase page is unavailable

[!CAUTION] Official Hardware Compatibility Notice: The v0.6.x and v0.7.x reference firmware builds are only compatible with integrated hardware produced after January 1, 2026. If your official kit or same-reference hardware was purchased/produced before this date, please use v0.6.0 or earlier firmware versions.

✅ Current Version Features (v0.7.0)

• Firmware Version: v0.7.0
  • Compatible with new hardware produced after January 1, 2026.
• OSAIG Mobile Client: First standalone Android mobile client release for connecting, managing, and debugging OSAIG glasses.
• Device Management: Device binding, device list management, online/local connection status, device detail page, status trends, and basic device information including battery, storage, memory, and system version.
• RTSP Real-Time Video Streaming: The glasses can expose the camera feed through RTSP for the mobile client and for cloud-side video AI recognition pipelines. The reference main stream is rtsp://<device_lan_ip>:554/live/0.
• BLE Text Interaction: Adds a BLE text channel and mobile BLE debug entry for sending preset commands and text content to the glasses.
• Audio Device Handling Refactor: Improves audio input/output coordination and supports simultaneous recording and playback.
• SDK Updates: The SDK documentation now covers recording control, AI-Core physical action control/query, media resource arbitration, display submission, text event listening, BLE text send/receive, and updated example build entries.
• Release Assets:
  • Firmware_RV1106B_RK962_IMX219.AudioVersion.img
  • Firmware_RV1106B_RK962_IMX219.DisplayVersion.img
  • osaig-0.3.0+7-release.apk
• Release Details: https://github.com/Iam5tillLearning/OpenSource-Ai-Glasses/releases/tag/v0.7.0

🔍 Hardware Layout

✨ Key Features

• 🖥️ Display: 30°FOV 640×480 monocular display (optional)
• 📸 Camera: 1080P video recording and RTSP real-time video streaming
• 🔊 Audio: microphone + speaker system
• 📡 Connectivity: WiFi 802.11b/g/n, Bluetooth 5.3, USB 2.0
• ⚡ Performance: Single Cortex-A7 core, 8GB storage
• 🔋 Battery: 180mAh, 2hr music, 3hr display, 45min recording
• ⚖️ Lightweight: Only 43g
• 🧠 Sensors: Optional geomagnetic sensor, IMU
• 🐧 OS: Embedded Linux-based system

🚀 Quick Start

💡 Note: If you purchased the official AI glasses kit linked above, it comes with pre-installed firmware and is ready to use. If you are developing on your own RV1106B board, start from the host setup and flashing guide below.

Using Native Host Environment (Recommended)

We recommend developing directly on Ubuntu/Debian hosts, or Ubuntu/Debian running in WSL2 on Windows.

Single source for development environment: https://github.com/makevary/AIGLASS_DEV_ENV

# Get development environment
git clone https://github.com/makevary/AIGLASS_DEV_ENV.git
cd AIGLASS_DEV_ENV

# Fix/install build dependencies (Debian/Ubuntu and WSL2 Debian/Ubuntu)
./setup_build_env.sh

# Build firmware (choose one command only)
./build.sh
./build.sh --without-display

# Firmware output
ls -lh output/image/update.img

Build option selection:

• Device with display capability: ./build.sh
• Device without display capability: ./build.sh --without-display

Note: setup_build_env.sh relies on apt-get, so it supports Debian/Ubuntu systems (including Ubuntu/Debian in WSL2).

See Development Environment Setup Guide for details.

Firmware Flashing: After compilation, please refer to Firmware Flashing Guide to flash the firmware to your device.

Application Development: The host environment provides a complete development workflow for user-level applications. See Application Development Guide for details.

📖 Usage Guide

1. Network Configuration

• Automatic: The device automatically enters network configuration mode on boot.
• Manual: At any time, continuously short press the button 10 times to manually enter network configuration mode.
• Current method: Use an Android phone to share a Wi-Fi QR code, then let the glasses scan the QR code to connect.
• Band limitation: Only 2.4GHz Wi-Fi is supported. 5GHz/5G Wi-Fi is not supported.
• App status: The standalone OSAIG Android mobile client is available in the v0.7.0 release assets. It currently focuses on device binding, management, debugging, update guidance, BLE debugging, and real-time video viewing.

2. RTSP Video Streaming for Cloud AI

• Main stream: rtsp://<device_lan_ip>:554/live/0
• Sub stream: rtsp://<device_lan_ip>:554/live/1
• Typical use: cloud-side video AI recognition, remote inspection, visual debugging, and mobile live preview.
• Guide: RTSP Video Streaming Guide

3. AI Conversation

• Operation: After configuration, long press the left temple button to speak, release to send, and wait for the AI response.

📦 SDK Development

This project provides a complete C/C++ SDK, allowing developers to easily access underlying hardware capabilities and build their own applications.

SDK Core Features:

• GPIO Event Subscription: Low-latency access to button presses and other GPIO events
• Camera Access: Zero-copy image data retrieval via shared memory
• Audio Control: audio playback, TTS, recording start/stop/status, and improved recording/playback coexistence
• Physical Action Control: enable, disable, and query AI-Core physical-button business actions while keeping GPIO event delivery available
• Media Resource Arbitration: request AI-Core to release or reclaim camera/audio resources for external applications
• Display Submission: shared-memory framebuffer submission and display focus management
• Text Event Listening: subscribe to ASR, LLM, and system text events
• BLE Text Channel: send and receive UTF-8 JSON text messages through the glasses BLE gateway
• Inter-Process Communication: Reliable communication based on Unix Domain Sockets

v0.7.0 SDK Documentation Updates:

• Added BLE text client API documentation and the ai_ble.h header entry.
• Added recording control APIs: ai_audio_record_start(), ai_audio_record_stop(), and ai_audio_record_get_status().
• Added physical action control/query APIs: ai_audio_set_disable_aicore_physical_actions() and ai_audio_get_disable_aicore_physical_actions().
• Added media resource arbitration APIs for camera/audio release, resume, and status query.
• Added query-only physical action example to avoid changing runtime state during field checks.
• Updated examples to link against the prebuilt SDK libraries and use the unified build output directory.

SDK Location: SDK/ai_glass_sdk

Resource Navigation:

• 📖 SDK Documentation - Detailed SDK instructions and usage
• 📚 API Reference - Detailed API documentation
• 💡 Examples - Complete example code for GPIO, camera, audio, etc.

Integration Guide: Please refer to the "Integration" section in the SDK README to learn how to link and use the SDK in your application.

🏗️ System Architecture

📚 Documentation

• 📖 User Manual - Read this after glasses assembly
• 🧰 Development Environment Setup Guide | 中文
• 💻 Application Development Guide | 中文
• 📡 RTSP Video Streaming Guide | 中文
• ⚡ Firmware Flashing Guide | 中文

🛠️ Development

Build from Source

# Get the full development workspace
git clone https://github.com/makevary/AIGLASS_DEV_ENV.git
cd AIGLASS_DEV_ENV

# Install/fix build dependencies
./setup_build_env.sh

# Build firmware
./build.sh

# Output image
ls -lh output/image/update.img

Development Tools

• IDE: VS Code with C/C++ extension
• Debugger: GDB + OpenOCD
• Profiler: perf, valgrind
• Version Control: Git

API Overview

#include "ai_glasses_api.h"

// Initialize device
int device_init(device_config_t *config);

// Capture photo
int capture_photo(const char *filename);

// Display text
int display_text(const char *text, int x, int y);

// Play audio
int play_audio(const char *filename);

// Get sensor data
int get_sensor_data(sensor_data_t *data);

🎯 Use Cases

🏥 Medical Applications

🏭 Industrial Applications

🎓 Educational Applications

🤝 Contributing

We welcome contributions!

How to Contribute

[!IMPORTANT] Development environment single source: obtain it from https://github.com/makevary/AIGLASS_DEV_ENV.

Clone Directory Requirement: If you need to compile code in src or samples, clone this project under the AIGLASS_DEV_ENV workspace root to satisfy build-time relative path dependencies.

Correct directory structure example:

/path/to/AIGLASS_DEV_ENV/
├── OpenSource-Ai-Glasses/    # This project
├── luckfox-pico/             # System SDK
└── ...

1. 🍴 Fork the repository
2. 📥 Clone your fork locally (run under the AIGLASS_DEV_ENV workspace root), and initialize submodules

   cd /path/to/AIGLASS_DEV_ENV  # Navigate to the development workspace root
   git clone https://github.com/YOUR_USERNAME/OpenSource-Ai-Glasses.git
   cd OpenSource-Ai-Glasses
   git submodule update --init --recursive

3. 🌿 Create a feature branch (git checkout -b feature/AmazingFeature)
4. 💻 Commit your changes (git commit -m 'Add some AmazingFeature')
5. 📤 Push to the branch (git push origin feature/AmazingFeature)
6. 🔃 Create a Pull Request

Development Areas

• 🐛 Bug Fixes: Report and fix issues
• ✨ New Features: Propose and implement new capabilities
• 📚 Documentation: Improve guides and API docs
• 🧪 Testing: Add tests and improve coverage
• 🌐 Internationalization: Add language support

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🙏 Acknowledgments

• Linux Foundation for Linux OS support
• OpenCV for computer vision capabilities
• BlueZ for Bluetooth protocol stack
• Community contributors and testers

📞 Contact

• Project Maintainer: Iam5tilllearning
• Issues & Bugs: GitHub Issues
• Discussions: GitHub Discussions

🌟 Star History

Last Updated: 2026-05-18 | Version: v0.7.0