代码库
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
C++
aiandroidarmedgeedge-aiframeworkiosllamacppllmllm-inferencellmsmobilemobile-inferenceon-device-aiquantizragsmartphonespeechtransformerwhisper
Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Python
cactusedge-aifoundation-modelfunction-callingllmon-device-aitinymltool-calling