Edge AI

Edge AI runs model inference on or near the device that produces the data — phones, browsers, IoT sensors, factory gateways — instead of round-tripping to a cloud API. The wins are latency (no network hop, so sub-50ms responses are feasible), privacy (raw data never leaves the device), offline resilience, and zero per-request inference bills. The constraint is capability: edge hardware fits quantized small language models, distilled vision models, and speech models, not frontier LLMs, so accuracy trails the cloud. Apple Intelligence's on-device tier, Gemini Nano on Android, and browser inference via WebGPU and ONNX Runtime are the mainstream examples. The dominant production pattern is hybrid: handle common, latency-sensitive, or sensitive requests locally and escalate hard ones to a cloud model. For SaaS builders, edge AI is also a pricing story — shifting inference cost from your margin onto hardware the customer already owns.