LiteRT, Google’s “modern” on-device inference framework evolved from TensorFlow Lite (TFLite), has introduced advanced acceleration capabilities, based on a ”next-generation GPU engine” called ML Drift.
Google said that this milestone, announced January 28, solidifies LiteRT as a universal on-device framework and represents a significant leap over its predecessor, TFLite. LiteRT delivers 1.4x faster GPU performance than TFLite, provides a unified workflow for GPU and NPU acceleration across edge platforms, supports superior cross-platform deployment for generative AI models, and offers first-class PyTorch/JAX support through seamless model conversion, Google said. The company previewed LiteRT’s new acceleration capabilities last May.
Found on GitHub, LiteRT powers apps used every day, delivering low latency and high privacy on billions of devices, Google said. Via the new ML Drift GPU engine, LiteRT supports OpenCL, OpenGL, Metal, and WebGPU, allowing developers to deploy models across, mobile, desktop, and web. For Android, LiteRT automatically prioritizes when available for peak performance, while falling back to OpenGL for broader device coverage. In addition, LiteRT provides a unified, simplified NPU deployment workflow that abstracts away low-level, vendor-specific SDKs and handles fragmentation across numerous SoC (system on chip) variants, according to Google.
