An American startup has developed what it claims is the world’s smallest artificial intelligence (AI) supercomputer. Packed with high-performance hardware and plenty of RAM, company reps say it can run “Ph.D. intelligence” AI models — despite being compact enough to fit in your pocket. This means that they are capable of autonomous problem solving, abstract reasoning and strategic planning.
“AI Pocket Lab”, as the creators of Tiiny AI have labeled the device, is capable of running a complex 120 billion parameter large language model (LLM) locally, without any dependence on internet connection. You’ll typically need data center-class infrastructure to run these systems, allowing for local expert-level coding, document review and refinement, or multi-step reasoning.
It’s built around a 12-core ARM processor, the kind commonly found in smartphones, laptops and tablets. Despite the small frame – the device measures just 5.59 × 3.15 × 1.00 inches (14.2 × 8 × 2.53 cm) – it has 80 GB of LPDDR5X RAM. Most current laptops come with between 8GB and 32GB of RAM, by comparison.
A massive 48GB of Pocket Labs RAM is also reserved exclusively for neural processing unit (NPU), a chip optimized for AI-related calculations. Both Intel and AMD have been producing processors for a few years that include dedicated NPUs to handle AI workloads and to meet Microsoft’s 40 trillion operations per second (TOPS) threshold for running AI features on Windows 11.
The Pocket Lab qualifies as a supercomputer (rather than a standard mini-PC or workstation) because of its computational power, capable of running workloads—particularly local inference on 100 billion-plus parameter language models—that would normally require multi-GPU, data center-class systems. Current models the device can drive include GPT-OSS 120B, large Phi models and high parameter Llama family models.
This is part of a recent push towards edge computing for AI, in an effort to reduce some of the power limitations and environmental impact of distributed AI processing.
Pocket power
Although it is far from competing with the world’s most powerful supercomputersAI Pocket Lab is capable of delivering 190 TOPS of computing power between NPU and CPU. It represents another step towards miniaturization in the wake of Nvidia’s recently announced Project Digits mini PC. While it doesn’t have the same horsepower as the Nvidia project, it’s a fraction of the size.
To pack so much power into such an unassuming chassis, the Tiiny AI team relied on a number of technologies and optimizations. Key among them was something the company calls TurboSparse — an innovation that allows massive LLMs to run faster on more limited hardware by ensuring that a system only calls on the parts of a model that it needs at any given time. While traditional models use each parameter for each processing/output word, a TurboSparse model uses only specific parameters per step.
Another important feature is PowerInfer, which allows for heterogeneous scheduling of the device’s CPU, GPU and NPU. This means that each processor only receives the workload it is most capable of handling, making the entire system more efficient overall and reducing power consumption. PowerInfer also includes intelligent power management, which determines when full power is needed and when it is possible to use less, in part by eliminating unnecessary calculations.
The implications of a miniature AI supercomputer go beyond reducing our reliance on environmentally damaging data centers. It’s a boon to privacy, with users able to deploy the power of a sophisticated LLM without being connected to the internet and without their data being processed in the cloud by third parties, while enabling AI access in fieldwork situations such as remote research stations, or on ships or aircraft out of connectivity range.
