← Blog

How to Run an LLM Locally on Android (Apps, Models and RAM)

November 17, 2026

The fastest way to run an LLM locally on Android is to install an app that downloads a small model and runs it on your phone: Google AI Edge Gallery, PocketPal AI or ChatterUI. Pick a 1B to 4B parameter model in 4-bit form (Gemma 4 E2B, Gemma 3 1B, Llama 3.2, Qwen3 or Phi-4-mini), and you can chat with it offline in under ten minutes. If you're comfortable with a terminal, llama.cpp and Ollama both install in Termux with one command.

What to expect, in short:

  • RAM decides everything. Google's own app asks for 6 GB of device RAM for 1B-class models, 8 GB for Gemma 4 E2B and 12 GB for Gemma 4 E4B.
  • Speed is usable on recent phones. Google's published numbers for Gemma 4 E2B on a Galaxy S26 Ultra are roughly 47 to 52 tokens per second. Older and midrange phones are much slower.
  • Quantization makes it possible. A 4-bit GGUF model is about a quarter the size of the full-precision one.
  • Everything stays on the phone once the model is downloaded. That's the main reason to bother.
  • Battery and heat are the real limits. Long sessions drain the battery and phones throttle when they get hot.

This guide is about phones. For running models on a laptop or desktop, see our guide to offline AI on your computer.

Quick Comparison: Android Apps for Local LLMs

OptionSkill levelModel formatGet it fromLicenseBest for
Google AI Edge GalleryBeginnerLiteRT models from Google's list, plus your ownGoogle Play, or APK from GitHubApache-2.0Trying Gemma 4 with image and audio input
PocketPal AIBeginnerAny GGUF from Hugging FaceGoogle Play, App Store or GitHub APKMITPicking your own models and benchmarking
ChatterUIIntermediateGGUF, or remote APIsAPK from GitHubAGPL-3.0Character chat, mixing local and remote
MLC Chat (MLC LLM)Intermediate to developerMLC-compiled modelsDemo APK, or build your ownApache-2.0GPU-accelerated inference
llama.cpp in TermuxAdvancedGGUFpkg install llama-cppMITFull control, a local server
Ollama in TermuxAdvancedOllama library modelspkg install ollamaMITUsing Ollama's familiar commands

Before You Start: RAM, Model Size and Quantization

Most failed attempts come from picking a model too big for the phone. Do this math first.

How much RAM you need

Google AI Edge Gallery's model list sets a minimum device RAM for each model it offers. It's the best official guide there is:

ModelDownload size in GalleryMinimum device RAM
Gemma 3 1Babout 0.6 GB6 GB
Qwen2.5 1.5Babout 1.6 GB6 GB
Gemma 4 E2Babout 2.6 GB8 GB
Gemma 3n E2Babout 3.7 GB8 GB
Gemma 4 E4Babout 3.7 GB12 GB

The rule of thumb: with 6 GB of RAM, stay at 1B to 1.5B. With 8 GB, a 2B to 3B model is comfortable. With 12 GB or more, 4B models work. Android and your other apps need memory too, so a model that technically fits can still crash or crawl. Close heavy apps before a long session.

Which models are worth trying

These are the small open models people actually run on phones, with parameter counts from each model's official page:

  • Gemma 4 E2B and E4B. Google built these "effective 2B and 4B" sizes for mobile and edge devices. The Gemma 4 model overview lists them as the only sizes with mobile memory figures, and they accept text, image and audio input. Google also publishes official 4-bit GGUF versions on Hugging Face.
  • Gemma 3 1B and Gemma 3n. The older generation, still in Gallery's list. Gemma 3 1B is the safest choice for 6 GB phones.
  • Llama 3.2 1B and 3B. Meta's small models, about 1.2B and 3.2B parameters. Good general chat; downloads require accepting Meta's license on Hugging Face.
  • Qwen3 0.6B, 1.7B and 4B. Apache-2.0 licensed. The 0.6B model runs on almost anything, though it's limited.
  • Phi-4-mini. Microsoft's 3.8B-parameter model, MIT-licensed. Strong at reasoning for its size, tight on 8 GB phones.

What quantization means

A model's weights are normally stored at 16 bits each. Quantization shrinks them to fewer bits, usually 4, at a small cost in quality. That's what lets a 3B model fit on a phone at all.

On Android you'll mostly see two formats:

  • GGUF, the llama.cpp format used by PocketPal, ChatterUI and Termux. Files are labeled by quantization, like Q4_0 or Q4_K_M. Start with a Q4 file. ChatterUI's README specifically recommends Q4_0 on newer chipsets.
  • LiteRT (.litertlm or .task files), Google's format used by AI Edge Gallery.

A quick size estimate for 4-bit GGUF: roughly 0.6 GB per billion parameters, plus a bit more for the context window. A 1B model lands under 1 GB, a 3B model around 2 GB.

Google AI Edge Gallery is Google's open-source showcase app for on-device AI. It's labeled an experimental beta, but it's the most polished option for beginners. It's on Google Play, and the GitHub releases also offer an APK for phones without Play access. It needs Android 12 or newer.

What it does:

  • AI Chat with on-device speech recognition and text-to-speech
  • Ask Image, for questions about a photo or the camera view
  • Audio Scribe, for transcription and translation
  • Prompt Lab, for testing prompts with settings like temperature
  • Agent Skills, which let the model call tools such as Wikipedia lookups
  • A built-in benchmark and a live metrics display

The app picks models from Google's list and downloads them for you. Recent releases, through version 1.0.23 in November 2026, added Gemma 4 E2B and on-device speech models. You can also load your own compatible model. Once a model is downloaded, the README says no internet is required.

Best for: first-timers, and anyone who wants to try images or voice, not just text.

The Most Flexible: PocketPal AI

PocketPal AI is an open-source, MIT-licensed app that runs GGUF models with llama.cpp. It's on Google Play and the App Store, and there's an APK on GitHub.

The killer feature is direct Hugging Face search. You can browse and download any GGUF model from the app, including gated ones like Llama if you add your access token. That means you're not limited to a curated list.

It also has:

  • Pals, custom assistants with their own model, system prompt and personality
  • A benchmark page that measures tokens per second and memory use on your phone, with an optional public leaderboard
  • Fully offline chat after download

Best for: people who want to try many models and see exactly how fast each one runs on their device.

For Character Chat and Remote APIs: ChatterUI

ChatterUI is a mobile front end with two modes. Local mode runs GGUF models on the phone through llama.cpp. Remote mode connects to backends like Ollama, koboldcpp or text-generation-webui, or APIs such as OpenAI, Claude and OpenRouter.

That second mode is handy: run a small model on the phone when you're offline, and point it at a bigger model on your home computer when you're on Wi-Fi.

ChatterUI is Android only and installs as an APK from its GitHub releases page. It's licensed under AGPL-3.0. Its roots are in character roleplay, so it has character cards and chat-editing tools that general chat apps lack.

Best for: roleplay users and anyone who wants one app for local and remote models.

For GPU Speed: MLC Chat and MLC LLM

MLC LLM is an open-source engine that compiles models to run on a phone's GPU. Its Android deployment guide links a prebuilt demo APK, built for a Snapdragon 8 Gen 2 phone. The guide says MLC needs "an actual mobile GPU to meaningfully run at an accelerated speed", and it won't run in an emulator.

The catch is flexibility. Models must be compiled for MLC, and adding new ones means editing a config file and building the app yourself. The guide's example configs estimate around 1.9 GB of memory for a 3B model and 3 GB for Gemma 2B.

Best for: developers who want to build a local LLM into their own Android app.

For Full Control: llama.cpp and Ollama in Termux

Termux is a Linux terminal for Android that doesn't need root. The Termux project recommends installing it from F-Droid or GitHub. The Google Play build is experimental and missing features. Don't mix sources, because apps signed with different keys won't work together.

llama.cpp

llama.cpp is the engine under most local model apps. Termux packages it, so installing it is one line:

pkg install llama-cpp

Then download a Q4 GGUF file and run llama-cli with it for chat, or llama-server to get a local web interface and an OpenAI-compatible API on the phone. The llama.cpp Android guide also covers building from source in Termux and cross-compiling with the Android NDK. It suggests starting with a context size around 4096, since larger values can run you out of memory.

Ollama

Ollama also has an official Termux package, maintained by the Termux team rather than by Ollama. It's 64-bit only.

pkg install ollama
ollama serve &
ollama run llama3.2:1b

Swap in gemma4:e2b or qwen3:1.7b if your phone has the RAM. Older guides that build Ollama from source with Go are outdated now that the package exists. If you use Ollama on a computer too, our comparison of LM Studio vs Ollama and our roundup of Ollama alternatives cover the desktop side.

Best for: developers, and anyone who wants a local API server on the phone for other apps or scripts.

How Fast Is It? Real Speeds

Speed is measured in tokens per second. A token is about three-quarters of a word, so 10 tokens per second reads like a fast typist, and 30 or more feels instant.

Google's published LiteRT-LM benchmarks for Gemma 4 on a Galaxy S26 Ultra (1,024 tokens in, 256 out) give a sense of the ceiling:

ModelBackendGeneration speedTime to first token
Gemma 4 E2BCPUabout 47 tokens/sec1.8 sec
Gemma 4 E2BGPUabout 52 tokens/sec0.3 sec
Gemma 4 E4BCPUabout 18 tokens/sec5.3 sec
Gemma 4 E4BGPUabout 22 tokens/sec0.8 sec

Those numbers come from the Gemma 4 E2B LiteRT-LM model card and its E4B sibling. That's a current flagship. On a midrange phone or one that's a few years old, expect a fraction of that. Doubling the model size roughly halves the speed, and the GPU mostly helps with how fast the model reads your prompt.

The honest way to know is to test your own phone. Run the benchmark in PocketPal or AI Edge Gallery with the model you plan to use.

Battery, Heat and Privacy

Battery. Running an LLM pushes the CPU or GPU hard. Short questions are fine. Long chats, big documents or image input drain the battery quickly, so plug in for long sessions.

Heat. Phones throttle when they get hot, and speed can drop noticeably after a few minutes of steady generation. A cooler phone is a faster phone. Avoid running models while charging fast or sitting in the sun.

Storage. Models are big. Three or four of them can eat 10 GB. Delete the ones you don't use.

Privacy. This is the point of the whole exercise. Once a model is downloaded, your prompts, photos and audio never leave the phone. You can check for yourself: switch on airplane mode and keep chatting. On a de-Googled phone, you can go further. GrapheneOS lets you deny an app the Network permission entirely. Our comparison of GrapheneOS vs LineageOS covers which de-Googled system suits local AI best.

What local models can't do. A 2B to 4B model is far behind the large cloud models at hard reasoning, long documents and fresh facts. It has no web access unless a tool provides it. Use it for drafts, summaries, quick answers, translation and offline help, not for anything where a wrong answer is costly.

Where Taku Fits

Running a model on your phone is a great way to learn what small models can do. Most people, though, just want AI tools that already work, without picking quantizations or reading benchmark tables. Taku doesn't replace any of the apps here. If you'd rather run AI tools someone has already built on your Mac than set up your own, that's where Taku comes in.

Taku Apps and Sites, built by the best AI minds. Find one, run it, and keep it in your Mac Dock. Taku is in Beta, and the Mac app is available now.

Key Points

  • The easiest way to run an LLM locally on Android is an app: Google AI Edge Gallery, PocketPal AI or ChatterUI.
  • Match the model to your RAM: 1B to 1.5B on 6 GB phones, 2B to 3B on 8 GB, 4B on 12 GB or more.
  • Use 4-bit quantized models (Q4 GGUF or Google's LiteRT files). They're about a quarter of full size.
  • Gemma 4 E2B runs at roughly 47 to 52 tokens per second on a current flagship. Older phones are much slower, so benchmark your own.
  • llama.cpp and Ollama install in Termux with pkg install, for full control and a local API.
  • Local models keep your data on the phone, but they drain battery, get hot, and can't match cloud models on hard tasks.

FAQ

Can I run an LLM locally on my Android phone?

Yes. Install Google AI Edge Gallery, PocketPal AI or ChatterUI, download a small model like Gemma 3 1B or Gemma 4 E2B, and chat with it offline. A phone with 6 GB of RAM can run 1B-class models, and 8 GB or more is better.

What is the best app to run local LLMs on Android?

For beginners, Google AI Edge Gallery, because it picks compatible models for you and handles images and audio. For choosing any model from Hugging Face, PocketPal AI. For roleplay or mixing local and remote models, ChatterUI.

Which LLM is best for Android?

On an 8 GB or larger phone, Gemma 4 E2B is a strong default: built for mobile and multimodal. On a 6 GB phone, try Gemma 3 1B, Llama 3.2 1B or Qwen3 1.7B. Phi-4-mini is a good reasoning model if you have the memory.

How much RAM do I need to run an LLM on Android?

Google AI Edge Gallery requires 6 GB of device RAM for 1B to 1.5B models, 8 GB for Gemma 4 E2B and 12 GB for Gemma 4 E4B. Other apps follow similar limits, since the model has to fit in memory alongside Android.

Can I run Ollama on Android?

Yes, through Termux. Install Termux from F-Droid or GitHub, run pkg install ollama, start the server with ollama serve, then run a small model like llama3.2:1b. The package is maintained by Termux, not by Ollama.

How do I run llama.cpp on Android?

In Termux, run pkg install llama-cpp, download a Q4 GGUF model, and run it with llama-cli or llama-server. You can also build it from source in Termux or cross-compile with the Android NDK, as the llama.cpp docs describe.

Do local LLMs on Android work offline?

Yes. After the model downloads, apps like AI Edge Gallery, PocketPal AI and ChatterUI in local mode run without an internet connection, so your prompts stay on the phone.

Does running an LLM drain my phone's battery?

Yes, noticeably during long sessions. Generation keeps the processor busy, and the phone heats up and may slow down. Keep sessions short or plug in.