Hello, everyone! I wanted to share my experience of successfully running LLaMA on an Android device. The model that performed the best for me was llama3.2:1b on a mid-range phone with around 8 GB of RAM. I was also able to get it up and running on a lower-end phone with 4 GB RAM. However, I also tested several other models that worked quite well, including qwen2.5:0.5b , qwen2.5:1.5b , qwen2.5:3b , smallthinker , tinyllama , deepseek-r1:1.5b , and gemma2:2b. I hope this helps anyone looking to experiment with these models on mobile devices!
Step 1: Install Termux
- Download and install Termux from the Google Play Store or F-Droid
Step 2: Set Up proot-distro and Install Debian
-
Open Termux and update the package list:
pkg update && pkg upgrade -
Install proot-distro
pkg install proot-distro -
Install Debian using proot-distro:
proot-distro install debian -
Log in to the Debian environment:
proot-distro login debianYou will need to log-in every time you want to run Ollama. You will need to repeat this step and all the steps below every time you want to run a model (excluding step 3 and the first half of step 4).
Step 3: Install Dependencies
-
Update the package list in Debian:
apt update && apt upgrade -
Install curl:
apt install curl
Step 4: Install Ollama
-
Run the following command to download and install Ollama:
curl -fsSL https://ollama.com/install.sh | sh -
Start the Ollama server:
ollama serve &After you run this command, do ctrl + c and the server will continue to run in the background.
Step 5: Download and run the Llama3.2:1B Model
- Use the following command to download the Llama3.2:1B model:
This step fetches and runs the lightweight 1-billion-parameter version of the Llama 3.2 model .ollama run llama3.2:1b
Running LLaMA and other similar models on Android devices is definitely achievable, even with mid-range hardware. The performance varies depending on the model size and your device’s specifications, but with some experimentation, you can find a setup that works well for your needs. I’ll make sure to keep this post updated if there are any new developments or additional tips that could help improve the experience. If you have any questions or suggestions, feel free to share them below!
– llama
I’ve run a local LLM on my PC for a while, so I’m familiar enough with Ollama to understand what’s going on. I’ve tried this with my Samsung Tracfone, not really expecting a lot. Surprisingly I’ve gotten all the way to getting a prompt, but then things crash and I’m kicked back to the starting terminal. Pretty sure it’s memory, so I’m now trying to use virtual memory to bump it up to the 4GB you’ve had success with (the phone looks to have 3GB actual memory, plenty of storage though).
If it doesn’t work, I’ll try some of the others, perhaps they’re a bit smaller.
I did get the 0.5 Qwen to run well. I’m surprised how fast it is even using CPU mode, but maybe being smaller also helps with the processing.
Just a tip (maybe obvious to experienced users): while you do have to run the terminal, login to debian, start the server and then run the model, remember that you can use the arrow keys in the terminal to repeat past commands, so it’s pretty quick to do. I actually missed the arrow keys the first time around because they aren’t very distinct or highlighted, but then when I had to look for how to do CTRL, I realized they were right in front of me.
I have tried on more or less 5 spare phones. None of them have less than 4 GB of RAM, however.