What to know
- Install an app like MLC Chat on your Android device to experiment and chat with LLMs locally.
- Download one of the available models and tap on the ‘Chat’ icon to start chatting with your chosen LLM.
- The MLC Chat app lets you download Gemma 2b, RedPajama, Llama3, Phi-2, Mistral, and Llama2.
There are various LLMs that you can run locally on your PC. But when it comes to smartphones, the options are slightly limited. You have to run a small language model or have a premium device with enough processing power to tackle LLMs. Either way, it’s possible to chat with language models locally and offline. Here’s everything you need to know about running LLMs locally on Android.
How to run LLMs locally on Android
Large language models are, well, large, and require some heavy computational prowess. But even if your Android device has enough resources to run SLMs (small language models) as well as LLMs, you still need an app that lets you experiment with them with a user friendly interface.
This is where an app like MLC Chat comes in handy. Use the steps below to run LLMs locally on your Android using the MLC Chat app.
Step 1: Install the MLC Chat app
- MLC Chat app | Download Link
- Firstly, click on the link above and download the APK for the MLC Chat app. When prompted, tap Download anyway.
- Install the MLCChat app. Once done, Open it.
Step 2: Download an LLM
On the MLCChat app, you’ll have a list of available models that you can download. Tap on the download link next to a model you like and wait for it to download.
Step 3: Run the installed LLM
Once downloaded, tap on the chat icon next to it to start the chat. Wait for the model to initialize. Once ready, go ahead and start chatting with the AI.
Note that from the list of available models on the MLCChat app, there will be some, like Llama3, that will require lot of processing power. But you can always install the models and test to see if they’re running smoothly enough. If not, try a smaller language model.
In our testing (on Nothing Phone 1), Gemma-2b failed to load, while the Llama3 killed the System UI. The ones that worked well were Llama-2 and Phi-2. The main difference between LLMs and other smaller language models is the breadth of information that they are trained on (parameters). So choose your models according to your needs and the capabilities of your Android device.
We hope you were able to install and run LLMs on your Android device locally. Until next time!
