small language models are getting really good, and very tiny. from last year, google already had been using gemma 3 1b as on-device models that even have tool-calling capabilities. apple has also launched openelm, a family of models targeting local inferencing, with models as small as 270 million parameters.

for fun, here are some tokens per second benchmarks for a few slms running on my samsung galaxy a15 (an almost 3 years old phone).

setup

termux is an "android terminal emulator and linux environment app". it can be installed through f-droid or from the realeases on github. i do not recommed the playstore version, as it is unmaintained.

termux uses pkg as its package distributor. we can install ollama with:

pkg i ollama

with ollama, we can easily run a model with:

ollama run [model]:[parameters]

for example, running the 0.8b qwen 3.5 is just calling:

ollama run qwen3.5:0.8b

results

i only asked the models what is the capital of france?.

model	size	avg tokens per second	tokens	reasoning time
qwen 3.5	0.8b	5.5	1340	5 minutes
lfm2.5	1.2b	4	260	1 minutes
gemma3	1b	7	30	non-reasoning
gemma3	270m	10	30	non-reasoning

of course, this simple question isn't necessarily a good benchmark for actual practical use. qwen 3.5 0.8B, for instance, is great at tool calling. it is impressive that it can run on my 5 year old galaxy book at all, and can do basic calls (read, edit, write) on python scripts.

the gemma3 models are non-reasoning, and because of that they answer very fast. the 270m model in particular gets to 11 t/s at times. of course, it is the least capable at well in terms of encyclopedic knowledge. but i do like how straight to the point it is. unlike gemma3 1b, the smaller cousin does not use emojis at all. the smaller models are mostly useful as cognitive cores that pull information from elsewhere and call tools.