small language models are getting really good, and very tiny. from last year, google already had been using gemma 3 1b as on-device models that even have tool-calling capabilities. apple has also launched openelm, a family of models targeting local inferencing, with models as small as 270 million parameters.
for fun, here are some tokens per second benchmarks for a few slms running on my samsung galaxy a15 (an almost 3 years old phone).
the real cost of llm use is already hitting some large companies like a truck. and i expect individual developers will soon start to feel the costs too.
i have been talking about the end of the subsidies and how to prepare for the actual costs of llms with my peers for a while now. and at the information and data team at cejam, i have helped us prepare for it in three major ways: implementing a governance gateway for cost control, providing training on llm usage, and deploying cheaper models.
my team at cejam are increasingly integrating llm agents into the work we do. we have been hitting a problem: each agent session knew nothing of the others. someone's agent solved a bug one way, mine had to figure out how to solve it again and we had no record of why decisions were made.
i needed a shared memory for our llms.
went to sesc paulista. last time i visited was years ago, soon after it first opened. the top floor now seems to be accessible only through reservations now.
this was the first time i tried connecting to wifi with this new arch installation.
took me a while, but i managed to figure out how to show a preview of a post instead of the whole post.
(actual prod code for airflow)
[
(
a_data := get_users.partial(
connection_meta=connection_meta
).expand(code=code_a)
),
(
b_data := get_users.partial(
connection_meta=connection_meta
).expand(code=code_b)
),
] >> (fetch_content := content_fetch(a_data, b_data))
last year, masp's new annex opened to the public. i had the opportunity to visit at opening week. a few weeks before that, i bought a tiny thermal camera that prints on receipt paper.
while using pandoc on arch, i got an error as i tried to convert a markdown to a pdf.
conversion snippet:
pandoc document.md -o document.pdf
microsoft's documentation for between doesn't mention inclusivity at all until section b, which suggests that between is inclusive. the information is only really explicitly stated all the way down in section d, where it states that (for datetime values, at least), between is inclusive.
however, as i have come to find out (not before a whole hour of debugging): it ... depends?
The finest qualities of our nature, like the bloom on fruits, can be preserved only by the most delicate handling. Yet we do not treat ourselves nor one another thus tenderly.
i tried using the auto builder, but either i did something wrong, or there is a bug with it.
nikola depends on the python package watchdog for the continuous auto-build.
i might try it again some other time.
checking if a task has failed on airflow is simple enough, and may be done with:
from airflow.operators.python import get_current_context
@task
def check_task_failed(task_id: str) -> bool:
context = get_current_context()
return (
context["dag_run"]
.get_task_instance(task_id=task_id)
.state == State.FAILED
)
looking at the commit history for the repository containing the source for this blog, you may notice that this was originally my personal website/cv.
my initial intention was to setup the present blog on a separate branch, but i forgot to switch branches before deleting everything.
in any case, i suppose i'll have to commit to this now. so here is a test post.