Ollama models not loading after update (dual RTX 3060)

19 Oct 2025 - tsp
Last update 23 Oct 2025
Reading time 4 mins

TL;DR

On a box with two NVIDIA GeForce RTX 3060 (12 GB) GPUs, Ollama auto-updated to 0.12.4 and model loads silently stopped working, upgrading to 0.12.5 did not solve the problem. Downgrading to Ollama 0.12.3 fixed it immediately. NVIDIA driver upgrade from 572.82 to 581.29 did not change the behavior (other CUDA apps were fine)

When Ollama auto-updated on my small dual RTX 3060 headless setup, model loading suddenly stopped working - no errors, just silence and hanging clients. The GPUs were detected, yet nothing generated and loading of tensors failed from one day to the other. Upgrading drivers and even moving from version 0.12.4 to 0.12.5 did not help, while other CUDA applications ran perfectly.

After a few hours of debugging, the fix turned out to be simple: rolling back to Ollama 0.12.3 instantly restored normal behavior. If you are seeing lines like llama_model_load: vocab only - skipping tensors and key with type not found ... general.alignment, this post walks through what happened and how to get your models running again without tearing your hair out.

Setup

Symptoms and what I saw in the logs

Server was up and GPUs detected:

Listening on [::]:8182 (version 0.12.5)
inference compute ... CUDA0 / CUDA1 ... NVIDIA GeForce RTX 3060 ... total="12.0 GiB"

When a clients hit /api/show (metadata probe), logs looked normal but gave the impression models don’t load:

POST "/api/show"
llama_model_load: vocab only - skipping tensors

This is expected for /api/show, but in my case real generations never kicked in either.

Extra noise that wasn’t fatal but confused debugging:

key with type not found key=general.alignment default=32

and many lines like:

load: control token: 128... '<|reserved_special_token_...|>' is not marked as EOG

What I changed (that did not fix it)

Tried a driver update from 572.82 to 581.29. This yielded no change (at least OpenCL and CUDA in other apps remained OK).

Root cause (practical)

I didn’t dig deeper into whether it’s scheduling, multi-GPU runner selection, or a llama.cpp interface edge case in 0.12.4/0.12.5 - the point here is the quick fix.

Fix

Downgrade Ollama to 0.12.3. The models load and generate again.

Unfortunately on some platforms like Windows the system tray application of ollama will perform auto-updates. As it appears from various bug reports, even though the feature to disable them is desired by many people for different reasons (prevent breaking upgrades, prevent fetching huge amounts of data over metered connections, not unknowingly pull new code into your network, etc.) the developers actually do not intend to allow disabling of auto update. A workaround is completly getting rid of the system tray app.exe manually or supressing access to Internet by this application. On Windows this can be done in the powershell when executes as administrator using the following command also mentioned in the referenced bug report:

New-NetFirewallRule -DisplayName "Block Ollama App (disable autoupdate)" -Direction Outbound -Program "$env:LOCALAPPDATA\Programs\Ollama\ollama app.exe" -Action Block

On other applications you can perform similar blocks, block on DNS level - though this will also prevent fetching of models - or remove the tray application.

Notes that might help your debugging

Hitting /api/show only opens the GGUF header and tokenizer. It prints:

llama_model_load: vocab only - skipping tensors

This is expected behaviour. You need /api/generate or /api/chat to actually allocate and load tensors into VRAM.

The general.alignment default=32 line is a benign warning from newer llama.cpp when an optional GGUF key is absent.

Conclusion

If your dual-GPU RTX 3060 rig suddenly stops loading models after Ollama auto-updates and you see lines like:

llama_model_load: vocab only - skipping tensors
key with type not found key=general.alignment default=32

try rolling back to Ollama 0.12.3 first. That immediately restored normal model loading and inference for me. This may solve the immediate headache before trying to get resolved how to upgrade without breaking everything.

Note: These links are Amazon affiliate links, this pages author profits from qualified purchases

This article is tagged:

Related articles

Running Ollama on a CPU-Only Headless FreeBSD 14 System

This mini guide walks through compiling and running Ollama on a headless, CPU-only FreeBSD 14.2 system with an Intel Alder Lake-N97 processor. It covers why one might opt for CPU-based inference, the Vulkan-related issue preventing execution without a GUI, and how to disable Vulkan by recompiling from ports. The article provides step-by-step instructions for extracting, modifying, compiling, and installing Ollama, along with details on running inference and transferring models to air-gapped systems.

Coding with an AI Assistant: My Ongoing Journey into Vibe Coding

Pair programming with an AI might sound futuristic or flaky - but what if it actually works? In this short article (that evolves over time) I reflect on my personal evolving experience with vibe coding: a collaborative, conversational approach to software development using large language models. From architectural planning and boilerplate generation to debugging rituals and code reviews the article reveals how these tools can be both uncanny helpers and frustrating novices - often within the same session. This is not another contribution to the hype. Instead, you'll find some practical advice, mentions of the cost involved, tool recommendations and cautionary tales drawn from real-world projects. The article invites you to explore what it's like to co-create software with a machine that remembers nothing, hallucinates confidently, and - if you guide it well - helps you build better, faster, and with fewer headaches.

Setting Parameters like Context Length and Temperature in Ollama Models

This article provides a concise, practical overview of modifying Ollama models by editing or creating a MODELFILE. It explains how to adjust fundamental parameters such as num_ctx (context length), temperature, and sampling settings, as well as how to modify prompts, templates, and integrate LoRA adapters. The focus is on concrete configuration steps rather than retraining, making it a straightforward reference for those who need to adapt model behavior for specific workloads. The examples demonstrate increasing the context length for code-oriented models, altering generation variability, and attaching domain-specific LoRA modules. The discussion also addresses the technical implications of these changes, including VRAM requirements, inference speed, and the limitations imposed by a model’s original training context.

Expanding GPU Capabilities on Notebooks and Mini PCs Without PCIe Slots via M.2 NVMe Slots

Ever wondered if your compact notebook or mini PC could power serious AI, LLMs, or scientific simulations? In this guide, we show you how to retrofit systems without PCIe slots using M.2 NVMe adapters and GPU expansion boards—enabling multi-GPU setups even on budget hardware. While not a full desktop replacement, this approach offers surprising performance for specific workloads. Perfect for tinkerers, researchers, and hardware hackers.

mini-apigw: A Lightweight Gateway for Multi-Model AI Infrastructure

mini-apigw is a lightweight, locally controlled OpenAI-compatible API gateway that unifies multiple AI backends such as OpenAI, Anthropic, Ollama, vLLM, and Fooocus under one clean endpoint. It restores sanity in a world where every AI service assumes exclusive control of your GPU by adding arbitration, per-app governance, and transparent configuration through simple JSON files - no dashboards, no Kubernetes, no bloat. Designed for small labs, research environments, and hobby clusters, mini-apigw behaves like an old-school UNIX daemon: minimal, reliable, and easy to reason about. It lets you run modern LLMs and image models side by side with full control over access, security, and resources - all while keeping your infrastructure as lean and transparent as possible and using just a single API.

Getting started with STM32F401RE Nucleo starter board on FreeBSD

How to get started with the STM32F401RE nucelo development board on FreeBSD with gcc and openocd

Repairing corrupted RabbitMQ instance (VHost / experienced error, not a dets file)

Mini post about an unusual error when using RabbitMQ - or mnesia in general. This is something that shouldn't happen and was also not expected to happen.

Frama-C with wp-dynamic and function pointers

How to use Frama-C static analysis and proof assitant with ACSL annotations for function pointers in ANSI C.

Also on this blog

Getting started with WSGI and Python as well as uwsgi application server

A simple summary of hat WSGI can do for one, how to use it with Python and how to run the uwsgi application server (and how to solve the deployment problem there) from the viewpoint of someone who already developed web applications in many different languages and frameworks.

Some Gaussian integrals in 1 dimension

Short summary of various commonly used Gaussian integrals (1 dimensional) most of the time not shown in school textbooks.

How to use Frama-C to proof correctness of AVR microcontroller code

This is a short article that shows how one can use Frama-C and the WP plugin to proof correctness or some properties of ANSI C code written for AVR microcontrollers.

Recovering MySQL or MariaDB database via dump and restore

This blog post summarizes the steps for disaster recovery on MariaDB or MySQL in case the database system doesn't start up any more or contains damaged table definitions which might even prevent one from dropping tablespaces or databases.

Data protection policy

Dipl.-Ing. Thomas Spielauer, Wien (webcomplains389t48957@tspi.at)

This webpage is also available via TOR at http://rh6v563nt2dnxd5h2vhhqkudmyvjaevgiv77c62xflas52d5omtkxuid.onion/

Valid HTML 4.01 Strict Powered by FreeBSD IPv6 support