Introducing Curie by colibrì: A 17 Billion Parameter Model Running Seamlessly from SSD on Standard Hardware
A few weeks ago, I embarked on an ambitious project driven by a fundamental question:
Is it really necessary to invest tens of thousands of dollars in specialized hardware to run large-scale AI models, when their weights can be stored on a modest SSD?
This curiosity led to the creation of colibrì, an innovative approach to neural network inference that challenges conventional assumptions. Traditionally, models are designed with the expectation that their weights and data stay within high-speed VRAM or RAM, especially on GPUs or data centers. However, colibrì reimagines this paradigm by considering storage devices such as SSDs, RAM, and GPUs as parts of a unified memory hierarchy. This allows even massive models—containing hundreds of billions of parameters—to run on hardware most users already possess, by intelligently orchestrating data movement across these layers.
Rethinking Model Compatibility with Modern Hardware
In the process of developing colibrì, I observed a crucial obstacle: most open-source models are inherently optimized for GPU-centric environments. Their architecture, weight placement, and execution patterns are tailored for VRAM-centric workflows. While colibrì can enable these models to operate across different memory layers, the models themselves lack awareness of the underlying hardware configuration. Essentially, they are designed to expect constant high-speed access to their entire parameter set, which isn’t feasible in a memory hierarchy involving SSDs and other storage media.
This observation prompted a new question:
What if the model and the engine were designed from scratch, with each other in mind?
This inquiry inspired the development of Curie, a language model engineered to work optimally with a memory architecture built around SSD, RAM, CPU, and GPU as a cohesive system rather than isolated components.
Introducing Curie: A 17B Parameter Model Built from Scratch
The current alpha version of Curie features approximately 17 billion parameters. Notably, I built this model entirely in C, without leveraging existing inference frameworks, training it from scratch on a standard laptop without a GPU. Its weights are stored on SSD and are dynamically loaded based on the model’s execution path, demonstrating that large models can, in principle, operate efficiently within constrained hardware environments.
Performance Highlights and Limitations
In preliminary tests, Curie exhibits promising performance. For example, generating 33 tokens per second using a single CPU core, significantly outperforming traditional dense 12B models on the same hardware, which typically produce around 3-5 tokens per second.
However, it’s important to recognize that this model is still in development. Curie has been trained on only 11 million tokens, making it unqualified for serious language generation tasks at this stage. It is not yet considered a “smart” or high-quality language model, and further training with billions of tokens will be necessary to evaluate its true capability.
The Broader Implication: Hardware-Aware Model Design
While Curie remains a work in progress, it demonstrates a compelling architectural principle: models can be designed—and models should be designed—with the full memory hierarchy in mind. Moving away from the assumption that everything must fit into GPU VRAM opens new avenues for accessible and efficient AI deployment on everyday hardware.
Next Steps
Curie is not yet publicly available. I am currently focused on further training, validation, and benchmarking to better understand its performance and potential. The overarching goal is to inspire a shift toward hardware-aware model design, unlocking the possibility of running large-scale models on hardware that most users already own.
Stay tuned for updates as I continue to refine Curie and explore the limits of on-device AI inference.
