MIT’s LARS Cuts Memory Needed to Fine-Tune LLMs on Small Devices

Smartwatch and small computer board on a desk linked by a stream of compressed glowing data layers
AI-generated illustration

Running an AI model on a phone or a fitness watch sounds simple until you try to teach it something new. Training on the device itself, rather than in the cloud, can eat gigabytes of memory in moments, and the usual result is an out-of-memory error. A team at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), working with a Harvard researcher, says it has a way to ease that squeeze. The method is called LARS, and it works by keeping the model’s training “notes” short.

The problem: memory peaks, not parameter counts

According to MIT CSAIL’s announcement on November 19, 2026, engineers have long used Parameter-Efficient Fine-Tuning (PEFT) to make training lighter. PEFT changes only a fraction of a model’s parameters, on the assumption that fewer updates mean less memory use. The CSAIL and Harvard researchers argue that this assumption misses something.

While a large language model (LLM) learns from new data, it keeps an internal record of its calculations, a step the team calls “intermediate activation storage.” That record is where memory use peaks sharply. Trimming the number of parameters does not shrink it, so a device with a hard memory ceiling can still run out of room.

What LARS does differently

The new approach is named “Low-memory Activation Rank Subspace,” or LARS. Instead of stashing every calculation, it pools those details into a more concise form. Irene Tenison, an MIT electrical engineering and computer science PhD student and CSAIL researcher, compared it to a lecture: rather than transcribing everything you heard, you recap it in a couple of sentences.

The team reports that, compared with top PEFT techniques, LARS reduces an LLM’s memory footprint by over 23% on GPUs and about 47% on CPUs. Those are the researchers’ figures from their own tests, and they come with a trade-off, covered below.

Results on small hardware

To see what the savings look like in practice, the researchers tried LARS on a Raspberry Pi 5, a small edge computing device with an 8 GB limit. They fine-tuned an LLM with LARS and with LoRA, a leading PEFT approach. At a context length of 128,000, LARS saved 9% more memory, and the savings grew as the context got longer.

On chips with more memory available, LARS saved about 10 GB more than its strongest baseline on high-performance NVIDIA GPUs. On less powerful AMD CPUs it saved twice as much memory. The method performed most competitively on long-context prompts, where the model had to summarize detailed information.

There is a catch. When the team asked LLMs to answer more detailed questions on chips with more memory, LARS was slightly less accurate than LoRA. In other words, the approach gives up a little accuracy in some settings in exchange for a much smaller memory bill.

Energy savings and compatibility

Lower memory use also affected energy. The team ran short, intermediate and long prompts on a CPU and found that LARS used roughly 22% less power and energy than comparable baselines. “The data we’ve collected so far shows LARS may be a more sustainable LLM fine-tuning approach,” Tenison said.

LARS is also designed to stack with other tricks. Tools that optimize attention mechanisms, freeze certain layers or quantize a model are compatible with it, because those tools do not address a model’s intermediate, hidden calculations. Tenison said the approaches are complementary and that using them together should save even more energy.

What it means

The researchers point to several possible uses. A fitness watch could one day analyze changes in heart rate over a few minutes directly on the wrist. A factory could track different units in real time without sending all of its data to the cloud. A robot could process its surroundings locally. In each case, the appeal is the same: a model that adapts where the data is created, which can mean faster responses and less data leaving the device.

It is worth keeping expectations measured. These are research results presented at a conference, and the reported gains depend on the hardware and the task. The slight accuracy trade-off may matter for some uses and not for others. Still, the core insight is useful on its own: if you want smaller AI training footprints, look at the activations as well as the parameters.

The research team and paper

The project was introduced in a paper, “Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspace”, presented at the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) in November. Tenison wrote it with CSAIL colleagues Stella Ahn, visiting researcher and Harvard student Miriam Kim, and Broad Institute researcher Ebtisam Alshehri. Lalana Kagal, a Principal Research Scientist at the lab, is senior author.

Conclusion

LARS reframes a familiar problem: the memory that limits on-device AI training comes from the model’s running notes as much as from its parameters. By compressing those notes, the CSAIL team reports meaningful savings in memory and energy on small devices, with a small accuracy cost in some tests. If the approach holds up beyond the lab, it could make personalized, on-device AI more practical.

Sources