How Compressed LLMs Retain Knowledge: Insights from an Experimental Study

Large Language Models (LLMs) like GPT-4 and ChatGPT have introduced a new era of capabilities in natural language processing, facilitating tasks from simple text generation to complex reasoning. Yet, these advancements come with the substantial challenge of enormous model sizes, which increase computational costs and limit accessibility. In response, several model compression techniques, including quantization and sparsification, have been developed to reduce the size and resource demands of these models while maintaining their performance. Despite their promise, a critical concern has emerged: compressed LLMs may underperform on knowledge-intensive tasks, potentially “forgetting” the knowledge they once held.

The recent research study, “Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications,” addresses these concerns by questioning whether compressed LLMs indeed “forget” knowledge or if the knowledge is simply “displaced” within the model structure. Clarifying this fundamental issue has profound implications for the future use of LLMs.

Understanding Knowledge Retention in Compressed Models

Focusing on this critical dilemma, the study explores two primary hypotheses to elucidate the observed degradation of LLM performance post-compression:

• Knowledge Forgotten: This hypothesis likens knowledge loss to erasure, suggesting that compression permanently removes information, necessitating relearning through additional parameters to recover lost capabilities.
• Knowledge Displaced: Contrary to the first, this hypothesis posits that knowledge remains within the model but is relocated, causing established inference pathways to become ineffective. This scenario implies that performance might be regained via strategic input augmentation, such as prompting, to redirect and engage the displaced knowledge.

To validate these hypotheses, the authors utilized two well-known LLMs—Llama-7b and OPT-6.7b—employing various compression techniques, including GPTQ for quantization and SparseGPT for pruning. They particularly addressed post-training compression, a method suitable for very large models where comprehensive retraining is computationally prohibitive.

Innovations in Prompting as Recovery Solutions

Evaluation of compressed LLMs after fine-tuning involved three conventional methods—prompt-tuning, prefix-tuning, and LoRA—and introduced a novel approach: Inference-time Dynamic Prompting (IDP). Designed to overcome traditional prompting limitations, IDP automatically selects prompts based on input specifications and knowledge domains, aligning with input formats through one-shot selection. Critically, IDP dramatically outperforms previous techniques in several respects:

• Prompting as a Catalyst for Recovery: Experiments demonstrated significant recovery in LLM performance through prompt-based interventions, validating the “knowledge displaced” hypothesis. IDP, in particular, matched or exceeded results from parameter-heavy options like LoRA with reduced computational demands.
• Task-Specific Performance Regimes: The research categorized tasks into performance regimes. Simple input re-direction via prompting sufficed for factual knowledge recovery, whereas nuanced language-dense tasks benefited more from additional parameters as in LoRA.
• Efficiency of IDP: The IDP system highlighted a reduction up to 20-fold in parameter size vs. LoRA, alongside comparable or better performance across tasks, showcasing its robustness and efficiency for broader LLM applications.

Implications and Future Prospects

The compelling performance of IDP underscores the need to reconsider how knowledge is structured within LLMs post-compression. Attention and activation pattern analysis revealed meaningful distinctions in how prompted models differ from their baselines, reinforcing the notion of knowledge displacement rather than knowledge obliteration.

A broader examination of the research reveals intriguing commercial implications, particularly for tech giants like Apple—identified as premier contributors to this research alongside OpenAI—and solidifies their initiatives to advance LLM compression and application optimization.

Embracing Knowledge Redirection for Compressed LLMs

This study elucidates the crucial role of prompting, particularly IDP, in the efficient recovery of knowledge, potentially transforming compressed LLM deployment in various fields by reducing parameters and latency while maintaining efficacy. The insights gained mark a significant leap in making LLMs more accessible and functionally adaptable, heralding new avenues for research in facilitating LLM integration into real-world applications.

“It is clear that both LoRA and IDP improve the performance of compressed models. However, the patterns of attention and activation show that IDP is more effective in redirecting the latent knowledge within the model, while LoRA relies on adding new knowledge to the model. This suggests that IDP is a more efficient and effective method for recovering performance in compressed models.” – Duc Hoang, Minsik Cho, Thomas Merth, Mohammad Rastegari, Zhangyang Wang

Overall, this research provides vital insights into the intricate dynamics of LLM compression and knowledge retention, advocating intelligent prompting as an effective strategy for addressing knowledge recovery. This fills a crucial gap in the deployment and optimization of compressed LLMs, shaping a more proficient and economically viable future for utilizing these powerful tools.

For more detailed information, the full research can be found at Arxiv.