INFORMATION THEORY FOR LARGE LANGUAGE MODELS

About

Contemporary AI systems, particularly large language models (LLMs), have demonstrated remarkable capabilities, yet they often function as "black boxes," undermining trust, fairness, and efficiency. This workshop will explore information theory (IT) as a principled framework to advance both the capabilities and interpretability of LLMs. By integrating core IT concepts into the design and analysis of LLMs, we aim to deepen our understanding of their behavior, efficiency, factual accuracy, and inherent limitations. The workshop will bring together researchers from IT and AI to foster cross-disciplinary collaboration and catalyze innovation. Through presentations, panel discussions, and interactive sessions, participants will explore both theoretical foundations and practical applications of IT in LLMs, addressing key challenges in this rapidly evolving landscape. Our goal is to lay the foundation for more efficient, reliable, and transparent AI systems to address the most pressing challenges in this rapidly evolving field.