Compression is the Key
Taking a step back for a minute, and looking at our current time and space in the information systems and data industry (circa March 2024), I find a “silent thought” [1] hidden in plain sight - a signal amidst the noise.
Compression - of data, information and knowledge, is key to what is happening in our world with artificial intelligence (AI), machine learning (ML) and our current infatuation with large language models (LLM). Claude Shannon, attributed with establishing the concept of information theory [2] has never seemed more validated.
It’s not just me, have a look:
- An Elegant Equivalence Between LLMs and data compression
- Large Language Models as Data Compression Engines
- Language modelling is compression
- Semantic Compression with Large Language Models
- …and many more..
If you look at the history of mankind, information science, statistics, even mathematics, and for that...