100 years ago, on April 30th, 1916, Claude Elwood Shannon was born. While he does not have the name recognition of Albert Einstein, Jean-Baptist Fourier, James Clerk Maxwell or Marie Curie, he may have done more to define our modern digital age than any other scientist or engineer.

In 1948, Shannon published his groundbreaking work, “A Mathematical Theory of Communication” and for the first time engineers were able to quantify information.

Shannon identified two fundamental limits: how many bits were required to represent a signal noiselessly, and how much information could be transmitted over a medium without error. With this knowledge in hand, the world of modern communication was realized and the field of information theory was born.

Information Theory at USC

The Ming Hsieh Department of Electrical and Computer Engineering has always been a leader in the field of information theory. The highest accolade given by the IEEE Information Theory Society is the Claude E. Shannon Award – won by four distinguished USC professors: Irving Reed (1982), Andrew and Erna Viterbi Chair in Communications, Solomon Golomb (1985), Lloyd Welch (2003), and Presidential Chair Professor in Electrical Engineering, Andrew Viterbi (1991).

In fact, the picture above of Claude and Betty Shannon, Golomb and Bob Scholtz was taken by Scholtz’s wife, Lolly, at a reception organized by Golomb and his wife Bo at the 1985 International Symposium on Information Theory.

But USC’s leadership in the field did not end there. With the proposal of “Turbo” coding in 1993, a new era of coding emerged and again our engineers were on the front lines. The primary engine for the Turbo decoder was the forward-backward algorithm, invented by Prof. Lloyd Welch.

Soon, these modern codes were adopted in 3G digital cellular standards and variations in Wi-Fi and digital satellite broadcast. USC professors Peter Beerel and Keith Chugg developed a decoding architecture that provided an exponential speed-up to the forward-backward algorithm.

The Future of Information Theory

Information Theory is playing a role in defining the limits of compression and communication in ways that seem like science fiction.  Many USC faculty (Todd Brun, Daniel Lidar, Ben Reichardt) are pioneers in the new field of quantum information theory. Dr. Brun explains that quantum phenomena such as entanglement, quantum channels, and shared quantum states combined with classical information theory could enable “quantum teleportation, quantum key distribution, quantum authentication, and quantum communication.’’

Closer to current technological innovations, Prof. Salman Avestimehr notes, “I view the impact of Shannon’s information theory in revealing untapped resources that can tremendously improve the design of our information infrastructure. For example, we have recently discovered that the structure of computation assignments can be exploited to eliminate the communication bottleneck in distributed computing systems (e.g., Amazon’s server clusters).’’

As with all groundbreaking theories, Shannon’s work has greatly impacted arenas beyond digital communications and engineering.  While early applications included linguistics, economics and space exploration, today USC professors are applying his work to biological and natural systems.

Dean’s Professor Urbashi Mitra is the inaugural editor-in-chief for the IEEE Transactions on Molecular, Biological and Multi-scale Communications — a journal devoted to the interface of information technologies and biological systems.  Mitra observes, “new results on the information theory of biological systems  might provide us with clues as to how to solve significant problems such as the design of microbial fuel cells or even early detection of cancer.”

Claude Shannon, was a consummate scholar and tinkerer.  He famously created a chess playing computer, a robotic mouse which could navigate a maze, juggling robots, and of course, his “ultimate machine.”  Much of his work was well ahead of the times and his contributions continue to inspire and empower new generations of engineers.

Published on September 20th, 2016Last updated on August 2nd, 2023