Extensive applications arising from the theory of information
have been made in the obvious areas, such as television channels,
telephone lines, and computer networks. However, information theory's
applications in such frontier areas as speech processing, speech
synthesis and recognition, optical-character recognition, music
synthesis, and so on, are of even greater interest. Information
Theory for Technologists by M J Usher (see reference4) contains
a summary of these applications.
One shortcoming of information theory is its restriction to the
efficient transmission of messages without any regard for their
meaning. For instance, a choice between two equally probable messages
has an information content of 1 infobit, but the significance
of the messages may be vastly different from a human point of
view. For example, there is a vast difference in significance
between the two messages "rain tomorrow" and "no
rain tomorrow", and the two messages "earthquake tomorrow"
and "no earthquake tomorrow".
Saying that information theory deals only with the quantity of
information and not its quality may be valid criticism. However,
Shannon carefully pointed out that the theory was concerned only
with the efficient transmission of messages and not with their
semantics. Originally, the theory of probability dealt only with
the chances of alternative events. In due course, it was extended
to handle the relative values of the different events by incorporating
the ideas of profits and losses associated with the outcomes.
Similar extensions may arise as information theory is applied
more and more to the frontier areas.
The qualitative implications of information theory, the implications
of redundancy, and how these ideas have found application in areas
much wider than the transmission of information are discussed
in a fascinating book by Jeremy Campbell, Grammatical Man (see
reference 5).