FOA Home
The early days of cybernetics were heady, and Zipf was not alone in
seeking a grand, unifying theory that might explain the phenomena of
communication on computational grounds like those proving so successful
in physics. Benoit Mandelbrot was equally ambitious.
Mandelbrot's
background as a physicist is clear when he considers the message decoder
as a physical piece of apparatus, ``... cutting a continuous incoming
string of signs into groups, and recoding each group separately.'' This
``differentiator'' complements an ``integrator'' which reconstitutes new
messages from individual words. Within this model communication can be
considered ``fully analogous to the perfect gas of thermodynamics.''
Minimizing the cost of transmission corresponds to minimization of free
energy in thermodynamics.
Mandelbrot was also interested in how the
critical parameter \alpha \) varied from one vocabulary to another.
Extending the physical analogy of thermodynamic energy, the
``informational temperature'' or ``temperature of discourse'' is
proportional to \( 1/\alpha \), which Mandelbrot argues provides a much
better measure of the richness of a vocabulary than simply counting the
number of words it contains.
The value 1/\alpha \) can also be used to
relate our analysis of Zipf's Law to Mandelbrot's fractals. If the
letters of our alphabet are imagined to be digits of numbers base n + 1,
and a leading decimal point is placed before each word. Then each word
corresponds to a number between 0 and 1. \bq The construction amounts in
effect to cutting out of [0, 1] all the numbers that include the digit 0
otherwise than at the end. One finds that the remainder is a Cantor
dust, the fractal dimension of which is precisely \( 1/\alpha \) . []
(p. 346) [] (p. 346) ] (p. 346) (p. 346) . 346) 346) 46) ) \eq
Mandelbrot
proposed a more general form of Zipf's Law: F(r)={C\over{(r+b)^\alpha}}
that has proved important to analysis of the relationship between word
frequencies and their rank (cf. Section §3.2.1.1 ).
Mandelbrot also suggested
how this model might be applied within a model of cognition: Whatever
the detailed structure of the brain it recodes information many times.
The public representation through phonemes is immediately replaced by a
private one through a string of nerve impulses.... This recorded message
presumably uses fewer signs than the incoming one; therefore when a
given message reaches a higher level it will have been reduced to a
choice between a few possibilities only without the extreme redundancy
of the sounds. The last stages are ``idea'' stages, where not only the
public representation has been lost, but also the public elements of
information. (p. 488- 489) \eq He also makes other provocative
suggestions, for example that schizophrenics provide the best test of
his theory since these individuals impose fewest ``semantic
constraints'' on the random process (generating language) of interest?!
While
unsuccessful at his more ambitious goals of wedding a physical model of
communcation to models of semantics such as Saussure's, Mandelbrot was
probably the first to characterize the real truth underlying Zipfian
distributions. Before considering this derivation, one more historical
perspective, due to Herbert Simon, will be mentioned.
Top of Page
Benoit Mandelbrot's explanation