• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
Menu

Article

Study of Optimal Text Size Phenomenon in Zipf–Mandelbrot’s Distribution on the Bases of Full and Distorted Texts. Author’s Frequency Characteristics and derivation of Hapax Legomena

Journal of Quantitative Linguistics. 2020. Vol. 27. No. 2. P. 134-158.

This paper explores word frequency patterns when considering text length, authorship
and random distortion of texts. Through a series of experiments, we determined an
optimal text size, a phenomenon that was predicted by George Zipf, which sees a
minimal discrepancy between calculated and observed frequencies. A graphic
representation allowed a plausible explanation behind the existence of this
phenomenon. Working on the assumption that distorted texts might disobey Zipf's Law,
we explored correlations among frequencies and text entirety compared to text
distortions. Results revealed the crucial role of text length for maintaining Zipfian
distribution: randomly chosen sets of words and fragmentary texts of optimal size still
obey Zipf's Law. Findings show that authorship manifests itself through the author
constant, defined as the relative frequency of the most frequent words, which remains
constant throughout the works of any given author, including randomly chosen text
chunks and fragments of sentences of various sizes.