Long-Range Memory in Literary Texts: On the Universal Clustering of the Rare Words

PLoS One. 2016 Nov 28;11(11):e0164658. doi: 10.1371/journal.pone.0164658. eCollection 2016.

Abstract

A fundamental problem in linguistics is how literary texts can be quantified mathematically. It is well known that the frequency of a (rare) word in a text is roughly inverse proportional to its rank (Zipf's law). Here we address the complementary question, if also the rhythm of the text, characterized by the arrangement of the rare words in the text, can be quantified mathematically in a similar basic way. To this end, we consider representative classic single-authored texts from England/Ireland, France, Germany, China, and Japan. In each text, we classify each word by its rank. We focus on the rare words with ranks above some threshold Q and study the lengths of the (return) intervals between them. We find that for all texts considered, the probability SQ(r) that the length of an interval exceeds r, follows a perfect Weibull-function, SQ(r) = exp(-b(β)rβ), with β around 0.7. The return intervals themselves are arranged in a long-range correlated self-similar fashion, where the autocorrelation function CQ(s) of the intervals follows a power law, CQ(s) ∼ s-γ, with an exponent γ between 0.14 and 0.48. We show that these features lead to a pronounced clustering of the rare words in the text.

MeSH terms

  • Cluster Analysis
  • England
  • France
  • Germany
  • Humans
  • Ireland
  • Language
  • Linguistics / methods*
  • Mathematical Computing
  • Models, Theoretical*
  • Probability
  • Vocabulary

Grants and funding

KT is granted PRESTO (Precursory Research for Embryonic Science and Technology) fund of Japan Science and Technology Agency (JST, http://www.jst.go.jp/EN/index.html). AB has a grant of Deutsche Forschungsgemeinschaft (http://www.dfg.de). Both of the funders supported the study design, data analysis, and preparation of the manuscript. JST-PRESTO supported the data collection and decision to publish.