?
The construction and analysis of the Russian language models for a cryptographic algorithm research
The word and n-gram distribution of natural language is studied in many areas: linguistics, game theory, and molecular biology. Corpus analysis of the language is also important in cryptographic protection of information, including the effectiveness of cryptographic algorithms. The procedures for recovery of discrete message parts are based on n-gram dictionaries of various lengths. In this regard, it is of particular importance to study their statistical properties, test the completeness and adequacy of the corpus used. When studying the language corpus and compiling dictionaries, one of the main issues is the coverage of all possible text segments with the dictionary. The coverage problem is significantly more complex for inflectional languages such as French and German, and especially Russian, compared to analytical languages such as English. Such languages require a larger vocabulary to achieve the required coverage. The paper examines two language models of the Russian language: lexical and n-gram. In the lexical model of the language, the unit of analysis is tokens, that is, units of text, elements of separate writing. In the n-gram model, which is a special case of the lexical model, sequences of n characters or words are considered.