In this paper we explore to what extent text parameters, such as average number of words per sentence, syllables per word, nouns per sentence, frequency of content words, etc. can successfully rank Russian academic texts for different age and grade levels. We provide a brief overview of previous research on readability of Russian texts and describe the corpus of school textbooks on Social Studies (from 5-th to 11-th grade) compiled by the authors. We share our experience of using a variety of quantitative text complexity metrics and evaluate the measures of existing Russian text complexity formulas. Based on the tests of a set of extended text features, we propose one innovative metric for better prediction of Russian text complexity, i.e. the number of adjectives. As the results obtained compare favorably with the previously published results on the established complexity metrics for Russian texts, the study encourages the development of valid, reliable and transparent complexity tools for Russian texts.
AndersonR., DavisonA., Conceptual and empirical bases of readability formulas, Center for the Study of Reading Technical Report; no. 392, 1986.
2.
BezrukikhM., AdamovskayaO. and IvanovV., Visual and oculomotor activity of first-grade school students during reading texts of varying complexity, Human Physiology43(2) (2017), 168–176.
3.
BiberD., Methodological issues regarding corpus-based analyses of linguistic variation, Literary and Linguistic Computing5(4) (1990), 257–269.
4.
BiberD., Representativeness in corpus design, Literary and Linguistic Computing8(4) (1993), 243–257.
5.
BiberD., University language: A corpus-based study of spoken and written registers, volume 23. John Benjamins Publishing, 2006.
6.
BrownC., SnodgrassT., KemperS., HermanR. and CovingtonM., Automatic measurement of propositional idea density from part-of-speech tagging, Behavior Research Methods40(2) (2008), 540–545.
7.
CarterR. and McCarthyM., Grammar and the spoken language 1, Applied Linguistics16(2) (1995), 141–158.
8.
ChallJ., and DaleE., Readability revisited: The new Dale-Chall readability formula, Brookline Books, 1995.
9.
DaleE. and ChallJ., A formula for predicting readability: Instructions, Educational Research Bulletin (1984), 37–54.
DuranN., BellissensC., TaylorR. and McNamaraD., Quantifying text difficulty with automated indices of cohesion and semantics, In Proceedings of the Cognitive Science Society, volume 29, 2007.
12.
FleschR., How to write plain English: A book for lawyers and consumers, Harpercollins1979.
13.
FlowerdewL., Corpus-based analyses in eap. Academic discourse, pages 95–114, 2002.
14.
GrechikhinA., Sotsiologiya i psikhologiya chteniya [sociology and psychology of reading], Moscow:
MGUP, 2007.
15.
IvanovV., K voprocu o vozmožnosti ispol’zovanija lingvisticeskix xarakteristik složnosti teksta pri issledovanii okulomotornoj aktivnosti pri ctenii u podrostkov [toward using linguistic profiles of text complexity for research of oculomotor activity during reading by teenagers], Novye issledovanija [New studies]34(1) (2013), 42–50.
16.
KarpovN., BaranovaJ., VituginF., Single-sentence readability prediction in Russian. In International Conference on Analysis of Images, Social Networks and Texts, pages 91–100. Springer, 2014.
17.
KincaidJ., FishburneR.Jr, RogersR. and ChissomB.Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel, Naval Technical Training Command Millington TN Research Branch1975.
18.
KitsonH., The mind of the buyer: A psychology of selling, volume 21549. Macmillan, 1921.
19.
KlareG., Assessing readability. Reading research quarterly, pages 62–102,
1974.
20.
KoesterA., Building small specialised corpora, The Routledge Handbook of Corpus Linguistics1 (2010), 66–79.
21.
KrioniN., NikinA. and FillipovaA., Avtomatizirovannaja sistema analiza slozhnosti uchebnyh tekstov [the automated system of the analysis of educational texts complexity], Vestnik UGATU (Ufa) [Herald of UGATU]11(1) (2008), 28.
22.
LorgeI., Word lists as background for communication, Teachers College Record, 1944.
23.
LyashevskayaO., SharovS., The frequency dictionary of modern Russian language, Azbukovnik,
Moscow, 2009.
24.
ManthorpeJ., BiggsS., McCreadieC., TinkerA., HillsA., O’KeefeM., DoyleM., ConstantineR., ScholesS. and ErensB., The uk national study of abuse and neglect among older people, Nursing Older People19(8) (2007), 24–26.
25.
McNamaraD., GraesserA., CaiZh and KulikowichJ., Coh-metrix easability components: Aligning text difficulty with theories of text comprehension. In, annual meeting of the American Educational Research Association, New Orleans, LA, 2011.
26.
McNamaraD., GraesserA., McCarthyP., CaiZh., Automated evaluation of text and discourse with Coh-Metrix
, Cambridge University Press, 2014.
27.
NaumovI. and VykhovanetsV., Using syntactic text analysis to estimate educational tasks’ difficulty and complexity, Automation and Remote Control77(1) (2016), 159–178.
28.
ObobronevaI., Avtomatizirovannaya otsenka slozhnosti uchebnykh tekstov na osnove statisticheskikh parametrov [semiautomatic evaluation of the complexity of texts on the base of statistic parameters]. moscow: Rae institute of content and methods of teaching, M.: RAS Institut soderzhaniya i metodov obucheniya,
2006.
29.
OkladnikovaS., Model’kompleksnoj otsenki citabel’nosti testovyx materialov na etape razrabotki [A model of multidimensional evaluation of the readability of test materials at the development stage],urnal: Upravlenie i vysokie texnologii [Caspian Journal: Management and High Technologies]}, Prikaspijskij \v z3 (2010), 63–71.
30.
PearsonP. and LibenD., The progression of reading comprehension, Achieve the core2013.
31.
ShpakovskiyY.et al., Otsenka trudnosti vospriyatiya i optimizatsiya slozhnosti uchebnogo teksta [Evaluation of the difficulty of perception and optimization of the text complexity], PhD thesis 2007.
32.
SidorovG., Non-linear construction of n-grams in computational
linguistics, M’e: Sociedad Mexicana de Inteligencia Artificial, 2013.
33.
SidorovG., VelasquezF., StamatatosE., GelbukhA., Chanona-HernándezL., Syntactic dependency-based n-grams as classification features. In Mexican International Conference on Artificial Intelligence, pages 1–11. Springer,
2012.
34.
SolnyshkinaM., HarkovaE. and KiselnikovA., Comparative coh-metrix analysis of reading comprehension texts: Unified (Russian) state exam in English vs Cambridge first certificate in English, English Language Teaching7(12) (2014), 65.
35.
SolnyshkinaM. and KiselnikovA., Slozhnost’ teksta: Etapy izucheniya v otechestvennom prikladnom yazykoznanii [text complexity: Study phases in russian linguistics]. Vestnik Tomskogo gosudarstvennogo universiteta. Filologiya [Tomsk State University Journal of Philology]6(38), 2015.
36.
SolovyevV. and IvanovV., Knowledge-driven event extraction in Russian: Corpus-based linguistic resources, Computational intelligence and neuroscience2016 (2016), 16.
37.
ToldovaS., LyashevskayaO., Bonch-OsmolovskayaA., IonovM., Evaluation for morphologically rich language: Russian NLP. In Proceedings on the International Conference on Artificial Intelligence (ICAI), page 300. The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 2015.
38.
ToldovaS. and IonovM., Mention detection for improving coreference resolution in russian texts: A machine learning approach,n y Sistemas, Computación20(4), 2016.
39.
VaughanE., ClancyB., Small corpora and pragmatics. In Yearbook of Corpus Linguistics and Pragmatics 2013, pages 53–73. Springer,
2013.
