This article focuses on the quantitative phase of a multiphase mixed methods study investigating adults’ and families’ access to government food assistance. The research evaluates participants’ comprehension of, and ability to, adequately complete authentic complex texts—national food assistance application documents. Summative usability testing of the Supplemental Nutrition Assistance Program application was conducted with 12 adult participants in divergent literacy learning subgroups: 6 adults enrolled in adult literacy classes and 6 adults enrolled in a doctoral program in education. Protocol analysis and extended think alouds were used to evaluate the materials. Individual sessions were audio/video recorded and types, kinds, and rates of error metrics were compared by subgroup. The completion rate without critical errors for doctoral and adult literacy students was 66.6% and 0%, respectively. Results indicate significant concerns with the usability and comprehensibility of food assistance applications and point to the importance of including educationally/linguistically diverse participants in the usability/evaluation process for government assistance applications to prevent further marginalization.
AlexanderP. A. (2005). The path to competence: A lifespan developmental perspective on reading. Journal of Literacy Research, 37, 413–436.
2.
BakerL.BrownA. L. (1984). Metacognitive skills and reading. In PearsonP. D. (Ed.), Handbook of reading research (pp. 353–394). New York, NY: Longman.
3.
BarnumC. M. (2010). Usability testing essentials: Ready, set … test! Burlington, MA: Morgan Kaufmann.
4.
BartonD.HamiltonM.IvanicR. (2000). Situated literacies: Reading and writing in context. New York, NY: Routledge.
5.
CheekA. (2014, November). The US plain writing act of 2010: Ensuring the citizen’s right to clarity. Paper session presented at International Clarity Conference 2014, Brussels, Belgium.
6.
ClayM. M. (2000). Running records for classroom teachers. Portsmouth, NH: Heinemann.
7.
Coleman-JensenA.GregoryC.SinghA. (2014). Household food security in the United States in 2013 (USDA Economic Research Report No. 173). Washington, DC: U.S. Government Printing Office. Retrieved from United States Department of Agriculture, Economic Research Service website: http://www.ers.usda.gov/publications/err-economic-research-report/err173.aspx
8.
CromleyJ.AzevedoR. (2011). Measuring strategy use in context with multiple-choice items. Metacognition and Learning, 6, 155–177.
9.
de BeaugrandeR. (1987). Text, attention, and memory in reading research. In TierneyR. J.AndersP. L.MitchellJ. N. (Eds.), Understanding readers’ understanding: Theory and practice (pp. 15–58). New York, NY: Routledge.
10.
EricssonK. A.SimonH. A. (1993). Protocol analysis: Verbal reports as data(rev. ed.). Cambridge, MA: MIT Press.
11.
FlavellJ. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906–911.
12.
GeeJ. (2001). Identity as an analytic lens for research in education. In SecandaW. (Ed.), Review of research in education (Vol. 25, pp. 99–125). Washington, DC: American Educational Research Association.
13.
GoodmanY. M.WatsonD. J.BurkeC. L. (1987). Reading miscue inventory: Alternative procedures. Katonah, NY: Owen.
14.
GrayW. S. (1947). Progress in the study of readability. The Elementary School Journal, 47, 491–499.
15.
HinkleD. E.WiersmaW.JursS. G. (2003). Applied statistics for the behavioral sciences. Boston, MA: Houghton Mifflin.
16.
HolschuhJ. P.AultmanL. P. (2009). Comprehension development. In FlippoR. F.CaverlyD. C. (Eds.), Handbook of college reading and study strategies (2nd ed., pp. 121–144). New York, NY: Routledge.
17.
MartinezM. E. (2006). What is metacognition?The Phi Delta Kappan, 87, 696–699.
18.
MokhtariK.ReichardC. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94, 249–259.
19.
Nash-DitzelS. (2010). Metacognitive reading strategies can improve self-regulation. Journal of College Reading and Learning, 40, 45–63.
20.
NielsenJ.LandauerT. K. (1993). A mathematical model of the finding of usability problems. In AshlundS.MulletK.HendersonA.HollnagelE.WhiteT. (Eds.), Proceedings of ACM INTERCHI’93 Conference (pp. 206–213). Amsterdam, the Netherlands: ACM Press.
ParisS. G.LipsonM. Y.WixsonK. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293–316.
23.
PearsonP. D.CamperellK. (1981). Comprehension of text structures. In GuthrieJ. T. (Ed.), Comprehension and teaching: Research perspectives (pp. 27–55). Newark, NJ: International Reading Association.
24.
PintrichP. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41, 219–225.
PressleyM.Van EttenS.YokoiL.FreebernG.Van MeterP. (1998). The metacognition of college studentship: A grounded theory approach. In HackerD. J.DunloskyJ.GraesserA. C. (Eds.), Metacognition in educational theory and practice (pp. 347–381). Mahwah, NJ: Erlbaum.
27.
QuesenberyW. (2008). The five dimensions of usability. In AlbersM. J.MazurB. (Eds.), Content and complexity: Information design in technical communication (pp. 75–95). Mahwah, NJ: Taylor & Francis.
28.
QuesenberyW. (2012). Evaluation in government environments. In GreenT. (Ed.), Usability in government systems (pp. 317–330). Boston, MA: Morgan Kauffman.
29.
RubinJ.ChisnellD. (2008). The handbook of usability testing: How to plan, design, and conduct effective tests. Indianapolis, IN: Wiley.
30.
RumelhartD. E. (1981). Schemata: The building blocks of cognition. In GuthrieJ. T. (Ed.), Comprehension and teaching: Research perspectives (pp. 27–55). Newark, NJ: International Reading Association.
31.
ShadishW. R.CookT. D.CampbellD. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
32.
SperlingR. A.HowardB. C.StaleyR.DuBoisN. (2004). Metacognition and self-regulated learning constructs. Educational Research and Evaluation, 10, 117–139.
33.
St. ClairR.BelzerA. (2010). Adult basic education. In KaswormC. E.RoseA. D.Ross-GordonJ. M. (Eds.), Handbook of adult and continuing education (pp. 189–199). Thousand Oaks, CA: Sage.
34.
StreetB. (1993). Cross-cultural approaches to literacy. New York, NY: Cambridge University Press.
35.
StringfieldS. (1994). Outlier studies of school effectiveness. In ReynoldsD.CreemersB. P. M.NesselrodtP. S.ShafferE. C.StringfieldS.TeddlieC. (Eds.), Advances in school effectiveness research and practice (pp. 78–84). Thousand Oaks, CA: Sage.
VirziR. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough?Human Factors: The Journal of Human Factors in Ergonomics Society, 34, 457–468.
41.
Web Accessibility Initiative Site Task Force of Education and Outreach Working Group and American Institutes for Research. (2003, October 24). WAI site usability testing questions. Retrieved fromhttps://www.w3.org/WAI/EO/Drafts/UCD/questions.html
