The dangers of a single story in current U.S. large-scale writing assessment are that assessment practice does not align with theory and this practice has negative effects on instruction and students. In this article, I analyze the connections or lack of connections among writing theory, writing assessment, and writing instruction, critique the construct and consequential validity of direct writing assessment and portfolio assessment, and reframe reliability as local consensus among experts. A new vision of large-scale sociocultural writing portfolios in K–12 education is offered that builds on the practices of past large-scale portfolio assessment but also encourages students to write in multiple languages/dialects and modes for multiple purposes. Another key feature of sociocultural portfolios is that students are encouraged to write for impact. These additional components will mitigate the dangers of a single story by ensuring that (a) assessment practice matches sociocultural writing theory, resulting in high construct validity, and (b) teachers are encouraged to utilize culturally sustaining pedagogy, resulting in high consequential validity.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
3.
AuW. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36, 258–267.
4.
BakhtinM. M. (1981). The dialogic imagination: Four essays by M. M. Bakhtin. Austin: The University of Texas Press.
5.
BalesterV. (2012). How writing rubrics fail: Toward a multicultural model. In InoueA. B.PoeM. (Eds.), Race and writing assessment (pp. 63–77). New York, NY: Peter Lang.
6.
BallA. F.EllisP. (2008). Identity and the writing of culturally and linguistically diverse students. In BazermanC. (Ed.), Handbook of research on writing: History, society, school, individual, text (pp. 499–513). New York, NY: Lawrence Erlbaum.
7.
BanduraA. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman.
8.
BeckS.JefferyJ. (2007). Genres of high-stakes writing assessments and the construct of writing competence. Assessing Writing, 12, 60–79.
9.
BehizadehN. (2012, November). Making middle school writing meaningful: An analysis of factors increasing or decreasing perceptions of authenticity in writing. Paper presented at the 2012 Annual Meeting of the Literacy Research Association (LRA), San Diego, CA.
10.
BehizadehN.EngelhardG. (2011). Historical view of the influences of measurement and writing theories on the practice of writing assessment in the United States. Assessing Writing, 16, 189–211.
11.
BurkeA.HammettR. F. (2009). Assessing new literacies: Perspectives from the classroom. New York, NY: Peter Lang.
12.
CalfeeR.FreedmanS. (1996). Classroom writing portfolios: Old, new, borrowed, blue. In CalfeeR.PerfumoP. (Eds.), Writing portfolios in the classroom: Policy and practice, promise and peril (pp. 3–26). Mahwah, NJ: Lawrence Erlbaum.
13.
CarbonaroW. J.GamoranA. (2002). The production of achievement inequality in high school English. American Educational Research Journal, 39, 801–827.
14.
ColumbiniC. B.McBrideM. (2012). “Storming and norming”: Exploring the value of group development models in addressing conflict in communal writing assessment. Assessing Writing, 17, 191–207.
15.
Common Core State Standards Initiative. (2012). Common Core state standards for English language arts & literacy in history/social studies, science, & technical subjects. Retrieved from http://www.corestandards.org/assets/CCSSI_ELA%20Standards.pdf
16.
CondonW. (2013). Large-scale assessment, locally-developed measures, and automated scoring essays: Fishing for red herrings?Assessing Writing, 18, 100–108.
17.
CopeB.KalantzisM. (Eds.). (2000). Multiliteracies: Literacy learning and the design of social futures. New York, NY: Routledge.
18.
DappenL.IsernhagenJ.AndersonS. (2008). A statewide writing assessment model: Student proficiency and future implications. Assessing Writing, 13, 45–60.
19.
Darling-HammondL.SnyderJ. (1992). Reframing accountability: Creating learner-centered schools. In LiebermanA. (Ed.), The changing contexts of teaching (91st yearbook of the National Society for the Study of Education). Chicago, IL: University of Chicago Press.
20.
DeaneP. (2013). On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing, 18, 7–24.
21.
DelpitL. (1995). Other people’s children: Cultural conflict in the classroom. New York, NY: Free Press.
DiederichP. B.FrenchJ. W.CarltonS. T. (1961). Factors in judgments of writing ability (Research Bulletin No. RB-61-15). Princeton, NJ: Educational Testing Service.
24.
ElbowP.BelanoffE. (1997). Reflections on an explosion: Portfolios in the 90’s and beyond. In YanceyK.WeiserI. (Eds.), Situating portfolios: Four perspectives (pp. 21–34). Logan, UT: Utah State University Press.
25.
EngeströmY. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14, 133–156.
26.
FreedmanS. W. (1993). Linking large-scale testing and classroom portfolio assessments of student writing. Educational Assessment, 1, 27–52.
27.
GeeJ. P. (2001). Literacy, discourse, and linguistics: Introduction. In CushmanE.KintgenE. R.KnollB. M.RoseM. (Eds.), Literacy: A critical sourcebook (pp. 525–554). Boston, MA: Bedford/St. Martin’s.
28.
GeeJ. P. (2008). A sociocultural perspective on opportunity to learn. In MossP.PullinD. C.GeeJ. P. (Eds.), Assessment, equity, and opportunity to learn (pp. 76–108). Cambridge: Cambridge University Press.
29.
GeeJ. P. (2012). Social linguistics and literacies: Ideology in discourses (4th ed.). New York, NY: Routledge.
GonzalezN.MollL. (2002). Cruzando el Puente: Building bridges to funds of knowledge. Educational Policy, 16, 623–641.
32.
GutierrezK. D.Baquedano-LopezP.TejadaC. (1999). Rethinking diversity: Hybridity and hybrid language practices in the third space. Mind, Culture, and Activity, 6, 286–303.
33.
Hamp-LyonsL. (2002). The scope of writing assessment. Assessing Writing, 8, 5–16.
34.
Hamp-LyonsL.CondonW. (2000). Assessing the portfolio: Principles for practice, theory, and research. Cresskill, NJ: Hampton Press.
HeathS. B. (1983). Ways with words: Language, life, and work in communities and classrooms. Cambridge: Cambridge University Press.
37.
HermanJ.GearhartM.AschbacherP. (1996). Portfolios for classroom assessment: Design and implementation issues. In CalfeeR.PerfumoP. (Eds.), Writing portfolios in the classroom: Policy and practice, promise and peril (pp. 27–59). Mahwah, NJ: Lawrence Erlbaum.
38.
HesterV.O’NeillV.NealM.EddingtonA.HuotB. (2007). Adding portfolios to the placement process: A longitudinal perspective. In O’NeillP. (Ed.), Blurring boundaries: Developing writing, researchers, and teachers (pp. 61–90). Creskill, NJ: Hampton Press.
39.
HillocksG. (2002). The testing trap: How state writing assessments control learning. New York, NY: Teachers College Press.
40.
HuotB. (1990). The literature of direct writing assessment: Major concerns and prevailing trends. Review of Educational Research, 60237–263.
41.
InoueA. B.PoeM. (2012). Introduction. In InoueA. B.PoeM. (Eds.), Race and writing assessment (pp. 1–11). New York, NY: Peter Lang.
42.
IRA/NCTE Joint Task Force on Assessment. (2010). Standards for the assessment of reading and writing (Revised edition). Newark, DE: International Reading Association and National Council of Teachers of English.
43.
IrvineJ. J.ArmentoB. J. (2001). Culturally responsive teaching: Lesson planning for elementary and middle grades. New York, NY: McGraw-Hill.
44.
JefferyJ. V. (2009). Constructs of writing proficiency in US state and national writing assessments: Exploring variability. Assessing Writing, 14, 3–24.
45.
KamilM. L.BormanG. D.DoleJ.KralC. C.SalingerT.TorgesenJ. (2008). Improving adolescent literacy: Effective classroom and intervention practices: A practice guide (NCEE #2008–4027). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc.
46.
KimberK.Wyatt-SmithC. (2009, September). Rethinking quality assessment for 21st century learning: How students use and create knowledge online. Paper presented at Assessment for a Creative World, 35th Annual Conference of International Association of Educational Assessment (IAEA), Brisbane, Australia.
47.
KinlochV. F. (2005). Revisiting the promise of “students’ right to their own language”: Pedagogical strategies. College Composition and Communication, 57, 83–113.
48.
KoretzD.StecherB.KleinS.McCaffreyD. (1994). The Vermont Portfolio Assessment Program: Findings and implications. Educational Measurement: Issues and Practice, 13(3), 3–16.
49.
LeMahieuP.GitomerD.EreshJ. (1995). Portfolios in large-scale assessment: Difficult but not impossible. Educational Measurement: Issues and Practice, 14(3), 11–16, 25–28.
50.
Ladson-BillingsG. (2009). The dreamkeepers: Successful teachers of African American children (2nd ed.). San Francisco, CA: Jossey-Bass.
51.
LeeC. D. (2006). “Every good-bye ain’t gone”: Analyzing the cultural underpinnings of classroom talk. International Journal of Qualitative Studies in Education, 19, 305–327.
52.
McGrailE.BehizadehN. (2014). Evaluating multimodal assessment frameworks: The interactive audience as a missing element of online multimodal composition. Manuscript submitted for publication.
53.
McHenryE.HeathS. B. (1994). The literate and the literary: African Americans as writers and readers—1830–1940. Written Communication, 11, 419–444.
54.
McWhorterJ. (2000). Spreading the word: Language and dialect in America. Portsmouth, NH: Heinemann.
55.
MessickS. (1996). Validity and washback in language testing. Language Testing, 13, 241–256.
56.
MintropH.SundermanG. L. (2009). Predictable failure of federal sanctions-driven accountability for school improvement: And why we may retain it anyway. Educational Researcher, 38, 353–364.
57.
MorrellE. (2008). Critical literacy and urban youth: Pedagogies of access, dissent, and liberation. New York, NY: Routledge.
58.
MorrellE.Duncan-AndradeJ. M. (2002). Promoting academic literacy with urban youth through engaging hip-hop culture. English Journal, 91, 88–93.
59.
MossP. A. (1994). Can there be validity with reliability?Educational Researcher, 23, 229–258.
NewmannF. M.MarksH. M.GamoranA. (1996). Authentic pedagogy and student performance. American Journal of Education, 104, 280–312.
62.
NietoS.BodeP. (2012). Affirming diversity: The sociopolitical context of multicultural education (6th ed.). Boston, MA: Pearson.
63.
PachecoM. (2012). Learning in/through everyday resistance: A cultural-historical perspective on community resources and curriculum. Educational Researcher, 41, 121–132.
64.
ParisD. (2012). Culturally sustaining pedagogy: A needed change in stance, terminology, and practice. Educational Researcher, 41(3), 93–97.
PetruzziA. (2008). Articulating a hermeneutic theory of writing assessment. Assessing Writing, 13, 219–242.
67.
PriorP. (2006). A sociocultural theory of writing. In MacArthurC. A.GrahamS.FitzgeraldJ. (Eds.), Handbook of writing research (pp. 54–66). New York, NY: Guilford.
68.
Purcell-GatesV.DukeN. K.MartineauJ. A. (2007). Learning to read and write genre-specific text: Roles of authentic experience and explicit teaching. Reading Research Quarterly, 42, 8–45.
69.
Salahu-DinD.PerskyH.MillerJ. (2008). The nation’s report card: Writing 2007 (NCES 2008–468). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://nces.ed.gov/nationsreportcard/
70.
SmithermanG.VillanuevaV. (2003). Language diversity in the classroom: From intention to practice. Urbana, IL: National Council of Teachers of English.
71.
SolórzanoR. W. (2008). High stakes testing: Issues, implications, and remedies for English language learners. Review of Educational Research, 78, 260–329.
72.
StreetB. V. (1984). Literacy in theory and practice. Cambridge: Cambridge University Press.
73.
StreetB. V. (2001). The new literacies studies. In CushmanE.KintgenE. R.KrollB. M.RoseK. M. (Eds.), Literacy: A critical sourcebook. Boston, MA: Bedford/St. Martin’s.
VannemanA.HamiltonL.Baldwin AndersonJ.RahmanT. (2009). Achievement gaps: How Black and White students in public schools perform in mathematics and reading on the National Assessment of Educational Progress (NCES 2009–455). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.
76.
WardleE.RoozenK. (2012). Addressing the complexity of writing development: Toward an ecological model of assessment. Assessing Writing, 17, 106–119.
77.
WeigleS. C. (2002). Assessing writing. Cambridge: Cambridge University Press.
78.
WhiteE. (1985). Teaching and assessing writing. San Francisco, CA: Jossey-Bass.
79.
WinnM. T.BehizadehN. (2011). The right to be literate: Literacy, education, and the school-to-prison pipeline. Review of Research in Education, 35, 147–173.
80.
YanceyK. B. (1999). Looking back as we look forward: Historicizing writing assessment. College Composition and Communication, 50, 483–503.
81.
YanceyK. B. (2002). Portfolios in the writing classroom: A final reflection. In YanceyK. B. (Ed.), Portfolios in the writing classroom: An introduction (pp. 102–116). Urbana, IL: National Council of Teachers of English.
82.
YanceyK. B. (2009). Reflection and electronic portfolios: Inventing the self and reinventing the university. In CambridgeD.CambridgeB.YanceyK. (Eds.), Electronic portfolios 2.0: Emergent research on implementation and impact (pp. 5–16). Sterling, VA: Stylus.
83.
YanceyK. B. (2012). College admissions and the Insight Resume. In InoueA. B.PoeM. (Eds.), Race and writing assessment (pp. 171–185). New York, NY: Peter Lang.
