October 28, 2019 | Blog

The Reality of “Norming”

Anya Derr is a former middle school and high school English teacher and has a Masters Degree in education. Before joining MZD, she was active in the K12 curriculum market. She is dedicated and passionate about empowering teachers with the right tools to better meet the needs of their students.

In the words of comedian, Jim Gaffigan, “You know what it’s like having a fourth kid? Imagine you’re drowning, then someone hands you a baby.” Although I don’t have four kids, this image of being overwhelmed has stayed with me over the years. In all honesty, this is how overwhelmed I felt teaching high school English my first year. I loved my students but the sheer volume of paperwork to grade, seeing 130 students a day, was daunting. I remember spending my weekends at local coffee shops evaluating and scoring the work of my students for hours on end, only to come home to more paperwork. Due to limited time and resources, assigning authentic performance tasks to truly gauge meaningful student learning could only be done so often. 

In the district I taught at, creating performance tasks was left to each teacher’s discretion. This resulted in inconsistency within each grade level and across the district as a whole. At the district level, these inconsistencies led to skewed and often unreliable data making appropriate professional learning opportunities for teachers difficult to identify and information regarding equity across student populations inaccurate. At the school level, the lack of more frequent, meaningful assessment, left teachers without the guidance needed to inform instruction and provide support to students in a timely manner.

The one exception, was a benchmark performance task given to all ninth graders: writing a five paragraph persuasive essay. A traditional paper and pencil writing task resulting in approximately 1,200 physical pieces of student work that needed to be manually compared to a rubric and assigned a score. To ensure all ninth grade teachers across the district would be assessing this student work consistently and objectively, norming sessions for each ninth grade team was held at each high school.

Most teachers are familiar with the norming process. Gathering together to review exemplars, evaluating additional anonymous student samples, comparing scores as a group and discussing discrepancies to ensure that all scores fall in a consistent range regardless of whom may be scoring. It was a lengthy, manual process and it left me with questions. 

When all ninth grade teachers across the district met to score 1,200 student responses, how would we manage the paperwork? Could we ensure all teachers from each school were scoring the same way? Given the diverse area that the districts encompassed, what may be considered a 2 at a more affluent district may be considered a 3 at an urban, Title 1 school. How will scorers be held accountable for their evaluations? It would be impossible to review each essay to see trends in a scorer’s behavior.  

With technology, norming and training teachers for scoring, can now be done online in a distributed fashion. They no longer have to be sequestered in a room, huddled around piles of paperwork. Regardless of where teachers work within the district, they can all participate in one, uniform process, ensuring higher quality scoring and consistency.  This consistency allows districts to gather more relevant, reliable data and gain important oversight into scoring practices and scorer behavior.  

And because OSCAR Classroom can anonymously route student responses for a second or third, resolution score, students are ensured more equity and teachers have a greater opportunity to collaborate for Professional Learning. 

MZD provides technology makes performance tasks easier to manage and evaluate. This results in more consistent data and oversight for districts and additional resources for teachers to assign more frequent authentic performance tasks. Teachers become better equipped to meet the needs of their students, improve student outcomes and improve their instructional practice. Having this technology in my own classroom would have greatly decreased how overwhelmed I felt, but please, don’t throw me any babies!