Brian Cohen
  • Making the Grade Blog
  • About Me
  • Tutoring
  • Press
  • Resources to Share

The "best laid plans" of Value-added Modeling

11/27/2012

2 Comments

 
The 18th century poet and lyricist Robert Burns is known for the phrase: "The best laid plans of mice and men often go awry." The book Of Mice and Men credits its title to the poem. This book tells the story of two men who move from job to job, town to town, hoping to make it "big," only to find themselves often back where they started. 

In the national narrative of education reform it can be difficult to discern which of the plans promoted by politicians or researchers will be able to improve the future of the American education system and which will simply "go awry." Foundations and organizations are crafting policy statements and reports that advocate one position or another, often with dire implications students, teachers, parents, and community members. While education reformers may genuinely have good intentions for students and communities, this effort to make it "big" is destined to fail, while leaving students to pay the price.

Recently, the results of teacher evaluation systems in New York City and Los Angeles have caused a flurry of anxiety across the country. Teachers worry about their livelihood, parents are concerned about their children's future, and community members point fingers of blame at each side. The Strategic Data Project and other organizations provide reports using this data, often called Value-added modeling (VAM), in which teachers are compared with prior test scores in order to discover how much they, in particular, influence student achievement. Sometimes their skill and experience is even summarized in one number or grade. Advocates of VAM contend that it holds the key to determining which teachers should be retained and which let go, allowing better-qualified teachers to take their place. Organizations like The New Teacher Project create reports like the recent Irreplaceables, a piece emphasizing the need for more control over teacher hiring and firing in order to retain high-quality teachers, demonstrated by high VAM scores. Through reports like these, teachers are being unfairly compared using these metrics and demoralized in the process.
Picture
When analyzed on a deeper level studies show that Value-Added Modeling has many flaws, some acknowledged by its own users. They recognize the problem but emphasize the need for a metric to be used, regardless of how well it works. In fact, these flaws undermine the modeling system itself and prove it to be of little use when comparing specific teachers year after year. On his blog, "School Finance 101," Dr. Bruce Baker analyzes the VAM scores of New York City teachers over two years and demonstrates the minute correlation between teacher scores. Teachers who were once in the top 20% of their cohort (red points) are scattered across the spectrum the next year. 

A research briefing from September 2011 produced similar negative results in its analysis of teacher evaluation being used across the country. One study showed "a small minority (25%) [of teachers] stayed in the same rating band the following year while most scores moved to other parts of the distribution," meaning only one in four teachers had consistent scores from year to year. Another showed an "experienced English teacher in the study whose rating went from the very lowest category in one year to the very highest in the next year." This was due to a significant decrease in the number of English Language Learners in her class, making it significantly easier to communicate. In addition, since a "teacher who works in a well-resourced school with specialist supports may appear to be more effective than one whose students do not receive these supports," teachers in low-income schools across the country, much like Philadelphia, are less experienced with higher turnover, causing problems down the line. 

One of the more perplexing issues, however, is that VAM is being used to boil down the expertise of an educator to one number - a feat most would argue is impossible and de-professionalizes the career path that needs motivated individuals. Even if VAM were to show more promise in its purpose, the sharing of this data will inherently influence the outcome of its use. Instead of allowing for time to improve, parents and communities would pull their support for teachers who could potentially grow in skill if allowed the time. With thousands of teacher retiring and leaving the classroom each year, there is not much room for this to happen.

Value-Added Modeling is currently being misused as the "silver bullet" in education reform. Unfortunately, not only is there no one policy that can have the effect desired to change everything, but this one is also inaccurate and dangerous. Alternatives with more proven (but more expensive) results do exist. In Montgomery County, MD teachers are evaluated using the Peer Assistance and Review (PAR) Program, which fosters collegial and collaborative processes to improve performance of teachers or dismiss those who cannot improve. Teachers are referred by their principal and then evaluated by a team of trained educators (including other mentor teachers). Philadelphia is currently piloting a program like this in schools. 

Programs like PAR may be difficult to implement due to high costs in labor and demands on time. However, they are effective in creating a professional community that can thrive and sustain itself, warrants their use. In contrast, Value-Added Modeling is more often used punitively and neglects the need for high-quality teachers judged by a fair standard.

Metrics used today like VAM are touted as a way of quantifying the work that teachers do but, in the end, prove only to quantify what reformers think of teachers, while demoralizing them in the process. Instead of focusing on the negative and working against teachers, focus on the positive and work with them. Treat them not as cogs in a wheel but as partners on a journey.

2 Comments
Gamal Sherif link
11/27/2012 07:09:56 pm

Arne Duncan estimates 10% of California's teachers don't belong in the classroom. The AFT's Randi Weingarten says the national average is more like 3%. In either case, what are we doing to support the other 90-97%?

Teachers need effective working conditions so that they and their students can flourish. It's up to teachers to build the coalition that can advocate for such a worthwhile transformation.

What are effective working conditions? What are the top three things you would change?

Reply
Claire Landau
12/2/2012 02:46:07 am

I agree!

If my work is going to be effectively evaluated it also must be effectively supported.

I am a first grade teacher. Some basic basic supports I believe every early elementary student needs (yet many schools do not provide) are: access to recess at least once a day, adequate access to a counselor, adequate access to a school nurse.

I would love to see more dialog included on the lack of these basic supports as we examine how to measure the work of a teacher.

Reply

Your comment will be posted after it is approved.


Leave a Reply.

    Author

    I am a math teacher in the New York Department of Education. I infuse technology and real-world problems into my curriculum in order to prepare my students for the future. I would love for people across the country to recognize we teachers can't do it alone. If you don't believe me, come visit my classroom!

    Picture

    Contact Me

    Picture

    Email Updates

    Enter your email address:

    Delivered by FeedBurner

    Archives

    March 2022
    September 2021
    August 2021
    September 2020
    August 2020
    February 2020
    January 2020
    December 2019
    November 2019
    October 2019
    August 2018
    June 2018
    April 2018
    September 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    May 2016
    April 2016
    March 2016
    February 2016
    January 2016
    December 2015
    November 2015
    October 2015
    September 2015
    August 2015
    July 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012
    January 2012
    December 2011
    November 2011
    October 2011
    September 2011
    August 2011
    July 2011
    June 2011
    May 2011
    April 2011
    March 2011
    February 2011
    January 2011

    RSS Feed

Powered by Create your own unique website with customizable templates.