Brian Cohen
  • Making the Grade Blog
  • About Me
  • Tutoring
  • Press
  • Resources to Share

FiveThirtyEight enters the Value-Added Modeling argument

7/30/2015

0 Comments

 
A friend recently sent me an article written on the FiveThirtyEight blog (remember, Nate Silver's organization?) supporting the use of value-added modeling (VAM) in order to evaluate teachers across the country. I find it truly interesting that so many researchers continue to use this data despite such vilifying evidence against it. 

In this article, the author explains a new method being used that allegedly makes the use of VAM more reliable and valid than ever before. 

As a brief background summary: VAM is a method whereby statistical analysis of standardized tests is used in combination with demographic backgrounds of students and teachers in order to calculate how much a teacher should improve a student's ability vs. how much they a actually do. I have found this method very tenuous ever since reading Gary Rubenstein's critique of it back in 2012. 
Picture
One of the best pieces of evidence is the graph to the left. It shows the correlation between teacher ratings in 2008 (x-axis) and 2009 (y-axis) in New York City. If teachers were consistent year-to-year, one would expect the ratings to be fairly linear. As can be seen here, that is patently false. In fact, the correlation is rated as 0.35, something incredibly low for us to think about. If you read further into his report, you will find out even more scary pieces of data and graphs for those who might want to use this method for evaluation.

What the FiveThirtyEight author argues in this article, however, is that there are new method to get around these issues of consistency because there are more randomized trials that can be performed. Even though students are not randomly selected and sent to classrooms across the USA, teachers move somewhat randomly (or, at least, in a way that can be calculated as random) so that the difference in one class and the next can be quantified in connection with the teacher. Unfortunately, because of the myriad studies demonstrating the lack of consistency of tests and students, it is impossible to designate the teacher as the sole reason for student achievement. 

The top comment on this article comes from a gentleman named BIll Honig, who accurately states (and cites):
Studies have shown that under current tests, a teacher who tests rank her at the 50th percentile could be anywhere from the 85th percentile to the 15th A significant number of teachers bounce from top to bottom or vice versa from year to year. (Reseach cited below) A recent report from the Federal Department of Education (of all places) found very high misidentifications even with three years of data per teacher. One fourth of teachers identified as needing special treatment were actually at the mid-range of performance and one-fourth of teachers who were deemed as average were actually in need of special treatment. http://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf
I really hope that those advocating value-added modeling read these studies carefully and spend some time in a classroom to see the effects of these policies.
0 Comments

Your comment will be posted after it is approved.


Leave a Reply.

    Author

    I am a math teacher in the New York Department of Education. I infuse technology and real-world problems into my curriculum in order to prepare my students for the future. I would love for people across the country to recognize we teachers can't do it alone. If you don't believe me, come visit my classroom!

    Picture

    Contact Me

    Picture

    Email Updates

    Enter your email address:

    Delivered by FeedBurner

    Archives

    March 2022
    September 2021
    August 2021
    September 2020
    August 2020
    February 2020
    January 2020
    December 2019
    November 2019
    October 2019
    August 2018
    June 2018
    April 2018
    September 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    May 2016
    April 2016
    March 2016
    February 2016
    January 2016
    December 2015
    November 2015
    October 2015
    September 2015
    August 2015
    July 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012
    January 2012
    December 2011
    November 2011
    October 2011
    September 2011
    August 2011
    July 2011
    June 2011
    May 2011
    April 2011
    March 2011
    February 2011
    January 2011

    RSS Feed

Powered by Create your own unique website with customizable templates.