The 18th century poet and lyricist Robert Burns is known for the phrase: "The best laid plans of mice and men often go awry." The book Of Mice and Men credits its title to the poem. This book tells the story of two men who move from job to job, town to town, hoping to make it "big," only to find themselves often back where they started.
In the national narrative of education reform it can be difficult to discern which of the plans promoted by politicians or researchers will be able to improve the future of the American education system and which will simply "go awry." Foundations and organizations are crafting policy statements and reports that advocate one position or another, often with dire implications students, teachers, parents, and community members. While education reformers may genuinely have good intentions for students and communities, this effort to make it "big" is destined to fail, while leaving students to pay the price.
Recently, the results of teacher evaluation systems in New York City and Los Angeles have caused a flurry of anxiety across the country. Teachers worry about their livelihood, parents are concerned about their children's future, and community members point fingers of blame at each side. The Strategic Data Project and other organizations provide reports using this data, often called Value-added modeling (VAM), in which teachers are compared with prior test scores in order to discover how much they, in particular, influence student achievement. Sometimes their skill and experience is even summarized in one number or grade. Advocates of VAM contend that it holds the key to determining which teachers should be retained and which let go, allowing better-qualified teachers to take their place. Organizations like The New Teacher Project create reports like the recent Irreplaceables, a piece emphasizing the need for more control over teacher hiring and firing in order to retain high-quality teachers, demonstrated by high VAM scores. Through reports like these, teachers are being unfairly compared using these metrics and demoralized in the process.
In the national narrative of education reform it can be difficult to discern which of the plans promoted by politicians or researchers will be able to improve the future of the American education system and which will simply "go awry." Foundations and organizations are crafting policy statements and reports that advocate one position or another, often with dire implications students, teachers, parents, and community members. While education reformers may genuinely have good intentions for students and communities, this effort to make it "big" is destined to fail, while leaving students to pay the price.
Recently, the results of teacher evaluation systems in New York City and Los Angeles have caused a flurry of anxiety across the country. Teachers worry about their livelihood, parents are concerned about their children's future, and community members point fingers of blame at each side. The Strategic Data Project and other organizations provide reports using this data, often called Value-added modeling (VAM), in which teachers are compared with prior test scores in order to discover how much they, in particular, influence student achievement. Sometimes their skill and experience is even summarized in one number or grade. Advocates of VAM contend that it holds the key to determining which teachers should be retained and which let go, allowing better-qualified teachers to take their place. Organizations like The New Teacher Project create reports like the recent Irreplaceables, a piece emphasizing the need for more control over teacher hiring and firing in order to retain high-quality teachers, demonstrated by high VAM scores. Through reports like these, teachers are being unfairly compared using these metrics and demoralized in the process.
When analyzed on a deeper level studies show that Value-Added Modeling has many flaws, some acknowledged by its own users. They recognize the problem but emphasize the need for a metric to be used, regardless of how well it works. In fact, these flaws undermine the modeling system itself and prove it to be of little use when comparing specific teachers year after year. On his blog, "School Finance 101," Dr. Bruce Baker analyzes the VAM scores of New York City teachers over two years and demonstrates the minute correlation between teacher scores. Teachers who were once in the top 20% of their cohort (red points) are scattered across the spectrum the next year.
A research briefing from September 2011 produced similar negative results in its analysis of teacher evaluation being used across the country. One study showed "a small minority (25%) [of teachers] stayed in the same rating band the following year while most scores moved to other parts of the distribution," meaning only one in four teachers had consistent scores from year to year. Another showed an "experienced English teacher in the study whose rating went from the very lowest category in one year to the very highest in the next year." This was due to a significant decrease in the number of English Language Learners in her class, making it significantly easier to communicate. In addition, since a "teacher who works in a well-resourced school with specialist supports may appear to be more effective than one whose students do not receive these supports," teachers in low-income schools across the country, much like Philadelphia, are less experienced with higher turnover, causing problems down the line.
One of the more perplexing issues, however, is that VAM is being used to boil down the expertise of an educator to one number - a feat most would argue is impossible and de-professionalizes the career path that needs motivated individuals. Even if VAM were to show more promise in its purpose, the sharing of this data will inherently influence the outcome of its use. Instead of allowing for time to improve, parents and communities would pull their support for teachers who could potentially grow in skill if allowed the time. With thousands of teacher retiring and leaving the classroom each year, there is not much room for this to happen.
Value-Added Modeling is currently being misused as the "silver bullet" in education reform. Unfortunately, not only is there no one policy that can have the effect desired to change everything, but this one is also inaccurate and dangerous. Alternatives with more proven (but more expensive) results do exist. In Montgomery County, MD teachers are evaluated using the Peer Assistance and Review (PAR) Program, which fosters collegial and collaborative processes to improve performance of teachers or dismiss those who cannot improve. Teachers are referred by their principal and then evaluated by a team of trained educators (including other mentor teachers). Philadelphia is currently piloting a program like this in schools.
Programs like PAR may be difficult to implement due to high costs in labor and demands on time. However, they are effective in creating a professional community that can thrive and sustain itself, warrants their use. In contrast, Value-Added Modeling is more often used punitively and neglects the need for high-quality teachers judged by a fair standard.
Metrics used today like VAM are touted as a way of quantifying the work that teachers do but, in the end, prove only to quantify what reformers think of teachers, while demoralizing them in the process. Instead of focusing on the negative and working against teachers, focus on the positive and work with them. Treat them not as cogs in a wheel but as partners on a journey.
A research briefing from September 2011 produced similar negative results in its analysis of teacher evaluation being used across the country. One study showed "a small minority (25%) [of teachers] stayed in the same rating band the following year while most scores moved to other parts of the distribution," meaning only one in four teachers had consistent scores from year to year. Another showed an "experienced English teacher in the study whose rating went from the very lowest category in one year to the very highest in the next year." This was due to a significant decrease in the number of English Language Learners in her class, making it significantly easier to communicate. In addition, since a "teacher who works in a well-resourced school with specialist supports may appear to be more effective than one whose students do not receive these supports," teachers in low-income schools across the country, much like Philadelphia, are less experienced with higher turnover, causing problems down the line.
One of the more perplexing issues, however, is that VAM is being used to boil down the expertise of an educator to one number - a feat most would argue is impossible and de-professionalizes the career path that needs motivated individuals. Even if VAM were to show more promise in its purpose, the sharing of this data will inherently influence the outcome of its use. Instead of allowing for time to improve, parents and communities would pull their support for teachers who could potentially grow in skill if allowed the time. With thousands of teacher retiring and leaving the classroom each year, there is not much room for this to happen.
Value-Added Modeling is currently being misused as the "silver bullet" in education reform. Unfortunately, not only is there no one policy that can have the effect desired to change everything, but this one is also inaccurate and dangerous. Alternatives with more proven (but more expensive) results do exist. In Montgomery County, MD teachers are evaluated using the Peer Assistance and Review (PAR) Program, which fosters collegial and collaborative processes to improve performance of teachers or dismiss those who cannot improve. Teachers are referred by their principal and then evaluated by a team of trained educators (including other mentor teachers). Philadelphia is currently piloting a program like this in schools.
Programs like PAR may be difficult to implement due to high costs in labor and demands on time. However, they are effective in creating a professional community that can thrive and sustain itself, warrants their use. In contrast, Value-Added Modeling is more often used punitively and neglects the need for high-quality teachers judged by a fair standard.
Metrics used today like VAM are touted as a way of quantifying the work that teachers do but, in the end, prove only to quantify what reformers think of teachers, while demoralizing them in the process. Instead of focusing on the negative and working against teachers, focus on the positive and work with them. Treat them not as cogs in a wheel but as partners on a journey.
RSS Feed
