Fact: For many schools, only the name is the same.
Comparing the performance of individual New Orleans public schools with the same name pre- and post-Katrina is not a valid analysis.
Why? We shuffled the deck.
Schools with the same name may:
- be in a different location
- have different grade configurations
- no longer serve the surrounding neighborhood
- have different admission requirements
Comparing apples to bananas:
As an example, let’s say we wanted to compare pre- and post-Katrina performance of Abramson, currently an RSD charter school. Which post-Katrina school serves as the most appropriate comparison for pre-Katrina Abramson?
Abramson School & Abramson Students
Pre- and Post Katrina
The answer is, ”none of the above.” We have no way of knowing which of these three schools, or if any of these three schools, are enrolling the students who would have gone to the old Abramson High School.
Let’s look at two other high schools: McMain and McDonogh 35.
These two Orleans Parish School Board (OPSB) direct-run high schools have the same name, are in the same buildings, and continue to draw students from across the city just as they did before Katrina. There is one important difference, however. Each school lowered its admission requirements when it reopened in 2006.
McMain & McDonogh 35
School Performance Pre- and Post-Katrina
It would be unfair to use the drop in SPS at these schools as evidence of failure of the OPSB. It is equally counterproductive to compare individual schools pre- and post-Katrina to evaluate the performance of the RSD.
BESE policy recognizes that individual school populations may change.
In 1999, when Louisiana adopted school accountability, BESE realized that when a school’s student population changed by more than 50%, comparing School Performance Scores was no longer valid, and it put in place policies for reconfigured schools.
As a result, after Hurricanes Katrina and Rita, BESE allowed all schools in Jefferson, Orleans, St. Bernard, Calcasieu (Lake Charles) and parts of St. Tammany Parishes to start over with new accountability base lines.
How can we measure progress?
- We can compare the overall performance of all of our students and the rate of improvement. (See Myth 3 for a comparison of all students.)
- We can compare the performance of all schools. While we don’t know which school that former Abramson high school student would choose today, we know that s/he is at some school. So, while we cannot compare each school to itself, we can compare the entire portfolio of schools pre- and post-Katrina. (See Myth 2 for a comparison of all schools.)
- We can use the District Performance Score, which is the best, most accurate way to compare performance. The District Performance Score includes all students, all tests, and all achievement levels on the tests. Student test results are calculated as if all public school students in the city were in one school. Hence, every student is equally weighted, and every student’s performance counts, even if his or her school does not yet have a performance score.
Compelling Results: District Performance Scores
All Students, All Tests, All Performance Levels
The 2009 District Performance Score shows the impressive improvement in learning that has occurred in New Orleans since 2005. The 2010 DPS should be released this month. Based on the spring 2010 tests, New Orleans will again show robust growth in its DPS and will have more improvement than the state.
FACT: Data analysis is critical to evaluation and planning but only if you are asking the right questions and making accurate comparisons.
FACT: It is not valid to compare a school to itself post-Katrina to measure growth.
FACT: It is valid to use the performance of all students and all schools to compare pre- and post-Katrina results.
See below for sources.
_______________________________________
REMINDER: Educate Now! is giving away $3,000 to New Orleans Public Schools.
Educate Now! is giving three New Orleans public schools $1,000 each through its Share to Win competition. To win $1,000 for your favorite N.O. public school, simply go to www.sharetowinnola.com and register. Then use the website to share the contest information with your friends. The three people who generate the most entries win! Start sharing today and win big for New Orleans public schools!
Educate Now!
Leslie Jacobs
Founder
Sources:
2004-05 OPSB Student Enrollment and School Performance
OPSB and RSD 2006-10 Student Enrollment and Most Recent School Performance
District Performance Scores:
2005 District Performance Score
2009 District Performance Score – The Department of Education calculates the 2009 District Performance Score for OPSB and RSD in its October 2009 press release.