The Adjacent Possible, Part III: The Problem with Most Early-Warning Systems for Schools
The adjacent possible is all about creating new forms out of previous, related forms. For the data team at New Visions, this translates into how do we expand and reframe our work developing an early-warning system for schools? Before we could expand this system, we had to think long and hard about the problem that frames current early-warning systems for tracking students who are at-risk of a bad outcome, such as dropping out. We had to retrace our steps.
As I mentioned in my previous post, the problem that underpins most early-warning systems is that when students fail to hit critical benchmarks (e.g., high grades, high attendance rates, continuous credit accumulation), they are more likely to drop out of high school, fail to graduate on time, and to not enroll or persist in college.
It is no surprise, then, that the focus of early warning is on a particular type of student, one who is on the verge of dropping out or not graduating on time. This narrow definition of "at risk" has important implications for our early-warning systems. They are designed to catch something that is about to happen. They are designed to prevent problems from getting worse. But this type of early-warning system does not get at the root cause of the problems: the structures and systems in schools that are producing risk. In fact, these traditional early-warning systems may actually support reactive school behaviors in which well-intentioned school staff are acting -- ineffectively -- after the fact.
Shifting the Burden
I tend to believe that the emphasis on a certain type of accountability has resulted in a proliferation of early-warning systems that provide important but limited information that inform school priorities and then drive school behavior.
When we define at-risk in the narrowest possible sense -- kids who are at risk of dropping out or failing to graduate on time -- and when we then couple that with intense regulatory requirements, an interesting phenomenon unfolds: what, in systems thinking is called shifting the burden
Let me illustrate with an example.
Beginning in 2009, New York State Board of Regents began phasing out the less rigorous local diploma. By 2012, all general education NYC students had to meet the requirements for the Regents diploma if they were to graduate from high school.
Since 2008, NYC schools have increased the number of kids graduating with a Regents diploma as the Local diploma is phased out. In 2008, 25.7 percent of students earned a Regents diploma. By 2011, that number was 39.2 percent.
The increased percentage of students graduating with a Regents diploma is important. But, when viewed out of context, it masks a critical reality. When we look "under the hood" of the Regents diplomas, we see that not all Regents diplomas are created equal.
It makes intuitive sense that two kids who end in the same place (ie. graduating with a Regents diploma) but who have different progress pathways are not identical. We see this very clearly in the figure below, which shows student progress to graduation over 8 semesters of study in high school. Although both students achieved the same outcome, which student do you think is more prepared to tackle college-level material?
And when we begin to follow these kids into college, the kids with patterns of lower performance will not perform as well. We have shifted the burden to the two and four-year institutions. A great report released by the Center for an Urban Future speaks to this:
"Academic preparation is a serious problem, as one would expect for young adults (and many older ones as well) who struggled in high school. Of those who arrived from a public high school in New York City, 74 percent place into remediation, 32 percent fail their math proficiency exam, 36 percent fail in writing proficiency, and 18 percent fail in reading proficiency. These students must take developmental courses that provide no credit towards completing a degree and cost as much as college-level courses, and their chances of dropout are far higher than students who test into college-level courses immediately. A recent CUNY analysis found that even for students who reach the last remedial course, 'pass rates on CUNY's exit tests are discouraging – just 58 percent for reading, 52 percent for writing, and 65 percent for math.'" (p. 13)
Some districts are beginning to incorporate college success (the 13th year) into their accountability requirements. Merely adding onto or modifying accountability structures and current early-warning system will not make the problem of shifting the burden go away. What is required is a dramatically different framework (and mindset) that will radically alter how we frame "at risk" and the systems required to support this new conceptualization of risk.
My next post will begin to explore using systems thinking as a framework for recasting risk – and the new data attributes that begin to emerge when we do so.
Susan Fairchild is director of program analysis and applied research at New Visions. Follow her on Twitter at @SKFchild.