The Stock-Flow Tool: Demonstrating that All Parts of a School are Connected
In a previous post, I suggested that our current conceptualization of "at-risk" is overly narrow. That is, we tend to characterize students who are on the verge of dropping out or on the verge of not graduating on time as at-risk. Certainly, this is true. But when we limit risk in this way, we see only a small portion of a deeply interconnected system.
At New Visions, we believe it is necessary to expand our definition of at-risk. When we do so, we inherently expand the possibilities for our early warning systems (we push the boundaries of our adjacent possible). The first question, then, is how do we reconceptualize at-risk?
We think it requires a different mindset. This is where our application of systems thinking comes into play. Systems thinking underscores the interconnectedness of a school.
The questions that frame traditional early warning systems, namely "Which students are at-risk of dropping out? Which are at-risk of not graduating on time?" open up into a more provocative question that forces us to look for patterns that previously were hidden. "What do a school’s performance patterns tell us about the implications and interconnections of a school's resource allocation?" This reframing treats the school as a system.
All Parts of a School are Connected
Throughout their eight semesters in high school, students can and do flow between higher and lower performance levels. A school's job is to induce positive "volatility" (moving lower performing students into higher performance brackets) and to sustain high performing students.
But in trying to move or sustain students, a school's actions can be both purposeful and inadvertent. The most obvious example of purposely induced volatility occurs when schools intervene to improve failing students' performance (e.g., a school may adopt intensive credit recovery for seniors who are at risk for failing to graduate on time). Less obvious, though, is the inadvertently induced volatility that may occur when those same schools, while focusing on interventions that help improve failing students, draw focus from students less obviously at risk.
Recall our color-coded system for tracking student performance, as seen in the figure above (blue means college ready; green, on-track to graduate; yellow, almost on-track; and red, off-track.) There is a systematic relationship between the blue, college-ready student who slips to green and the yellow, almost on track student who is moved up to green. Is it possible that one student rises at the expense of another falling? This very question underpins a new generation early warning system – one that helps us see and understand that all parts of a school are connected.
Vicious Cycles Across and Within Student Cohorts
Assume a school, through the best of intentions, decides to allocate intensive resources to intervene with those seniors on the cusp of not graduating. Assume also that there are limited total resources in a school. The more resources applied to the at-risk 12th grade students, the fewer remaining resources can be applied to students in earlier grades.
As resources to freshmen are reallocated to seniors, the rate of moving off-track freshmen into higher performance categories is reduced; the fewer the resources, the more students can fall off track. This means that by the time freshmen become sophomores, there will likely be a higher percentage of off-track sophomores than would otherwise have occurred. Assuming everything else stays the same, the school will eventually end up with a higher percentage of off-track seniors – leading the school to allocate even more resources to seniors.
This creates a vicious cycle (or a reinforcing feedback loop). Fewer students may enter their senior year prepared to graduate; and the more resources applied to the ever-present cadre of at-risk 12th graders, the less likely low-performing students in earlier grades will ever move into higher performing categories.
This same pattern of resource allocation also happens within cohorts. A school decides to allocate resources to helping off-track and almost on-track seniors graduate on time. This likely comes at the expense of the higher performing students. A blue, on-track to college readiness student may slip into a lower performance category because insufficient resources were allocated to helping him/her stay high performing.
This presents an uncomfortable and complex dilemma for schools and creates a tension between short-term and long-term strategies. Schools that are in fire-fighting mode are more likely to intervene later, when the problem is already visible (seniors at risk of not graduating on time), and less effectively (see my post on "shifting the burden"). When we intervene earlier and apply more resources during freshman year and those resources also build stronger foundational skills, we've increased the percentages of students in school who are on track.
We need new tools that make these interconnections transparent. To help our network of schools begin to visualize how all parts of a school are connected, we have created a new interactive tool.
The School-Level Stock-Flow Tool
The Stock-Flow tool uses our Progress to College Readiness Metric to classify students within different performance levels. What's different about this tool, however, is that it shows how students' risk profiles change not just from one semester to the next but between semesters. In this way, we are visualizing how students drain out of and fill up different performance categories at different moments in time.
Each graph plots the movement of a single cohort of students over four years. Comparing two or more of these figures across years (and cohorts) begins to give us important insights into school structures. Do flows of students change across years, where do they change, and how do they change? When we map a school's programmatic timeline against these stock-flow maps, we can begin to consider the effectiveness of certain interventions.
We can also see subtle patterns within cohorts. For example, in the image below we see how blue and green students drain into lower performance categories between the 7th and 8th semester. At the same time, almost on track students (the yellow and orange flows) and a smaller proportion of off-track students (red flow) are raised to a higher performance category. This poses hard questions for schools. Did one group of students do better at the expense of another group? If the intervention to move almost on-track and off-track students had happened earlier in the students' trajectories than the 7th semester, what would that move mean for other students in the school? Seeing this phenomenon play out allows us to ask a new set of questions about the structures within the school that are producing this dynamic.
You can learn more about the Stock-Flow Tool here and explore its interactive features. We're indebted to infographic specialist Sarah Slobin and her partner, Andrew Garcia Phillips, for providing their expertise, creativity and thought-partnership in helping us design these tools.
Over the next several months, we will be working to implement this tool in our schools and will be posting case studies that reflect deeper analysis of these maps.comments powered by Disqus