A Capital Idea: Washington, DC, Builds a System for LEA Determinations

00:00:04.25 >> For the IDEA Data Center, I'm Amy Bitterman, and this is A Date with Data. Every month, I sit down with data quality influencers from around the country to share their stories about special education data and the work they do to improve outcomes for children with disabilities. Hello, I'm Amy Bitterman, and this is IDC's "A Date with Data." On this episode, I have with me Victoria Glick, who is the Special Education Director, and Karen Morgan-Donaldson, a Supervisory Education Policy and Compliance Specialist, who are both with the District of Columbia's Office of the State Superintendent of Education. With differentiated monitoring and support, or DMS 2.0, and the general supervision Q and A that came out from OSEP back in July of 2023, LEA determinations is an area of high interest and priority by states and their districts, and we are so excited to have Victoria and Karen here to share the story of their SEA's LEA Determination System. Welcome, Victoria and Karen. Thank you so much for being on.

00:01:11.07 >> Hey, Amy. Thank you for having us today. We're very excited to be here. I'm Victoria Glick. I'm the Director of Special Education at the Office of the State Superintendent of Education, which is the District of Columbia State Education Agency.

00:01:25.04 >> Great. Thank you, Victoria.

00:01:26.89 >> Karen, do you want to introduce yourself, too?

00:01:28.95 >> Of course. Thank you for having us. My name is Karen Morgan-Donaldson, and I oversee OSSE's IDEA Part B Monitoring and Compliance Team, and we're excited to be here.

00:01:42.43 >> Great. So to start us off, can you talk about your LEA Determination System? What does it include and some of the ways that you use it?

00:01:51.88 >> Of course. Let me go ahead and start with some background on the evolution of our LEA determinations in the District of Columbia and how we got to our current state. So a while back, OSSE decided to shine the light on special education. And as a part of that work, we examined our special education data and developed a theory of action that connected the necessary actions required to improve outcomes for students with disabilities. OSSE embarked on projects in several priority areas, including one focused on setting clear and high expectations. So we took on the project of updating our framework for OSSE's IDEA General Supervision for Special Education.

00:02:38.99 >> Mm-hmm.

00:02:40.07 >> And our goal was to transition from a system that was primarily focused on compliance, meeting the bare minimum requirements under IDEA to a results-based system that emphasized outcomes for students with disabilities. We really wanted to reflect the national trends that we've been seeing towards results-based accountability systems for special education. And so out of this work came a strategic plan focusing on improving results for students with disabilities, which led to our new and improved terminations, which we actually call our Special Education Performance Report. We actually call it SEPR for short. And SEPR is designed to evaluate the performance of an LEA's special education program, like I said, with an emphasis on student outcomes. And what this allows us to do as a state agency is to pinpoint supports for the LEA for areas improvement to build educator and system capacity to serve students with disabilities. So we kind of like to consider this report a needs assessment for us at the state agency to help us do our job better in regards to giving LEAs that specific and very detailed support that they need. So a couple of key points that I want to talk about. The first year we issued SEPR was in 2023, based on our federal fiscal year '21 data, APR. And we are coming up in our third year of issuance in this format. Another important point is we publicly post the results. There is a large parent engagement piece to this work and SEPR. So this report is designed to provide LEAs and families with meaningful information about LEA progress towards improving outcomes for students with disabilities. This is really important in DC because school choice has a very large presence in DC with roughly half of school age students attending independent public charter schools. We have found that parents really need to have the right information in order to make those informed decisions. Another key point that I want to make is our data division after our first rollout of the report analyzed the reports and showed that LEA SEPR scores are not significantly related to two important things. And that is, one, they are not significantly related to an LEA's percent of students who are economically disadvantaged. And they are also significantly not related to the percent of students who have disabilities. So this was a really key point that we make to LEAs in particular, but of course, the public as well. But this point demonstrates that even despite challenges in student demographics, LEAs can achieve success with special education programming and outcomes.  

00:05:45.72 >> That's really interesting that you've looked into that and ...

00:05:48.10 >> Yeah.

00:05:48.50 >> ... especially can then kind of go to those LEAs maybe that are economically disadvantaged or do have higher proportion of students with disabilities that are doing well and ask them, "What are you doing that's working that we can share with the others?"

00:06:03.31 >> Yeah. It was a good point for us to make in order to kind of level the playing field, but also just to help us shift the mindset, which I think we're going to talk about a little bit later, but we're all in the same boat and here we are. So we also, just another important point is that we use this data from our SEPR to prioritize which LEAs we monitor on an annual basis. So we prioritize LEAs based on their score as part of our cyclical IDEA Part B monitoring. And then, of course, we also use this information to inform tiered technical assistance.

00:06:41.38 >> It's a very robust system. So it's not just merely for issuing those required determinations, but it sounds like you've really taken steps to make it more robust and useful and so excited to hear more about how that's been going. Given that it is .. It sounds like it is a very useful and probably complicated system. What are some ways that you support the LEAs and the schools also to help them understand and be able to use the system?

00:07:12.14 >> First, we solicited a lot of stakeholder feedback from educational stakeholders in the district. This included LEAs. And we started this work at the very beginning, at the conceptual level. So we engaged our stakeholders. We sought feedback, and we listened. We made some changes along the way based on feedback, and so we really wanted to include LEAs so that we could have buy-in specifically from LEAs, of course. We also provide technical-assistance sessions at the beginning of each school year where we pair up with our data team and the program team partners up with the data team to kind of just go over what SEPR is, how do we look at our data. We provide individual LEA data-literacy sessions, where LEAs can meet with, again, program and data folks at our agency to learn and ask questions either about their individual LEA SEPR or just SEPR in general. This coming year, we are very excited that we will be rolling out a SEPR dashboard, which will provide LEAs with real-time data whenever possible for each metric. Of course, we'll provide a great deal of training on this, and this new support tool is something that we get the sense that LEAs are definitely interested in learning more about and kind of engaging with that level of support. Additionally, when we publicly post all LEA SEPRs, we provide resources for LEAs to not only share with parents, but also their staff. These resources allow them to identify areas of strengths and areas for improvement and also gives them guiding guardrails for how to create strategies for improvement and share with their LEA community. And then as I mentioned earlier, family engagement is a big component of SEPR. We also collaborate with DC's Special Education Hub, which helps families understand special education in the district. And of course, that includes CIRP.

00:09:24.76 >> Wow, that dashboard sounds really exciting. And being able to provide that sort of live snapshot of some data, I'm sure that's something that LEAs will be really appreciative of.

00:09:37.17 >> Yeah. We're really excited about the dashboard. And, of course, states are always issuing their determinations a year and a half after the fact.

00:09:44.66 >> Mm-hmm, mm-hmm.

00:09:45.50 >> So we really think that the dashboard is going to be key to changing practice and changing performance on these metrics.

00:09:54.21 >> Yeah. Will those be publicly available or just for the LEA to see for themselves?

00:10:00.96 >> Just the LEA.

00:10:02.06 >> Okay. I'm wondering, too, can you say a little bit more about in terms of the parents and stakeholders, and what are some of the ways that they're able to use SEPR and some of the data maybe that they look at?

00:10:14.67 >> Yeah, of course. Well, they're publicly posted, of course. And we do have plans to include our public SEPR reports on our school report cards.

00:10:24.29 >> Okay.

00:10:24.52 >> So this is something that we've talked about, and that is something we plan to do. In terms of supporting parents and public stakeholders, again, as I stated, we do have our Special Education Hub, which offers family trainings, which we partner with to provide just a readability of the report. What does this mean? We're constantly striving to make the report as meaningful and easy to use as possible. And so that's something that I might talk about during when we talk about challenges later, but when we are engaging with parents, our Special Education Hub does a good job of helping us make this really accessible for our parents.

00:11:13.90 >> Yeah. Having that close collaboration and connection is really important to that hub, so I would imagine that that's a big benefit to families in DC.

00:11:25.13 >> Yes.

00:11:26.36 >> So what have been some of the challenges that you've encountered along the way? And I'm sure over the course of the 3 years that you've been implementing the system, you've learned lessons and had to address them. So can you just talk about some of some of what you've encountered and how you've sort of dealt with it?

00:11:43.10 >> So I think one of the biggest challenges that we have seen has been the culture shift and mindset shift that SEPR has presented. Many LEAs have a very different picture of their performance level in special education now that we have put more emphasis and more weight on student outcomes. For example, the number of LEAs that went from meeting requirements to needing assistance was large. So we are resetting our baseline to align with our ultimate goal of preparing our students with disabilities to be successful, so I think that that is the biggest challenge that we've seen. But we are working with LEAs to address that. We also, in terms of data, we have several very small independent charter schools here in the district that don't meet the end sides for many of the indicators.

00:12:40.07 >> Okay.

00:12:40.97 >> And so unfortunately, those LEA's reports don't necessarily provide the best picture of the health of the LEA special education programs. And that is something that we're working with our data team to address. But of course, we have suppression rules, and there's only so many things that we can include.

00:12:59.68 >> Right.

00:13:00.42 >> And then data literacy, this is something we are working on with our LEAs through technical assistance, through our upcoming SEPR dashboard, SEPR data trainings, office hours. We expect to see an increase in data literacy at the LEA level. So, you know, I think some LEAs are very proficient in accessing their data, using their data to make data-informed decisions, and some of our LEAs are not. And in order to move the needle as a state, we really need all LEAs to be able to do that.

00:13:35.74 >> Yeah.

00:13:35.99 >> And so that kind of aligns with what we were talking about earlier with the readability of the report. Of course, parents are going to have difficulty reading a report that's very data heavy and special education heavy. But even LEA staff and leaders will sometimes look at the report and not have a good grasp on exactly what it means. So we are constantly making improvements. We have a SEPR technical guide that we provide to LEAs where it gives all of the business rules. We are constantly updating that technical guide just to make this more accessible to LEAs. For example, when we're looking at an indicator, for example, secondary transition, what exactly are we looking at? And so we have an actual technical guide that they can go and look at, and it tells them exactly where we get the data, which students are included in the universe, which ones are excluded, what's the equation that we use. And so we are trying to be as transparent as possible. And so I think, again, that the SEPR dashboard is going to make that really improve our efforts when it comes to being proactive, transparent and clear.

00:14:59.26 >> Yeah. It's a balance when you want to be as transparent as possible, but then given who your audience is and may not have all of the data and technical expertise to understand, it can be really challenging. So it sounds like ...

00:15:13.41 >> It absolutely is. Yeah.

00:15:15.13 >> Yep. We work with a lot of data folks on this project, and they have to make it accessible to me sometimes. So it is a balance, and going into our third year, I can say that every year, the support documents and the technical guide and the report itself just becomes more and more readable.

00:15:39.08 >> Mm-hmm, great. Any other things you want to mention about your system?

00:15:45.06 >> Yeah. I think I'll just add that, Karen, thank you so much for the robust discussion about all the work that has gone into this. In the 2-year lead up to the delivery of SEPR and this new accountability framework to our community, there were a couple of big lessons learned that I think that other states could benefit from. And the first one that comes to mind for me is really improving data chops within the agency. Right? We have a data team that's really committed to special education, data analysis and research. But what we also knew we needed was to build our own programmatic data capacity as well. And in a lot of ways that meant Karen and I digging in, understanding our data much better and also growing the team to have a program-based data analyst. So I would say to other states, don't be afraid of expanding your data chops ahead of this kind of endeavor. The second is that measure things that matter to your state. U.S. Ed prescribes for us a list of measures for program evaluation, but U.S. Ed also charges us with general supervision that exceeds that list of measures. 

00:17:07.84 >> Mm-hmm, mm-hmm.

00:17:08.64 >> And so one of the questions we really asked ourselves was, "What matters for our district in terms of measurement? What areas do we know are unique and can be uniquely served by the folks who have set up shop to educate 50 percent of our kids here in the charter sector? And how can we measure them to get them to that space of improved outcomes for students with disabilities?" So we really thought critically. And then the third and the last one is to take risks. And we really presented this to the field in the initial sort of sales years, as I call them, those first few years of build-out where Karen worked tirelessly with her team to engage stakeholders, not once, not twice, but three times to really ask them, "What is it that you want to see, to be really open to the feedback of the community and to genuinely not arrive to those stakeholder engagement sessions with an agenda about what this report should and shouldn't say?" And I think that's a hard selling point for state education agency leaders when you're crafting your stakeholder-engagement models. But what we knew was that if the field, meaning LEAs and our families, don't buy into this framework, then it will be for naught. So those are just some tips for states who are also moving in this direction.

00:18:34.20 >> Great. Well, thank you for sharing those tips and Karen for walking us through your system and the support you've provided, the challenges. It sounds like you've all put so much time and effort, and I hope it's really paying off now and seems like such a useful and beneficial system for everyone in your LEA. So kudos to all of you, and I know it'll just continue to get better. Well, thank you, Victoria and Karen, so much for being on and sharing the story of your SEA's LEA Determination System. It was an amazing journey. And thank you for partaking and letting us in on it, really appreciate it.

00:19:13.38 >> Thanks so much.

00:19:14.61 >> Thank you.

00:19:17.39 >> A Date with Data is produced by the IDEA Data Center, which is funded by the U.S. Department of Education. Have a story about special education data that you'd like to share? We'd love to hear from you. Reach out to us at ideadata@westat.com. To learn more about our center and our work, visit us at ideadata.org.