DMS in MS: Mississippi Talks Differentiated Monitoring and Support 2.0, Part One

00:00:04.25  >> For the IDEA Data Center, I'm Amy Bitterman, and this is A Date with Data. Every month, I sit down with data-quality influencers from around the country to share their stories about special education data and the work they do to improve outcomes for children with disabilities.

00:00:21.69  >> Welcome to A Date with Data. On this episode, I am joined by Sharon Coon, who is the Director of Data and Compliance, and Jennifer Boykin, who is the Executive Director of the Office of Special Education at the Mississippi Department of Education. Mississippi was part of the second cohort of states to go through OSEP's Differentiated Monitoring and Support, or DMS 2.0. Sharon and Jennifer are going to share their state's story, starting with how they prepared for the data portions of DMS, all the way to reflecting on how the process has really helped to improve the quality of their IDEA data. Welcome to both of you. So glad to have you on. So to start off with, could you just each say hello and tell us a little bit about yourself? Jennifer, do you want to go first?

00:01:09.19  >> Sure. I'm Jennifer Boykin. I'm the Executive Director of the Office of Special Education for theMississippi Department of Education, and we were part of the DMS Cohort 2. Excited today to share a little bitabout the preparation that we went through to begin the discovery process and then on into the engagementprocess and how that specifically relates to our data processes.

00:01:29.62  >> Thank you. All right. And Sharon?

00:01:32.70  >> I am Sharon Coon. I am the Director of Data and Compliance in the Office of Special Education, and that bureau, the Bureau of Data Compliance, houses three of our offices. One is the Office of Integrated Monitoring Systems. So that office takes care of all of our cyclical, intensive, targeted, and universal monitoring. Our second office is the Office of Dispute Resolution and Parent Engagement, so formal state complaints, dispute resolutions, mediations, things of that nature. And then our third office is the Office of Data and Reporting, which is responsible for all of our data collection and reporting for federal data and state data.

00:02:13.46  >> Great. Thank you. So, Sharon, you had your hands in a lot of the DMS work, it sounds like.

00:02:18.22  >> Yes.

00:02:18.43  >> Kind of every component was somehow related to your ...

00:02:24.27  >> It was. But each of our offices have a director, an office director, so I have a really great team that works really well together, and so I just get to be the one to help out with that. They do all the real work.

00:02:37.93  >> To come on podcasts, you get perks like that.

00:02:40.28  >> Yeah. Exactly.

00:02:42.35  >> All right. So let's start at the beginning. Can you talk about what you did to really prepare for DMS, in particular getting ready for the data portions?

00:02:52.40  >> Yeah. So I'll start and give a high-level overview, and then I'll let Sharon go into more specifics about the data components. So as a leadership team, which includes our three bureaus ... Sharon just described her bureau ... we have two other bureaus within our office. We have fiscal support, which handles all of the funds that flow through our office, both federal and state, and then we also have our bureau that handles all of our outcomes, our technical assistance and things like that. And within each of those bureaus, there are offices, and there are directors there, too. And so those ... That's a little bit about our structure, so you'll know what comprises our leadership team. And so we began early meeting pretty frequently with our leadership team and just really pouring into those protocols. And it was a really unique opportunity to do some inter-bureau collaboration. And so, like Sharon said before, so much of the DMS preparation and activities and engagement pieces were ... fell within her office, the vast majority of them. So it was really important for our teams to come together to begin the preparation process because so much of what happens in the other bureaus inform our data processes and populate that data for us. And so when we were going through those data protocols, the different components, the monitoring piece, the fiscal piece even, those components were just as important to the data and reporting piece as anything else. And so it was really a great opportunity for our team to see how integrated all of our work actually is, even though when you really think about it high level, it seems very siloed, it's actually not. And so that process, that preparation process as an entire leadership team, was super important to being able to begin to bridge that understanding about the importance ofeveryone's activities feeding into the pieces and components that we actually report. And, Sharon, do you wantto kind of go into ... specifically to the data-preparation pieces?

00:04:42.00  >> Yeah, I think it was important that we all worked together for a couple of reasons, just like Jen said. Prior to having the Bureau of Data and Compliance, each of those offices were very siloed. Monitoring did their thing. Data and reporting did their own thing. Dispute resolution did their own thing. So by putting all of those into one bureau, those three teams work together daily, very, very integrated, so that data that we're having to get from monitoring is all flowing through one office. So our actual data from our student package system, our data from our monitoring and data from dispute resolution is flowing through one place. So that was one place that really, really helped us not to have silos. And then to speak to what Jen said, even larger was bringing in the other bureaus and letting ... and getting them to work with us, and so they could see exactly what data pieces impacted them and then what data that we were getting from them impacted our work. One of the most important things that we did though, as soon as we realized or that we were notified thatwe were in cohort two, we began working very closely with IDC and using their data-processes tool, and thatwas probably the singular most helpful thing for that office of data and reporting and really for the other officestoo because we had to very clearly document every single step of each process that we used for each datacollection piece. So of course we did those data processes for all of our indicator data, but we were also ableto create the processes for significant disproportionality, our processes for our district determinations. And I know child account is not necessarily separate from our indicator data all the time, but that process and have itvery clear, we outlined what we were going to do and how we were going to do it. And it also helped us toidentify some deficits that we may have in our collection processes and pieces that we needed to work on. So before we ever even jumped into the monitoring protocols with the team, we were able to identify some places where we needed to grow and put in some more supports. So that ... I think that was probably, for the data and reporting side, probably the most helpful piece of work that we were able to do prior to the visit.

00:07:14.20  >> Absolutely. And I would add to that just that these changes and these revisions to our processes and things, these were things that were going to happen anyway. And it was ... We used the DMS process to kind of speed it along a little bit. It gave us a sense of more urgency, I guess, is the best way to say it. And so it allowed us to provide a little bit more focus and kind of shift our energies into that work. And also, too, I think it's fair to bring up here, Sharon, that, you know, in that whole process, we were also beginning to transition into a new statewide data system. And so all of the work kind of just piled on at one time, and in some ways it seemed very overwhelming, but it was a necessary process for us to go through to clarify some of our processes. That kind of sounded redundant, but nonetheless, it was a useful ... it was a useful thing forus because we were in the midst of doing that anyway with this transition to a new statewide collection system.And so just using the urgency that was given to us by the ... by being in DMS 2, which really just kind of spurred everything along, which was, in retrospect, very helpful. At the time it was very stressful, but in retrospect, we can see that that was definitely a positive thing for us to engage in.

00:08:30.91  >> Great. So it sounds like this process really helped to get some things ... important things done that you knew you needed to have done anyway, and it wasn't just kind of OSEP coming in and imposing this additional work on top of you, but really as a kind of trigger to make changes and improvements, so that's good to hear.

00:08:53.09  >> That's for sure.

00:08:54.23  >> So can you talk about the actual DMS visit with OSEP, in particular kind of the data components? How did it go overall?

00:09:04.02  >> So I'll start, Sharon, if that's okay. The visit itself was actually very good. We prepared. We felt like we prepared well for it. And when I say prepared well, we don't mean by ... that we had all the right answers. We knew we had identified some areas of deficit that we had. We also knew some areas of strength that we had. And so we were well prepared to answer those. And I will say that we have a very, very goodstate lead. And one of the things that she helped us understand throughout the preparation process was justbe clear about what you are doing and where you are going. And so we were able to articulate that in our on-site. There were a lot of questions about how do you report it? What do you do with it? And I feel like they already knew because of the preparation activities and some of the documentation we sent, they already knew where ... Picking on us is not the right word because it's factual. It's not picking on you if it's not true. But they knew our areas of need and they were asking specific questions about that, which then, you know, we had some great note takers with our TA centers, and that we used that information that maybe we were a little too nervous to capture on our own, but our TA providers were able to capture it. And then we can use the questions that they asked us during the on-site to then make sure that our processes ... that we were still in theprocess ... still revising were addressing the questions that they had during the visit. And so because we werein that ... the midst of creating things still and in the midst of clarifying some of our processes and procedures,we could use their line of questioning to just further solidify or make an adjustment where we saw thatapparently is important to them, and so we need to make sure that our procedures clearly indicate that it isalso important to us. And so that was kind of the takeaway, the large 30,000-foot view takeaway for me on thedata pieces from the on-site visit.

00:11:08.74  >> I think Jennifer hit the nail on the head. I think there were lots of questions, and they could get kind of deep. They could kind of get into being very detailed. But it was very clear when they were questioning us that they had really reviewed the documents that we had sent ahead of time and really knew where to drill down so that they could identify where we might have some pain points and things like that. Also, to Jennifer's point, I think one of the most important pieces was we started ahead of time identifying issues that we had and we didn't wait. When we realized that there was a data process that needed to be corrected or we may havenot been collecting it in the correct manner, we may not have been completing the verification process exactly the way we should have been, we went ahead and started changing it right then and didn't wait until theycame, and we got the report. So I would say that just about every question that was asked, we would have athree-part answer of this is how it has been done, this is what we are doing now, and this is where we plan tobe in 6 months from now because we can't get everything complete. We can't make all the changes at one time. But here's the process for change. And that was particularly true for our monitoring and the collection ofour monitoring data. Very true for some of our indicator data where we had a process, as we went through thedata processes toolkit, we realized that there was a better way to do things. And so we began working on that better way. We may not have had enough time to cycle through and actually implement everything that we wanted to. So it was very helpful to know that they wanted that information. Here's what we did. Here's whatwe're doing now, and here's where we plan to be.

00:13:10.56  >> And what was really neat about that was, as we were talking through those three componentsof things, they would say to us things like this, "You may want to consider, or you may want to think about," andevery time that would happen, we would look up at our TA providers. They're like, "We got it." And so they're taking notes for us because when OSEP says to you, "You might want to think about X, Y, Z," you might want to think about X, Y, Z.

00:13:35.89  >> Yeah.

00:13:36.50  >> So go back to your processes and say, "Yes, we've addressed that," or, "No, and they asked aquestion about it. What what do we need to do?" And so that was really helpful. And in many ways, there were sometimes they would say things like, "You might want to think about," but we couldn't in the moment, in that on-site environment, which is a little nerve-wracking. Our team ... the OSEP team was great. Your brain can't wrap around all the components of that, and to have those TA providers there taking those notes and going ahead and putting their thoughts on paper so that we can have their thoughts already ready for us to think about and follow up, that was ... I cannot even begin to tell you how helpful that was. To feel confident that you didn't have to remember every single thing they said, that your TA providers are going to help you with that, oh, my gosh, that was the best thing.

00:14:24.74  >> It was super helpful, too, in that if OSEP said, "You might want to consider X, Y, Z," and then when we would debrief at the end of the day, like she said, you can't wrap your mind around it in the moment. The OSEP team was amazing. But anybody's going to get weary after three days of ...

00:14:42.28  >> Yeah.

00:14:43.42  >> ... asking intense questions. And so ... But it would help at the end of the day for that debrief to go back over it with those TA providers, and some of those things that they said you may want to consider, we could look back more objective and objectively and say, oh, wait, no, we are doing that. Maybe we didn't explain it well enough. Maybe we didn't make ourselves clear. And so it gave us the opportunity to go back the next day and say, hey, you asked about this and asked us to consider this. Let me give you a little more detail or let me provide some clarification so that maybe now you'll see ... you have a better idea of what we're talking about and what we're doing. Because, again, after three days ...

00:15:28.54  >> You're fried.

00:15:30.18  >> ... you are fried, and your words don't always work.

00:15:33.34  >> Yes.

00:15:33.65  >> And also too, our data processes are so detailed, and there are so many components. And we live in this world every day in our own little world, so to speak, and so we know all the components. And when Sharon and I are having a conversation about different processes or a data capture or our reporting piece or ... We don't have to give background on that because we already understand our own system. And so sometimes whenever they would ask us those, you might want to consider, it's just because they don't have allof the surrounding pieces that we assume they know because we know it.

00:16:10.40  >> Yeah.

00:16:10.81  >> And so a lot of times it wasn't even that we didn't explain it well. It's that we assume they knewthings that they didn't because they don't live in our world every day. And so that often times led to even more questions, which was super helpful, because then it would often times answer other questions they had asked or things they were going to ask and now didn't have to ask. And so I never felt ... And they told us this. The OSEP team told us up front, "If you answer something today and then you think on it and you want to follow up," like what Sharon said earlier, we did that a number of times. We're like, "Oh, okay, thanks so much for bringing that back up. We appreciate that." They liked that we thought about it, and we came back and said, "I don't know that we explained this fully, or there's also this component that we didn't really talk about, but it's kind of a big deal." And they really appreciated that. And they gave us the freedom and flexibility to be able to do that because they truly seemed to want to understand our system.

00:17:04.91  >> Yeah. Wow. Well, you shared some really great lessons and tips in all of that. I think one is how important working with the TA centers is and having them be part of this process and support with you.And then also, I really liked how you talked about answering the questions, sort of thinking of it in those three parts, that I think being really transparent and since you did so much work ahead of time and really understood, here's probably the areas that they might ask us questions about, you were really prepared and already had a response ready that, yes, we're aware of that, and this is what we're doing and planning on doing to address it.

00:17:42.84  >> Please listen to next month's episode for part two of Mississippi's DMS Data Story. Sharon and Jennifer will share lessons learned and tips for states embarking on their DMS journey.

00:17:56.83  >> A Date with Data is produced by the IDEA Data Center, which is funded by the U.S.Department of Education. Have a story about special education data that you'd like to share? We'd love to hear from you. Reach out to us at ideadata@westat.com. To learn more about our center and our work, visit us at ideadata.org.