Great PR: Puerto Rico Builds a Rich Data System

00:00:04.30 >> For the IDEA Data Center, I'm Amy Bitterman, and this is "A Date With Data." Every month, I sit down with data quality influencers from around the country to share their stories about special education data and the work they do to improve outcomes for children with disabilities.

00:00:21.61 >> Hello. On this episode of "A Date With Data," I am joined by Irving Feliciano and César Gracia from the Puerto Rico Department of Education, who will be talking about the data system that they use to collect their 618 and SPP/APR data. Welcome. Can you start by just saying a little bit about your role at the Department of Education? Irving, do you want to go first? 

00:00:44.84 >> Yeah, of course. My name is Irving Feliciano. I've been the Part B Data Manager for Puerto Rico for the last 18 years.

00:00:53.26 >> Oh, wow. Thanks.  

00:00:55.18 >> Yeah. I'm César Gracia. I'm a Project Manager for Inclusion, which, we are the developers of the data system that PRDE has implemented.

00:01:08.06 >> Mm-hmm.

00:01:08.37 >> I've been here with Irving for the last 14 years.

00:01:11.73 >> Wow, long time. Great. Well, as we like to talk about on the podcast, we know that every state and territory has a unique data story. Can you share with us something that's unique about Puerto Rico's data story?

00:01:27.26 >> Well, actually, I think that something that is really particular of Puerto Rico is that we are a unitary system.

00:01:35.43 >> Mm-hmm.

00:01:35.80 >> But we have over 800 schools. So for being a unitary system, we have a lot of schools. That's one of the parts that I think is ... 

00:01:48.12 >> In addition to that, something that's very particular about our data is that over 30 percent of our students, of the public students in Puerto Rico, are from special ed programs. So that's a very high percentage of students enrolled in the public system that are part of our data. We have over 104,000 students on the special ed program. And the data collection process in Puerto Rico, it's very unique because we have a state class action act in addition to the APR and the 618 data that we have to submit.

00:02:25.21 >> Mm-hmm.

00:02:26.03 >> It has 87 indicators, right? So that's a lot of data that our users or teachers and everyone involved has to be able to put in the system so PRDE can comply on submitting that data.

00:02:41.50 >> Wow.

00:02:42.00 >> In addition, I believe our data has evolved significantly. We recently, in the past 5, 6 years, had the Hurricane Maria, which was a huge storm that happened in Puerto Rico that everything got destroyed. But that helped us. The system was our main source of information to continue our services for students, and we were able to ... back in track, real, real agile because of having a system that where all the LEAs or ACAs, because we are a unitary system, can use. So the technology and the data system that we have helped us real, real good to be able to continue the services to our students.

00:03:28.57 >> Yeah. Also, I think that having a system that is custom-made to Puerto Rico ...  

00:03:34.80 >> Mm-hmm.

00:03:35.81 >> ... helped us a lot to get the necessary elements from the data system so we can have ... And we can change whenever we want to.

00:03:47.21 >> Mm-hmm. Yeah.  

00:03:48.49 >> Yeah.

00:03:49.53 >> Flexibility.

00:03:51.12 >> Yeah. It gives us the flexibility, especially ... PRDE owns the source code, right?

00:03:55.78 >> Mm-hmm.

00:03:56.10 >> So we as a developer worked in accordance with PRDE, in this case, Irving, and he established the requirements, and especially in the state side of the class action act, we get a lot of changes of new data that has to be collected for corrective action plans and all this stuff. So we are able to custom-made as the needs of PRDE as.  

00:04:18.02 >> Yeah. I think that's definitely a unique part to your data story, is having that flexibility and the source code and developing your own system and really having the ownership and ability to get in there and make those kind of more on-the-fly changes, and that can be more challenging in states that maybe have contractors or more set systems that don't have all of those, that amount of flexibility, so ...

00:04:43.58 >> Totally agree. 

00:04:46.15 >> Well, glad you were talking about your data system, because that's really what we're here to hear more about. So can you tell us a little bit more about this data system that you started to share about? What are some of the strengths and the capabilities that it has?

00:05:02.49 >> Being a customized system, it helps us a lot, so that's one of our main advantage, also that we ... On the develop side, we integrate with all other PRDE data system, finals and all other Department of Health. So it helps us, that part.

00:05:26.73 >> And one of the ... I think when we talk about this, when we're doing brainstorming sessions and I'm working to, where are we moving, and what's the next step with the system, we always call it our ecosystem, right, because it's an application that has different type of users.

00:05:42.91 >> Mm-hmm.

00:05:43.25 >> We have over 74,000 users. Fifty-two-thousand parents have accounts in our system, right?

00:05:54.78 >> Oh.

00:05:54.94 >> So we have to be able to use the technology and the data to connect and to get the family engagement also with our community. And in addition to that, I think the main part when we talk about this is that everything start with part C, right?

00:06:12.43 >> Mm-hmm.

00:06:12.64 >> Because in this system, we have the transition from part C in our system, right? In MIPE, they start the referral process, and we get the students in MIPE since they are in part C coming to part B, right?

00:06:25.88 >> Yeah.

00:06:26.08 >> So that allowed us to have all the history of that student with us.

00:06:29.27 >> Mm-hmm.

00:06:31.04 >> Irving, I don't know if you want to talk about the 618 data, right?

00:06:33.70 >> Yeah, of course. I think that this is kind of a few years back, but we integrate or make all the reports that are required by the federal government. All the existing data is already customized on our system, so we collect all the data there. And we produce and generate all those files to be ready to submit. So that's something that I think is really great, because if I win the lottery tomorrow, somebody can go ahead and keep doing ...

00:07:10.51 >> Mm-hmm.

00:07:11.12 >> ... do the reporting part also.

00:07:14.48 >> Yeah.

00:07:14.61 >> Something that we ...

00:07:16.68 >> In addition to that, just to sum it up, when we were developing this system that we have been developing for the past 12 years, we always made it a compliance-based system, right? So we are trying to make sure that our data and our processes are done in the compliance way, right?

00:07:40.57 >> Mm-hmm.

00:07:40.86 >> So let's say we have two processes. We have our billing system. We have our student system, the IEP, all those areas. For example, all the APR data is collected through the system. We do Indicator A. We do ... All those processes are developed within the system, which allow Irving to have a governance of the data, knowing where everything is coming from. And also, having this amount of users in our system allows Irving to create roles. And they all have different roles within the system, so our data is secured.

00:08:24.49 >> Yeah.

00:08:24.59 >> And it's managed properly based on the location that you are. We have school level, student level or a very admin process. Also, that Irving loves this part of the system, we use ... We believe in the data. We believe in making decisions based on data. So Irving developed with us ... We have a report model or application where you have Power BI dashboards where the users can understand why their data or what their data means, right? So he always says, "We have to make sure our users consume the data so they know how important the data is." And we use Power BI, and we also have different, other reports.

00:09:10.59 >> Yeah, and I think also development of the system, we have alerts that the system provides to our users If something is going to be wrong or an inside attention of somebody. The system will alert the user so they can make the correct arrangement to be able to complete the data on the system.

00:09:35.86 >> Yeah. We have all the lifestyle of the student in our system. We get the student from the registration process until the student exits, right? But we've got all that data. We even have our providers. They go into the system, too, and they submit their invoice based on the data that they enter in the system. 

00:09:55.56 >> Yeah, and I think having just one system, it will allow just ... It's just one and one only system that ... It's only one source, so it's not different sources of data. It's just in the same system. So that, I think, is a benefit for us.

00:10:13.47 >> Yeah. It's a huge benefit because it allowed us to be able to ... When we are going to generate the data, if it's going to be to the 618 data, if it's going to be APR, if it's going to be the state class action act that we have to submit the report annually or we're just getting petitions of data, or we're looking at data to be able to do public policy regarding the tendencies that we can see on the data. It's all recollected in the same system, even if you are a school teacher or you are a speech and pathology specialist, right? Everyone enters the data in the same system where everything is validated to make sure our students' IEPs are in compliance, that whatever we stated in that IEP, we are actually offering this student those services.

00:11:03.06 >> Oh, my gosh. I'm speechless. I don't know if I've heard of another system that is a one-stop shop, and not only that, but it's integrated across other state systems. And, wow, that must make your job feel a little easier in some ways, I would guess, Irving, that you're, like you said, dealing with the inconsistencies you can get if data ... you're pulling discipline data from one system and child count from another. And the race ethnicities maybe aren't the same, for example, or just all of that, those data pieces, just having that one source of truth. among all the other positives and strengths you mentioned.

00:11:41.47 >> Yeah.

00:11:42.04 >> That's amazing.  

00:11:42.64 >> Also we require all our teachers to develop the IEP for a student on the system.

00:11:48.54 >> Mm-hmm.

00:11:48.94 >> So we have like 97 percent of ...  

00:11:52.62 >> Yearly, yeah, in Puerto Rico, that's a unique area that is very important that we share it because, in Puerto Rico, we have to revise the IEPs yearly as every state. But in Puerto Rico, because of the class action act, we have to revise them in a short period of time. Most of the states, we heard, they revise their IEPs on their student birthday.  

00:12:20.35 >> Mm-hmm.

00:12:20.59 >> But in Puerto Rico, they all have to be revised between March 15 and May 30th.

00:12:28.58 >> Mm-hmm.

00:12:28.99 >> So we're talking about 2 1/2 months to revise 105,000 IEPs, so we have to make sure our teachers have the engine and the appropriate tools ...  

00:12:41.72 >> Yeah.  

00:12:41.87 >> ... to be able to do these processes and do it in compliance, because we don't want to just get data. We want quality in our data. We want our data that makes sense, right? 

00:12:50.83 >> Mm-hmm.  

00:12:51.04 >> And it's actually the reality of what is happening in our classrooms.  

00:12:54.94 >> And to have a system that can handle that much traffic, doing that heavy a load in such a short time, too ...  

00:13:01.49 >> Exactly.  

00:13:02.17 >> ... probably wasn't the easiest thing to design, but ...  

00:13:05.70 >> Yeah. We do escalators for the ... We know the big processes that we have.  

00:13:15.32 >> Yeah.

00:13:15.54 >> Actually, for us, it will vary monthly and by the time of the month because, as we were saying, we have to build the invoice process that is submitted by the vendors for the related services.

00:13:28.18 >> Mm-hmm.

00:13:28.94 >> And we get monthly over 130,000 PDF files on this system. 

00:13:35.33 >> Jeez.

00:13:35.57 >> So there's a lot of data, a lot of information, so we had to manage. We are based on the cloud. The system is on the cloud in Microsoft Azure.

00:13:46.75 >> Mm-hmm.

00:13:47.32 >> That allows us to be able to have more capability at certain times of the day and dates to have more resources for our servers.

00:13:57.70 >> Hmm. Oh, my gosh. Well, I know you mentioned 800 schools that are in Puerto Rico. So how do you work with them to build their capacity to be able to use the data system and enter high-quality data?

00:14:13.52 >> That's a really hard task, but we do all different types of ... We do trainings in person, so multiple training at the school level, so we do that. Also, we have short videos, quick guides. I don't know. What else, Cesar?

00:14:35.73 >> We also do the ... We do all asynchronic trainings where we prerecord some of the main topics that we're going to do.

00:14:43.83 >> Mm-hmm.

00:14:44.81 >> And then we proceed to prerecord it, not as a short video. More of a formal training and make it available to our teachers, to our school. We have over 5,000 teachers in in those 800 schools, right?

00:14:59.43 >> Mm-hmm.

00:14:59.85 >> So whenever we try to make every change that we do on the system, we try to do the training. But we know that that's not going to be possible to do in a face-to-face type of thing because it's a lot of person ...

00:15:12.89 >> Yeah.

00:15:12.98 >> ... a lot of ... and we don't want to affect the students' teaching time with the students in the classroom, right?  

00:15:19.62 >> Mm-hmm.

00:15:19.92 >> So in addition, Irving has, I would say, a ticket management system where when students ... When the teachers have incidents or situations within the system, they will create this ticket that will go all the way to the Irving and his personnel to validate. That helps us a lot, to know what we need to get better, right?

00:15:47.53 >> Mm-hmm. Mm-hmm.

00:15:48.18 >> Or what are the areas that we have to make sure we do our training based on those type of situations? Because most of those tickets that come in are actually questions regarding how to do processes within the system.

00:16:04.67 >> Mm-hmm. Mm-hmm.

00:16:05.95 >> Also, we do vocal groups so we can get what is needed for some regions or some areas in specific. And because it's a custom-made system, we can do improvement. We identify any kind of improvement that needs to be done, so ...

00:16:27.21 >> Yeah. In addition, I think how we build the capacity in schools primarily, I would say, within the pandemic, all teachers got, or most of the school personnel including students, teachers, everyone got computers. So got new computers, got more infrastructure to be able to access this system. PRDE has encouraged on getting better Internet in the schools.

00:16:57.30 >> Mm-hmm.

00:16:57.63 >> So that's all going to add up to getting a better system. In addition, the data requirement process is required, right? The teachers have to input this data in the system.

00:17:09.53 >> Mm-hmm. 

00:17:10.05 >> It's part of their duties in the day-to-day basis. So most of the stuff that we do is ... which has helped us a lot, I believe, in the past couple of years, is that whenever we need data, for example, for, let's say, Indicator 13, right, or Indicator Seven, which are data that you've got to compare to two processes, right, one that happens, for example, in Indicator Seven, it will start in their first eligibility and then whenever it comes into the school age, right? 

00:17:42.97 >> Mm-hmm. Yeah.  

00:17:43.42 >> So what we do is, when we identify that we need to collect this data in certain places or times, we make sure that we do it at the moment of the IEP revision, right?

00:17:54.97 >> Mm-hmm. Mm-hmm.

00:17:55.52 >> Because I revise it yearly, so I make sure whenever I'm going to do that, the system will present to you all the different areas of the process where we're going to collect that data ...

00:18:06.92 >> Yeah.

00:18:07.22 >> ... to be sure that we collect the data in a timely manner to be able to submit the reports yearly. But I think this is one of the main areas to improve because we always want to keep improving the process.

00:18:18.92 >> Of course.

00:18:19.48 >> One of the things that we want to do and we are currently developing, and it should be released very, very soon, I think probably this month, is the inclusion of artificial intelligence in the process of assisting the users with the different processes that they do within MIPE or the process that they're going to do in MIPE, but they require prior knowledge of what they're going to start doing in the process.

00:18:49.43 >> I also want to mention, Amy, that when we do the focus groups, we'll include all the stakeholders. We try to include parents, teachers. So we got the input from all the areas, so we just don't make any decisions based on a central level. We have a decision that it will involve everybody on these changes.

00:19:17.22 >> Yeah, and having the end users like the teachers and the families who are the ones hopefully that will be, in the end, consumers of the data and using it to make decisions, having their buy-in and input, too, is so important.

00:19:34.30 >> Yeah.

00:19:34.60 >> Yeah. Now that you mentioned that, we also develop a part for the parents. So we have an app for the parents so they can see all the students' information, so it's something that is really ...

00:19:53.07 >> Yeah, it's something that is being ready, is ... Everyone is very happy regarding that, that we developed a mobile app that makes parents ... It's focused 100-percent right now on the parents because we have a parent portal where parents can see most of their students' information like evaluations ...

00:20:17.29 >> Yeah.

00:20:17.42 >> ... like appointments for evaluations, IEPs ...

00:20:20.92 >> Mm-hmm.

00:20:22.05 >> ... the detail of the service provider. They could ... If they want to do or establish a claim on the due process, they could do it through the system.

00:20:30.93 >> Mm-hmm. 

00:20:31.58 >> But in addition, this last October, we launched an app, a mobile app, for the parents, and that has helped us to increase their engagement with the parents. But all that was done within the parents, right? We talked to the parents. We showed them what we were going to do, and they pretty much ...

00:20:53.70 >> Yeah, they gave us feedback.

00:20:54.51 >> ... gave feedback, yeah.

00:20:56.72 >> Helped with the development kind of and giving their thoughts.  

00:21:01.92 >> And we made the parents be aware of the data so they can see if anything is wrong.  

00:21:08.10 >> Mm-hmm.

00:21:09.31 >> They will go really fast to the teacher and ... 

00:21:11.87 >> Mm-hmm. Mm-hmm.

00:21:12.75 >> ... and make sure that ... 

00:21:14.72 >> Everything is ... We ... When we were going to ... When we were explaining to everyone, we were using the following concept. We were taking away the walls from the student's file, right? It's going to be in a crystal-clear ... 

00:21:31.20 >> Yeah.  

00:21:32.27 >> ... square where everyone can see it.

00:21:34.15 >> See. Mm-hmm.

00:21:34.69 >> And when I say "Everyone," I'm saying the people that should have access, which is the parents specifically. And they go through evaluation process to make sure that they could have access to the student's information. But they do see real-time all the information that is input in the system for their students to make sure that they ... We have an extra validation process, right, that our parents can see the data. That helped us a lot in the process of ... We have an electronic signature application within the system, too.

00:22:08.17 >> Mm-hmm.

00:22:08.43 >> And the parents now, they sign all the documents that they want to. They don't have to do it through any other alternative signature element. They could use our system, which is validated by all the rules and legislations that have to be validated. So they even have an account where they could sign and protect their signature to place it on the documents, and they automatically get a copy of the documents signed by the parents or whoever is part of the process. So we keep evolving. We're trying to keep evolving.  

00:22:42.01 >> Mm-hmm. For sure. Oh, my gosh. Well, I think you've  touched on several ways probably already that this data system has really helped to improve the quality of your IDEA data. Do you have some other examples that you can discuss? 

00:22:59.60 >> I will say that one of ... I think the best or the most important thing, we have four things. First is the integration with the different systems in PRDE, right? That allows us to make sure that the data ... it's properly entered within their system, which, we don't want people doing things twice, right?

00:23:20.48 >> Yeah. Yeah.

00:23:20.77 >> So I just want you to do it one time. If you did it one time, I could collect the data. 

00:23:24.56 >> Mm-hmm.

00:23:25.18 >> Second, our system is a transactional system, right? So we make sure that the process, for example, let's say the registration process, where you cannot be eligible to the program if you are not registered, right? So you have to do a process one first. Then you do the second process, then the third process, and the system within the business role that is there built, makes sure that the data that is being collected, it's the correct data. For example, when we do the invoice process, right, our specialists, they enter the data in the system. And whenever they're going to ... They have a process that is called its validation process that, within the system, what we do is we have a series of business rules that allows the system to tell the users that the data is not correct. So you have to have to fix it. So we don't allow them to submit invoice, for example, with issues on their data.

00:24:19.90 >> Mm-hmm.

00:24:20.12 >> We make sure that the data is in compliance with their IEP. That's just a small sample. And at the end, this will transition us to the next process, which will be a 618 data in the process where we have a model where we create child count data.

00:24:40.00 >> Mm-hmm.  

00:24:40.47 >> We create environment data. We create the discipline. We create the personnel. We create exiting, right? We have a single model for every single 618 report that it has to be submitted from the special ed side. Irving and I, we developed this series of business rules ...

00:24:59.62 >> Mm-hmm.

00:24:59.93 >> ... to make sure that our data that we're going to be submitting to the next step, which will be the EDPass, is valid and is accurate. And it has all the elements that they have to have to be able to generate that report, so if you want to talk about the alerts that you get in that process.

00:25:21.01 >> And I think that something that will really help us is that, based on the capability of the system, we have a really ... say, a trust with the EDPass coordinator that he knows that our files will be complete, accurate and will pass all the requirements. So that's something that I think we're really proud of that, because I know all the ORE areas. They have some concern for their files. But our EDPass coordinator is really at ease with us because she knows that when we're going to submit, there's going to be a complete, valid and reliable data.

00:26:09.54 >> Mm-hmm.

00:26:09.89 >> And I believe that starts with Irving believing in the data, right? If Irving doesn't believe in the data and what the data can do to the program, none of this happens, right?  

00:26:23.06 >> Yeah.

00:26:23.16 >> Because you have to believe how important that data is, and Irving is a data-driven person. So he's always trying, looking at ways to improve that data collection process.  

00:26:33.93 >> Yeah. The EDPass piece of things, and ... so it sounds like from what you've described in the system, it creates within the system your files that are ready to submit to EDPass. They're in the right format that they're all ready to go.

00:26:49.83 >> Yes.

00:26:50.17 >> That's incredible, and ...  

00:26:52.89 >> And we use all ... Once we generate the files, we do some validation.

00:26:58.70 >> Mm-hmm. We use your ... the IDC validation tools.

00:27:02.89 >> Mm-hmm.

00:27:03.44 >> So we make sure ... So we make the extra ...

00:27:08.57 >> Validations.

00:27:09.09 >> Yep.

00:27:09.24 >> ... validations to make sure everything is correct.

00:27:11.98 >> Yeah.

00:27:12.34 >> Yeah, because he has ... We have a model where Irving, once that year ends, he creates the provisional file so we could start validating the data. Once the data is validated, as is right now, we still submit the same file as an SEA or an LEA. Just the difference will be the zeros. And we only have one code as an LEA. 

00:27:36.70 >> Mm-hmm.

00:27:36.92 >> So it will be just one general LEA.

00:27:39.18 >> Mm-hmm.

00:27:40.48 >> So once Irving is ready, he has a little button where he prints out ... He creates the files, and he will get an output file with all the columns that he has to submit in EDPass.

00:27:53.41 >> Mm-hmm.

00:27:53.80 >> He downloads that CSV file. He passes them to ... All the IDC data checks were credible for us. They help us revalidate the data.

00:28:04.80 >> Mm-hmm.

00:28:05.26 >> And once it's ready, that he submit that data in the EDPass, he has a functionality in the system where that he clicks it. And once he does that, the data is available for the public on a public system, a public page that we have ... 

00:28:23.00 >> Oh. 

00:28:23.27 >> ... that we have info like infographics where you can see the data that was submitted on EDPass, automatically will be uploaded to this page so the general population can consume the data.

00:28:39.80 >> Oh, my gosh.

00:28:40.68 >> Also ...

00:28:41.16 >> I can't believe how fast that is. That's incredible. So it ... and you really put it out there that quickly? There's really not ...

00:28:47.89 >> Yes, we actually do.  

00:28:48.77 >> Oh.

00:28:48.84 >> As long as we know the file pass, it's okay on the EDPass ...  

00:28:53.67 >> Yep.

00:28:53.81 >> We just publish, like Cesar said, that click of the button, and we publish on our public site.

00:29:00.70 >> Hmm.

00:29:00.98 >> Also, it really ... It's been really helpful. There are your IDC data checks because when we do the comparison for the past few years, so we know, sadly, we need to submit in a data note ...  

00:29:15.63 >> Mm-hmm. 

00:29:16.00 >> ... or we need to confirm something that data has changed for.

00:29:21.58 >> Yeah.  

00:29:21.99 >> So that's really helped us.  

00:29:24.36 >> Oh, good. So it sounds like you probably had a pretty smooth transition to using EDPass because ... Did you already have all of those validations kind of built in at that phase since a lot of the data quality checks that had been happening kind of later on, right after the submission, are now happening when you submit initially in EDPass, and it ... Maybe you've doing a lot of those already. So, yeah, can you say a little bit about when you started using the EDPass system and how that went?

00:29:54.30 >> Yeah, of course. I think there's really a benefit, using EDPass, because now I can submit the data right away. And back then, we had ... I had to submit it to the EDPass coordinator ...

00:30:09.44 >> Mm-hmm.

00:30:09.96 >> ... and wait for their input to make sure if everything was right or everything was correct or no error was found or something like that. Right now, as soon as we submit it, we can validate if there is any errors or not.  

00:30:26.70 >> Hmm.

00:30:26.84 >> So that's really something that we really enjoy or really are ... all of that.

00:30:33.13 >> Yeah. I think in addition to that, Irving now has the capability to submit himself the data ... 

00:30:40.84 >> Mm-hmm.

00:30:41.35 >> ... which, that allows Irving to have a little bit more control on when the data is going to be submitted. I think this year we were waiting for the system to be ready, right?

00:30:51.04 >> You were ready well before the system.

00:30:53.04 >> Yeah, we were ready to submit. But I think it's giving Irving even more ownership over the data, which he already has, but it allows him to be able to work on his own schedule ... 

00:31:04.74 >> Mm-hmm.

00:31:04.96 >> ... and be ahead of the stuff.  

00:31:07.08 >> Yeah, and it really ... What you're saying is sort of the goal of EDPass, right, was to get the data out there quicker into the hands of the public and the districts, and that's exactly what you all are doing. So I feel like that's ...

00:31:21.75 >> Yeah.

00:31:21.98 >> ...yeah, a real good ... a real testament to the EDPass system as well as ...

00:31:25.29 >> It was like a learning curve, but ...

00:31:26.78 >> Yeah.  

00:31:27.48 >> ... because back then, we submitted a survey before and then the data files.

00:31:34.73 >> Now, it's like, right, both are in the same ...

00:31:38.85 >> Mm-hmm.

00:31:39.88 >> So you have to enter the metadata and then submit the files. That's really helpful.

00:31:46.41 >> Yeah.

00:31:46.60 >> Pretty cool.

00:31:47.82 >> Well, I know that IDC has been providing some support to Puerto Rico in particular around documenting your data processes for the 618 and SPP/APR indicators. Can you say a little bit about maybe how that documentation and the support has helped improve the quality of your data system and also your IDEA data?

00:32:09.35 >> I think that's been a really helpful part because it's preparing us also for the DMS.

00:32:17.12 >> Mm-hmm.

00:32:19.84 >> And doing that, we can see if there are any gaps or something that we need to improve. So it has been really helpful. I don't know if, Cesar, you can ...  

00:32:31.85 >> Yeah. Especially I think it has made us visualize what we have ...

00:32:38.54 >> Mm-hmm.

00:32:39.13 >> ... and how we can get better with the processes of documenting what we're already doing, even with business rules ...

00:32:46.63 >> Yeah. 

00:32:46.87 >> ... and all the processes that are taking place in that data collection area that we have facilitated for the users. But by the way that we ... I would say maybe not on purpose, but we facilitated that process in a school-level type of way that allowed us to be able to, whenever we started documenting the processes with the guys, we were like, "Okay. We've got this, so we're doing this. We're going to the next step," and everything was already lined up whenever we went to document the protocols. But it also gave us the formality that we needed in that process because we know we're going to be in a monitoring process soon. 

00:33:27.72 >> Mm-hmm. 

00:33:28.09 >> And it allowed us to make sure to validate that what we thought that we were doing correctly, it was actually done correctly. But in addition, for example, for child count, we talked a bit. PRDE is starting to transition to move from an SEA to an SEA/LEA to have both instead of just a unitary system.

00:33:52.13 >> Mm-hmm. 

00:33:52.70 >> So we were able to ask the guys in the process of documenting the data protocols how it would look if we have to do it this other way. And we already started preparing for that. We have already started, and actually, we made some changes to this year's child count process to be able to have our certifications at the LEA level.

00:34:13.45 >> And you know, Amy, that documenting something is the last thing that we do.  

00:34:19.74 >> That we feel like we have time to do, right?

00:34:22.58 >> But it has really helped us because we know that we have made a lot of changes because this system has been 13 years, so 13 years of development has been changes a lot. So this documenting process helped us understand that if we do any change, we have to do the corrections on the documentation part also, so ...

00:34:46.50 >> Yeah.  

00:34:48.05 >> And we have a base now that, in the past, we didn't have. It's just a memory process.  

00:34:55.09 >> Mm-hmm.

00:34:55.34 >> And I think also the compliance officers here at PRDE use IDC for their APR revisions, too.

00:35:03.93 >> Mm-hmm.

00:35:05.15 >> And we do technical data consultations sometimes when we have issues or we have just doubts regarding some topic, and it's been so helpful to us. And so we believe they're part of our family on our process of making ...

00:35:19.85 >> Yes.

00:35:21.49 >> ... as correct as we want it.

00:35:23.46 >> Wow. Well, I've so enjoyed hearing about all this. You have such a robust and rigorous system and process, and I would imagine other states might reach out to you after hearing this, certain territories, to kind of learn more or get some details. And, yeah, I would love to have you back on, too, at some point when you do make those changes and have the SEA/LEA system in place, and love to hear how that's going. So just thank you both so much for your time and sharing with us your story.

00:35:56.99 >> Thank you so much for the opportunity. We are very, very grateful.

00:36:00.73 >> Yeah, thank you. We really appreciate that.

00:36:07.16 >> A date with data is produced by the IDEA Data Center, which is funded by the US Department of Education. Have a story about special education data that you'd like to share? We'd love to hear from you. Reach out to us at IDEAdata@westat.com. To learn more about our center and our work, visit us at IDEAdata.org