For the past several years, pursuant to S.L. 2021-180 Section 7.27(a)(6), the North Carolina Department of Public Instruction (NCDPI) and SAS Institute Inc. (SAS EVAAS) collaborated to provide educators, policymakers, and other stakeholders with insight into the impacts of the pandemic on the K-12 public school system. The “Lost Instructional Time Impact Analysis” and “Recovery Analysis” presented average differences between students’ expected and actual performance during 2021 and 2022, respectively, and define recovery as progress back towards those expectations.
As part of our continued commitment to evidenced-based policymaking and transparency, NCDPI worked with SAS EVAAS to develop and share a new report for North Carolina education leaders known as "2022 Year-Over-Year State Analysis Report." The analysis focuses on how state achievement changed from one year to the next from 2013 to 2022. This new analysis looks at achievement trends prior to the pandemic, impacts of the pandemic, and benchmarks for recovery. Districts received local reports to help inform local discussions about recovery from the pandemic.
Its purpose is to provide insight into understanding the extent to which students have recovered across the state since the pandemic and answer a common question -- how will we know when students have fully recovered from negative impacts due to the pandemic?
This report includes new visualizations and tables that help us answer this question in new ways. For example, in the graph below (see Figure 1), light blue shading indicates the time period for the Pre-Pandemic Trend Line. The orange shading indicates the time period for the Pandemic Impact, and the purple shading indicates the time period for the Recovery Thresholds.
Figure 1. Year-Over-Year Trends and Impacts from the Pandemic (EOG Math Grade 3) with Shading to Indicate Three Key Time Periods
When reviewing the results, consider the following questions for reflection:
- During the pre-pandemic period, was achievement increasing, decreasing, or holding steady?
- Assuming the Pre-Pandemic Trend had continued, what was the average magnitude of the pandemic’s impact on state achievement across different assessments?
- Which assessments had the largest negative impact from the pandemic?
- In comparison to the Pre-Pandemic Trend extended to 2022, what is the continued gap in achievement?
- In comparison to the achievement threshold based on the most recent three-years prior to the pandemic, what is the continued gap in achievement?
- For each assessment, how does the magnitude of the pandemic compare to the remaining impact?
- Which assessments have closed the gap in achievement, and which are furthest from a full recovery?
The Year-Over-Year report has been extremely useful for our district to understand how our performance compares to the rest of the state and the extent to which recovery has begun. We particularly value the inclusion of multiple comparison data points.
~Lindsay Messinger, Ph.D., Director of Research, Evaluation, and Analytics, Charlotte-Mecklenburg Schools
- 2022 Year-Over-Year State Analysis Report to the General Assembly
- Top Takeaways and FAQ report
- NCDPI and EVAAS Presentation on Statewide 2022 Year-Over-Year (YoY) Analysis Report passcode: @7.EYRr4