This report is part of a series of analyses of publicly available North Carolina Department of Public Instruction student achievement data, known as the disaggregated data files. These FERPA-compliant files contain data on all the state’s public traditional and charter schools beginning in school year 2013-14. My intent in this series is to provide the public with discussions of the usefulness of the NCDPI data, and with descriptive statistics that I feel may be of interest. I will not perform modeling or hypothesis testing.

Performance by Race in CHCCS Public Schools for Grades 3, 4 and 5 in 2018-19

NCDPI publicly available data includes the percentage of grade 3, 4 and 5 students, by race (subgroup), achieving Grade Level Proficiency (GLP) and Career and College Ready (CCR) scores in standardized End of Grade tests. These tests score students into five Levels, from Level 1 as the lowest scores to Level 5 as the highest. Students scoring into Level 3 or above are designated GLP, while those scoring into Levels 4 or 5 are CCR. This means that the GLP student count will contain the CCR count, and thus will be greater than the CCR count. For instance, 30% of students might score into GLP, and perhaps 25% score into CCR. Somewhat confusingly, GLP will still be 30% and CCR will be 25%, which leaves 5% in Level 3. This, in effect, counts CCR students twice but it does make it convenient to talk about the percentage of GLP students without having to break that up into pieces. The NCDPI publicly available data does not identify AIG students by race, so any comparisons of numbers of students in CCR by race with those in AIG must be by imputation, not directly.

Seeking changes in the aggregate, such as in the average over all schools, masks the marked variations between schools.

Statewide Average Mathematics Percentage GLP, CCR and Level 5 in 2018-19

In order to provide a context, Figure 1 shows the statewide average percentage GLP, CCR, and Level 5 for 2018-19 by race (subgroup) by grade. Grade 8 is included for reference but is not the subject of this report.

In Figure 1, the percentage is that of the particular group of students. That is, if Asian shows 60% in grade 3 for the GLP category, then sixty percent of all the Asian students in grade 3 are GLP.

Comparing Percentage CCR for Grades 3, 4, and 5 in 2018-19

Figure 2 shows End of Grade percent CCR (the highest achieving students) by race by grade for each CHCCS school for 2018-19. This figure responds to the question, “Does student achievement vary from grade to grade?” These plots compare percentage CCR within each race, for each elementary school. While the percentage CCR “is what it is” for each school, the reliability of this percentage depends on the number of students of each race. It is misleading to compare a percentage CCR for two hundred students against one for twenty students because, in the likely scenario of a shift in just one or two students into or out of the CCR scoring range, the percentage CCR for the larger group would change hardly at all, while the percentage CCR for the smaller group might change considerably. The plot indicates this by having error bars. The larger the spread in the error bars, the smaller the number of students of that race for that school. When the error bars overlap, percent CCR cannot be considered significantly different. For instance, for Northside (681300) Asian students’ grade 3 and grade 5 percent CCR are not significantly different.

This plot is for 2018-19, and would look different in the details for other years. By way of summary, it is evident that even for the schools that presented challenges for the other races, White students do well. Also, the average percentage CCR over all schools for students of each race, shown in the far right of each plot as school 681LEA, is not very helpful as it obscures the variations among schools.

The report on CHCCS AIG students, part of this series, contains information that I will use here. It demonstrates that Seawell (681310) consistently has a high percentage of AIG students in grades 4 and 5. Northside (681300) provides a contrasting picture and has a consistently low percentage of AIG students. Of course, AIG classification should strongly relate to percent CCR.



Comments on the 2018-19 Data

Looking at Figure 2 in detail, I will pay special attention to two aspects. Are there specific schools that seem to be doing particularly well or poorly regarding percentage CCR values? And does there appear to be an overall effect in regards to percentage CCR?

For Asian students, in the 2018-19 year, Northside (681300) seems to be struggling in grades 3 and 5. Ephesus Road (681311) has a low grade 4 achievement.

For Black students, Northside (681300) has a consistently low percentage CCR, Seawell (681310) has a remarkable progression from grade 3 to 5, while Ephesus Road (681311) has high achievement in grade 4. The “extra wide” bars indicate an absence of Black students in some grades for some schools. Some of these are also associated with wide error bars, which indicate small numbers of Black students.

For Hispanic students, Northside (681300) is similar to that for Black students. Carrboro (681304) has a higher grade 3 than either grade 4 or 5. Schools Ephesus Road (681311) and Scroggs (681330) have notable increase from grades 3 to 4 to 5, which is similar to that for the Black students. Other schools are characterized by having “ups and downs” that surely warrant closer looks.

For White students, there is a substantial consistency of performance across all schools. However, grade 3 in Seawell (681310) is noticeably low, which is confirmed below in Figure 3D.

It is evident that a substantial ampunt of variability is intrinsic to percentage CCR, as well as to GLP and Level 5. This is fair warning that looking at one year is not a sound basis for making evaluations of performance or improvement.

Racial Composition of Elementary Schools by Year for All Grades

Public Schools of North Carolina Statistical Profile Table 10.1 which can be reached from here.

Figure 2 says nothing about the proportion of students by race at any school. The year by year racial composition of the CHCCS elementary schools is shown in Figure 3. In 2018-19 students reported as Asian, Black, Hispanic, White, and Multiracial. The increasing number of multiracial students suggests further explorarion of their role in the traditionally single-race construction of race.

Comparison of 2016-17 to 2018-19 Percentage CCR by School for Grades 3, 4, and 5

Figures 4 present a greater level of detail than was in Figure 2. These plots show percentage CCR data by race by school for three years, 2016-17, 2017-18, and 2018-19. Foremost, they show the year to year variability of achievement scores. The extent of this variability mitigates against over-interpreting change from a preceding year, and also raises concerns about the use of school or system-wide averages for policy decisions. I discuss this further in two other reports in this series, Overview and Position Statement.








Another Way to Look at Variability

Figures 4 show a lot of ups and downs in percentage CCR across the three years looking back from 2018-19. There is a way to focus on the nature of the changes, by comparing the change from 2017-18 to 2018-19 against the change from 2016-17 to 2017-18. This is called first differences. Figure 5 shows this for all students, that is, irrespective of race. Plots for Asian, Black, Hispanic, and White students are in Appendix A. Remember that Figure 5 is using eactly the same data as Figures 4; it is just presenting it in a different form.

Each point is labeled with the school code, where LEA is for the rolled up elementary schools. The overall impression is that there is a substantial amount of back and forth, and up and down. Some changes are of course expected, but looking only at the overall LEA in each grade completely misses the school by school variations.

Level 5 Performance by Race in CHCCS Public Schools for Grades 3, 4 and 5 in 2018-19

Level 5 is the highest achievement classification for the grade 3, 4 and 5 standardized tests, which have five levels. Due to the need for FERPA compliance, the publicly available NCDPI data is far from complete by race and by school. For any of the percentages, if a percentage of students, by race and grade, is less than five they are given a percentage of “<5”, and if greater than ninety-five percent they are given a percentage of “>95”. I have assigned numeric values of 2.5 and 97.5 in order to do calculations. An alternative would be to assign a uniformly distributed random number between zero and five, or between ninety-five and one hundred. In some cases schools decide not to report their Level 5 percentage, and I have left those as missing. It is my opinion that the Level 5 percentages are less meaningful than those for GLP or CCR, but they are at least of some value.

Figure 6 shows percentage Level 5 for 2018-19 by race. For whatever reason, Seawall (681310) did not report End of Grade percentage Level 5 in grade 4 and 5 for 2018-19 for White students, or for grade 4 Asian students, or for grade 5 Hispanic students. Considering the high values for percentage GLP and CCR, those for Level 5 2018-19 also have been high. Seawall shows a percentage Level 5 that increases with grade for Black students in 2018-19.



I will not show plots similar to Figures 3, that go across recent years by school, for percentage Level 5. The extent of missing data makes the data confusing and less than helpful.