Beginning of the Year Teacher Data Strategies

First Steps

As the school year begins, teachers are greeted by a new class or classes of students. The teacher will get to know the students and begin identifying students' learning and social interaction styles. Frequently teachers will do some beginning year informal assessments to identify skill strengths and needs. This is a terrific first step, and it begins the process of collecting and interpreting student performance data.

A step that principals sometimes require is that the teacher reviews each student file and records previous test data. I had one principal tell me during a data consultation that the teachers are required to record on 3 by 5 cards the previous scores. The principal wanted the scaled score, the achievement level, and the percentile rank. I questioned that approach because. In North Carolina, the scaled scores are not on a continuous scale and they are not equivalent to the same percentile rank from grade to grade, in both reading and math.   

What I suggest is that the teacher writes down the percentile rank scores for each year and CONVERT these scores to Normal Curve Equivalent (NCE) using a table that is readily available online. 

The Analysis

When the teacher examines the NCE differences from year to year, it will reveal the growth made by the student. Students with lower NCE scores than the previous year have not made a year's growth and should be a student who may need extra attention. students who have a pattern of negative growth will need some very special attention. 

The Importance

 With the increased emphasis on growth as part of the emerging North Carolina teacher career ladder proposal, it is essential to identify students who are not performing at grade level AND who are not making an annual growth target. In the next blog entry, the increased emphasis on EVAAS growth will be explained. 

Interpreting State Test Score Data

As teachers across the county receive the end of year test scores for their students, teachers need to be aware of the various scores they receive and hoe to interpret the scores. 

The score report received by North Carolina teachers will include a developmental scaled score. This is a three digit number such as 554. The first digit indicates the test edition. So this is the 5th edition of the test. The other two digits are the score the student received on the test. At times for some subjects for some years, this was a t-score with an mean of 50 and a standard deviation of 10 points. This is not the case anymore. Therefore, these scores have no real meaning, other than they can be compared within the same test in the same subject. For example a 550 in math is not the same as a 550 in reading. Also, because these scores are onto on a continuous scale they cannot be compared from year to year. For example a 550 in grade 4 is not comparable to a 550 in grade 5. 

So, with this information the teacher needs to look at other information that may be helpful in understanding a student's performance. Achievement Levels for proficiency is the basis for determining the status of a student and school for accountability purposes. In north Carolina, the proficiency rating goes from "non proficient" to level 3 Grade Level Proficient (GLP) to level 4 and 5. Students who score a level 4 or 5 are considered College and Career Ready Proficient (CCR). The percentage of students who achieve a CCR proficiency is used to report federal-level proficiency. It is important to not that these achievement level cut-offs are not determined by or connected to a given percentile rank score of the student. 

The next score a teacher may receive is a percentile rank score. If you are not certain what a percentile rank score is I suggest reading the information at this NWEA site.  An added feature of this page is the explanation that percentile ranks scores are not helpful in measuring growth. This is especially true when trying to aggregate growth of a group of students.

Percentile rank scores are NOT to be confused with percent correct.

Percentile ranks are determined by an analysis of the array of student scores for a given test on the year the test was developed. Generally, test that have a normal distribution of scores are easily used to computing percentile rank equivalent scores for each student raw score.  

Sometimes, As it happened in North Carolina this year, The state used the 2021 test score data and created a net percentile rank score for each corresponding scaled score. This makes comparing percentile ranks from last year's test reports impossible to do with any accuracy. 

Here is a chart of grade 4 and 5 scores from 2021 and 2022 to demonstrate the problem of comparing scores. 

																			5	554	4	84

What is Data Literacy?

 Educational data literacy is the ability to collect, organize, interpret, and use data both qualitative and quantitative to:

• plan instructional programs,
• provide student performance feedback to student, teachers and administrators
• understand the impact of instructional programs, and 
• evaluate the impact or programs, resources and conditions on student performance.
  
  and  

Data literacy involves using multiple sources of data, understanding various assessments and the assessment's value and limitations, and understanding the types of resulting scores. Ongoing professional development for teachers and administrators is necessary to keep educational staff up to date on assessments and data use.

It is important to understand the context in which data is collected. For example an assessment might be viewed as valid and reliable in a particular school that follows the intended curriculum that is represented in the assessment. However, if another school using the same assessment is not following the curriculum  then the assessment is not content valid and will result in unreliable scores. 

In all honesty, I have told clients that no data is better than data that is invalid and unreliable. 

Testing Anyone?

My granddaughter in grade who lives with us has taken 4 assessments in the three weeks. Probably two in math and two in reading. These tests are "interim" assessments. One I think was the NWEA test. We have no idea how she performed on the assessments. We have no idea if she performed below or above her peers, so we have no idea if what we are doing to help her at home is benefitting her. 

Unfortunately, teachers do not set the assessment policies and practices used in the schools to provide information that can help the teachers and the parents do better. So why are these assessments being administered? Certainly, it is not to inform parents of their child's performance or progress, if no reports are sent home. 

One assessment used in North Carolina is the NC Check-In Tests. While the teachers do get item-level response information, the tests frequently in math only test a sliver of what was taught in the 9 weeks leading up to the testing. Also, in some grades, the average percent correct on these 25 item tests may be somewhere near 40% on some tests. So what does that information tell a teacher? It is too late after the scores come back to do anything about the lack of mastery on a skill six weeks after the skill was taught. 

 Another assessment is the I_READY test. It is an adaptive online test that provides math scores in four strands. Teachers receive no information about how a student performs on each of the state curriculum strands, nor does it provide item-level results. It seems that the primary purpose of this test administered three times a year is to meet the universal screener requirement for the MTSS process. Again, teachers really don't know how to interpret the results.     

In a recent data focus session with a school improvement team, the focus turned to what can teachers do to use assessment to really drive instruction and identify instructional and learning problems. The answer is simple and it has been in use by high-performing teachers for the last two decades or more. The "silver bullet" is informative assessment, which really is just collecting and ACTING on daily data collection, or more precisely, data collected DURING instruction. 

So in the next post, I will explain assessment and the practice of informative assessment. 

Recent Data Coaching Session

 Do you know about different scores?

This week I visited a school to speak with lead teachers and the administrators about their interim test data. There are several data sources in the report that include I-Ready scores, DIBELS scores, and state non-standardized test scores. All of this data is collected three times per school year and put into one file.

When I prepare data in the data system, I generally convert percentile rank and percent correct scores into normal curve equivalent (NCE) scores.  As I discussed the data with the group, I began to sense that they did not know what NCE scores were and then as I questioned further, some were not clear about the difference between percentile rank scores and percent correct. 

The interpretation of scores is a critical understanding necessary for all teachers who use data to measure the learning performance of students and to identify students who are at-risk. We examined some students who were at the 45 NCE on the first test and then scored at the 30th NCE on the second interim test. While some students did better on the second test and the majority scored within a few NCE points on both tests. The decline of the 15 NCE point difference was substantial and significant at a practical level.  

Readers may wish to review this information located on this website: 

    Learn about scores and the normal curve distribution of scores

   

 

What Do Teachers Need to Know About Data?

 The question is "What do teachers need to know about data?"

Schools administer many tests and create many school reports. The administrators think this practice will help improve student performance and increase accountability test scores. This practice does not improve student performance. It is an "after the fact" activity that by the time the results are presented, the instructional train has departed the station and is now heading for the next destination. So, the question is What do teachers need to know about these tests and assessments for instruction?

Understanding Assessments

• Content validity
• Reliability
• Item analysis
• Error analysis
• Score interpretation
• Predictive value of the assessment

 Curriculum Analysis

• Intended versus enacted curriculum
• Test item weighting alignment
• Pacing guides 

Informal Assessment

• Curriculum-based assessment
• Daily direct data collection
• Reporting performance information to students

Do Teachers Need Data Skills?

 Over the past 5 years, I have trained administrators at the district and school level to understand and use student performance data (assessment  and test scores) to improve district and school performance. What i have experienced is that the information I provide to the administrators RARELY trickles down to the school lead teachers and the classroom teachers. 

Also, I have had conversations with these administrators about what professional development the district provides on the topic of data use for teachers. Unfortunately, there is very little school districts do to help teachers understand and use data and assessment strategies to improve student learning and performance. 

This blog is dedicated to the mission of informing teachers directly about how to understand and use data to make a significant positive impact on student learning. 

Welcome to my Blog,

Lewis R. Johnson, Ed.D.

  Retired Director of Accountability, Testing and Research

  Retired professor of assessment and special education

  CEO of Data Smart LLC

     Data-Smart.net