What is Data Literacy?

 Educational data literacy is the ability to collect, organize, interpret, and use data both qualitative and quantitative to:

• plan instructional programs,
• provide student performance feedback to student, teachers and administrators
• understand the impact of instructional programs, and 
• evaluate the impact or programs, resources and conditions on student performance.
  
  and  

Data literacy involves using multiple sources of data, understanding various assessments and the assessment's value and limitations, and understanding the types of resulting scores. Ongoing professional development for teachers and administrators is necessary to keep educational staff up to date on assessments and data use.

It is important to understand the context in which data is collected. For example an assessment might be viewed as valid and reliable in a particular school that follows the intended curriculum that is represented in the assessment. However, if another school using the same assessment is not following the curriculum  then the assessment is not content valid and will result in unreliable scores. 

In all honesty, I have told clients that no data is better than data that is invalid and unreliable. 

Testing Anyone?

My granddaughter in grade who lives with us has taken 4 assessments in the three weeks. Probably two in math and two in reading. These tests are "interim" assessments. One I think was the NWEA test. We have no idea how she performed on the assessments. We have no idea if she performed below or above her peers, so we have no idea if what we are doing to help her at home is benefitting her. 

Unfortunately, teachers do not set the assessment policies and practices used in the schools to provide information that can help the teachers and the parents do better. So why are these assessments being administered? Certainly, it is not to inform parents of their child's performance or progress, if no reports are sent home. 

One assessment used in North Carolina is the NC Check-In Tests. While the teachers do get item-level response information, the tests frequently in math only test a sliver of what was taught in the 9 weeks leading up to the testing. Also, in some grades, the average percent correct on these 25 item tests may be somewhere near 40% on some tests. So what does that information tell a teacher? It is too late after the scores come back to do anything about the lack of mastery on a skill six weeks after the skill was taught. 

 Another assessment is the I_READY test. It is an adaptive online test that provides math scores in four strands. Teachers receive no information about how a student performs on each of the state curriculum strands, nor does it provide item-level results. It seems that the primary purpose of this test administered three times a year is to meet the universal screener requirement for the MTSS process. Again, teachers really don't know how to interpret the results.     

In a recent data focus session with a school improvement team, the focus turned to what can teachers do to use assessment to really drive instruction and identify instructional and learning problems. The answer is simple and it has been in use by high-performing teachers for the last two decades or more. The "silver bullet" is informative assessment, which really is just collecting and ACTING on daily data collection, or more precisely, data collected DURING instruction. 

So in the next post, I will explain assessment and the practice of informative assessment. 

Recent Data Coaching Session

 Do you know about different scores?

This week I visited a school to speak with lead teachers and the administrators about their interim test data. There are several data sources in the report that include I-Ready scores, DIBELS scores, and state non-standardized test scores. All of this data is collected three times per school year and put into one file.

When I prepare data in the data system, I generally convert percentile rank and percent correct scores into normal curve equivalent (NCE) scores.  As I discussed the data with the group, I began to sense that they did not know what NCE scores were and then as I questioned further, some were not clear about the difference between percentile rank scores and percent correct. 

The interpretation of scores is a critical understanding necessary for all teachers who use data to measure the learning performance of students and to identify students who are at-risk. We examined some students who were at the 45 NCE on the first test and then scored at the 30th NCE on the second interim test. While some students did better on the second test and the majority scored within a few NCE points on both tests. The decline of the 15 NCE point difference was substantial and significant at a practical level.  

Readers may wish to review this information located on this website: 

    Learn about scores and the normal curve distribution of scores

   

 

What Do Teachers Need to Know About Data?

 The question is "What do teachers need to know about data?"

Schools administer many tests and create many school reports. The administrators think this practice will help improve student performance and increase accountability test scores. This practice does not improve student performance. It is an "after the fact" activity that by the time the results are presented, the instructional train has departed the station and is now heading for the next destination. So, the question is What do teachers need to know about these tests and assessments for instruction?

Understanding Assessments

• Content validity
• Reliability
• Item analysis
• Error analysis
• Score interpretation
• Predictive value of the assessment

 Curriculum Analysis

• Intended versus enacted curriculum
• Test item weighting alignment
• Pacing guides 

Informal Assessment

• Curriculum-based assessment
• Daily direct data collection
• Reporting performance information to students

Do Teachers Need Data Skills?

 Over the past 5 years, I have trained administrators at the district and school level to understand and use student performance data (assessment  and test scores) to improve district and school performance. What i have experienced is that the information I provide to the administrators RARELY trickles down to the school lead teachers and the classroom teachers. 

Also, I have had conversations with these administrators about what professional development the district provides on the topic of data use for teachers. Unfortunately, there is very little school districts do to help teachers understand and use data and assessment strategies to improve student learning and performance. 

This blog is dedicated to the mission of informing teachers directly about how to understand and use data to make a significant positive impact on student learning. 

Welcome to my Blog,

Lewis R. Johnson, Ed.D.

  Retired Director of Accountability, Testing and Research

  Retired professor of assessment and special education

  CEO of Data Smart LLC

     Data-Smart.net