Home
About Us
Our Donors
Publications
Research
Education Analyst
Policy Watch
Media & Events
Order Publications
Contact Us
Site Map
The Society for the Advancement of Excellence in Education provides non-partisan education research and information to policy-makers, education partners and the public. Our purpose is to encourage higher performance throughout Canada's public education system.
|
Beyond the Snapshot
Helen Raham, Winter 2003
A major shift is underway in how we collect student achievement data and use it to make decisions. As traditionally reported, assessment results provide a snapshot of performance at a given moment. Powerful new information-management systems, however, have the capacity to provide a much more meaningful picture of achievement progress.
These systems enable a range of performance indicators for individual students, cohort groups and various subsets to be tracked over time. This development is important for educators and policymakers because longitudinal data are more useful for examining school and program effectiveness and identifying promising practices.
Potential Benefits
Education can benefit in number of ways from information systems offering a rich store of data for evaluating programs and communicating the relative progress of groups of students. With this information, for example, we can:
* Track students from high school to post-secondary education, to find out what school programs and courses are most strongly associated with future success.
* Identify the relationship between early achievement and later academic success, providing a basis for evaluating and investing in early-intervention initiatives.
* Control for student background, prior attainment, and mobility in reporting school results so that educators can benefit from knowing how much of the learning gains the school is responsible for.
* Improve our knowledge of promising practices by identifying consistently high-performing schools in various socioeconomic categories so that other schools may learn from their success.
Such examples of the power of longitudinal data collection and analysis explain the growing trend towards its use in school systems.
Eighteen US states have the ability to collect longitudinal student data and three more are constructing such systems. The No Child left Behind Act of 2001 requires every state to test all students in Grades 3-8 annually in reading and mathematics by 2005-06, and in science two years later. The Act also provides federal funds for administering assessments and developing longitudinal data management systems.
School districts in 21 states are attempting to measure school effects on learning through value-added assessment. This sophisticated statistical tool moves beyond raw scores to gauge gains or losses over time compared with the rate of progress of matching populations in other schools, adjusting for prior levels of attainment and variables such as student background. Large-scale value-added assessment pilots involving over 100 districts are underway in Colorado, Pennsylvania, and Ohio ahead of state-wide implementation in 2005-06.
The UK Department for Education has recently published the first results of a pilot study involving 495 schools to measure how much the individual schools contribute to their pupilsâ progress in reading, writing, spelling and math. The achievement of each 11 year-old student in the study this year was compared with the scores of 11 year-olds nationally with similar scores four years earlier. Individual progress scores were then averaged to yield a value-added score for the school, providing added dimension of information to traditionally published raw scores.
Obstacles
Devising data systems to make longitudinal student records available to schools and districts is not easy or cheap. Many Canadian provinces still rely on the snapshot approach. Some of the obstacles to change are:
Current Reporting Systems. Most provinces report annual achievement based on different cohorts each year, with different characteristics and abilities. Comparing the performance of last yearâs students to that of this yearâs students in a given grade cannot provide an accurate measure of actual change in performance.
Privacy Concerns. The creation of province-wide longitudinal student databases entails assigning each student a permanent identification number that can be used to match records as students change schools. Steps must be taken to safeguard student privacy.
Limited Assessment. Most provinces collect achievement information at intervals such as grades 3, 6 and 9, limiting the capacity to track student progress from one year to the next and to examine school and classroom effects operating at the levels in-between.
Cost. Administering assessments across more grade levels and creating the necessary databases and reporting systems is estimated at one-tenth of one percent of the education budget. 1
Canadian Developments
Despite these challenges, strides are being made in the methods of data collection and use in Canada. The federal governmentâs National Longitudinal Survey of Children and Youth (NLSCY) along with data available from our participation in national and international assessments "provides for the first time a series of rich databases that that will allow researchers, and educators to explore how resources, schools and classroom conditions, - as well as individual and family circumstances ö affect variations in achievement." 2
BC has constructed a reliable ten-year database of individual student achievement using personal identification numbers (PIN). The PIN system allows the province to examine and report performance levels by gender, aboriginal status, language, age, and some special education classifications. Alberta Learning presents longitudinal data on provincial assessments to jurisdictions and schools, with predicted achievement based on prior levels of attainment. Thus, for example, Grade 3 results are used to predict the performance of the same group of students in Grade 6, and are to be considered by schools in setting achievement targets in their Annual Education Results Reports (AERR). Differences in predicted versus actual attainment by the cohort are then calculated by Alberta Learning and sent to the schools.
Quebec schools prepare annual Success Plans based in part on demographic and achievement data provided by the government. To assist in longitudinal tracking, each Quebec student receives a permanent code on entry to the system. The ability to track cohorts is still in early stages, but research is underway following one cohort over 5 years. New Brunswickâs database contains achievement results at the student level for Grades 8 and 11. A PIN system is under development; the system currently permits tracking of students by gender and language of instruction. No province yet has the capacity to assess all students at every grade annually, but Ontario has announced plans to test Grades 3-11 annually by 2006, enabling year-on-year progress analysis for each student.
Future Prognosis
Making sense of student assessment is challenging and will always be a work in progress. The ultimate purpose is not merely to determine the achievement gaps, but to find patterns that might suggest why they occur and how they can be bridged. Upgraded data-management systems will be instrumental to the school improvement efforts of educators and policy makers. Such thoughtful use of longitudinal results will assist schooling become a research-based, data-informed enterprise.Ê
1. ÎMore Than a Snapshotâ. Dohorty, C. Education Week. May 2, 2001. Vol 20. No. 33, pp.42
2. Measuring Up: The Performance of Canadaâs Youth in Reading, Mathematics and Science. (Nov 2002) HRDC Statistics Canada. p 25
♦ Policy Watch ♦ Home
| 
|
THEMES
Assessment
At-Risk Students
Breaking Traditional Boundaries
Literacy
Management, Administration and Governance
Parents and Community
School Choice
School Improvement
Teaching Profession
A Future in the Process of Arrival
Moving Forward
Sharing Our Success
District Practices and Student Achievement
♦ Disclaimer
♦ Terms and Conditions
♦ Privacy
|
 Society for the Advancement of Excellence in Education
225 - 1889 Springfield Road, Kelowna British Columbia V1Y 5V5 Canada
Telephone 250.717.1163 | Fax 250.717.1134 | Email info@saee.ca |
