Long Name, Solid
Results for Evaluation Tool
In another article in a periodic series on evaluation, Pre-K Primer examines
innovative approaches taken in Oklahoma and other states.
Finding a ‘gold standard’ for program evaluation is often difficult.
But in pre-kindergarten, researchers of programs in Oklahoma and elsewhere
have a model that may provide one of the most effective approaches to
examine success.
It has an intimidating name – regression discontinuity – but researchers
have a shorthand to describe the process. Similar to K-12 education,
pre-k also has strict age cutoff dates for admission in a certain school
year. As a result, one 4-year-old may go to pre-k while his friend, a few
days shy of age 4, does not.
A year later, the 5-year-old heads off to kindergarten while the slightly
younger child goes to pre-k. It’s at that point that researchers use
"regression discontinuity" to assess both children for an unbiased view
of the benefits of pre-k on children of similar ages and stages.
"We think it’s a terrific strategy to replicate," said William
Gormley, Georgetown University professor and co-director of the Center for
Research on Children in the U.S. (CROCUS). An approach previously found
mainly in economic studies, the model is a sound way to address selection
bias since it can include all children who want to enroll and, eventually,
participate. The only difference is the year they receive service.
"It’s really a ‘silver standard,’" Gormley told Pre-K Primer. He says
the "gold" would be an experimental study comparing pre-k children
against those who want to be in the program but are never allowed to
participate for the validity of the study.
"In the real world, that’s hard to do," he added.
But regression discontinuity provides an ideal comparison group for
researchers without jeopardizing access to services.
"One group has completed PreK and the other is just entering,"
said Jason Hustedt and colleagues from the National Institute for Early
Education Research in a new paper assessing the effects of New Mexico’s pre-k
program. By making just this type of comparison, Hustedt, assistant professor
at the NIEER, found statistically significant gains from children in pre-k.
Gormley pioneered the approach in 2004 research of Tulsa, Okla.’s pre-k
program, where he and his colleagues documented widespread gains for
participants.
In a recent presentation entitled Small
Miracles in Tulsa, Gormley said the research approach "helps
to ensure that the students are alike in their talent and motivation –
intangibles that are extremely difficult to measure."
For the research, CROCUS administered three subtests of the
Woodcock-Johnson Achievement Test covering letter-word identification,
spelling and applied problems. It also controlled for gender, race and
income. Overall, compared with those about to start pre-k, recent pre-k
"graduates" had these gains:
· a
52 percent increase in letter-word recognition;
· a
27 percent gain in spelling;
· a
21 percent increase in applied problems.
Progress was evident across all populations and income levels. In
addition, the research has broad implications, since Oklahoma has one of the
most comprehensive pre-k programs in the nation serving 70 percent of
eligible youngsters.
Since that time, NIEER has utilized this approach in evaluating pre-k not
only in New Mexico but also in New Jersey and Arkansas.
"As programs become more widely available, it is more difficult to
find a comparable group of children who did not attend," Hustedt says in
the new paper, Impacts of
New Mexico PreK on School Readiness (profiled in the column at
right).
Even when researchers can track down those who never participate, there
are likely to be other contributing factors to the pre-k enrollment decision,
such as income, parent education and family factors.
The advantage in this approach is that "both sets of students have
parents who affirmatively chose" pre-k for their youngsters, Gormley
says.
Regression discontinuity may be more cost-effective than some research
approaches, he added, since researchers can administer assessments to all
children – treatment and control groups – at the same time. One drawback,
however, is that longitudinal analysis over time is difficult because all
children in the study will have received pre-k services.
More information on NIEER studies is available from the web site at www.nieer.org. More information about the Tulsa studies
is online at www.crocus.georgetown.edu.
Pre-K News Across
the Nation
Early Start Program Helps Washington Students
More students get a preview of school.
Pre-K Can Help Reduce Dropout Rate
Maine advocates seek stronger early childhood programs.
New Literacy Push for 4-Year-Olds
Grand Rapids grant will promote pre-reading skills.
School Entry Decision Has Long-Term Implications
Analysts weigh 'redshirting' of children for one year.
Public Online Service Offers Wealth of Information
Early Childhood Asset Map helps families, policymakers. |
News Briefs
Congress Approves
College Bill With Early Childhood Provisions
In a long-awaited 1,100-page bill to renew the federal
government’s higher education programs, Congress has inserted several
provisions key to early childhood educators.
The Higher Education Act renewal bill sets a definition
of a "highly competent" early educator, much as Congress earlier
defined a "highly qualified" K-12 teacher. The HEA bill includes
funding to support teacher training, including those in pre-k and early
education.
Under lawmakers’ definition, a "highly
competent" early educator would possess:
· specialized
training and education in the development and education of children from
birth to kindergarten;
· a
bachelor’s degree in the arts and sciences or an associate degree in a
related field;
· and
a high level of knowledge of content and pedagogy in areas relevant to
quality early childhood education.
Early education programs also would gain access to
federal Teacher Quality Enhance-ment Grants. These grants traditionally have
gone to partnerships of colleges and universities and high-need school
districts. However, the bill’s new language would add "high-need"
early education programs to the list of potential applicants and authorize
$300 million in funding.
States also could receive competitive grants to set up
task forces to improve early childhood education. Another provision has loan
forgiveness of up to $2,000 a year to early childhood educators who work in
eli-gible programs in low-income communities.
Both the House and Senate overwhelmingly approved the
higher education bill, and President Bush signed it on Aug. 14.
* * *
Mass. Approves
Measure On Pre-K Expansion
Massachusetts Gov. Deval Patrick (D) has signed into law
a measure that may enhance the prospects of 'pre-k for all' in the state.
The Universal Pre-Kindergarten Program would expand
accessible, high-quality care and support strategies to encourage early care
providers to improve their education. Signed July 31, the plan had won
unanimous votes in the state House and Senate.
"With this bill, early education and care is firmly
established as the foundation for a lifelong education journey," said
Paul Reville, state education secretary.
A state Department of Early Education and Care will play
a lead role. A funding timeline is not yet available, however.
* * *
Second Year
Brings More Gains For New Mexico Pre-K Program
A pre-k program in New Mexico is showing student gains in
its second year of operation, with youngsters making progress in language,
literacy and math development, new research shows.
Children’s vocabulary scores increased by six raw score
points due to their participation in the program, says the National Institute
for Early Education Research at Rutgers University. Early math scores were up
by more than two raw score points. NIEER said both gains were statistically
significant.
In addition, early literacy scores increased by about 14
percentage points for children attending school compared with
non-participants. Children in New Mexico PreK knew more letters, letter-sound
associations and words, says the report, Impacts
of New Mexico PreK on Children’s School Readiness at Kindergarten Entry:
Results from the Second Year of a Growing Initiative.
New Mexico PreK serves more than 3,500 4-year-old
children, and it meets nine of 10 quality benchmarks designed by NIEER for
pre-k services.
For its study, researchers used a regression discontinuity
approach, a research frame-work that compares children who recently finished
pre-k with children of nearly a similar age who are just about to enter this
program. Contact: NIEER, www.nieer.org. |
