Tuesday, June 16th, 2009
The following is a summary of our results to date in a study in progress. This is a collaborative research effort with Jigsaw Learning (TeachTown), Los Angeles Unified School District, and Cal State University, Los Angeles. The study will conclude in June, 2009 and final results will be posted this summer. In addition, the results will be submitted to a peer-reviewed journal this fall.
To address the increasing need for solutions for serving children with Autism Spectrum Disorders (ASD) in the schools, it is important to consider options that are more accessible and affordable such as Computer-Assisted Instruction (CAI). However, it is even more essential that these solutions are effective and research is needed to address this issue. In this collaborative study with TeachTown, Los Angeles Unified School District, and California State University, Los Angeles, a CAI program which targets language, cognitive, academic, social, and life skills will be assessed in a large public school system. Approximately 50 preschool and kindergarten-1st grade children with ASD are participating with 25 children in a treatment group and 25 children in a control group. Children in the treatment group received 50-100 minutes per week of CAI and 50-100 minutes per week of supplementary off-computer activities designed to enhance generalization to the natural environment. The CAI and off-computer activities were provided through TeachTown: Basics, which is currently being used in many schools across the United States, but which has not yet been tested in a randomized research study. In addition to the automatic data collection provided by the software, students were assessed using behavioral and standardized outcome measures. It was anticipated that the classrooms using the TeachTown: Basics program would demonstrate significantly higher rates of acquisition across learning areas and would also show more improvement in their spontaneous language and social interaction. It was also expected that children in the treatment group would exhibit less inappropriate behaviors following treatment than their peers in the control group. Following this Gen Webinar study, it is also anticipated that teachers and parents involved in the treatment group will show higher satisfaction ratings with their child’s program than those parents and teachers associated with the control group. The results of this research will help demonstrate to school districts the effectiveness and social validity of implementing CAI, and will help districts such as LAUSD secure funding for these types of programs by having data to demonstrate the effectiveness. The behavioral observation data is currently being coded by graduate students at California State University, Los Angeles, and children in both groups will use TeachTown: Basics through the rest of the school year to assess effectiveness between groups and within the control group.
About LAUSD Population
District Population: 688,138
Special Education Students: 82,326
English Language Learners (ELL) Students: 39,455
Autism Spectrum Disorder (ASD) students: 8,516
- Primary eligibility
- Additional 55 with ASD as secondary eligibility
LAUSD Autism Programs
Preschool Autism Special Day Programs (SDPs) - 1/2 day
Intensive Comprehensive Autism Program (ICAP) (ages 3-6)
Autism SDP (primary – high school)
Autism SDP for students with High Functioning Autism (HFA) /Asperger
Any/all other options
Issues in Autism Programs
- Staff-student ratios
- Staff training and implementation
- Behavior problems of students
- Access to general curriculum
- Rapid increases in ASD
- Access to evidence-based interventions
- Paucity of appropriate staff (have to contract out)
Potential Benefits of TeachTown: Basics for LAUSD Students
Evidenced-based intervention with built-in data collection
Collaboration with parents
Curricular guidance for teachers
Motivation for “hard to teach” students
Flexibility as students transition to different settings
- TeachTown: Basics Curriculum (Dev Ages 2-7 years)
- Academic/Cognitive Skills
- Social Understanding
- Receptive Language
- Life & Community Skill Understanding
- Daily computer sessions on school days for 20 minutes/day (can be done in 2
- 10-minute sessions)
- Daily off-computer activities on school days for 20 minutes/day (1:1, small group or circle activity)
- 3 months of intervention
Purpose and Design of Research
- Purpose: To assess the efficacy of the TeachTown: Basics program in self-contained special education classrooms in a large, urban school district.
- Design: Between and Within-Subjects Group Design, 4 schools – each randomly assigned in the fall to Treatment or Control, in spring, Control classrooms begin treatment.
About the TeachTown: Basics Program (the Intervention)
Check out the website for information about the program: www.teachtown.com
TeachTown: Basics includes the following:
- On-computer lessons where the child gets on the computer and completes lessons in an ABA (Applied Behavior Analysis) format with engaging reward games to keep them motivated;
- Off-computer activities to work on skills that are not targeted on the computer (e.g. Expressive Language, Play, Imitation, Social Interaction, Motor Skills) and to enhance generalization of skills learned on the computer to the natural environment;
- Automated data collection and tracking to assess the child’s progress as they move through the computer program and for school staff to use to assess the effectiveness of the intervention and to determine which skills may need more work off the computer;
- Note taking system for school staff to jot down anecdotal information about the child’s performance or any other relevant information to the child’s success with the program; and
- Synchronization and updating of data which allows the teacher to eventually share information with the families (not in this study) and for the child to be able to use the program at home (not in this study). In addition, this feature allows the research team to look at data on a regular basis to determine how the study is progressing and to conduct final data analysis.
Participants Results: TeachTown: Basics Software Program
Fifteen of the 22 students mastered lessons using the TeachTown:Basics software program. This does not mean that the other students did not make progress on the program, it just means that some of the children are still working toward mastery on their lessons, which will likely result in some mastered lessons by the end of the school year for most students. It is not unusual for students to not master lessons in only 3 months time. Students not meeting mastery are those with more severe cognitive delays, and those that were unable to complete 20 minute sessions. Data below is shown for the 15 students who did master lessons in 3 months time. There was statistical significance at the p>0001 level from the Pre tests to the Post tests, which are a part of the TeachTown: Basics program and test the child’s knowledge of concepts using a different set of stimuli from the training to ensure that the children are learning the concepts (i.e. targets) and not just memorizing pictures. In 3 months, students, on average, mastered lessons in about 43 minutes (Preschool) to 52 minutes (K-1) and mastered 5-6 lessons (20-24 concepts/targets).
Results: Language Changes on the Brigance Assessment from Pre (November, 2008) to Mid-Treatment (Feb, 2009)
The Brigance is a standardized developmental assessment that is frequently used to identify deficits and track progress in various developmental areas including language, cognition, social skills, and motor skills. LAUSD uses the
measure in their ICAP and other autism programs to asess the progress of the children enrolled in their programs. This measure aligns well with the TeachTown: Basics curriculum and was used in this study to measure progress for students
in the Treatment and the Control groups. Body Parts measures the knowledge of body parts; Receptive Language measures comprehension and vocabulary; and Expressive Language measures labeling and expressive communication.
All classrooms demonstrated improvement in language areas on the Brigance, but
– The TeachTown Treatment Group showed much bigger changes in Body Image (i.e. identification of body parts) and Expressive Language.
– The students in the preschool groups performed similarly on Receptive Language, but,
– the K-1 students in the TeachTown Treatment group showed greater change than the Control K-1 students
Results: Cognitive and Social Skill Changes on the Brigance Assessment from Pre (November, 2008) to Mid-Treatment (Feb, 2009)
The Preschool students had similar improvements in Matching on the Brigance, but TTB students (Tx Grp) made bigger gains than the Control group in Auditory Memory, General Concepts, and Social Skills.
The K-1 Students had similar improvements in General Concepts and Social Skills, but the TTB students (Tx Grp) made bigger gains in Matching and Auditory Memory.
Matching measures the child’s ability to match objects and categorize, Auditory Memory measures the child’s ability to understand and follow directions and to remember information that was presented to them, General Concepts measure the child’s basic early academic abilities (e.g. letters, numbers, etc.), and Social Skills measure the child’s knowledge of social situations (e.g. emotions, friendship, etc.).
Results: Language Changes on the PPVT-III and EVT Assessments from Pre (November, 2008) to Mid-Treatment (Feb, 2009)
The Peabody Picture Vocabulary Test (PPVT-III) and the Expressive Vocabulary Test (EVT) were used to further measures changes in Receptive and Expressive Language skills. Age-Equivalents are not shown because many students did not establish basal in Oct. In Feb, there was a larger increase in the number of Preschool students establishing basal in the TeachTown group (4 additional students on PPVT, 5 additional students on EVT) compared to the Control group (1 additional student on PPVT & EVT). The TeachTown (Tx) group also had slightly more students establishing basal in Feb (2 additional students on PPVT & EVT) compared to the Control group (1 additional student on PPVT & EVT).
Summary of Results
The Treatment group demonstrated much bigger increases in
- Receptive and Expressive language using the PPVT-4, EVT-2, and the Brigance
- Auditory Memory, General Concepts, and Social Skills for the Preschool students using the Brigance
- Matching and Auditory Memory for the K-1 students using the Brigance
- They also showed slightly larger increases in
- Matching for the Preschool students using the Brigance
- General Concepts and Social Skills for K-1 students using the Brigance
- After 3 months of using the TeachTown: Basics program, students in the Treatment group:
- Made significant gains from Pre to Post Tests in the TeachTown: Basics software
- Learned 34-39 target concepts (on average) in the TeachTown: Basics software with the largest gains in Receptive Vocabulary
This is a summary of our results to date in a study in progress. This is a collaborative research effort with Jigsaw Learning (TeachTown), Los Angeles Unified School District, and Cal State University, Los Angeles. The study will conclude this summer and final results will be posted later this summer.
Posted in TeachTown, General Thoughts, Resources, Technology | 1 Comment »
Friday, January 9th, 2009
It is sometimes difficult to make sense of all of the data that is collected on a child in a special education program. The first issue is consistency and standardization. There is no excepted assessment protocol that is used in ALL schools for ALL children, it varies immensely from school district to school district. Another issue is that the data is often not presented to the families in a meaningful way and the assessment is often not directly tied to IEP goals, so at times, it is not clear what is being assessed or why. Similarly, it is not always clear which assessments to use to get meaningful outcome data.
If your child does get assessed, the terminology in these reports can be daunting. Here is a brief overview of some of the terms you may see:
* A raw score is almost always the number of items that the child answered correctly on the assessment. By itself, this has no meaning if you are not familiar with the specific assessment.
* A standard score is one that has been calculated from the raw score to fit into a normal distribution. In most cases, the mean is 100, and the standard deviation is 15. In special education, children are often at extreme ends of the distribution (either way above or way below the mean) in areas of exceptionality (for instance, a child with autism will likely score below the mean in social communication).
* The z score is the number of standard deviations above and below the mean. If a z score is -1.9, that means the child scored 1.9 standard deviations below the mean.
* Percentile rank is the percentage of scores in a particular group of people. Percentile ranks range from 99th (highest) to 1st (lowest). If your child has a percentile rank of 82, they did better than 82% of the population (could be their class or district, could be the general population for that age group, etc, depends on the assessment).
* The age equivalent is estimate of the age level that matches how your child did on the assessment. This is almost always shown in years and months. If your child is 10 years old and receives an age equivalent score on a language assessment of 6 years, 2 months. That means that your child’s language (as measured by that assessment) is similar to a typically developing child at the age of 6 years, 2 months. This is probably one of the more useful pieces of data, as this can help guide your decisions for what content is most appropriate for teaching your child and building her language skills.
* The grade equivalent is similar to age equivalent, but by grade level, rather than a specific age.
* A report card often has a completely different set of scoring than standardized measures. These vary so much, that it is impossible to review them all here. The most common (with older kids) is A (highest grade), B, C (pass), D, or F (fail). Younger grades often use things like S (satisfactory), E (excellent), U (unsatisfactory) or other types of grading. Report cards are done at a state, district, or sometimes even school level (especially in private schools). The report card shows how your child is doing compared to other children at that grade level.
* The IEP (see Manya’s Posting) is not really a report of how well the child is doing, but what needs to be worked on to make the child successful, so the IEP is not the outcome measure, per se, but the plan for improving the child’s skills.
TYPES OF ASSESSMENT
Intellectual, Educational: The purpose of these assessments is to determine the child’s overall, verbal, or non-language intellectual ability. Skills that are typically measured include language skills, processing speed, memory, abstract thinking, planning, motor skills, spatial abilities, organization, social understanding and judgment, and common sense. IQ scores are often (but not always) associated with these assessments. IQ scores show a child’s intellectual ability compared to other children their age. IQ scores are more stable for older children than for younger children and change from childhood to adulthood. Many factors may contribute to IQ, so it is important to take them for what they are and to not make more of them than what they are. IQ and academic achievement are highly correlated, but success in life is not as highly correlated with high IQs, and many researchers believe that success may relate more to social-emotional intelligence than to IQ. An IQ score can be very helpful though in determining a child’s ability to do well in a mainstream classroom. Here is a breakdown of IQ scores, the classification, and the percent included (this is taken from a table in the book THE SPECIAL EDUCATOR’S BOOK OF LISTS: 2nd EDITION by Roger Pierangelo, PhD (Wiley Publishers) on page 219) (great book, btw, I highly recommend it!):
IQ Range Classification % Included
130 and over Very superior 2.2
120-129 Superior 6.7
110-119 High Average 16.1
90-109 Average 50.0
80-89 Low Average 16.1
70-79 Borderline 6.7
69 and below Intellectually deficient 2.2
Some of the most commonly used intelligence assessments include (list from the above referenced book - click on the link above to order the book from Amazon) (the book also gives a nice review of what is included in each of these assessments and the authors insights regarding benefits and weaknesses of each assessment):
1) The Wechsler Scales of Intelligence (WPPSI for Preschool; WISC for school age; WAIS for adults)
2) The Stanford Binet
3) Kaufman Assessment Battery for Children (K-ABC)
4) Kaufman Brief Intelligence Test (KBIT)
5) Columbia Mental Maturity Scale (CMMS)
6) McCarthy Scales of Children’s Abilities
7) Slosson Intelligence Test
8) Comprehension Test of Nonverbal Intelligence (CTONI)
9) Woodcock-Johnson Test of Cognitive Ability (WJ)
10) Brigance Diagnostic Inventory of Basic Skills
11) Kaufman Test of Educational Achievement (KTEA)
12) Peabody Individual Achievement Test (PIAT)
13) Wechsler Individual Achievement Test (WIAT)
14) Wide Range Achievement Test (WRAT)
15) Woodcock-Johnson Tests of Achievement
There are a few other measures that I have come across when working in schools and clinics that were not listed in the book:
16) The Leiter Non-Verbal Intelligence Scale
17) Developmental Profile 3 (DP-3)
18) Developmental Assessment of Young Children (DAYC)
19) Reynolds Intellectual Assessment Scales (RIAS)
20) Universal Nonverbal Intelligence Test (UNIT)
21) The Assessment of Basic Language and Learning Skills (ABLLS)
Reading Assessments: These assessments are excellent for determining a child’s grade level for reading and identifying strengths and limitations for program planning.
1) Gates-MacGinitie Silent Reading Test (GMRT)
2) Gray Oral Reading Test (GORT)
3) Durrell Analysis of Reading Difficulty (DARD)
4) Gates-McKillop-Horowitz Diagnostic Reading Tests
5) Gilmore Oral Reading Test
6) Slosson Oral Reading Test (SORT)
7) Spache Diagnostic Reading Scales
8) Woodcock Reading Mastery Tests (WRMT)
9) Test of Reading Comprehension (TORC)
10) Nelson-Denny Reading Test (NDRT)
Written Language: These assessments refer to the child’s ability to put their thoughts down on paper and includes the motor act of handwriting and the cognitive ability to put thoughts into writing.
1) Picture Story Language Test (PSLT)
2) Test of Early Written Language (TEWL)
3) Test of Written Language (TOWL)
Math: These tests measure the child’s abilities to solve problems, interpret results, and apply math skills.
1) Key Math Diagnostic Arithmetic Test (KEY MATH)
2) Test of Early Mathematics Ability (TEMA)
3) Test of Mathematical Abilities (TOMA)
Problem Behaviors: These tests are used to assess the level of problem behaviors exhibited by a child and to measure progress from interventions targeted at reducing these behaviors.
1) Behavioral Observations (this is the most commonly used measure - often done through a Functional Behavioral Analysis (FBA) on a specific behavior problem)
2) Interview Methods are often used to supplement behavioral observations and help the behavior analyst or psychologist get a better understanding of the environment and possible behavioral triggers surround a particular behavior - can also be used to diagnosis
3) Psychological Tests are administered by the school psychologist and are used to properly place the child in an appropriate classroom and to identify issues to work on with the child in his IEP. These can include projective drawing, apperception tests, sentence completion tests, and rating scales.
Adaptive Behaviors are the life skills needed for the child to function in school, home, and in the community. These include things like communication, health, safety, self-care, leisure, work, social understanding, fine and gross motor, functional academics, and community knowledge. Here are some popular assessments (also from book referenced above):
1) AAMR Adaptive Behavior Scale - Residential and Community
2) AAMR Adaptive Behavior Scale - School
3) Adaptive Behavior Evaluation Scale (ABES)
4) Vineland Adaptive Behavior Scale (VABS)
Visual and Auditory Perception: These are extremely important measures for children who may have difficulty responding to the teaching materials typically available in classrooms and to qualify children for assistive technology or other tools to help give them access to the teaching materials. These are typically administered by an occupational or speech therapist.
1) Developmental Test of Visual-Motor Integration (VMI)
2) Test of Gross-Motor Development (TGMD)
3) Bender Visual-Motor Gestalt Test (BVMGT)
4) Developmental Test of Visual Perception (DTVP)
5) Motor-Free Visual Perceptual Test (MVPT)
6) Goldman-Fristoe-Woodcock Test of Auditory Discrimination
7) Test of Auditory Perceptual Skills (TAPS)
8) Wepman Test of Auditory Discrimination (ADT)
9) Detroit Tests of Learning Aptitudes (DTLA)
10) Slingerland Screening Tests for Identifying Children with Specific Language Disability
Early Childhood Assessments are developed specifically for children under the age of 5 years (often used with a new diagnosis).
1) Bayley Scales of Infant Development
2) Preschool Language Scale (PLS)
3) Metropolitan Readiness Tests (MRT)
4) Boehm Test of Basic Concepts (BTBC)
5) Bracken Basic Concept Scale (BBCS)
6) Preschool Evaluation Scale (PES)
7) Kindergarten Readiness Tests (KRT)
8) Batelle Developmental Inventory (BDI)
9) Communication and Symbolic Behaviors Scale (CSBS)
10) Mullen Scales of Early Learning
Motor skills are often assessed to identify the need for an occupational therapist and to develop specific programs to help children with motor skill difficulties. Gross motor skills include those that require larger movements (e.g. running, dancing, balance, etc.) and fine motor skills include those require more finger movements (e.g. writing, cutting, musical instruments, etc.). Here are some assessments that are often used (from the book):
1) Milani-Comparetti Motor Development Test
2) Miller Assessment for Preschoolers (MAP)
3) Quick Neurological Screening Test (QNST)
4) Sensory Integration and Praxis Test (SIPT)
5) Purdue Perceptual Motor Survey (PPM)
Autism/Asperger Severity measurements are used to make an initial diagnosis, but area also used to confirm the child’s diagnosis at various points in their development. Here are some of the measurements that I have used personally in my research, clinical, and education work:
1) Childhood Autism Rating Scale (CARS)
2) Gilliam Autism Rating Scale (GARS)
3) Gilliam Asperger’s Disorder Scale (GADS)
4) Autism Diagnostic Observation Scale (ADOS)
5) Autism Diagnostic Interview (ADI)
6) Modified Checklist for Autism in Toddlers (M-CHAT)
7) Asperger Syndrome Diagnostic Scales (ASDS)
8) Autism Screening Instrument for Educational Planning
9) Differential Assessment of Autism and Other Developmental Disorders
10) Pervasive Developmental Disorders Screening Test
There are also a great deal of language assessments that are administered to identify speech, language disorders, and to identify speech-language and behavioral services that might be needed to help the child advance in their language and communication. These assessments are extremely common with children who have an ASD diagnosis. I am not a speech therapist, and this list is by no means comprehensive, but here are some of the language measurements that I have used or seen used by speech therapists:
1) Peabody Picture Vocabulary Test
2) Peabody Expressive Vocabulary Test
3) Receptive One-Word Vocabulary Test
4) Expressive One-Word Vocabulary Test
5) Assessment of Sound Awareness and Production (ASAP)
6) Hodson Computerized Analysis of Phonological Patterns (HCAPP)
7) Language Proficiency Test (LPT)
8) Lindamood Auditory Comprehension Test
9) Oral and Written Language Scales (OWLS)
10) Test for Auditory Comprehension of Language (TACL)
11) Test of Pragmatic Language (TOPL)
12) Woodcock Language Proficiency Battery
13) Boehm 3- Preschool
14) Boehm Test of Basic Concepts
15) Clinical Evaluation of Language Fundamentals (CELF)
16) MacArthur Communication Development Inventories
17) Preschool Language Scale
18) Reynell Developmental Language Scales
19) Test of Early Language Development (TELD)
20) Test of Narrative Language (TNL)
21) Utah Test of Language Development
There are not a great deal of assessments for testing social skills, but there are a few that I think are great to use with children with ASD:
1) Social Communication Questionnaire (SCQ)
2) Social Responsiveness Scale (SRS)
3) Social Skills Rating System (SSRS)
4) Social-Communication, Emotional Regulation, Transactional Supports Assessments (SCERTS)
There you have it, a not so brief overview of assessment in special education. If anyone knows of other assessments that they use in their professional work, or that have been used on their children, please share, particularly if you find that one has really helped you. I am currently looking for good measurements for social skills, if you know of some, please let me know!
Happy New Year Everyone!!!
Posted in General Thoughts, Resources | 2 Comments »