Applied longitudinal data analysis to measure the impact of a program
Understanding impact of viewership on children’s skill levels (tested through an instrument devised by leading child psychologists) after controlling for location, socio-economic levels, gender, pre-exposure levels across a panel of over 5000 children across urban and rural India.
Using a naturalistic prospective longitudinal design, the study surveys children and their caregivers in several Indian states and urban/rural locations over three years with a baseline wave of data collection occurring prior to exposure, and two waves occurring after children had the opportunity to be exposed to the show. Each child acts as his/her own baseline, allowing us to track development and feedback in relation to viewing over time.
We use data on children’s viewing program to develop a measure of exposure to the show, and conduct analyses to examine the association of viewership with educational outcomes in literacy, math, and socio-emotional domains, after accounting for children’s baseline performance on these same outcomes.
To evaluate the series’ educational impact, we modelled the children’s scores as a function of a set of predictors: time, SEC, age, viewership (little or no exposure vs. high exposure) and language spoken at home (where relevant). These were hierarchical models. The expression “hierarchical” is used as the model is created at two levels: The Level 1 model predicts the child’s score as a result of Time (i.e., the child’s score at three time points: baseline, midline and endline). This model is expressed using the equation: Child score= Intercept + slope (Time) + error
In the Level 2 model, the analysis attempted to understand the variation in children’s scores attributable to the differences between children and that attributable to differences within children. To find out what predicts the variation in scores between children, explanatory factors (i.e., SEC, age, viewership, and language spoken at home) were introduced into the model sequentially to understand the impact that they had on both the initial status (child scores) and the rate of change over time.
The objective was to assess if viewership or program exposure made any significant contribution to educational outcomes over time. If the model created did not explain any variation associated with rate of change, researchers concluded that none of the Level 2 predictors had any significant impact on the development process of the child. If a predictor affected the initial status (before launch or just at launch of program) but not the rate of change, it was not seen to contribute to the development process of the child.
These results indicated that exposure to program was related to significant gains in literacy skills, and that exposure seemed to offer greater benefit to those from less privileged backgrounds. Taken together, this longitudinal study provides evidence that viewing the series can have positive effects on children’s skills both concurrently and over time.