The accuracy of school indicators depends on the reliability of the measure used as well as the size and diversity of the sample group that completes the survey.
For most of the measures used in OurSCHOOL (formerly Tell Them From Me), after a minimum of 30 children complete the survey, we have reasonably accurate results. If we have a larger sample size — 100 students, for example — the results are slightly more accurate. Statistically speaking, the size of the sampling error is a function of 1/n2, where n represents the size of the sample. As n increases (more students complete the survey), the sampling error gets smaller. In other words, as the number of students completing the survey increases, so too does the accuracy of the results.
The same argument applies to drill-downs. The reported estimate scores of drill-downs are reasonably accurate for most measures, so long as at least 30 student responses are used to estimate the results. With the OurSCHOOL survey, the minimum number of children required for reporting a result is set at 5. This means, for drill-downs, the results for small groups are not as accurate.
Results can also be inaccurate if the sample of students that completes the survey is not representative of the school population. This is called bias. For example, if students who are absent the day the survey is administered tend to be from lower socioeconomic backgrounds, then the sample is not representative of the full school population. In this case, the results would likely be biased.
In order to contribute to the accuracy of the results and avoid bias as much as possible, schools should follow up with students who were absent when the survey was administered and have them to take complete the survey. The accuracy of reported measures is best achieved with a sample of 100 or more children, but a sample of 30 students provides a reasonable estimate for most purposes. When interpreting results for small samples, especially for drill-downs, the results should be interpreted with caution.