Sampling Student Work
What are the advantages of sampling student work?
Why sample?
Most programs will utilize a sampling method to obtain student work – especially when using the method of course-embedded assessment. Sampling is used to keep the assessment process manageable when the program has large numbers of students. It is also keeps the process manageable where there are multiple or lengthy artifacts to review.
If a program is small, assessing the entire population (i.e., using the census) is advised to be able to draw more accurate conclusions about student learning. It may take a program longer to gather the relevant evidence for all students in the program (e.g., instead of taking work from a given quarter of the year, it may take a year to accumulate the work on the entire population of students).
Sample size considerations
Programs should determine the appropriate sample size and method as they plan an assessment. Four factors should be considered: the percentage of the student population you wish to include; the length and complexity of the artifacts (student work products); the number of faculty who will be involved in evaluating student work; and the expected variation in student work.
If your program is quite large, you may wish to sample between 10 and 20 percent of the students. If the work samples are large or complex (for example, multiple pieces of student work per student), you may sample only 10 percent of the students—especially if you don't have too many faculty raters available. If the variation in student work is considerable, you may find you wish to expand the sample so that the program is better able to detect the trends in student performance.
The curriculum alignment matrix will identify the relevant courses to include in an assessment. Sometimes only one course is relevant and then it's a good idea to include multiple sections of that course. If multiple courses are relevant, it's best to sample student work from each. This can be done in any typical quarter or if course offerings are less frequent, sampling can be done throughout the year. It is important to consider the representativeness of the sample.
The sample sizes used for assessment are typically smaller than those used for social science research. Guidelines for appropriate sample sizes can be found in Mary Allen's, Assessing General Education Programs (available in the Office of Assessment). Mary Allen notes that generally if almost all the students in a program are doing well or poorly, small samples (e.g., 25 students) are sufficient, and that many general education assessments draw upon approximately 50-75 students.