Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College)

White paper distributed at special session C3B, "Writing and Assessing Student Learning Objectives: Tips, Techniques, and What Our Community Needs," facilitated by Andrew Fraknoi (Foothill College), at the Astronomy Society of the Pacific Cosmos in the Classroom: A Hands-on Symposium on Teaching Introductory Astronomy and Related Science, July 23, 2013, 10:00-11:00 AM, Student Union Costanoan Room, San José State University, San Jose, CA.

The Student Assessment of Skills Survey (SASS, Patrick M. Len, in development) and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) are used to measure achievement of student learning outcomes, such that these results guide "continuous quality improvement" to instruction at Cuesta College, as mandated by the Accrediting Commission for Community and Junior Colleges (ACCJC). The SASS is a broad indirect assessment survey asking students to self-report their achievement of learning outcomes; in contrast the SPCI is a direct assessment instrument specifically evaluating student understanding of fundamental properties of stars. Programs at Cuesta College utilize student self-reports to assess learning outcomes, while also using direct assessment on selected student learning outcomes in order to comply with the ACCJC Rubric for Evaluating Institutional Effectiveness.

The SASS is assigned as homework during the last week of instruction, to be completed online before the final exam. For each of the 24 learning outcomes listed below, students are asked to mark the level of achievement that best describes their learning at the completion of the course, using a five-point Likert scale (1 = very poor; 2 = below average; 3 = average; 4 = above average; 5 = excellent).

1. Predict positions and cycles of stars, using a starwheel.
2. Explain sun cycles and seasons.
3. Explain and predict lunar phases and times.
4. Relate planets in the sky to a solar system map.
5. Explain differences between models of planetary motion.
6. Explain evidence for the heliocentric model of planetary motion.
7. Describe how optical telescopes work.
8. Describe different powers of optical telescopes.
9. Explain which telescopes should be funded based on relevant criteria.
10. Explain how stars produce energy.
11. Explain the relationship between star brightness and distances.
12. Predict the size of a star based on brightness and temperature.
13. Explain different stages a star will go through, based on its mass.
14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
15. Explain evidence for how our Milky Way galaxy came to be.
16. Explain how the speed of light affects observations of distant objects.
17. Explain evidence for the expansion of the universe.
18. Describe characteristics of the universe a long time ago.
19. Explain evidence for how our solar system came to be.
20. Describe key features of terrestrial planets.
21. Describe key features of jovian planets.
22. Explain why Pluto is not currently categorized as a planet.
23. Describe plausible requirements for life.
24. Explain difficulties in investigating the possibility for extraterrestial life.

For each of these learning outcomes, the percentage of achieved students is calculated from the consolidation of "average," "above average," and "excellent" responses, and the percentage of unachieved students from the consolidation of "very poor" and "below average" responses. The learning outcomes with the lowest percentage of achieved students are then noted for guiding "continuous quality improvement" to instruction in subsequent semesters.

Learning outcomes 10, 11, 12, and 13 are also directly assessed using the SPCI administered in class during the last week of instruction (excluding negative informed consent form responses) strictly as a post-instruction test. The average SPCI class score is then compared to results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007). These results are also used to guide "continuous quality improvement" to instruction in subsequent semesters.

This document, and further discussion of SASS and SPCI results at Cuesta College are posted at:

http://tinyurl.com/astronomySASS.