20120614

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College, spring semester 2012)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development), and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam. The SPCI is administered in class during the last week of instruction.

The SASS results from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").
Cuesta College
Student Assessment of Skills Survey (SASS)
Astronomy 210 spring semester 2012 sections 30674, 30676
N = 52

The questions below are designed to characterize your achievement of each
of the learning outcomes by filling in a bubble on the rating scale
provided to the right of each statement.

Mark the level of achievement that best describes your learning at the
completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 90%, unachieved: 10%)
1. Very poor 1 : *
2. Below average 4 : ****
3. Average 20 : ******************** [3.6 +/- 0.9]
4. Above average 18 : ******************
5. Excellent 9 : *********

2. Explain sun cycles and seasons.
(Achieved: 96%, unachieved: 4%)
1. Very poor 0 :
2. Below average 2 : **
3. Average 21 : *********************
4. Above average 21 : ********************* [3.7 +/- 0.8]
5. Excellent 8 : ********

3. Explain and predict lunar phases and times.
(Achieved: 86%, unachieved: 14%)
1. Very poor 0 :
2. Below average 7 : *******
3. Average 17 : *****************
4. Above average 17 : ***************** [3.6 +/- 1.0]
5. Excellent 10 : **********

4. Relate planets in the sky to a solar system map.
(Achieved: 92%, unachieved: 8%)
1. Very poor 0 :
2. Below average 4 : ****
3. Average 23 : *********************** [3.6 +/- 0.9]
4. Above average 16 : ****************
5. Excellent 9 : *********

5. Explain differences between models of planetary motion.
(Achieved: 87%, unachieved: 13%)
1. Very poor 0 :
2. Below average 7 : *******
3. Average 21 : ********************* [3.5 +/- 0.9]
4. Above average 16 : ****************
5. Excellent 8 : ********

6. Explain evidence for the heliocentric model of planetary motion.
(Achieved: 80%, unachieved: 20%)
1. Very poor 1 : *
2. Below average 9 : *********
3. Average 22 : *********************** [3.3 +/- 1.0]
4. Above average 12 : ************
5. Excellent 7 : *******

7. Describe how optical telescopes work.
(Achieved: 90%, unachieved: 10%)
1. Very poor 0 :
2. Below average 5 : *****
3. Average 20 : ******************** [3.6 +/- 0.8]
4. Above average 20 : ********************
5. Excellent 7 : *******

8. Describe different powers of optical telescopes.
(Achieved: 88%, unachieved: 12%)
1. Very poor 0 :
2. Below average 6 : ******
3. Average 17 : *****************
4. Above average 20 : ******************** [3.6 +/- 0.9]
5. Excellent 9 : *********

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 90%, unachieved: 10%)
1. Very poor 0 :
2. Below average 5 : *****
3. Average 13 : *************
4. Above average 20 : ******************** [3.8 +/- 0.9]
5. Excellent 13 : *************

10. Explain how stars produce energy.
(Achieved: 92%, umachieved: 8%)
1. Very poor 0 :
2. Below average 4 : ****
3. Average 16 : ****************
4. Above average 24 : ************************ [3.7 +/- 0.8]
5. Excellent 8 : ********

11. Explain the relationship between star brightness and distances.
(Achieved: 96%, unachieved: 4%)
1. Very poor 0 :
2. Below average 2 : **
3. Average 11 : ***********
4. Above average 20 : ******************** [4.1 +/- 0.9]
5. Excellent 19 : *******************

12. Predict the size of a star based on brightness and temperature.
(Achieved: 96%, unachieved: 4%)
1. Very poor 0 :
2. Below average 2 : **
3. Average 13 : *************
4. Above average 21 : ********************* [4.0 +/- 0.8]
5. Excellent 16 : ****************

13. Explain different stages a star will go through, based on its mass.
(Achieved: 88%, unachieved: 12%)
1. Very poor 0 :
2. Below average 6 : ******
3. Average 13 : *************
4. Above average 21 : ********************* [3.8 +/- 0.9]
5. Excellent 12 : ************

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 87%, unachieved: 13%)
1. Very poor 1 : *
2. Below average 6 : ******
3. Average 16 : ****************
4. Above average 22 : ********************** [3.5 +/- 0.9]
5. Excellent 7 : *******

15. Explain evidence for how our Milky Way galaxy came to be.
(Achieved: 87%, unachieved: 13%)
1. Very poor 1 : *
2. Below average 6 : ******
3. Average 19 : ******************* [3.5 +/- 1.0]
4. Above average 17 : *****************
5. Excellent 9 : *********

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 87%, unachieved: 13%)
1. Very poor 0 :
2. Below average 6 : ******
3. Average 13 : *************
4. Above average 24 : ************************ [3.7 +/- 0.9]
5. Excellent 9 : *********

17. Explain evidence for the expansion of the universe.
(Achieved: 88%, unachieved: 12%)
1. Very poor 0 :
2. Below average 7 : *******
3. Average 19 : ******************* [3.6 +/- 1.0]
4. Above average 15 : ***************
5. Excellent 11 : ***********

18. Describe characteristics of the universe a long time ago.
(Achieved: 83%, unachieved: 17%)
1. Very poor 2 : **
2. Below average 7 : *******
3. Average 22 : ********************** [3.4 +/- 1.1]
4. Above average 12 : ************
5. Excellent 9 : *********

19. Explain evidence for how our solar system came to be.
(Achieved: 87%, unachieved: 13%)
1. Very poor 0 :
2. Below average 7 : *******
3. Average 18 : ******************
4. Above average 22 : ********************** [3.5 +/- 0.8]
5. Excellent 5 : *****

20. Describe key features of terrestrial planets.
(Achieved: 94%, unachieved: 6%)
1. Very poor 1 : *
2. Below average 2 : **
3. Average 15 : **************
4. Above average 20 : ******************** [3.8 +/- 0.9]
5. Excellent 14 : **************

21. Describe key features of jovian planets.
(Achieved: 90%, unachieved: 10%)
1. Very poor 2 : **
2. Below average 3 : ***
3. Average 21 : ********************* [3.6 +/- 1.0]
4. Above average 15 : ***************
5. Excellent 11 : ***********

22. Explain why Pluto is not currently categorized as a planet.
(Achieved: 86%, unachieved: 14%)
1. Very poor 3 : ***
2. Below average 4 : ****
3. Average 8 : ********
4. Above average 12 : ************
5. Excellent 24 : ************************ [4.0 +/- 1.3]

23. Describe plausible requirements for life.
(Achieved: 98%, unachieved: 2%)
1. Very poor 1 : *
2. Below average 0 :
3. Average 18 : ******************
4. Above average 18 : ****************** [3.9 +/- 0.9]
5. Excellent 15 : ***************

24. Explain difficulties in investigating the possibility for extraterrestial life.
(Achieved: 96%, unachieved: 4%)
1. Very poor 1 : *
2. Below average 1 : *
3. Average 12 : ************
4. Above average 21 : ********************* [4.0 +/- 0.9]
5. Excellent 17 : *****************

Of the 24 student learning outcomes in the SASS, 22 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
23. Describe plausible requirements for life. (98%)
2. Explain sun cycles and seasons. (96%)
11. Explain the relationship between star brightness and distances. (96%)
12. Predict the size of a star based on brightness and temperature. (96%)
24. Explain difficulties in investigating the possibility for extraterrestial life. (96%)
20. Describe key features of terrestrial planets. (94%)
4. Relate planets in the sky to a solar system map. (92%)
10. Explain how stars produce energy. (92%)
1. Predict positions and cycles of stars, using a starwheel. (90%)
7. Describe how optical telescopes work. (90%)
9. Explain which telescopes should be funded based on relevant criteria. (90%)
21. Describe key features of jovian planets. (90%)
8. Describe different powers of optical telescopes. (88%)
17. Explain evidence for the expansion of the universe. (88%)
13. Explain different stages a star will go through, based on its mass. (88%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (87%)
15. Explain evidence for how our Milky Way galaxy came to be. (87%)
16. Explain how the speed of light affects observations of distant objects. (87%)
19. Explain evidence for how our solar system came to be. (87%)
5. Explain differences between models of planetary motion. (87%)
3. Explain and predict lunar phases and times. (86%)
22. Explain why Pluto is not currently categorized as a planet. (86%)

However, two student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
18. Describe characteristics of the universe a long time ago. (83%)
6. Explain evidence for the heliocentric model of planetary motion. (80%)

Of note is that these student learning outcomes self-reported as not being achieved (6, 18) are not the same as in the previous fall semester 2011 (4, 7, 8).

Student learning outcomes 10, 11, 12, and 13 for Cuesta College students were directly assessed using the Star Properties Concept Inventory (excluding negative informed consent form responses):
Star Properties Concept Inventory v3.0
Astronomy 210 spring semester 2012 sections 30674, 30676
N = 49
ave ± stdev = 46% ± 10%
These SPCI scores are comparable to results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007), where the average was 51% (no further statistics provided); and also comparable to SPCI results from earlier semesters at Cuesta College.

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this indirect assessment SASS tool, along with the direct assessment SPCI tool will be used for course/program improvement by increasing emphasis on these lowest three learning outcomes in instruction in future semesters.

Previous posts:

No comments: