Education research: SPCI statistics (spring semester 2017)

Students at Cuesta College (San Luis Obispo, CA) were administered the Star Properties Concept Inventory (SPCI version 3.0, developed by Janelle Bailey, University of Nevada-Las Vegas) during the first and the last week of instruction. Astronomy 210 is a one-semester introductory general science course, with a separate optional adjunct laboratory (Astronomy 210L).

The pre- to post-test gain for this semester at Cuesta College (excluding students with negative informed consent forms (*.pdf), and missing pre- or post-tests) is:

Astronomy 210 spring semester 2017 section 30674
N = 18 (matched-pairs)
<initial%>	= 36% ± 14%
<final%>	= 64% ± 16%
<g>	= 0.43 ± 0.22 (matched-pairs); 0.43 (class-wise)

Astronomy 210 spring semester 2017 section 30676
N = 35 (matched-pairs)
<initial%>	= 33% ± 17%
<final%>	= 57% ± 16%
<g>	= 0.32 ± 0.24 (matched-pairs); 0.36 (class-wise)

This semester's SPCI pre-instruction scores scores are comparable to results from previous semesters at Cuesta College, while the post-instruction scores and gains for section 30674 are slightly higher than those of section 30676, and some previous semesters.

Education research: SPCI statistics (spring semester 2016)

Students at Cuesta College (San Luis Obispo, CA) were administered the Star Properties Concept Inventory (SPCI version 3.0, developed by Janelle Bailey, University of Nevada-Las Vegas) during the first and the last week of instruction. Astronomy 210 is a one-semester introductory general science course, with a separate optional adjunct laboratory (Astronomy 210L).

The pre- to post-test gain for this semester at Cuesta College (excluding students with negative informed consent forms (*.pdf), and missing pre- or post-tests) is:

Astronomy 210 spring semester 2016 sections 30674, 30676
N = 59 (matched-pairs)
<initial%>	= 32% ± 15%
<final%>	= 54% ± 14%
<g>	= 0.32 ± 0.14 (matched-pairs); 0.33 (class-wise)

This semester's SPCI pre- and post-instruction scores are comparable to results from previous semesters at Cuesta College.

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College, spring semester 2015)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development), and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam. The SPCI is administered as a post-test in class during the last week of instruction.

The SASS results from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").

Cuesta College
Student Assessment of Skills Survey (SASS)
Astronomy 210 spring semester 2015 sections 30674, 30676
N = 46

The questions below are designed to characterize your achievement of each of the learning outcomes by filling in a bubble on the rating scale provided to the right of each statement.

Mark the level of achievement that best describes your learning at the completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 93%, unachieved: 7%)

Very poor.	[0]
Below average.	*** [3]
Average.	*************** [15]
Above average.	************* [13]
Excellent.	************ [12]

2. Explain sun cycles and seasons.
(Achieved: 98%, unachieved: 2%)

Very poor.	* [1]
Below average.	[0]
Average.	************** [14]
Above average.	*************** [15]
Excellent.	************* [13]

3. Explain and predict lunar phases and times.
(Achieved: 88%, unachieved: 12%)

Very poor.	* [1]
Below average.	**** [4]
Average.	******** [8]
Above average.	*********** [11]
Excellent.	******************* [19]

4. Relate planets in the sky to a solar system map.
(Achieved: 86%, unachieved: 14%)

Very poor.	* [1]
Below average.	***** [5]
Average.	**************** [16]
Above average.	*********** [11]
Excellent.	********** [10]

5. Explain differences between models of planetary motion.
(Achieved: 81%, unachieved: 19%)

Very poor.	[0]
Below average.	******** [8]
Average.	*************** [15]
Above average.	************ [12]
Excellent.	******** [8]

6. Explain evidence for the heliocentric model of planetary motion.
(Achieved: 74%, unachieved: 26%)

Very poor.	[0]
Below average.	*********** [11]
Average.	************* [13]
Above average.	************* [13]
Excellent.	****** [6]

7. Describe how optical telescopes work.
(Achieved: 88%, unachieved: 12%)

Very poor.	* [1]
Below average.	**** [4]
Average.	**************** [16]
Above average.	****************** [13]
Excellent.	********* [9]

8. Describe different powers of optical telescopes.
(Achieved: 88%, unachieved: 12%)

Very poor.	* [1]
Below average.	*** [3]
Average.	******** [8]
Above average.	******************** [20]
Excellent.	*********** [11]

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 91%, unachieved: 9%)

Very poor.	* [1]
Below average.	*** [3]
Average.	******** [8]
Above average.	******************** [20]
Excellent.	*********** [11]

10. Explain how stars produce energy.
(Achieved: 91%, unachieved: 9%)

Very poor.	** [2]
Below average.	** [2]
Average.	***************** [17]
Above average.	**************** [11]
Excellent.	**************** [11]

11. Explain the relationship between star brightness and distances.
(Achieved: 98%, unachieved: 2%)

Very poor.	[0]
Below average.	* [1]
Average.	********** [10]
Above average.	************ [12]
Excellent.	******************** [20]

12. Predict the size of a star based on brightness and temperature.
(Achieved: 95%, unachieved: 5%)

Very poor.	[0]
Below average.	** [2]
Average.	******** [8]
Above average.	**************** [16]
Excellent.	***************** [17]

13. Explain different stages a star will go through, based on its mass.
(Achieved: 88%, unachieved: 12%)

Very poor.	[0]
Below average.	***** [5]
Average.	**************** [16]
Above average.	************** [14]
Excellent.	******** [8]

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 86%, unachieved: 14%)

Very poor.	* [1]
Below average.	***** [5]
Average.	*************** [15]
Above average.	***************** [17]
Excellent.	***** [5]

15. Explain evidence for how our Milky Way galaxy came to be.
(Achieved: 79%, unachieved: 21%)

Very poor.	* [1]
Below average.	******** [8]
Average.	*************** [15]
Above average.	************** [14]
Excellent.	***** [5]

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 81%, unachieved: 19%)

Very poor.	** [2]
Below average.	****** [6]
Average.	*********** [11]
Above average.	********* [9]
Excellent.	*************** [15]

17. Explain evidence for the expansion of the universe.
(Achieved: 84%, unachieved: 16%)

Very poor.	** [2]
Below average.	***** [5]
Average.	*************** [15]
Above average.	************** [14]
Excellent.	******* [7]

18. Describe characteristics of the universe a long time ago.
(Achieved: 84%, unachieved: 16%)

Very poor.	** [2]
Below average.	***** [5]
Average.	************** [14]
Above average.	*************** [15]
Excellent.	******* [7]

19. Explain evidence for how our solar system came to be.
(Achieved: 79%, unachieved: 21%)

Very poor.	[0]
Below average.	********* [9]
Average.	*************** [15]
Above average.	*************** [15]
Excellent.	**** [4]

20. Describe key features of terrestrial planets.
(Achieved: 86%, unachieved: 14%)

Very poor.	* [1]
Below average.	***** [5]
Average.	************ [12]
Above average.	***************** [17]
Excellent.	******** [8]

21. Describe key features of jovian planets.
(Achieved: 86%, unachieved: 14%)

Very poor.	* [1]
Below average.	***** [5]
Average.	*************** [15]
Above average.	*************** [15]
Excellent.	******* [7]

22. Explain why Pluto is not currently categorized as a planet.
(Achieved: 95%, unachieved: 5%)

Very poor.	* [1]
Below average.	* [1]
Average.	******* [7]
Above average.	***** [5]
Excellent.	***************************** [29]

23. Describe plausible requirements for life.
(Achieved: 98%, unachieved: 2%)

Very poor.	* [0]
Below average.	* [1]
Average.	***************** [17]
Above average.	************ [12]
Excellent.	************* [13]

24. Explain difficulties in investigating the possibility for extraterrestrial life.
(Achieved: 93%, unachieved: 7%)

Very poor.	[0]
Below average.	*** [3]
Average.	*************** [15]
Above average.	************* [13]
Excellent.	************ [12]

Of the 24 student learning outcomes in the SASS, 18 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
2. Explain sun cycles and seasons. (98%)
11. Explain the relationship between star brightness and distances. (98%)
23. Describe plausible requirements for life. (98%)
12. Predict the size of a star based on brightness and temperature. (95%)
22. Explain why Pluto is not currently categorized as a planet. (95%)
1. Predict positions and cycles of stars, using a starwheel. (93%)
24. Explain difficulties in investigating the possibility for extraterrestrial life. (93%)
9. Explain which telescopes should be funded based on relevant criteria. (91%)
10. Explain how stars produce energy. (91%)
3. Explain and predict lunar phases and times. (88%)
7. Describe how optical telescopes work. (88%)
8. Describe different powers of optical telescopes. (88%)
13. Explain different stages a star will go through, based on its mass. (88%)
4. Relate planets in the sky to a solar system map. (86%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (86%)
20. Describe key features of terrestrial planets. (86%)
21. Describe key features of jovian planets. (86%)

However, six student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
18. Describe characteristics of the universe a long time ago. (84%)
5. Explain differences between models of planetary motion. (81%)
16. Explain how the speed of light affects observations of distant objects. (81%)
15. Explain evidence for how our Milky Way galaxy came to be. (79%)
19. Explain evidence for how our solar system came to be. (79%)
6. Explain evidence for the heliocentric model of planetary motion. (74%)

Compare these student learning outcomes self-reported as not being achieved (5, 6, 15, 16, 18, 19) with those from previous semesters (fall semester 2014: (7, 21); spring semester 2014: (4, 6, 14, 15, 18, 24); fall semester 2013: (6, 9, 14, 15, 17, 18); spring semester 2012: (6, 18); fall semester 2011: (4, 7, 8)).

Student learning outcomes 10, 11, 12, and 13 for Cuesta College students were directly assessed using the Star Properties Concept Inventory (excluding negative informed consent form responses):

Star Properties Concept Inventory v3.0
Astronomy 210 spring semester 2015 sections 30674, 30676
N = 68
ave ± stdev = 56% ± 19%

This semester's SPCI scores are slightly higher than results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007), where the average was 51% (no further statistics provided); and comparable to SPCI results from earlier semesters at Cuesta College.

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this indirect assessment SASS tool, along with the direct assessment SPCI tool will be used for course/program improvement by increasing emphasis on the lowest learning outcomes in instruction in future semesters.

Previous posts:

• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2014).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2014).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2013).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2012).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2010).

Education research: SPCI statistics (spring semester 2015)

Students at Cuesta College (San Luis Obispo, CA) were administered the Star Properties Concept Inventory (SPCI version 3.0, developed by Janelle Bailey, University of Nevada-Las Vegas) during the first and the last week of instruction. Astronomy 210 is a one-semester introductory general science course, with a separate optional adjunct laboratory (Astronomy 210L).

The pre- to post-test gain for this semester at Cuesta College (excluding students with negative informed consent forms (*.pdf), and missing pre- or post-tests) is:

Astronomy 210 spring semester 2015 sections 30674, 30676
N = 68 (matched-pairs)
<initial%>	= 31% ± 12%
<final%>	= 56 ± 19%
<g>	= 0.38 ± 0.23 (matched-pairs); 0.36 (class-wise)

This semester's SPCI pre- and post-instruction scores are comparable to results from previous semesters at Cuesta College.

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College, fall semester 2014)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development), and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam. The SPCI is administered as a post-test in class during the last week of instruction.

The SASS results from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").

Cuesta College
Student Assessment of Skills Survey (SASS)
Astronomy 210 fall semester 2014 sections 70158, 70160
N = 46

The questions below are designed to characterize your achievement of each of the learning outcomes by filling in a bubble on the rating scale provided to the right of each statement.

Mark the level of achievement that best describes your learning at the completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 98%, unachieved: 2%)

Very poor.	[0]
Below average.	* [1]
Average.	*************** [15]
Above average.	************* [13]
Excellent.	***************** [17]

2. Explain sun cycles and seasons.
(Achieved: 91%, unachieved: 9%)

Very poor.	[0]
Below average.	**** [4]
Average.	****************** [18]
Above average.	********* [9]
Excellent.	*************** [15]

3. Explain and predict lunar phases and times.
(Achieved: 93%, unachieved: 7%)

Very poor.	* [1]
Below average.	** [2]
Average.	****** [6]
Above average.	*************** [15]
Excellent.	********************** [22]

4. Relate planets in the sky to a solar system map.
(Achieved: 89%, unachieved: 11%)

Very poor.	[0]
Below average.	***** [5]
Average.	******************* [19]
Above average.	************* [13]
Excellent.	********* [9]

5. Explain differences between models of planetary motion.
(Achieved: 87%, unachieved: 13%)

Very poor.	[0]
Below average.	****** [6]
Average.	******************* [19]
Above average.	************* [13]
Excellent.	******** [8]

6. Explain evidence for the heliocentric model of planetary motion.
(Achieved: 87%, unachieved: 13%)

Very poor.	[0]
Below average.	****** [6]
Average.	******************** [20]
Above average.	**************** [16]
Excellent.	**** [4]

7. Describe how optical telescopes work.
(Achieved: 78%, unachieved: 22%)

Very poor.	*** [3]
Below average.	******* [7]
Average.	***************** [17]
Above average.	*************** [14]
Excellent.	***** [5]

8. Describe different powers of optical telescopes.
(Achieved: 85%, unachieved: 15%)

Very poor.	* [1]
Below average.	****** [6]
Average.	***************** [17]
Above average.	*************** [15]
Excellent.	******* [7]

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 87%, unachieved: 13%)

Very poor.	[0]
Below average.	****** [6]
Average.	************** [14]
Above average.	************** [14]
Excellent.	************ [12]

10. Explain how stars produce energy.
(Achieved: 93%, unachieved: 7%)

Very poor.	[0]
Below average.	*** [3]
Average.	************** [14]
Above average.	******************* [19]
Excellent.	********** [10]

11. Explain the relationship between star brightness and distances.
(Achieved: 89%, unachieved: 11%)

Very poor.	* [1]
Below average.	**** [4]
Average.	******* [7]
Above average.	************** [14]
Excellent.	******************** [20]

12. Predict the size of a star based on brightness and temperature.
(Achieved: 93%, unachieved: 7%)

Very poor.	[0]
Below average.	*** [3]
Average.	****************** [18]
Above average.	************ [17]
Excellent.	******* [7]

13. Explain different stages a star will go through, based on its mass.
(Achieved: 87%, unachieved: 13%)

Very poor.	* [1]
Below average.	***** [5]
Average.	************** [14]
Above average.	***************** [17]
Excellent.	********* [9]

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 91%, unachieved: 9%)

Very poor.	* [1]
Below average.	*** [3]
Average.	****************** [18]
Above average.	***************** [17]
Excellent.	******* [7]

15. Explain evidence for how our Milky Way galaxy came to be.
(Achieved: 91%, unachieved: 9%)

Very poor.	* [1]
Below average.	*** [3]
Average.	******************* [19]
Above average.	**** [14]
Excellent.	********* [9]

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 96%, unachieved: 4%)

Very poor.	* [1]
Below average.	* [1]
Average.	************** [14]
Above average.	***************** [17]
Excellent.	*********** [11]

17. Explain evidence for the expansion of the universe.
(Achieved: 91%, unachieved: 9%)

Very poor.	[0]
Below average.	**** [4]
Average.	************** [14]
Above average.	***************** [17]
Excellent.	*********** [11]

18. Describe characteristics of the universe a long time ago.
(Achieved: 93%, unachieved: 7%)

Very poor.	[0]
Below average.	*** [3]
Average.	********************** [22]
Above average.	************** [14]
Excellent.	******* [7]

19. Explain evidence for how our solar system came to be.
(Achieved: 87%, unachieved: 13%)

Very poor.	[0]
Below average.	****** [6]
Average.	******************** [20]
Above average.	************* [13]
Excellent.	******* [7]

20. Describe key features of terrestrial planets.
(Achieved: 89%, unachieved: 11%)

Very poor.	[0]
Below average.	***** [5]
Average.	******************** [20]
Above average.	*********** [11]
Excellent.	********** [10]

21. Describe key features of jovian planets.
(Achieved: 80%, unachieved: 20%)

Very poor.	** [2]
Below average.	******* [7]
Average.	**************** [16]
Above average.	************* [13]
Excellent.	******** [8]

22. Explain why Pluto is not currently categorized as a planet.
(Achieved: 93%, unachieved: 7%)

Very poor.	[0]
Below average.	**** [4]
Average.	********** [10]
Above average.	********* [9]
Excellent.	*********************** [23]

23. Describe plausible requirements for life.
(Achieved: 93%, unachieved: 7%)

Very poor.	* [1]
Below average.	** [2]
Average.	*************** [15]
Above average.	******************* [19]
Excellent.	********* [9]

24. Explain difficulties in investigating the possibility for extraterrestrial life.
(Achieved: 93%, unachieved: 7%)

Very poor.	* [1]
Below average.	** [2]
Average.	***************** [17]
Above average.	****************** [18]
Excellent.	********* [9]

Of the 24 student learning outcomes in the SASS, 22 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
1. Predict positions and cycles of stars, using a starwheel. (98%)
16. Explain how the speed of light affects observations of distant objects. (96%)
3. Explain and predict lunar phases and times. (93%)
10. Explain how stars produce energy. (93%)
12. Predict the size of a star based on brightness and temperature. (93%)
18. Describe characteristics of the universe a long time ago. (93%)
22. Explain why Pluto is not currently categorized as a planet. (93%)
23. Describe plausible requirements for life. (93%)
24. Explain difficulties in investigating the possibility for extraterrestrial life. (93%)
2. Explain sun cycles and seasons. (91%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (91%)
15. Explain evidence for how our Milky Way galaxy came to be. (91%)
17. Explain evidence for the expansion of the universe. (91%)
4. Relate planets in the sky to a solar system map. (89%)
11. Explain the relationship between star brightness and distances. (89%)
20. Describe key features of terrestrial planets. (89%)
5. Explain differences between models of planetary motion. (87%)
6. Explain evidence for the heliocentric model of planetary motion. (87%)
9. Explain which telescopes should be funded based on relevant criteria. (87%)
13. Explain different stages a star will go through, based on its mass. (87%)
19. Explain evidence for how our solar system came to be. (87%)
8. Describe different powers of optical telescopes. (85%)

However, two student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
21. Describe key features of jovian planets. (80%)
7. Describe how optical telescopes work. (78%)

Compare these student learning outcomes self-reported as not being achieved (7, 21) with those from previous semesters (spring semester 2014: (4, 6, 14, 15, 18, 24); fall semester 2013: (6, 9, 14, 15, 17, 18); spring semester 2012: (6, 18); fall semester 2011: (4, 7, 8)).

Student learning outcomes 10, 11, 12, and 13 for Cuesta College students were directly assessed using the Star Properties Concept Inventory (excluding negative informed consent form responses):

Star Properties Concept Inventory v3.0
Astronomy 210 fall semester 2014 sections 70158, 70160
N = 74
ave ± stdev = 60% ± 16%

This semester's SPCI scores are slightly higher than results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007), where the average was 51% (no further statistics provided); and also slightly higher than SPCI results from earlier semesters at Cuesta College.

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this indirect assessment SASS tool, along with the direct assessment SPCI tool will be used for course/program improvement by increasing emphasis on these lowest learning outcomes in instruction in future semesters.

Previous posts:

• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2014).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2013).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2012).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2010).

Education research: SPCI statistics (fall semester 2014)

Students at Cuesta College (San Luis Obispo, CA) were administered the Star Properties Concept Inventory (SPCI version 3.0, developed by Janelle Bailey, University of Nevada-Las Vegas) during the first and the last week of instruction. Astronomy 210 is a one-semester introductory general science course, with a separate optional adjunct laboratory (Astronomy 210L).

The pre- to post-test gain for this semester at Cuesta College (excluding students with negative informed consent forms (*.pdf), and missing pre- or post-tests) is:

Astronomy 210 fall semester 2014 sections 70158, 70160
N = 74 (matched-pairs)
<initial%>	= 34% ± 15%
<final%>	= 60% ± 16%
<g>	= 0.39 ± 0.22 (matched-pairs); 0.39 (class-wise)

This semester's SPCI pre- and post-instruction scores are slightly higher than results from previous semesters at Cuesta College, and this semester's gain is much higher than those in previous semesters.

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College, spring semester 2014)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development), and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam. The SPCI is administered in class during the last week of instruction.

The SASS results from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").

Cuesta College
Student Assessment of Skills Survey (SASS)
Astronomy 210 spring semester 2014 sections 30674, 30676
N = 52
The questions below are designed to characterize your achievement of each of the learning outcomes by filling in a bubble on the rating scale provided to the right of each statement.

Mark the level of achievement that best describes your learning at the completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 98%, unachieved: 2%)

Very poor.	* [1]
Below average.	[0]
Average.	*************** [15]
Above average.	************************ [24]
Excellent.	************ [12]

2. Explain sun cycles and seasons.
(Achieved: 100%, unachieved: 0%)

Very poor.	[0]
Below average.	[0]
Average.	********************** [22]
Above average.	********************* [21]
Excellent.	********* [9]

3. Explain and predict lunar phases and times.
(Achieved: 92%, unachieved: 8%)

Very poor.	[0]
Below average.	**** [4]
Average.	***************** [17]
Above average.	**************** [16]
Excellent.	*************** [15]

4. Relate planets in the sky to a solar system map.
(Achieved: 83%, unachieved: 17%)

Very poor.	* [1]
Below average.	******** [8]
Average.	******************* [19]
Above average.	****************** [18]
Excellent.	****** [6]

5. Explain differences between models of planetary motion.
(Achieved: 92%, unachieved: 8%)

Very poor.	[0]
Below average.	**** [4]
Average.	******************************* [31]
Above average.	************* [13]
Excellent.	**** [4]

6. Explain evidence for the heliocentric model of planetary motion.
(Achieved: 83%, unachieved: 17%)

Very poor.	[0]
Below average.	********* [9]
Average.	************************* [25]
Above average.	************* [13]
Excellent.	***** [5]

7. Describe how optical telescopes work.
(Achieved: 88%, unachieved: 12%)

Very poor.	[0]
Below average.	****** [6]
Average.	************************** [26]
Above average.	**************** [16]
Excellent.	**** [4]

8. Describe different powers of optical telescopes.
(Achieved: 87%, unachieved: 13%)

Very poor.	[0]
Below average.	******* [7]
Average.	*********************** [23]
Above average.	*************** [15]
Excellent.	******* [7]

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 92%, unachieved: 8%)

Very poor.	[0]
Below average.	**** [4]
Average.	***************** [17]
Above average.	****************** [18]
Excellent.	************* [13]

10. Explain how stars produce energy.
(Achieved: 92%, unachieved: 8%)

Very poor.	[0]
Below average.	**** [4]
Average.	******************* [19]
Above average.	********************* [21]
Excellent.	******** [8]

11. Explain the relationship between star brightness and distances.
(Achieved: 96%, unachieved: 4%)

Very poor.	[0]
Below average.	** [2]
Average.	************ [12]
Above average.	*************************** [27]
Excellent.	*********** [11]

12. Predict the size of a star based on brightness and temperature.
(Achieved: 96%, unachieved: 4%)

Very poor.	[0]
Below average.	** [2]
Average.	***************** [17]
Above average.	************************ [24]
Excellent.	********* [9]

13. Explain different stages a star will go through, based on its mass.
(Achieved: 94%, unachieved: 6%)

Very poor.	[0]
Below average.	*** [3]
Average.	******************** [20]
Above average.	******************** [20]
Excellent.	********* [9]

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 83%, unachieved: 17%)

Very poor.	* [1]
Below average.	******** [8]
Average.	***************************** [29]
Above average.	************ [12]
Excellent.	** [2]

15. Explain evidence for how our Milky Way galaxy came to be.
(Achieved: 77%, unachieved: 23%)

Very poor.	[0]
Below average.	************ [12]
Average.	*************************** [27]
Above average.	************ [12]
Excellent.	* [1]

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 88%, unachieved: 12%)

Very poor.	[0]
Below average.	****** [6]
Average.	************************ [24]
Above average.	***************** [17]
Excellent.	***** [5]

17. Explain evidence for the expansion of the universe.
(Achieved: 85%, unachieved: 15%)

Very poor.	* [1]
Below average.	******* [7]
Average.	*************************** [27]
Above average.	************** [14]
Excellent.	*** [3]

18. Describe characteristics of the universe a long time ago.
(Achieved: 81%, unachieved: 19%)

Very poor.	* [1]
Below average.	********* [9]
Average.	************************ [24]
Above average.	************* [13]
Excellent.	***** [5]

19. Explain evidence for how our solar system came to be.
(Achieved: 85%, unachieved: 15%)

Very poor.	* [1]
Below average.	******* [7]
Average.	************************** [26]
Above average.	************** [14]
Excellent.	**** [4]

20. Describe key features of terrestrial planets.
(Achieved: 96%, unachieved: 4%)

Very poor.	[0]
Below average.	** [2]
Average.	******************** [20]
Above average.	************************ [24]
Excellent.	****** [6]

21. Describe key features of jovian planets.
(Achieved: 90%, unachieved: 10%)

Very poor.	* [1]
Below average.	**** [4]
Average.	********************** [22]
Above average.	********************** [22]
Excellent.	*** [3]

22. Explain why Pluto is not currently categorized as a planet.
(Achieved: 98%, unachieved: 2%)

Very poor.	* [1]
Below average.	[0]
Average.	********* [9]
Above average.	************************ [24]
Excellent.	****************** [18]

23. Describe plausible requirements for life.
(Achieved: 98%, unachieved: 2%)

Very poor.	[0]
Below average.	* [1]
Average.	**************** [16]
Above average.	************************** [26]
Excellent.	********* [9]

24. Explain difficulties in investigating the possibility for extraterrestial life.
(Achieved: 81%, unachieved: 19%)

Very poor.	* [1]
Below average.	********* [9]
Average.	**************** [16]
Above average.	***************** [17]
Excellent.	********* [9]

Of the 24 student learning outcomes in the SASS, 18 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
2. Explain sun cycles and seasons. (100%)
1. Predict positions and cycles of stars, using a starwheel. (98%)
23. Describe plausible requirements for life. (98%)
22. Explain why Pluto is not currently categorized as a planet. (98%)
20. Describe key features of terrestrial planets. (96%)
11. Explain the relationship between star brightness and distances. (96%)
12. Predict the size of a star based on brightness and temperature. (96%)
13. Explain different stages a star will go through, based on its mass. (94%)
3. Explain and predict lunar phases and times. (92%)
5. Explain differences between models of planetary motion. (92%)
9. Explain which telescopes should be funded based on relevant criteria. (92%)
10. Explain how stars produce energy. (92%)
21. Describe key features of jovian planets. (90%)
7. Describe how optical telescopes work. (88%)
16. Explain how the speed of light affects observations of distant objects. (88%)
8. Describe different powers of optical telescopes. (87%)
17. Explain evidence for the expansion of the universe. (85%)
19. Explain evidence for how our solar system came to be. (85%)

However, six student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
4. Relate planets in the sky to a solar system map. (83%)
6. Explain evidence for the heliocentric model of planetary motion. (83%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (83%)
18. Describe characteristics of the universe a long time ago. (81%)
24. Explain difficulties in investigating the possibility for extraterrestial life. (81%)
15. Explain evidence for how our Milky Way galaxy came to be. (77%)

Compare these student learning outcomes self-reported as not being achieved (4, 6, 14, 15, 18, 24) those from a previous semesters (fall semester 2013: (6, 9, 14, 15, 17, 18); spring semester 2012: (6, 18); fall semester 2011: (4, 7, 8)).

Student learning outcomes 10, 11, 12, and 13 for Cuesta College students were directly assessed using the Star Properties Concept Inventory (excluding negative informed consent form responses):

Star Properties Concept Inventory v3.0
Astronomy 210 spring semester 2014 sections 30674, 30676
N = 55
ave ± stdev = 55% ± 16%

This semester's SPCI scores are comparable to results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007), where the average was 51% (no further statistics provided); and also comparable to SPCI results from earlier semesters at Cuesta College.

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this indirect assessment SASS tool, along with the direct assessment SPCI tool will be used for course/program improvement by increasing emphasis on these lowest three learning outcomes in instruction in future semesters.

Previous posts:

• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2013).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2012).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2010).

Education research: SPCI statistics (spring semester 2014)

Students at Cuesta College (San Luis Obispo, CA) were administered the Star Properties Concept Inventory (SPCI version 3.0, developed by Janelle Bailey, University of Nevada-Las Vegas) during the first and the last week of instruction. Astronomy 210 is a one-semester introductory general science course, with a separate optional adjunct laboratory (Astronomy 210L).

The pre- to post-test gain for this semester at Cuesta College (excluding students with negative informed consent forms (*.pdf), and missing pre- or post-tests) is:

Astronomy 210 spring semester 2014 sections 30674, 30676
N = 55 (matched-pairs)
<initial%>	= 29% ± 12%
<final%>	= 55% ± 16%
<g>	= 0.35 ± 0.25 (matched-pairs); 0.37 (class-wise)

This semester's SPCI post-instruction score is slightly higher than results from previous semesters at Cuesta College, and this semester's gain is much higher than those in previous semesters.

SPCI pre-test comparison: Cuesta College SLO vs. NC campuses

The Star Properties Concept Inventory (SPCI, developed by Janelle Bailey, University of Nevada-Las Vegas) was administered to Astronomy 210 (one-semester introductory astronomy) students at Cuesta College, San Luis Obispo, CA during the first week, at both the main San Luis Obispo campus and the North County campus at Paso Robles.

	NC campus (section 30674)	SLO Campus (section 30676)
N	31 students*	50 students*
low	3	0
mean	6.4 ± 2.0	6.5 ± 3.2
high	11	16

*Excludes students with negative informed consent forms (*.pdf)

Student's t-test of the null hypothesis results in p = 0.88 (t = -0.416, sdev = 2.78, degrees of freedom = 79), thus there is no significant difference between the pre-test scores of these two sections.

Later this semester (spring semester 2014), a comparison will be made between the pre- to post-test gains for both sections, separate and combined.

Previous posts:

SASS, SPCI and student learning outcomes assessment (fall semester 2013) .

SASS, SPCI and student learning outcomes assessment (spring semester 2012).

SPCI and student learning outcomes (fall semester 2011).

SPCI gains (spring semester 2009).

SPCI gains (spring semester 2008).

SPCI gains (fall semester 2007).

SPCI gains (summer session 2007).

SPCI gains (spring semester 2006-spring semester 2007).

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College, fall semester 2013)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development), and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam. The SPCI is administered in class during the last week of instruction.

The SASS results from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").

Cuesta College
Student Assessment of Skills Survey (SASS)
Astronomy 210 fall semester 2013 sections 70158[*], 70160
N = 50
[*] Questions (20)-(22) were not covered for this section, and results from these students (N = 32) are not posted below.

The questions below are designed to characterize your achievement of each of the learning outcomes by filling in a bubble on the rating scale provided to the right of each statement.

Mark the level of achievement that best describes your learning at the completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 94%, unachieved: 6%)

Very poor.	[0]
Below average.	*** [3]
Average.	*************** [15]
Above average.	*********************** [23]
Excellent.	********* [9]

2. Explain sun cycles and seasons.
(Achieved: 98%, unachieved: 2%)

Very poor.	[0]
Below average.	* [1]
Average.	***************** [17]
Above average.	************************ [24]
Excellent.	******** [8]

3. Explain and predict lunar phases and times.
(Achieved: 90%, unachieved: 10%)

Very poor.	[0]
Below average.	***** [5]
Average.	************ [17]
Above average.	*************** [15]
Excellent.	************* [13]

4. Relate planets in the sky to a solar system map.
(Achieved: 90%, unachieved: 10%)

Very poor.	* [1]
Below average.	**** [4]
Average.	*************************** [27]
Above average.	***************** [17]
Excellent.	* [1]

5. Explain differences between models of planetary motion.
(Achieved: 90%, unachieved: 10%)

Very poor.	* [1]
Below average.	**** [4]
Average.	************************* [25]
Above average.	**************** [16]
Excellent.	**** [4]

6. Explain evidence for the heliocentric model of planetary motion.
(Achieved: 80%, unachieved: 20%)

Very poor.	[0]
Below average.	********** [10]
Average.	***************** [17]
Above average.	***************** [17]
Excellent.	****** [6]

7. Describe how optical telescopes work.
(Achieved: 90%, unachieved: 10%)

Very poor.	* [1]
Below average.	**** [4]
Average.	********************* [21]
Above average.	******************* [19]
Excellent.	***** [5]

8. Describe different powers of optical telescopes.
(Achieved: 88%, unachieved: 12%)

Very poor.	* [1]
Below average.	***** [5]
Average.	******************** [20]
Above average.	****************** [18]
Excellent.	******** [8]

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 82%, unachieved: 18%)

Very poor.	** [2]
Below average.	******* [7]
Average.	**************** [16]
Above average.	****************** [18]
Excellent.	******* [7]

10. Explain how stars produce energy.
(Achieved: 88%, unachieved: 12%)

Very poor.	[0]
Below average.	****** [6]
Average.	****************** [18]
Above average.	**************** [16]
Excellent.	********** [10]

11. Explain the relationship between star brightness and distances.
(Achieved: 96%, unachieved: 4%)

Very poor.	[0]
Below average.	** [2]
Average.	************** [14]
Above average.	********************** [22]
Excellent.	******* [12]

12. Predict the size of a star based on brightness and temperature.
(Achieved: 96%, unachieved: 4%)

Very poor.	[0]
Below average.	** [2]
Average.	**************** [16]
Above average.	******************** [20]
Excellent.	************ [12]

13. Explain different stages a star will go through, based on its mass.
(Achieved: 88%, unachieved: 12%)

Very poor.	[0]
Below average.	****** [6]
Average.	******************* [19]
Above average.	*************** [15]
Excellent.	********** [10]

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 82%, unachieved: 18%)

Very poor.	[0]
Below average.	********* [9]
Average.	************************** [26]
Above average.	*************** [15]
Excellent.	[0]

15. Explain evidence for how our Milky Way galaxy came to be.
(Achieved: 78%, unachieved: 22%)

Very poor.	[0]
Below average.	*********** [11]
Average.	************************* [25]
Above average.	************ [12]
Excellent.	** [2]

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 90%, unachieved: 10%)

Very poor.	[0]
Below average.	***** [5]
Average.	************************* [25]
Above average.	************** [14]
Excellent.	****** [6]

17. Explain evidence for the expansion of the universe.
(Achieved: 84%, unachieved: 16%)

Very poor.	* [1]
Below average.	******* [7]
Average.	********************* [21]
Above average.	***************** [17]
Excellent.	**** [4]

18. Describe characteristics of the universe a long time ago.
(Achieved: 80%, unachieved: 20%)

Very poor.	* [1]
Below average.	********* [9]
Average.	*********************** [23]
Above average.	************** [14]
Excellent.	*** [3]

19. Explain evidence for how our solar system came to be.
(Achieved: 94%, unachieved: 6%)

Very poor.	* [1]
Below average.	*** [3]
Average.	****** [6]
Above average.	******* [7]
Excellent.	* [1]

20. Describe key features of terrestrial planets.
(Achieved: 94%, unachieved: 6%)

Very poor.	[0]
Below average.	* [1]
Average.	***** [5]
Above average.	********** [10]
Excellent.	** [2]

21. Describe key features of jovian planets.
(Achieved: 100%, unachieved: 0%)

Very poor.	* [1]
Below average.	[0]
Average.	****** [6]
Above average.	********* [9]
Excellent.	** [2]

22. Explain why Pluto is not currently categorized as a planet.
(Achieved: 94%, unachieved: 6%)

Very poor.	[0]
Below average.	[0]
Average.	**** [4]
Above average.	**** [4]
Excellent.	********** [10]

23. Describe plausible requirements for life.
(Achieved: 88%, unachieved: 12%)

Very poor.	[0]
Below average.	****** [6]
Average.	****************** [18]
Above average.	******************* [19]
Excellent.	******* [7]

24. Explain difficulties in investigating the possibility for extraterrestial life.
(Achieved: 85%, unachieved: 14%)

Very poor.	[0]
Below average.	******* [7]
Average.	****************** [18]
Above average.	**************** [16]
Excellent.	********* [9]

Of the 24 student learning outcomes in the SASS, 18 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
21. Describe key features of jovian planets. (100%)
2. Explain sun cycles and seasons. (98%)
11. Explain the relationship between star brightness and distances. (96%)
12. Predict the size of a star based on brightness and temperature. (96%)
1. Predict positions and cycles of stars, using a starwheel. (94%)
19. Explain evidence for how our solar system came to be. (94%)
20. Describe key features of terrestrial planets. (94%)
22. Explain why Pluto is not currently categorized as a planet. (94%)
3. Explain and predict lunar phases and times. (90%)
4. Relate planets in the sky to a solar system map. (90%)
5. Explain differences between models of planetary motion. (90%)
7. Describe how optical telescopes work. (90%)
16. Explain how the speed of light affects observations of distant objects. (90%)
8. Describe different powers of optical telescopes. (88%)
10. Explain how stars produce energy. (88%)
13. Explain different stages a star will go through, based on its mass. (88%)
23. Describe plausible requirements for life. (88%)
24. Explain difficulties in investigating the possibility for extraterrestial life. (85%)

However, six student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
17. Explain evidence for the expansion of the universe. (84%)
9. Explain which telescopes should be funded based on relevant criteria. (82%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (82%)
6. Explain evidence for the heliocentric model of planetary motion. (80%)
18. Describe characteristics of the universe a long time ago. (80%)
15. Explain evidence for how our Milky Way galaxy came to be. (78%)

Compare these student learning outcomes self-reported as not being achieved (6, 9, 14, 15, 17, 18) those from a previous semesters (spring semester 2012: (6, 18); fall semester 2011: (4, 7, 8)).

Student learning outcomes 10, 11, 12, and 13 for Cuesta College students were directly assessed using the Star Properties Concept Inventory (excluding negative informed consent form responses):

Star Properties Concept Inventory v3.0
Astronomy 210 fall semester 2013 sections 70158, 70160
N = 56
ave ± stdev = 52% ± 13%

These SPCI scores are comparable to results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007), where the average was 51% (no further statistics provided); and also comparable to SPCI results from earlier semesters at Cuesta College.

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this indirect assessment SASS tool, along with the direct assessment SPCI tool will be used for course/program improvement by increasing emphasis on these lowest three learning outcomes in instruction in future semesters.

Previous posts:

• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2012).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2010).

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College)

White paper distributed at special session C3B, "Writing and Assessing Student Learning Objectives: Tips, Techniques, and What Our Community Needs," facilitated by Andrew Fraknoi (Foothill College), at the Astronomy Society of the Pacific Cosmos in the Classroom: A Hands-on Symposium on Teaching Introductory Astronomy and Related Science, July 23, 2013, 10:00-11:00 AM, Student Union Costanoan Room, San José State University, San Jose, CA.

The Student Assessment of Skills Survey (SASS, Patrick M. Len, in development) and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) are used to measure achievement of student learning outcomes, such that these results guide "continuous quality improvement" to instruction at Cuesta College, as mandated by the Accrediting Commission for Community and Junior Colleges (ACCJC). The SASS is a broad indirect assessment survey asking students to self-report their achievement of learning outcomes; in contrast the SPCI is a direct assessment instrument specifically evaluating student understanding of fundamental properties of stars. Programs at Cuesta College utilize student self-reports to assess learning outcomes, while also using direct assessment on selected student learning outcomes in order to comply with the ACCJC Rubric for Evaluating Institutional Effectiveness.

The SASS is assigned as homework during the last week of instruction, to be completed online before the final exam. For each of the 24 learning outcomes listed below, students are asked to mark the level of achievement that best describes their learning at the completion of the course, using a five-point Likert scale (1 = very poor; 2 = below average; 3 = average; 4 = above average; 5 = excellent).

1. Predict positions and cycles of stars, using a starwheel.
2. Explain sun cycles and seasons.
3. Explain and predict lunar phases and times.
4. Relate planets in the sky to a solar system map.
5. Explain differences between models of planetary motion.
6. Explain evidence for the heliocentric model of planetary motion.
7. Describe how optical telescopes work.
8. Describe different powers of optical telescopes.
9. Explain which telescopes should be funded based on relevant criteria.
10. Explain how stars produce energy.
11. Explain the relationship between star brightness and distances.
12. Predict the size of a star based on brightness and temperature.
13. Explain different stages a star will go through, based on its mass.
14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
15. Explain evidence for how our Milky Way galaxy came to be.
16. Explain how the speed of light affects observations of distant objects.
17. Explain evidence for the expansion of the universe.
18. Describe characteristics of the universe a long time ago.
19. Explain evidence for how our solar system came to be.
20. Describe key features of terrestrial planets.
21. Describe key features of jovian planets.
22. Explain why Pluto is not currently categorized as a planet.
23. Describe plausible requirements for life.
24. Explain difficulties in investigating the possibility for extraterrestial life.

For each of these learning outcomes, the percentage of achieved students is calculated from the consolidation of "average," "above average," and "excellent" responses, and the percentage of unachieved students from the consolidation of "very poor" and "below average" responses. The learning outcomes with the lowest percentage of achieved students are then noted for guiding "continuous quality improvement" to instruction in subsequent semesters.

Learning outcomes 10, 11, 12, and 13 are also directly assessed using the SPCI administered in class during the last week of instruction (excluding negative informed consent form responses) strictly as a post-instruction test. The average SPCI class score is then compared to results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007). These results are also used to guide "continuous quality improvement" to instruction in subsequent semesters.

This document, and further discussion of SASS and SPCI results at Cuesta College are posted at:

http://tinyurl.com/astronomySASS.

Education research: SASS, SPCI and student learning outcomes assessment (Cuesta College, spring semester 2012)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development), and the Star Properties Concept Inventory (SPCI, Janelle M. Bailey, "Development of a Concept Inventory to Assess Students' Understanding and Reasoning Difficulties about the Properties and Formation of Stars," Astronomy Education Review, Vol. 6, No. 2, pp. 133–139, August 2007) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam. The SPCI is administered in class during the last week of instruction.

The SASS results from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").

Cuesta College
Student Assessment of Skills Survey (SASS) 
Astronomy 210 spring semester 2012 sections 30674, 30676
N = 52

The questions below are designed to characterize your achievement of each 
of the learning outcomes by filling in a bubble on the rating scale 
provided to the right of each statement. 

Mark the level of achievement that best describes your learning at the 
completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 90%, unachieved: 10%)
1. Very poor          1 : *
2. Below average      4 : ****
3. Average           20 : ********************  [3.6 +/- 0.9]
4. Above average     18 : ******************
5. Excellent          9 : *********

2. Explain sun cycles and seasons.
(Achieved: 96%, unachieved: 4%)  
1. Very poor          0 : 
2. Below average      2 : **
3. Average           21 : *********************
4. Above average     21 : *********************  [3.7 +/- 0.8]
5. Excellent          8 : ********

3. Explain and predict lunar phases and times. 
(Achieved: 86%, unachieved: 14%)  
1. Very poor          0 : 
2. Below average      7 : *******
3. Average           17 : *****************
4. Above average     17 : *****************  [3.6 +/- 1.0]
5. Excellent         10 : **********

4. Relate planets in the sky to a solar system map. 
(Achieved: 92%, unachieved: 8%)  
1. Very poor          0 : 
2. Below average      4 : ****
3. Average           23 : ***********************  [3.6 +/- 0.9]
4. Above average     16 : ****************
5. Excellent          9 : *********

5. Explain differences between models of planetary motion. 
(Achieved: 87%, unachieved: 13%)  
1. Very poor          0 : 
2. Below average      7 : *******
3. Average           21 : *********************  [3.5 +/- 0.9]
4. Above average     16 : ****************
5. Excellent          8 : ********

6. Explain evidence for the heliocentric model of planetary motion. 
(Achieved: 80%, unachieved: 20%)  
1. Very poor          1 : *
2. Below average      9 : *********
3. Average           22 : ***********************  [3.3 +/- 1.0]
4. Above average     12 : ************
5. Excellent          7 : *******

7. Describe how optical telescopes work. 
(Achieved: 90%, unachieved: 10%)  
1. Very poor          0 : 
2. Below average      5 : *****
3. Average           20 : ********************  [3.6 +/- 0.8]
4. Above average     20 : ********************
5. Excellent          7 : *******

8. Describe different powers of optical telescopes. 
(Achieved: 88%, unachieved: 12%)  
1. Very poor          0 : 
2. Below average      6 : ******
3. Average           17 : *****************
4. Above average     20 : ********************  [3.6 +/- 0.9]
5. Excellent          9 : *********

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 90%, unachieved: 10%)  
1. Very poor          0 : 
2. Below average      5 : *****
3. Average           13 : *************
4. Above average     20 : ********************  [3.8 +/- 0.9]
5. Excellent         13 : *************

10. Explain how stars produce energy. 
(Achieved: 92%, umachieved: 8%)  
1. Very poor          0 : 
2. Below average      4 : ****
3. Average           16 : ****************
4. Above average     24 : ************************  [3.7 +/- 0.8]
5. Excellent          8 : ********

11. Explain the relationship between star brightness and distances. 
(Achieved: 96%, unachieved: 4%)  
1. Very poor          0 : 
2. Below average      2 : **
3. Average           11 : ***********
4. Above average     20 : ********************  [4.1 +/- 0.9]
5. Excellent         19 : *******************

12. Predict the size of a star based on brightness and temperature. 
(Achieved: 96%, unachieved: 4%)  
1. Very poor          0 : 
2. Below average      2 : **
3. Average           13 : *************
4. Above average     21 : *********************  [4.0 +/- 0.8]
5. Excellent         16 : ****************

13. Explain different stages a star will go through, based on its mass. 
(Achieved: 88%, unachieved: 12%)  
1. Very poor          0 : 
2. Below average      6 : ******
3. Average           13 : *************
4. Above average     21 : *********************  [3.8 +/- 0.9]
5. Excellent         12 : ************

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 87%, unachieved: 13%)  
1. Very poor          1 : *
2. Below average      6 : ******
3. Average           16 : ****************
4. Above average     22 : **********************  [3.5 +/- 0.9]
5. Excellent          7 : *******

15. Explain evidence for how our Milky Way galaxy came to be. 
(Achieved: 87%, unachieved: 13%)  
1. Very poor          1 : *
2. Below average      6 : ******
3. Average           19 : *******************  [3.5 +/- 1.0]
4. Above average     17 : *****************
5. Excellent          9 : *********

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 87%, unachieved: 13%)  
1. Very poor          0 : 
2. Below average      6 : ******
3. Average           13 : *************
4. Above average     24 : ************************  [3.7 +/- 0.9]
5. Excellent          9 : *********

17. Explain evidence for the expansion of the universe. 
(Achieved: 88%, unachieved: 12%)  
1. Very poor          0 : 
2. Below average      7 : *******
3. Average           19 : *******************  [3.6 +/- 1.0]
4. Above average     15 : ***************
5. Excellent         11 : ***********

18. Describe characteristics of the universe a long time ago. 
(Achieved: 83%, unachieved: 17%)  
1. Very poor          2 : **
2. Below average      7 : *******
3. Average           22 : **********************  [3.4 +/- 1.1]
4. Above average     12 : ************
5. Excellent          9 : *********

19. Explain evidence for how our solar system came to be. 
(Achieved: 87%, unachieved: 13%)  
1. Very poor          0 : 
2. Below average      7 : *******
3. Average           18 : ******************
4. Above average     22 : **********************  [3.5 +/- 0.8]
5. Excellent          5 : *****

20. Describe key features of terrestrial planets. 
(Achieved: 94%, unachieved: 6%)  
1. Very poor          1 : *
2. Below average      2 : **
3. Average           15 : **************
4. Above average     20 : ********************  [3.8 +/- 0.9]
5. Excellent         14 : **************

21. Describe key features of jovian planets. 
(Achieved: 90%, unachieved: 10%)  
1. Very poor          2 : **
2. Below average      3 : ***
3. Average           21 : *********************  [3.6 +/- 1.0]
4. Above average     15 : ***************
5. Excellent         11 : ***********

22. Explain why Pluto is not currently categorized as a planet. 
(Achieved: 86%, unachieved: 14%)  
1. Very poor          3 : ***
2. Below average      4 : ****
3. Average            8 : ********
4. Above average     12 : ************
5. Excellent         24 : ************************  [4.0 +/- 1.3]

23. Describe plausible requirements for life. 
(Achieved: 98%, unachieved: 2%)  
1. Very poor          1 : *
2. Below average      0 : 
3. Average           18 : ******************
4. Above average     18 : ******************  [3.9 +/- 0.9]
5. Excellent         15 : ***************

24. Explain difficulties in investigating the possibility for extraterrestial life.
(Achieved: 96%, unachieved: 4%)  
1. Very poor          1 : *
2. Below average      1 : *
3. Average           12 : ************
4. Above average     21 : *********************  [4.0 +/- 0.9]
5. Excellent         17 : *****************

Of the 24 student learning outcomes in the SASS, 22 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
23. Describe plausible requirements for life. (98%)
2. Explain sun cycles and seasons. (96%)
11. Explain the relationship between star brightness and distances. (96%)
12. Predict the size of a star based on brightness and temperature. (96%)
24. Explain difficulties in investigating the possibility for extraterrestial life. (96%)
20. Describe key features of terrestrial planets. (94%)
4. Relate planets in the sky to a solar system map. (92%)
10. Explain how stars produce energy. (92%)
1. Predict positions and cycles of stars, using a starwheel. (90%)
7. Describe how optical telescopes work. (90%)
9. Explain which telescopes should be funded based on relevant criteria. (90%)
21. Describe key features of jovian planets. (90%)
8. Describe different powers of optical telescopes. (88%)
17. Explain evidence for the expansion of the universe. (88%)
13. Explain different stages a star will go through, based on its mass. (88%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (87%)
15. Explain evidence for how our Milky Way galaxy came to be. (87%)
16. Explain how the speed of light affects observations of distant objects. (87%)
19. Explain evidence for how our solar system came to be. (87%)
5. Explain differences between models of planetary motion. (87%)
3. Explain and predict lunar phases and times. (86%)
22. Explain why Pluto is not currently categorized as a planet. (86%)

However, two student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
18. Describe characteristics of the universe a long time ago. (83%)
6. Explain evidence for the heliocentric model of planetary motion. (80%)

Of note is that these student learning outcomes self-reported as not being achieved (6, 18) are not the same as in the previous fall semester 2011 (4, 7, 8).

Student learning outcomes 10, 11, 12, and 13 for Cuesta College students were directly assessed using the Star Properties Concept Inventory (excluding negative informed consent form responses):

Star Properties Concept Inventory v3.0
Astronomy 210 spring semester 2012 sections 30674, 30676
N = 49
ave ± stdev = 46% ± 10%

These SPCI scores are comparable to results from 1,100 large research university students that have completed introductory astronomy and earth sciences courses (Bailey, 2007), where the average was 51% (no further statistics provided); and also comparable to SPCI results from earlier semesters at Cuesta College.

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this indirect assessment SASS tool, along with the direct assessment SPCI tool will be used for course/program improvement by increasing emphasis on these lowest three learning outcomes in instruction in future semesters.

Previous posts:

• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, spring semester 2011).
  
• Education research: SASS and student learning outcomes (Cuesta College, fall semester 2010).