20110724

Education research: SASS and student learning outcomes (spring semester 2011)

Student achievement of course learning outcomes are assessed by administering an Student Assessment of Skills Survey (SASS), a five-point Likert scale questionnaire (Patrick M. Len, in development) to Astronomy 210 students at Cuesta College, San Luis Obispo, CA. This is a one-semester, introductory astronomy course (with an optional adjunct laboratory), and is taken primarily by students to satisfy their general education science transfer requirement.

The SASS is administered online during the last week of instruction, to be completed before the final exam.

The results for SASS from this semester are compiled below. Values for the mean and standard deviations are given next to the modal response category for each question. Also listed is the percentage of students who have self-assessed themselves as having successfully achieving a learning outcome (responding "average," "above average," or "excellent") as opposed to not achieving success with a learning outcome (responding "very poor" or "below average").

Cuesta College
Student Assessment of Skills Survey (SASS) 
Astronomy 210 spring semester 2011 sections 30674, 30676

The questions below are designed to characterize your achievement of each 
of the learning outcomes by filling in a bubble on the rating scale 
provided to the right of each statement. 

Mark the level of achievement that best describes your learning at the
completion of the course.

1. Predict positions and cycles of stars, using a starwheel.
(Achieved: 90%, unachieved: 10%) 
1. Very poor          0 : 
2. Below average      7 : *******
3. Average           30 : ******************************  [3.6 +/- 0.9]
4. Above average     19 : *******************
5. Excellent         16 : ****************

2. Explain sun cycles and seasons.
(Achieved: 92%, unachieved: 8%)  
1. Very poor          0 : 
2. Below average      6 : ******
3. Average           29 : *****************************  [3.6 +/- 0.9]
4. Above average     23 : ***********************
5. Excellent         14 : **************

3. Explain and predict lunar phases and times. 
(Achieved: 94%, unachieved: 6%)  
1. Very poor          2 : **
2. Below average      2 : **
3. Average           29 : *****************************  [3.6 +/- 0.9]
4. Above average     28 : ****************************
5. Excellent         10 : **********

4. Relate planets in the sky to a solar system map. 
(Achieved: 90%, unachieved: 10%)  
1. Very poor          1 : *
2. Below average      6 : ******
3. Average           35 : ***********************************  [3.4 +/- 0.8]
4. Above average     24 : ************************
5. Excellent          6 : ******

5. Explain differences between models of planetary motion. 
(Achieved: 89%, unachieved: 11%)  
1. Very poor          2 : **
2. Below average      6 : ******
3. Average           29 : *****************************  [3.4 +/- 0.9]
4. Above average     28 : ****************************
5. Excellent          7 : *******

6. Explain evidence for the heliocentric model of planetary motion. 
(Achieved: 83%, unachieved: 17%)  
1. Very poor          1 : *
2. Below average     11 : ***********
3. Average           34 : **********************************  [3.3 +/- 0.9]
4. Above average     16 : ****************
5. Excellent         10 : **********

7. Describe how optical telescopes work. 
(Achieved: 81%, unachieved: 19%)  
1. Very poor          3 : ***
2. Below average     11 : ***********
3. Average           31 : *******************************  [3.3 +/- 1.1]
4. Above average     15 : ***************
5. Excellent         12 : ************

8. Describe different powers of optical telescopes. 
(Achieved: 81%, unachieved: 19%)  
1. Very poor          2 : **
2. Below average     12 : ************
3. Average           26 : **************************  [3.4 +/- 1.0]
4. Above average     21 : *********************
5. Excellent         11 : ***********

9. Explain which telescopes should be funded based on relevant criteria.
(Achieved: 88%, unachieved: 13%)  
1. Very poor          1 : *
2. Below average      8 : ********
3. Average           23 : ***********************  [3.7 +/- 1.0]
4. Above average     21 : *********************
5. Excellent         19 : *******************

10. Explain how stars produce energy. 
(Achieved: 83%, umachieved: 17%)  
1. Very poor          1 : *
2. Below average     11 : ***********
3. Average           27 : ***************************  [3.4 +/- 1.0]
4. Above average     22 : **********************
5. Excellent         11 : ***********

11. Explain the relationship between star brightness and distances. 
(Achieved: 92%, unachieved: 8%)  
1. Very poor          1 : *
2. Below average      5 : *****
3. Average           19 : *******************
4. Above average     23 : ***********************  [3.9 +/- 1.0]
5. Excellent         24 : ************************

12. Predict the size of a star based on brightness and temperature. 
(Achieved: 97%, unachieved: 3%)  
1. Very poor          0 : 
2. Below average      2 : **
3. Average           22 : **********************
4. Above average     25 : *************************  [3.9 +/- 0.9]
5. Excellent         22 : **********************

13. Explain different stages a star will go through, based on its mass. 
(Achieved: 83%, unachieved: 17%)  
1. Very poor          3 : ***
2. Below average      9 : *********
3. Average           24 : ************************
4. Above average     23 : ***********************  [3.5 +/- 1.1]
5. Excellent         13 : *************

14. Explain evidence for the shape/size/composition of our Milky Way galaxy.
(Achieved: 85%, unachieved: 15%)  
1. Very poor          1 : *
2. Below average     10 : **********
3. Average           28 : ****************************  [3.5 +/- 1.0]
4. Above average     20 : ********************
5. Excellent         13 : *************

15. Explain evidence for how our Milky Way galaxy came to be. 
(Achieved: 80%, unachieved: 20%)  
1. Very poor          0 : 
2. Below average     14 : **************
3. Average           25 : *************************  [3.3 +/- 0.9]
4. Above average     24 : ************************
5. Excellent          6 : ******

16. Explain how the speed of light affects observations of distant objects.
(Achieved: 88%, unachieved: 13%)  
1. Very poor          0 : 
2. Below average      9 : *********
3. Average           23 : ***********************
4. Above average     21 : *********************  [3.7 +/- 1.0]
5. Excellent         19 : *******************

17. Explain evidence for the expansion of the universe. 
(Achieved: 85%, unachieved: 15%)  
1. Very poor          2 : **
2. Below average      9 : *********
3. Average           28 : ****************************  [3.4 +/- 1.0]
4. Above average     22 : **********************
5. Excellent         11 : ***********

18. Describe characteristics of the universe a long time ago. 
(Achieved: 83%, unachieved: 17%)  
1. Very poor          2 : **
2. Below average     10 : **********
3. Average           31 : *******************************  [3.3 +/- 0.9]
4. Above average     23 : ***********************
5. Excellent          6 : ******

19. Explain evidence for how our solar system came to be. 
(Achieved: 88%, unachieved: 13%)  
1. Very poor          1 : *
2. Below average      8 : ********
3. Average           38 : **************************************  [3.3 +/- 0.8]
4. Above average     18 : ******************
5. Excellent          7 : *******

20. Describe key features of terrestrial planets. 
(Achieved: 92%, unachieved: 8%)  
1. Very poor          1 : *
2. Below average      5 : *****
3. Average           23 : *************************
4. Above average     26 : **************************  [3.7 +/- 0.9]
5. Excellent         17 : *****************

21. Describe key features of jovian planets. 
(Achieved: 93%, unachieved: 7%)  
1. Very poor          1 : *
2. Below average      4 : ****
3. Average           25 : *************************
4. Above average     27 : ***************************  [3.7 +/- 0.9]
5. Excellent         15 : ***************

22. Explain why Pluto is not currently categorized as a planet. 
(Achieved: 94%, unachieved: 6%)  
1. Very poor          0 : 
2. Below average      4 : ****
3. Average           22 : **********************
4. Above average     24 : ************************  [3.9 +/- 0.9]
5. Excellent         22 : **********************

23. Describe plausible requirements for life. 
(Achieved: 93%, unachieved: 7%)  
1. Very poor          1 : *
2. Below average      4 : ****
3. Average           22 : **********************
4. Above average     28 : ****************************  [3.8 +/- 0.9]
5. Excellent         17 : *****************

24. Explain difficulties in investigating the possibility for extraterrestial life.
(Achieved: 93%, unachieved: 7%)  
1. Very poor          1 : *
2. Below average      4 : ****
3. Average           21 : *********************
4. Above average     28 : ****************************  [3.8 +/- 0.9]
5. Excellent         18 : ******************


Of the 24 student learning outcomes in the SASS, 17 were self-reported as being achieved by at least 85% of students, listed below in order of decreasing success:
12. Predict the size of a star based on brightness and temperature. (97%)
3. Explain and predict lunar phases and times. (94%)
22. Explain why Pluto is not currently categorized as a planet. (94%)
21. Describe key features of jovian planets. (93%)
23. Describe plausible requirements for life. (93%)
24. Explain difficulties in investigating the possibility for extraterrestial life. (93%)
2. Explain sun cycles and seasons. (92%)
11. Explain the relationship between star brightness and distances. (92%)
20. Describe key features of terrestrial planets. (92%)
1. Predict positions and cycles of stars, using a starwheel. (90%)
4. Relate planets in the sky to a solar system map. (90%)
5. Explain differences between models of planetary motion. (89%)
9. Explain which telescopes should be funded based on relevant criteria. (88%)
16. Explain how the speed of light affects observations of distant objects. (88%)
19. Explain evidence for how our solar system came to be. (88%)
14. Explain evidence for the shape/size/composition of our Milky Way galaxy. (85%)
17. Explain evidence for the expansion of the universe. (85%)

However, 7 student learning outcomes were self-reported as being achieved by less than 85% of students, listed below in order of decreasing success:
6. Explain evidence for the heliocentric model of planetary motion. (83%)
10. Explain how stars produce energy. (83%)
13. Explain different stages a star will go through, based on its mass. (83%)
18. Describe characteristics of the universe a long time ago. (83%)
7. Describe how optical telescopes work. (81%)
8. Describe different powers of optical telescopes. (81%)
15. Explain evidence for how our Milky Way galaxy came to be. (80%)

As per the ACCJC (Accrediting Commission for Community and Junior Colleges), results from this assessment tool will be used for course/program improvement by increasing emphasis on these lowest seven student learning outcomes in instruction in future semesters.

Previous post:
at

2 comments:

  1. Paul Martenis9:47 AM

    The results for each outcome look very similar. By asking in terms of "average" and below or above average, that seems to be inevitable.
    Suppose the outcomes were phrased as "I can ..." statements, and the students could say whether they could do this at the expected level, or could not, or could go beyond the expectations. Would the results look different? Would it make it easier to discriminate between areas of strength and weakness?

    ReplyDelete
  2. Patrick M. Len10:05 AM

    Inserting "I can..." to start out each statements sounds like a great idea to try on the next round. This SASS originated from a mandate to explicitly assess student understanding for accreditation purposes, but is definitely a work in progress, so thank you for your input. We'll see how results differ at the end of fall semester 2011.

    ReplyDelete