Education research: ALLS pre- to post-instruction attitude shifts (Cuesta College, spring semester 2012)

Student attitudes are assessed using an Astronomy Laboratory Learning Survey (ALLS), a five-point Likert scale questionnaire with demographic questions, and entry/exit evaluation questions (Patrick M. Len, in development) to Astronomy 210L students at Cuesta College, San Luis Obispo, CA. This laboratory course is a one-semester, adjunct course to Astronomy 210 lecture, and is taken primarily by students to satisfy their general education science laboratory transfer requirement.

The ALLS is administered as a pre-test on the first laboratory meeting, before any introduction/instruction took place; and as a post-test on the last laboratory meeting.

The results from the pre- and post-test questions follow below. Values for the mean and standard deviations are given next to the modal response category for each question, along with a Student t-test for the probability of null hypothesis rejection, and the class-wise Hake gain. (Matched-pair Hake gains were not calculated for each student, as pre-instruction values such as "5" would result in undefined values.) For statistical purposes, blank entries were treated as "3. Neutral," and multiply-circled entries such as "12," "23," "34," and "45" were treated as "1," "2," "4," and "5" respectively.

Cuesta College
Astronomy Laboratory Learning Survey (ALLS)
Pre- and Post-instruction results
Astronomy 210L spring semester 2012 sections 30678, 30679, 30680, 30682
(N = 57, matched-pairs only)

1. I am interested in using a telescope or binoculars for astronomy.
Pre-instruction
1. Strongly disagree      0 : 
2. Disagree               0 : 
3. Neutral               10 : **********
4. Agree                 26 : **************************  [4.2 +/- 0.7]
5. Strongly agree        21 : *********************

Post-instruction
1. Strongly disagree      0 :
2. Disagree               0 : 
3. Neutral               13 : *************
4. Agree                 33 : *********************************  [4.0 +/- 0.7]
5. Strongly agree        11 : ***********

Student t-test p = 0.079 (t = 1.77, sd = 0.687, dof = 112)
Class-wise <g> = -0.28

2. Astronomy has little relation to what I experience in the real world.
Pre-instruction
1. Strongly disagree      4 : ****
2. Disagree              26 : **************************  [2.6 +/- 1.0]
3. Neutral               18 : ******************
4. Agree                  6 : ******
5. Strongly agree         3 : ***

Post-instruction
1. Strongly disagree      7 : *******
2. Disagree              25 : *************************  [2.6 +/- 1.1]
3. Neutral               13 : *************
4. Agree                  9 : *********
5. Strongly agree         3 : ***

Student t-test p = 0.85 (t = 0.185, sd = 1.02, dof = 112)
Class-wise <g> = -0.01

3. I know where and how to look up astronomy information.
Pre-instruction
1. Strongly disagree      9 : *********
2. Disagree              21 : *********************  [2.7 +/- 1.2]
3. Neutral               11 : ***********
4. Agree                 10 : **********
5. Strongly agree         6 : ******

Post-instruction
1. Strongly disagree      0 :
2. Disagree               0 : 
3. Neutral                4 : ****
4. Agree                 27 : ***************************  [4.4 +/- 0.6]
5. Strongly agree        26 : **************************

Student t-test p < 0.0001 (t = -9.18, sd = 0.979, dof = 112) 
Class-wise <g> = +0.73 

4. I know where and how to find objects in the night sky. 
Pre-instruction 
1. Strongly disagree      9 : *********
2. Disagree              15 : *************** 
3. Neutral               18 : ******************  [2.7 +/- 1.1] 
4. Agree                 14 : **************
5. Strongly agree         1 : * 

Post-instruction 
1. Strongly disagree      0 :  
2. Disagree               1 : * 
3. Neutral               17 : ***************** 
4. Agree                 29 : *****************************  [3.8 +/- 0.7] 
5. Strongly agree        10 : **********  

Student t-test p < 0.0001 (t = -6.66, sd = 0.914, dof = 112) 
Class-wise <g> = +0.50 

5. I am interested in news that is related to astronomy. 
Pre-instruction 
1. Strongly disagree      0 :  
2. Disagree               7 : ******* 
3. Neutral               17 : ***************** 
4. Agree                 25 : *************************  [3.6 +/- 0.9] 
5. Strongly agree         8 : ********  

Post-instruction 
1. Strongly disagree      2 : **
2. Disagree               3 : ***
3. Neutral               18 : ****************** 
4. Agree                 24 : ************************  [3.6 +/- 1.0] 
5. Strongly agree        10 : **********  

Student t-test p = 0.76 (t = 0.306, sd = 0.920, dof = 112) 
Class-wise <g> = +0.04 

6. I often ask myself questions related to astronomy. 
Pre-instruction 
1. Strongly disagree      3 : ***
2. Disagree              14 : **************
3. Neutral               18 : ******************  [3.1 +/- 1.0] 
4. Agree                 19 : *******************
5. Strongly agree         3 : ***

Post-instruction 
1. Strongly disagree      3 : ***
2. Disagree               8 : ******** 
3. Neutral               14 : ************** 
4. Agree                 25 : *************************  [3.4 +/- 1.1]
5. Strongly agree         7 : *******  

Student t-test p = 0.071 (t = -1.82, sd = 1.03, dof = 112) 
Class-wise <g> = +0.18  

7. I understand what science research is like.
Pre-instruction 
1. Strongly disagree      2 : **
2. Disagree              14 : **************
3. Neutral               17 : ***************** 
4. Agree                 18 : ******************  [3.2 +/- 1.0] 
5. Strongly agree         6 : ******  

Post-instruction 
1. Strongly disagree      0 : 
2. Disagree               0 :  
3. Neutral                9 : ********* 
4. Agree                 33 : *********************************  [4.1 +/- 0.6] 
5. Strongly agree        15 : *************** 

Student t-test p < 0.0001 (t = -5.49, sd = 0.870, dof = 112) 
Class-wise <g> = +0.50  

8. I understand how science research questions are answered.
Pre-instruction 
1. Strongly disagree      4 : ****
2. Disagree              10 : ********** 
3. Neutral               20 : ********************  [3.2 +/- 1.1]
4. Agree                 16 : ****************
5. Strongly agree         7 : *******

Post-instruction 
1. Strongly disagree      0 : 
2. Disagree               0 :  
3. Neutral                7 : *******
4. Agree                 32 : ********************************  [4.2 +/- 0.6] 
5. Strongly agree        18 : ****************** 

Student t-test p < 0.0001 (t = -5.84, sd = 0.898, dof = 112) 
Class-wise <g> = +0.55

9. I prefer to work independently rather than in groups. 
Pre-instruction 
1. Strongly disagree      8 : ********
2. Disagree              11 : *********** 
3. Neutral               21 : *********************  [2.9 +/- 1.2] 
4. Agree                 12 : ************
5. Strongly agree         5 : *****

Post-instruction 
1. Strongly disagree      8 : ******** 
2. Disagree              20 : ********************  [2.6 +/- 1.1] 
3. Neutral               19 : *******************
4. Agree                  5 : *****
5. Strongly agree         5 : ***** 

Student t-test p = 0.19 (t = 1.32, sd = 1.13, dof = 112) 
Class-wise <g> = -0.13 

10. I can understand difficult concepts better if I am able to explain them to 
others. 
Pre-instruction 
1. Strongly disagree      2 : **
2. Disagree               9 : *********
3. Neutral               24 : ************************   [3.2 +/- 0.9] 
4. Agree                 19 : *******************
5. Strongly agree         3 : ***  

Post-instruction 
1. Strongly disagree      2 : **
2. Disagree               4 : ****
3. Neutral               22 : **********************  [3.5 +/- 1.0]
4. Agree                 19 : ******************* 
5. Strongly agree        10 : **********  

Student t-test p = 0.062 (t = -1.89, sd = 0.943, dof = 112) 
Class-wise <g> = +0.19 

11. I can understand difficult concepts better if I am able to ask lots of  
questions. 
Pre-instruction 
1. Strongly disagree      1 : *
2. Disagree               3 : ***
3. Neutral               14 : **************
4. Agree                 30 : ******************************  [3.8 +/- 0.9] 
5. Strongly agree         5 : *****  

Post-instruction 
1. Strongly disagree      0 :  
2. Disagree               9 : *********
3. Neutral                9 : *********
4. Agree                 26 : **************************  [3.8 +/- 1.0] 
5. Strongly agree        13 : *************

Student t-test p = 1.00 (t = 0.00, sd = 0.922, dof = 112) 
Class-wise <g> = 0  

12. Knowledge in astronomy consists of many pieces of information each of 
which applies primarily to a specific situation. 
Pre-instruction 
1. Strongly disagree     5 : *****
2. Disagree             19 : *******************
3. Neutral              27 : ***************************  [2.6 +/- 0.8] 
4. Agree                 5 : *****
5. Strongly agree        1 : *  

Post-instruction 
1. Strongly disagree     15 : ***************
2. Disagree              16 : ****************
3. Neutral               18 : ******************   [2.4 +/- 1.1] 
4. Agree                  6 : ******
5. Strongly agree         2 : **

Student t-test p = 0.18 (t = 1.34, sd = 0.976, dof = 112) 
Class-wise <g>> = -0.10

13. I am good at math.
Pre-instruction
1. Strongly disagree      7 : *******
2. Disagree              10 : **********
3. Neutral               19 : *******************  [3.0 +/- 1.1]
4. Agree                 16 : ****************
5. Strongly agree         5 : *****

Post-instruction
1. Strongly disagree      9 : *********
2. Disagree              10 : **********
3. Neutral               19 : *******************  [3.0 +/- 1.3]
4. Agree                 11 : ***********
5. Strongly agree         8 : ********

Student t-test p = 0.82 (t = 0.233, sd = 1.21, dof = 112)
Class-wise <g> = -0.03

14. I am good at science.
Pre-instruction
1. Strongly disagree      3 : ***
2. Disagree               9 : *********
3. Neutral               24 : ************************  [3.2 +/- 1.0]
4. Agree                 17 : *****************
5. Strongly agree         4 : ****

Post-instruction
1. Strongly disagree      4 : ****
2. Disagree               9 : *********
3. Neutral               20 : ********************  [3.2 +/- 1.0]
4. Agree                 19 : *******************
5. Strongly agree         5 : *****

Student t-test p = 0.85 (t = 0.186, sd = 1.01, dof = 112)
Class-wise <g> = 0.02

15. Astronomy lab will be/was difficult for me.
Pre-instruction
1. Strongly disagree      3 : ***
2. Disagree              24 : ************************  [2.6 +/- 0.8]
3. Neutral               22 : **********************
4. Agree                  7 : *******
5. Strongly agree         1 : *

Post-instruction
1. Strongly disagree     21 : *********************  [2.0 +/- 0.9]
2. Disagree              19 : *******************
3. Neutral               15 : ***************
4. Agree                  1 : *
5. Strongly agree         1 : *

Student t-test p = 0.0002 (t = 3.90, sd = 0.888, dof = 112)
Class-wise <g> = -0.27

This semester (spring semester 2012) was the third implementation of a backwards faded scaffolding curriculum (Slater, Slater, and Lyons, 2010), and second implementation of version 2.0 of the ALLS. Previous semesters (fall semester 2011 and earlier) used a conventional "cookbook" laboratory curriculum, and version 1.1 of the ALLS.

(The following results are similar for both this spring semester 2012, and the previous fall semester 2011.) A key difference between version 2.0 and version 1.1 of the ALLS is that the newer version has questions (7) and (8) specifically addressing gains towards achieving the student learning outcomes for the backwards faded scaffolding laboratory curriculum:

• Keeping abreast of present-day discoveries and developments in astronomy (current events).
• Developing scientific evidence-based research questions.
• Developing procedures to gather evidence in order to answer research questions.
• Making appropriate evidence-supported conclusions.
• Explaining research findings in a report, poster, or presentation.
• Evaluating evidence to determine whether or not it appropriately answers a research question.

Response to questions (7)-(8) show statistically significant, strong pre- to post-instruction gains, while all other responses show comparable results whether from the current BFS or conventional "cookbook" labs from a previous curriculum.

Previous posts:

• Education research: ALLS pre- to post-instruction attitude shifts (Cuesta College, fall semester 2011).
  
• Education research: Astronomy Laboratory Learning Survey (ALLS) version 2.0.
  
• Education research: ALLS pre- to post-instruction attitude shifts (Cuesta College, spring semester 2011).
  
• Education research: Astronomy Laboratory Learning Survey (ALLS), pre- and post-instruction forms (version 1.1).