20110729

Education research: ALLS pre- to post-instruction attitude shifts (Cuesta College, spring semester 2011)

Student attitudes are assessed using an Astronomy Laboratory Learning Survey (ALLS), a five-point Likert scale questionnaire with demographic questions, and entry/exit evaluation questions (Patrick M. Len, in development) to Astronomy 210L students at Cuesta College, San Luis Obispo, CA. This laboratory course is a one-semester, adjunct course to Astronomy 210 lecture, and is taken primarily by students to satisfy their general education science laboratory transfer requirement.

The ALLS is administered as a pre-test on the first laboratory meeting, before any introduction/instruction took place; and as a post-test on the last laboratory meeting.

The results from the pre- and post-test questions follow below. Values for the mean and standard deviations are given next to the modal response category for each question, along with a Student t-test for the probability of null hypothesis rejection, and the class-wise Hake gain. (Matched-pair Hake gains were not calculated for each student, as pre-instruction values such as "5" would result in undefined values.) For statistical purposes, blank entries were treated as "3. Neutral," and multiply-circled entries such as "12," "23," "34," and "45" were treated as "1," "2," "4," and "5" respectively.
Cuesta College
Astronomy Laboratory Learning Survey (ALLS)
Pre- and Post-instruction results
Astronomy 210L spring semester 2011 sections 30678, 30679, 30680, 30682
(N = 67, matched-pairs only)

1. I am interested in using a telescope or binoculars for astronomy.
Pre-instruction
1. Strongly disagree 4 : ****
2. Disagree 11 : ***********
3. Neutral 26 : ************************** [3.2 +/- 1.1]
4. Agree 15 : ***************
5. Strongly agree 9 : *********

Post-instruction
1. Strongly disagree 0 :
2. Disagree 1 : *
3. Neutral 17 : *****************
4. Agree 29 : ***************************** [4.0 +/- 0.8]
5. Strongly agree 18 : ******************

Student t-test p = 0.0001 (t = -4.65, sd = 0.944, dof = 128)
Class-wise <g> = +0.43

2. Astronomy has little relation to what I experience in the real world.
Pre-instruction
1. Strongly disagree 11 : ***********
2. Disagree 27 : *************************** [2.3 +/- 0.8]
3. Neutral 23 : ***********************
4. Agree 4 : ****
5. Strongly agree 0 :

Post-instruction
1. Strongly disagree 16 : ****************
2. Disagree 30 : ****************************** [2.2 +/- 0.9]
3. Neutral 11 : ***********
4. Agree 8 : ********
5. Strongly agree 0 :

Student t-test p = 0.38 (t = 0.889, sd = 0.888, dof = 128)
Class-wise <g> = -0.05

3. I know where and how to look up astronomy information.
Pre-instruction
1. Strongly disagree 8 : ********
2. Disagree 15 : ***************
3. Neutral 20 : ******************** [2.9 +/- 1.1]
4. Agree 17 : *****************
5. Strongly agree 5 : *****

Post-instruction
1. Strongly disagree 0 :
2. Disagree 1 : *
3. Neutral 6 : ******
4. Agree 31 : ******************************* [4.3 +/- 0.7]
5. Strongly agree 27 : ***************************

Student t-test p < 0.0001 (t = -8.14, sd = 0.949, dof = 128)
Class-wise <g> = +0.66

4. I know where and how to find objects in the night sky.
Pre-instruction
1. Strongly disagree 6 : ******
2. Disagree 19 : *******************
3. Neutral 26 : ************************** [2.8 +/- 1.0]
4. Agree 12 : ************
5. Strongly agree 2 : **

Post-instruction
1. Strongly disagree 0 :
2. Disagree 1 : *
3. Neutral 13 : *************
4. Agree 39 : *************************************** [4.0 +/- 0.7]
5. Strongly agree 12 : ************

Student t-test p < 0.0001 (t = -8.13, sd = 0.831, dof = 128)
Class-wise <g> = +0.53

5. I am interested in news that is related to astronomy.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 5 : *****
3. Neutral 17 : *****************
4. Agree 26 : ************************** [3.8 +/- 0.9]
5. Strongly agree 17 : *****************

Post-instruction
1. Strongly disagree 0 :
2. Disagree 5 : *****
3. Neutral 13 : *************
4. Agree 32 : ******************************** [3.9 +/- 0.9]
5. Strongly agree 15 : ***************

Student t-test p = 0.84 (t = 0.199, sd = 0.882, dof = 128)
Class-wise <g> = +0.03

6. I often ask myself questions related to astronomy.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 14 : **************
3. Neutral 22 : ********************** [3.4 +/- 1.0]
4. Agree 20 : ********************
5. Strongly agree 9 : *********

Post-instruction
1. Strongly disagree 1 : *
2. Disagree 11 : ***********
3. Neutral 19 : *******************
4. Agree 23 : *********************** [3.5 +/- 1.0]
5. Strongly agree 11 : ***********

Student t-test p = 0.48 (t = -0.703, sd = 0.998, dof = 128)
Class-wise <g> = +0.08

7. I am comfortable using a calculator to make complex calculations.
Pre-instruction
1. Strongly disagree 5 : *****
2. Disagree 13 : *************
3. Neutral 17 : *****************
4. Agree 22 : ********************** [3.2 +/- 1.1]
5. Strongly agree 8 : ********

Post-instruction
1. Strongly disagree 5 : *****
2. Disagree 11 : ***********
3. Neutral 14 : **************
4. Agree 24 : ************************ [3.4 +/- 1.2]
5. Strongly agree 11 : ***********

Student t-test p = 0.45 (t = -0.755, sd = 1.16, dof = 128)
Class-wise <g> = +0.09

8. I can make sense of equations and scientific notation numbers.
Pre-instruction
1. Strongly disagree 5 : *****
2. Disagree 11 : ***********
3. Neutral 14 : **************
4. Agree 27 : *************************** [3.3 +/- 1.1]
5. Strongly agree 8 : ********

Post-instruction
1. Strongly disagree 5 : *****
2. Disagree 6 : ******
3. Neutral 16 : ****************
4. Agree 27 : *************************** [3.5 +/- 1.1]
5. Strongly agree 11 : ***********

Student t-test p = 0.39 (t = -0.855, sd = 1.13, dof = 128)
Class-wise <g> = +0.10

9. I prefer to work independently rather than in groups.
Pre-instruction
1. Strongly disagree 7 : *******
2. Disagree 14 : **************
3. Neutral 29 : ***************************** [2.9 +/- 1.0]
4. Agree 11 : ***********
5. Strongly agree 4 : ****

Post-instruction
1. Strongly disagree 12 : ************
2. Disagree 20 : ********************
3. Neutral 23 : *********************** [2.5 +/- 1.1]
4. Agree 6 : ******
5. Strongly agree 4 : ****

Student t-test p = 0.085 (t = 1.74, sd = 1.06, dof = 128)
Class-wise <g> = -0.15

10. I can understand difficult concepts better if I am able to explain them to
others.
Pre-instruction
1. Strongly disagree 1 : *
2. Disagree 10 : **********
3. Neutral 22 : **********************
4. Agree 24 : ************************ [3.4 +/- 1.0]
5. Strongly agree 8 : ********

Post-instruction
1. Strongly disagree 1 : *
2. Disagree 2 : **
3. Neutral 15 : ***************
4. Agree 29 : ***************************** [3.9 +/- 0.9]
5. Strongly agree 18 : ******************

Student t-test p = 0.0020 (t = -3.16, sd = 0.917, dof = 128)
Class-wise <g> = +0.32

11. I can understand difficult concepts better if I am able to ask lots of
questions.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 5 : *****
3. Neutral 15 : ***************
4. Agree 35 : *********************************** [3.8 +/- 0.8]
5. Strongly agree 10 : **********

Post-instruction
1. Strongly disagree 0 :
2. Disagree 3 : ***
3. Neutral 14 : **************
4. Agree 29 : ***************************** [4.0 +/- 0.8]
5. Strongly agree 19 : *******************

Student t-test p = 0.14 (t = 0.822, sd = 0.822, dof = 128)
Class-wise <g> = +0.18

12. Knowledge in astronomy consists of many pieces of information each of
which applies primarily to a specific situation.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 5 : *****
3. Neutral 37 : ************************************* [3.3 +/- 0.7]
4. Agree 20 : ********************
5. Strongly agree 3 : ***

Post-instruction
1. Strongly disagree 0 :
2. Disagree 5 : *****
3. Neutral 24 : ************************
4. Agree 29 : ***************************** [3.6 +/- 0.8]
5. Strongly agree 7 : *******

Student t-test p = 0.046 (t = -2.02, sd = 0.740, dof = 128)
Class-wise <g>> = +0.16

13. I am good at math.
Pre-instruction
1. Strongly disagree 5 : *****
2. Disagree 16 : ****************
3. Neutral 28 : **************************** [2.9 +/- 1.0]
4. Agree 12 : ************
5. Strongly agree 4 : ****

Post-instruction
1. Strongly disagree 7 : *******
2. Disagree 16 : ****************
3. Neutral 19 : ******************* [3.0 +/- 1.2]
4. Agree 16 : ****************
5. Strongly agree 7 : *******

Student t-test p = 0.63 (t = -0.484, sd = 1.09, dof = 128)
Class-wise <g> = +0.04

14. I am good at science.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 10 : **********
3. Neutral 34 : ********************************** [3.2 +/- 0.7]
4. Agree 19 : *******************
5. Strongly agree 2 : **

Post-instruction
1. Strongly disagree 1 : *
2. Disagree 17 : *****************
3. Neutral 23 : *********************** [3.2 +/- 1.0]
4. Agree 18 : ******************
5. Strongly agree 6 : ******

Student t-test p = 0.203 (t = 0.203, sd = 0.864, dof = 128)
Class-wise <g> = -0.02

15. This course will be/was difficult for me.
Pre-instruction
1. Strongly disagree 2 : **
2. Disagree 14 : **************
3. Neutral 28 : **************************** [3.1 +/- 0.9]
4. Agree 18 : ******************
5. Strongly agree 3 : ***

Post-instruction
1. Strongly disagree 13 : *************
2. Disagree 25 : ************************* [2.4 +/- 1.0]
3. Neutral 18 : ******************
4. Agree 7 : *******
5. Strongly agree 2 : **

Student t-test p = 0.0001 (t = 4.19, sd = 0.964, dof = 128)
Class-wise <g> = -0.37

Note that this semester (spring semester 2011) was the first implementation of a backwards faded scaffolding curriculum (Slater, Slater, and Lyons, 2010), previous semesters (fall semester 2011 and earlier) used a conventional "cookbook" laboratory curriculum. The student learning outcomes for the conventional "cookbook" laboratory curriculum:
  • Keep abreast of present-day discoveries and developments in astronomy (current events).
  • Construct and use devices to measure locations and sizes on the celestial sphere (observational astronomy).
  • Apply laws of spectroscopy and gravitation to remotely determine properties of satellites, planets, and stars (astronometry).
  • Develop and test physical models of the properties of solar system bodies (planetology).
  • Collect data, evaluate the data using error analysis, draw conclusions from the data.
  • Explain the information in a laboratory report.
Compare to the student learning outcomes for the backwards faded scaffolding laboratory curriculum:
  • Keeping abreast of present-day discoveries and developments in astronomy (current events).
  • Developing scientific evidence-based research questions.
  • Developing procedures to gather evidence in order to answer research questions.
  • Making appropriate evidence-supported conclusions.
  • Explaining research findings in a report, poster, or presentation.
  • Evaluating evidence to determine whether or not it appropriately answers a research question.
The following results were all comparable between the BFS and conventional curriculum labs, suggesting that (this version of the) ALLS either (a) cannot distinguish how these two types of labs affected student attitudes differently, or (b) these two types of labs do not affect student attitudes differently. Future revisions to the ALLS may include questions more specifically designed to elicit student attitude changes that may (or may not) stem from the new backwards faded scaffolding laboratory curriculum.
A statistically significant (p < 0.05) positive shift was observed for the interest in using telescopes/binoculars in astronomy (question 1), which was comparable to the results from fall semester 2011. There was a statistically insignificant positive shift in spring semester 2010, and notably a statistically significant (p < 0.05) negative shift in fall semester 2009! Also no statistically significant (p > 0.05) shifts were observed for relating astronomy to personal experience (question 2), interest in astronomy-related news (question 5), pondering astronomy-related questions (question 6), self-efficacy in use of calculators and math in astronomy (questions 7 and 8), individual/group learning habits (questions 9 and 11), and self-efficacy in math/science (questions 13 and 14). However, there are statistically significant (p < 0.05) gains in being able to find astronomy-related news/information (question 3), finding night sky objects (question 4), understanding concepts better by explaining (question 10), disconnectedness in astronomy concepts (question 12), and rating the expected/experience difficulty of this course (question 15).

Previous posts:

No comments: