The ALLS is administered as a pre-test on the first laboratory meeting, before any introduction/instruction took place; and as a post-test on the last laboratory meeting.
The results from the pre- and post-test questions follow below. Values for the mean and standard deviations are given next to the modal response category for each question, along with a Student t-test for the probability of null hypothesis rejection, and the class-wise Hake gain. (Matched-pair Hake gains were not calculated for each student, as pre-instruction values such as "5" would result in undefined values.) For statistical purposes, blank entries were treated as "3. Neutral," and multiply-circled entries such as "12," "23," "34," and "45" were treated as "1," "2," "4," and "5" respectively.
Cuesta College
Astronomy Laboratory Learning Survey (ALLS)
Pre- and post-instruction results
Astronomy 210L Spring Semester 2010 sections 30678, 30679, 30680, 30682
1. I am interested in using a telescope or binoculars for astronomy.
Pre-instruction
1. Strongly disagree 4 : ****
2. Disagree 5 : *****
3. Neutral 16 : ****************
4. Agree 23 : *********************** [3.7 +/- 1.1]
5. Strongly agree 19 : *******************
Post-instruction
1. Strongly disagree 1 : *
2. Disagree 2 : **
3. Neutral 9 : *********
4. Agree 39 : *************************************** [4.0 +/- 0.8]
5. Strongly agree 16 : ****************
Student t-test p = 0.097 (t = -1.67, sd = 0.983, dof = 132)
Class-wise <g> = +0.22
2. Astronomy has little relation to what I experience in the real world.
Pre-instruction
1. Strongly disagree 8 : ********
2. Disagree 28 : **************************** [2.4 +/- 0.8]
3. Neutral 25 : *************************
4. Agree 6 : ******
5. Strongly agree 0 :
Post-instruction
1. Strongly disagree 15 : ***************
2. Disagree 23 : *********************** [2.4 +/- 1.1]
3. Neutral 20 : ********************
4. Agree 6 : ******
5. Strongly agree 3 : ***
Student t-test p = 0.79 (t = 0.271, sd = 0.955, dof = 132)
Class-wise <g> = -0.02
3. I know where and how to look up astronomy information.
Pre-instruction
1. Strongly disagree 6 : ******
2. Disagree 21 : *********************
3. Neutral 22 : ********************** [2.8 +/- 1.1]
4. Agree 14 : **************
5. Strongly agree 4 : ****
Post-instruction
1. Strongly disagree 0 :
2. Disagree 9 : *********
3. Neutral 8 : ********
4. Agree 38 : ************************************** [4.2 +/- 0.6]
5. Strongly agree 21 : *********************
Student t-test p < 0.0001 (t = -9.05, sd = 0.869, dof = 132)
Class-wise <g> = +0.63
4. I know where and how to find objects in the night sky.
Pre-instruction
1. Strongly disagree 7 : *******
2. Disagree 20 : ********************
3. Neutral 23 : *********************** [2.8 +/- 1.0]
4. Agree 15 : ***************
5. Strongly agree 2 : **
Post-instruction
1. Strongly disagree 0 :
2. Disagree 2 : **
3. Neutral 7 : *******
4. Agree 44 : ******************************************** [4.0 +/- 0.7]
5. Strongly agree 14 : **************
Student t-test p < 0.0001 (t = -8.59, sd = 0.855, dof = 132)
Class-wise <g> = +0.57
5. I am interested in news that is related to astronomy.
Pre-instruction
1. Strongly disagree 1 : *
2. Disagree 6 : ******
3. Neutral 22 : **********************
4. Agree 29 : ***************************** [3.6 +/- 0.79]
5. Strongly agree 9 : *********
Post-instruction
1. Strongly disagree 0 :
2. Disagree 6 : ******
3. Neutral 23 : ***********************
4. Agree 24 : ************************ [3.7 +/- 0.9]
5. Strongly agree 14 : **************
Student t-test p = 0.50 (t = -0.673, sd = 0.899, dof = 132)
Class-wise <g> = +0.07
6. I often ask myself questions related to astronomy.
Pre-instruction
1. Strongly disagree 5 : *****
2. Disagree 11 : ***********
3. Neutral 16 : ****************
4. Agree 30 : ****************************** [3.3 +/- 1.1]
5. Strongly agree 5 : *****
Post-instruction
1. Strongly disagree 3 : ***
2. Disagree 9 : *********
3. Neutral 26 : ************************** [3.3 +/- 1.0]
4. Agree 23 : ***********************
5. Strongly agree 6 : ******
Student t-test p = 0.93 (t = -0.0846, sd = 1.02, dof = 132)
Class-wise <g> = +0.009
7. I am comfortable using a calculator to make complex calculations.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 8 : ********
3. Neutral 14 : **************
4. Agree 30 : ****************************** [3.8 +/- 0.9]
5. Strongly agree 15 : ***************
Post-instruction
1. Strongly disagree 1 : *
2. Disagree 6 : ******
3. Neutral 9 : *********
4. Agree 30 : ****************************** [4.0 +/- 1.0]
5. Strongly agree 21 : *********************
Student t-test p = 0.28 (t = -1.086, sd = 0.955, dof = 132)
Class-wise <g> = +0.15
8. I can make sense of equations and scientific notation numbers.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 6 : ******
3. Neutral 21 : *********************
4. Agree 32 : ******************************** [3.6 +/- 0.8]
5. Strongly agree 8 : ********
Post-instruction
1. Strongly disagree 0 :
2. Disagree 7 : *******
3. Neutral 16 : ****************
4. Agree 29 : ***************************** [3.8 +/- 0.9]
5. Strongly agree 15 : ***************
Student t-test p = 0.32 (t = -0.996, sd = 0.867, dof = 132)
Class-wise <g> = +0.11
9. I prefer to work independently rather than in groups.
Pre-instruction
1. Strongly disagree 7 : *******
2. Disagree 22 : **********************
3. Neutral 28 : **************************** [2.7 +/- 1.0]
4. Agree 6 : ******
5. Strongly agree 4 : ****
Post-instruction
1. Strongly disagree 14 : **************
2. Disagree 22 : **********************
3. Neutral 24 : ************************ [2.4 +/- 1.1]
4. Agree 3 : ***
5. Strongly agree 4 : ****
Student t-test p = 0.15 (t = 1.43, sd = 1.03, dof = 132)
Class-wise <g> = -0.11
10. I can understand difficult concepts better if I am able to explain them
to others.
Pre-instruction
1. Strongly disagree 2 : **
2. Disagree 6 : ******
3. Neutral 23 : ***********************
4. Agree 27 : *************************** [3.5 +/- 0.9]
5. Strongly agree 9 : *********
Post-instruction
1. Strongly disagree 0 :
2. Disagree 5 : *****
3. Neutral 18 : ******************
4. Agree 31 : ******************************* [3.8 +/- 0.8]
5. Strongly agree 13 : *************
Student t-test p = 0.10 (t = -1.64, sd = 0.898, dof = 132)
Class-wise <g> = +0.17
11. I can understand difficult concepts better if I am able to ask lots of
questions.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 1 : *
3. Neutral 16 : ****************
4. Agree 34 : ********************************** [4.0 +/- 0.7]
5. Strongly agree 16 : ****************
Post-instruction
1. Strongly disagree 0 :
2. Disagree 1 : *
3. Neutral 12 : ************
4. Agree 32 : ******************************** [4.1 +/- 0.7]
5. Strongly agree 22 : **********************
Student t-test p = 0.25 (t = -1.16, sd = 0.744, dof = 132)
Class-wise <g> = +0.15
12. Knowledge in astronomy consists of many pieces of information each of which
applies primarily to a specific situation.
Pre-instruction
1. Strongly disagree 0 :
2. Disagree 3 : ***
3. Neutral 41 : ***************************************** [3.3 +/- 0.6]
4. Agree 21 : *********************
5. Strongly agree 2 : **
Post-instruction
1. Strongly disagree 1 : *
2. Disagree 6 : ******
3. Neutral 24 : ************************
4. Agree 28 : **************************** [3.5 +/- 0.9]
5. Strongly agree 8 : ***
Student t-test p = 0.11 (t = -1.60, sd = 0.756, dof = 132)
Class-wise <g>> = +0.13
13. I am good at math.
Pre-instruction
1. Strongly disagree 6 : ******
2. Disagree 10 : **********
3. Neutral 20 : ********************
4. Agree 27 : *************************** [3.2 +/- 1.1]
5. Strongly agree 4 : ****
Post-instruction
1. Strongly disagree 5 : *****
2. Disagree 10 : **********
3. Neutral 22 : ********************** [3.3 +/- 1.1]
4. Agree 22 : **********************
5. Strongly agree 8 : ********
Student t-test p = 0.69 (t = 0.400, sd = 1.08, dof = 132)
Class-wise <g> = +0.04
14. I am good at science.
Pre-instruction
1. Strongly disagree 3 : ***
2. Disagree 6 : ******
3. Neutral 41 : ***************************************** [3.1 +/- 0.8]
4. Agree 15 : ***************
5. Strongly agree 2 : **
Post-instruction
1. Strongly disagree 5 : *****
2. Disagree 7 : *******
3. Neutral 29 : ***************************** [3.2 +/- 1.0]
4. Agree 22 : **********************
5. Strongly agree 4 : ****
Student t-test p = 0.56 (t = -0.587, sd = 0.882, dof = 132)
Class-wise <g> = 0.05
15. This course will be/was difficult for me.
Pre-instruction
1. Strongly disagree 1 : *
2. Disagree 16 : ****************
3. Neutral 34 : ********************************** [3.0 +/- 0.8]
4. Agree 14 : **************
5. Strongly agree 2 : **
Post-instruction
1. Strongly disagree 13 : *************
2. Disagree 21 : ********************* [2.5 +/- 1.1]
3. Neutral 19 : *******************
4. Agree 3 : ***
5. Strongly agree 1 : *
Student t-test p = 0.0039 (t = 2.94, sd = 0.940, dof = 132)
Class-wise <g> = -0.24
No statistically significant (p > 0.05) shift was observed for the slight positive shift in interest in using telescopes/binoculars in astronomy (question 1); notably there was a statistically significant (p < 0.05) downward shift in Fall Semester 2009!
Also not statistically significant (p > 0.05) shifts were observed for relating astronomy to personal experience (question 2), interest in astronomy-related news (question 5), pondering astronomy-related questions (question 6), self-efficacy in use of calculators and math in astronomy (questions 7 and 8), individual/group learning habits (questions 9, 10, 11, and 12), and self-efficacy in math/science (questions 13 and 14).
However, there are statistically significant (p < 0.05) gains in being able to find astronomy-related news/information (question 3), finding night sky objects (question 4), and rating the expected/experience difficulty of this course (question 15). The first two of these positive self-reported shifts correlate well with two of the student learning outcomes for this course:
- Keep abreast of present-day discoveries and developments in astronomy (current events).
- Construct and use devices to measure locations and sizes on the celestial sphere (observational astronomy).
(For completeness, the remaining four student learning outcomes are listed below, and are measured primarily through grade-related assessment):
- Apply laws of spectroscopy and gravitation to remotely determine properties of satellites, planets, and stars (astronometry).
- Develop and test physical models of the properties of solar system bodies (planetology).
- Collect data, evaluate the data using error analysis, draw conclusions from the data.
- Explain the information in a laboratory report.
Other than the positive shift for question 1, these results are comparable to those from Fall Semester 2009.
Previous posts:
- Education research: ALLS post-instruction opinion results (Cuesta College, Spring Semester 2010).
- Education research: ALLS demographic results (Cuesta College, Spring Semester 2010).
- Education research: ALLS pre- to post-instruction attitude shifts (Cuesta College, Fall Semester 2009).
- Education research: ALLS post-instruction opinion results (Cuesta College, Fall Semester 2009).
- Education research: ALLS demographic results (Cuesta College, Fall Semester 2009).
- Education research: Astronomy Laboratory Learning Survey (ALLS), pre- and post-instruction forms.
No comments:
Post a Comment