Education research: pre-instruction survey protocol

Pre-instruction surveys are administered as students walk into the first day of class, without even mentioning the course the students are in (however, course and section information is already written on the whiteboard, in order to assure perplexed students that they have found the right classroom). The reasons for this immediacy are that:

1. Student preconceptions should be minimal, with so little exposure to the course and/or instructor.
   
2. Students immediately get acclimated to the policy that all assessment in this course starts promptly.
   
3. Students who come in a few minutes late (usually lost in locating the classroom) do not interrupt lecture time, and can usually make up the few minutes lost in taking the pre-instruction surveys.
   
4. Students who are trying "crash" the course can be given something to do while the instructor resolves their enrollment status.

After the end of the allotted time for the pre-instruction surveys is over (excepting any extremely late stragglers who are instructed to complete the surveys any time before they leave class that day (no surveys are taken out of class by students)); a general introduction to course policies and/or instruction begins.

Astronomy 10 pre-instruction surveys:

• SPCI: Stellar Properties Concept Inventory
  Bailey, J. M. (2006), "Development of a concept inventory to assess students' understanding and reasoning difficulties about the properties and formation of stars," unpublished doctoral dissertation, University of Arizona, Tucson, AZ.
  
• SATA: Student Attitudes Towards Astronomy
  Zeilik, M., & Morris, V. J. 2003, "An Examination of Misconceptions in an Astronomy Course for Science, Mathematics, and Engineering Majors," Astronomy Education Review, 2(1), 101.

Physics 5AB/8AB pre-instruction surveys:

• FCI: Force Concept Inventory
  D. Hestenes, M. Wells, and G. Swackhamer, Arizona State University, "Force Concept Inventory," Phys. Teach. 30, 141-158 (1992).
  
• MPEX: Maryland Physics Expectations Survey
  E. F. Redish, J. M. Saul, and R. N. Steinberg, "Student Expectations in Introductory Physics," American Journal of Physics, 1998, 66(3).

Astronomy clicker question: retrograde/prograde rise/set

Astronomy 10, Summer Session 2007
Cuesta College, San Luis Obispo, CA

Astronomy 10 learning goal Q3.2

Students were asked the following clicker question (Classroom Performance System, einstruction.com) near the end of their learning cycle:

[0.3 points.] As seen from San Luis Obispo, CA, when a planet is in retrograde motion, how will it move in a single night?
(A) It will rise from the east horizon, and set in the west horizon.
(B) It will rise from the east horizon, and shortly afterwards set in the east horizon.
(C) It will rise from the west horizon, and set in the east horizon.
(D) It will rise from the west horizon, and shortly afterwards set in the west horizon.

Correct answer: (A).

Most students are probably thinking that retrograde motion (planet moves a very small distance east-to-west relative to the background stars) is the reason why it will rise in the east horizon, and set in the west horizon. The next clicker question will test for this misconception.

Student responses
Section 8027
(A) : 11 students
(B) : 1 student
(C) : 2 students
(D) : 0 students

[0.3 points.] As seen from San Luis Obispo, CA, when a planet is in prograde motion, how will it move in a single night?
(A) It will rise from the east horizon, and set in the west horizon.
(B) It will rise from the east horizon, and shortly afterwards set in the east horizon.
(C) It will rise from the west horizon, and set in the east horizon.
(D) It will rise from the west horizon, and shortly afterwards set in the west horizon.

Correct answer: (A).

This follow-up question causes a lot of consternation for students, who must realize that in prograde motion, a planet moves a very small distance west-to-east relative to the background stars, which themselves over a single night rise from the east horizon, and set in the west horizon. Thus a planet will be seen to rise from the east horizon, and set in the west horizon in a single night, regardless if it is undergoing retrograde or prograde motion!

Student responses
Section 8027
(A) : 3 students
(B) : 1 student
(C) : 10 students
(D) : 0 students

Carousel teacup epicycle movie

070526 Carousel Teacup Epicycle
Venetian Carousel, K Street Mall
Sacramento, CA
http://www.youtube.com/watch?v=FIsiezgo7nY

Astronomy 10 learning goal Q3.2

Dramatization of planetary motion in the Ptolemaic model. Watch the girl in the teacup (planet on epicycle) as the merry-go-round (the deferent), well, goes around and around.

General relativity: warped spacetime

"Spacetime 2"
David J. Grossman (c) 1996
http://unpronounceable.com/graphics/raytraces/spactim2.html

What better way to depict the gravitational distortion of spacetime than with distorted Einsteins?

Astronomy clicker question: lunar phases

Astronomy 10, Summer Session 2007
Cuesta College, San Luis Obispo, CA

Astronomy 10 learning goal Q2.1

Students were asked the following clicker question (Classroom Performance System, einstruction.com) at the start of their learning cycle:

[0.3 points.] Shown below are eight different Moon phases.


What causes these different phases of the Moon?
(A) Different amounts of the far and near sides of the Moon as seen from the Earth.
(B) Different amounts of the day and night sides of the Moon as seen from the Earth.
(C) The Earth blocks light from the Sun, casting a shadow on the Moon.
(D) The slow, gradual rotation of the Moon about its own axis.

Correct answer: (B).

The border between the sides of the Moon that are illuminated or non-illuminated by the Sun is the terminator. As the Moon orbits the Earth (in approximately 28 days), the terminator moves across the face of the Moon. Students are instructed to diagram this, in order to construct a mental model of this phenomenon.

Student responses
Section 8027
(A) : 1 student
(B) : 4 students
(C) : 7 students
(D) : 1 student

Education research: Breaking out of "teaching by telling"

Instructional time at Cuesta College is beginning to shift away from exclusively lecture to incorporating electronic response systems (clickers) and in-class group activities, especially for Astronomy 10 (introductory astronomy, general education requirement), and starting fall 2007, for Physics 5AB (introductory physics, algebra-based). This trend towards an emphasis on more effective pedagogy merely follows from the results of physics and astronomy education research over the past twenty years.

The classic statement of the problem of traditional instruction is given by Alan Van Heuvelen at the Ohio State University, Columbus, OH (1991):

Historically we have relied on expository lectures--telling students the physical results that seem to guide the universe and demonstrating how to use the rules to solve problems... This is a very efficient method to transmit information in terms of the time interval needed. We know the concepts and techniques, and students do not. Why not just tell them? Study after study indicates that this expository method is very ineffective--the transmission is efficient but the reception is almost negligible.

Randall Knight at the California Polytechnic State University, San Luis Obispo, CA (2004) reiterates the problems with traditional instruction (which he calls "teaching by telling"), but does not dismiss it completely:

This is not to say that lectures are never effective... Even in the introductory class, short periods of instructor-centered discourse can clarify difficult issues or provide background information. But extended lectures, particularly formal lectures of deriving results, appear to be the least effective mode of instruction.

Eugenia Etkina at Rutgers University, New Brunswick, NJ, along with Alan Van Heuvelen offer this recent observation (2003) regarding the resistance of instructors to research-based pedagogy reform:

There is considerable evidence that students using various researched-based learning methods are more successful in courses than students taught traditionally in the same courses. Would something bad happen if all courses were based on research about learning? Perhaps this is the age-old problem about the resistance to the adoption of innovation--fluoridation, the use of electricity, mechanization of farming practices, adoption of the ideas of relativity and of evolution, to name a few. Eventually, progressive ideas that produce a better product find acceptance.

A. Van Heuvelen, "Learning to think like a physicist: A review of research-based instructional strategies," Am. J. Phys. 59, 891-897 (1991).

Knight, Randall D., Five Easy Lessons: Strategies for Successful Physics Teaching, Addison Wesley, 2004, p. 41.

E. Etkina, A. Van Heuvelen, "More on Education Reform: Author Response," Phys. Teacher, 41, 68-69 (2003).

Universe expansion movie

Continued Consistent Expansion (*.mov, 3.7 MB)
Hubblesite.org

Astronomy 10 learning goal Q12.2

This movie is not just a zoom-in shot! Note that as the universe expands, the distances between galaxies increases, while the size of the galaxies remains constant (due to gravitational interactions). This can be illustrated during the movie by covering one of the galaxies with the mouse cursor, and showing that that the size of the galaxy does not change relative to the cursor.

Geocentrism, redux

"Something's going wrong. Stop the global warming."

Dogal Hayati Koruma Dernegi (Turkish Nature Life Protection Association), www.dhkd.org
(Via shedwa.blogspot.com)

Astronomy 10 learning goal Q3.2

Ignorance of global warming is equated with the antiquated geocentric model of planetary motion.

Pirate displacement vectors

Bizarro, by Dan Piraro
www.bizarro.com
June 18, 2007

Physics 8A learning goal Q1.3

GPS is concerned with position vectors, here we have pirate displacement vectors!

Rifle bugle

Rifle Barrel Becomes Bugle for Musical Stunt
Popular Science, March 1938
blog.modernmechanix.com

Physics 8B learning goal Q1.4

Even though the bolt has been removed, don't try this at home?

Education research: Heterogeneous gender student groups

Patricia Heller and others at the University of Minnesota have general guidelines regarding the optimal formation of student groups for collaborative in-class activities in physics (such as those found in the Lecture-Tutorials for Introductory Astronomy (Adams, Prather, & Slater, 2005)), as reported by Randall Knight at the California Polytechnic State University, San Luis Obispo, CA (Knight, 2004):

...The initial groups [at the University of Minnesota] are formed at random. Once the first test scores are known, mixed ability groups are formed with one student each from the top, middle, and bottom thirds of the class. Students rotate to a different group every three weeks. Groups sizes of three are optimal, two are below critical mass, and five or more generally finds some members not engaged. As for gender balance, groups should be homogeneous or have two women, one man. Groups with a single women tend to not function well, even if the woman is the "top third" member.

Jeffery P. Adams et al. (2002) studied self-formed groups of four students in astronomy (as opposed to assigned groups), and found that:

Specifically, females in mixed-sex groups of unequal numbers (e.g., three females and one male or three males and one female) were less likely to display behavior indicating active engagement than females in all-female groups or in groups with equal numbers of males and females.

Thus the optimal formation of three (or four) students in a heterogeneous group should strive to have two-one (or two-two) females to males. Homogeneous groups are also allowed.

The assignment of student groups in Astronomy 10 (introductory astronomy) at Cuesta College in San Luis Obispo, CA is done alphabetically for the first class, even though this may violate the optimal heterogenous gender rules discussed above. This is mainly done to facilitate a spot-check on enrollment, and to take photos for identification by the instructor, and is only done on the first day of instruction.

Students take conceptual and demographic surveys at the start of the first class, and these results are compiled before the second class. Students are then sorted into new groups using top/middle/bottom thirds of their self-reported confidence in math and science for every subsequent class until after the first midterm. By then students are segregated into top/middle/bottom thirds of exam performance once a week (every other class meeting).

Near the end of the semester, some students have observed that they have never been in the same group as certain other students--these are typically the highest performing students in class, and thus being in the same top third, cannot be in the same group. They told that since they are as "equally smart" as those other students, they can never be in the same group. This is also the same answer given even if the students in question are in the same bottom third of exam performance!

Adams, J. P., Brissenden, G., Lindell, R. S., Slater, T. F., & Wallace, J. 2002, "Observations of Student Behavior in Collaborative Learning Groups," Astronomy Education Review, 1(1), 25.

Adams, J. P., Prather, E., & Slater, T. F., Lecture-Tutorials for Introductory Astronomy, Prentice Hall, 2005.

Heller, P., Hollabaugh, M., "Teaching problem solving through cooperative grouping Part 2: Designing problems and structuring groups," American Journal of Phyics 60, 1992, 637.

Knight, Randall D., Five Easy Lessons: Strategies for Successful Physics Teaching, Addison Wesley, 2004, p. 31.

Kudos: frequency of the electric wave...

(Front)

(Back)

"Frequency of the Electric Wave..." by Mary W. and Janet L.
Physics 5B
Spring Quarter 1991
University of California, Davis, CA

Education research: FCI concerns and protocols

Charles Henderson at Western Michigan University, Kalamazoo, MI addresses and lays to rest three common concerns regarding the Force Concept Inventory (Doug Hestenes et al., 1992) in a large population study published in 2002 (N = 1,856 students):

1. The FCI is not appropriate for use as a placement test [this supports the recommendations of Hestenes et al.].
   
2. There is little difference between FCI scores when the test is given graded versus ungraded.
   
3. Giving the FCI as a pre-test does not affect the post-test results.

Following Henderson's (2002) guidelines, the FCI is not used as a placement test for Physics 8A at Cuesta College, nor is it graded for credit. However, Physics 8A students who take the FCI in a serious and conscientious manner do receive credit towards their total quiz grade (it is counted as a "perfect quiz score" regardless of their actual pre- and/or post-instruction performance, which can then be used to replace their lowest quiz score).

Also, as Henderson (2002) finds that pre-instruction familiarity with the FCI does not affect post-instruction FCI scores, Cuesta College FCI pre-instruction scores and post-instruction scores are respectively pooled for each class as a whole, even though some students (typically late adds) did not take the FCI in the first week of instruction, and many students (due to drops) did not take the FCI in the last week of instruction.

C. Henderson, "Common Concerns About the Force Concept Inventory," Phys. Teach. 40, 542 (2002).

D. Hestenes, M. Wells, and G. Swackhamer, Arizona State University, "Force Concept Inventory," Phys. Teach. 30, 141-158 (1992).
Development of the FCI, a 30-question survey of basic Newtonian mechanics concepts.

Education research: clicker literature review

Some contemporary observations on the use of electronic response systems (clickers), and how they are implemented at Cuesta College.

Duncan (2006) discusses an idealized use of clickers in introductory astronomy:

The regular use of clickers can transform the classroom in a very positive way. Students become active participants, not merely passive listeners to a lecture. They ask more questions... If students are allowed to discuss their answers with the neighbors before responding, the impact is even stronger.

These aspirations come with qualifications, especially with the end goal (peer instruction) in mind:

Using clickers does not mean that your class will immediately achieve the results of an interactive education course... Particularly good results are achieved when clickers are used in conjunction with peer discussions. Peer instruction is based on two ideas: (1) ask conceptual questions that probes students' understanding of a topic, and (2) get students to discuss (argue, debate) and try to convince each other of the correct answer.

Judson and Sadawa (2002) reiterate the benefits of peer interaction over conventional lecturing by an instructor:

The only positive effects upon student academic achievement, related to incorporation of electronic response systems [clickers] into instruction, occurred when students communicated actively to help one another understand. It is more beneficial for a student, who has just arrived at a new conceptual understanding, to explain to peers how he/she struggled and arrived at his/her new rationale than it is for an instructor to simply explain the abstraction.

To this end, Beatty (2007) suggests how clicker questions can best be used to elicit peer interaction:

For effective use, [clickers] should be used in a low stakes environment where students are willing to explain their thinking and reasoning. The intent of these ideas is to reveal student thinking, not to assess their knowledge. Many of the answers to the [clicker questions] have been included because they are common but incorrect choices.

Similarly, Crouch and Mazur (2000) also stress the importance of clicker questions as a means to drive peer interaction, and not just for assessment:

[Clicker questions] should be designed to give students a chance to explore important concepts, rather than testing cleverness or memory, and to expose common difficulties with the material. For this reason, incorrect answer choices should be plausible, and when possible, based on typical student misunderstandings. A good way to write questions is by looking at students' exam or homework solutions from previous years to identify common misunderstandings, or by examining the literature on student difficulties.

The use of clickers in Astronomy 10 (introductory astronomy) at Cuesta College in San Luis Obispo, CA has been implemented with these guidelines in mind since summer 2003. Clickers will also be implemented for the first time (fall 2007) in Physics 5A (college physics, algebra-based).

• As per Duncan (2006) and Judson and Sadawa (2002), clicker questions at the end of a class previous to an exam for Astronomy 10 at Cuesta College are scored collaboratively, in order to maximize peer interaction among students. During this review session of five-ten questions, students are awarded points for each clicker question that they answer, regardless if their response is correct. However, their total points for the review session are doubled if the cumulative class average is above an 80% threshold, giving the students both a non-penalizing incentive to response freely, but also a strong motivating incentive to collaborate and cooperate in order to respond correctly en masse. Further discussion of this mode of clicker response scoring is discussed in Len (2006), as well as in a scheduled workshop presentation, "Formative, Summative, and Cooperative Clicker Instruction in Astronomy" at the 2007 Cosmos in the Classroom conference sponsored by the Astronomical Society of the Pacific at Pomona College, August 3-5, 2007.
  
• Following Beatty (2007) and Crouch and Mazur (2000), clicker questions for Astronomy 10 at Cuesta College typically include incorrect responses culled from previous student essay questions, questions raised by students during lecture, misconceptions from science-fiction films and other popular culture sources, and obsolete theories (which often are still incorporated in K-12 legacy curriculum materials).
  

Beatty, I., A2L: Assessing to Learn and A2L: Suggestions for Using Items webpages, retrieved June 10, 2007.

Crouch, C., & Mazur, E. 2001, "Peer Instruction: Ten Years of Experience and Results," American Journal of Physics 69(9).

Duncan, D. 2006, "Clickers: A New Teaching Aid with Exceptional Promise," Astronomy Education Review, 5(1), 70.

Judson, E., & Sawada, D. 2002, "Learning from Past and Present: Electronic Response Systems in College Lecture Halls," Journal of Computers in Mathematics and Science Teaching, 21(2), 167.

Len, P. M., 2006, "Different Reward Structures to Motivate Student Interaction with Electronic Response Systems in Astronomy," Astronomy Education Review, 5(2), 5.

Bon mot: on the shoulders of giants

Illustration by Matt Collins
www.mattcollins.com

"If I have seen a little further it is by standing on the shoulders of Giants."
--Isaac Newton

Elaboration of this metaphor in contemporary and earlier attributions is discussed on Wikipedia. Apparently it has been adopted as the byword of Google Scholar.

"Ultra Violet" wine

"Ultra Violet" Merlot wine bottle label
Casa de Caballos Vineyards
Paso Robles, CA

Would they happen to have an Infrared Zinfandel?

Kudos: P-A-T

Anonymous
Physics 7B course evaluation
Winter Quarter 2001
University of California, Davis, CA

Kudos: love and amazement of the universe

"Love and Amazement of the Universe" by Carley F.
Astronomy 10
May 2007
Cuesta College, San Luis Obispo, CA

Kudos: jigsaw puzzle

(Front)

(Back)

"Thanks!!!!!" by Charissa S.
Physics 7C
December 1998
University of California, Davis, CA

Overheard: pee-pee

Physics 7B, Fall Quarter 2002
University of California, Davis, CA

Excerpt from a student e-mail regarding placement of the origin (pivot point) for torque calculations:

Anyways I just had a thing [that] helped me in PHYS 7B. "Beware where you place your P.P." P.P. as in pivot point you know, but it's cool cause it sounds like pee-pee. Ha-ha, well anyways, thanks for being a good prof and bearing with my office hour visits.
--K. H.

Education research: student expectations in physics

E. F. Redish, J. M. Saul, and R. N. Steinberg (University of Maryland, College Park, MD) developed the Maryland Physics Expectations (MPEX) Survey in 1998 to quantify student attitudes, beliefs, and assumptions about physics using six question categories, rating responses as either favorable or unfavorable towards:

1. Independence--beliefs about learning physics--whether it means receiving information or involves an active process of reconstructing one's own understanding;
   
2. Coherence--beliefs about the structure of physics knowledge--as a collection of isolated pieces or as a single coherent system;
   
3. Concepts--beliefs about the content of physics knowledge--as formulas or as concepts that underlie the formulas;
   
4. Reality Link--beliefs about the connection between physics and reality--whether physics is unrelated to experiences outside the classroom or whether it is useful to think about them together;
   
5. Math Link--beliefs about the role of mathematics in learning physics--whether the mathematical;
   formalism is used as a way of representing information about physical phenomena or mathematics is just used to calculate numbers;
   
6. Effort--beliefs about the kind of activities and work necessary to make sense out of physics--whether they expect to think carefully and evaluate what they are doing based on available materials and feedback or not.

As a baseline, the MPEX was administered for calibration purposes by Redish, Saul, and Steinberg to teachers "committed to implementing an interactive engagement model of teaching in their classroom," and these results were taken to be the "expert" favorable versus unfavorable responses:

"Experts"
(N = 19)
Percentage of favorable:unfavorable responses
        Overall Indep.  Coher.  Concept Real.   Math   Effort
        87:6    93:3    85:12   89:6    93:3    92:4   85:4

The MPEX was also administered pre- and post-instruction by Redish, Saul, and Steinberg to students at an unidentified two-year college, in an introductory physics course where there is no adjunct laboratory during the first semester.

"Two-year college"
(Unidentified school, unknown course, N = 44)
Percentage of favorable:unfavorable responses
        Overall Indep.  Coher.  Concept Real.   Math    Effort
Initial 55:22   41:29   50:21   30:42   69:16   58:17   80:8 
Final   49:26   42:32   48:29   35:41   58:17   58:18   65:21

In comparision the MPEX was given to Physics 8A (university physics, calculus-based, mandatory adjunct laboratory) students at Cuesta College, San Luis Obispo, CA, during the first week of the semester, and then on the last week of the semester. Care was taken to match pre- and post-instruction statistics, as per Redish, Saul, and Steinberg:

In order to eliminate the confounding factor of differential drop-out rates, we only include students who completed the survey both at the beginning and at the end of the term. We say that the data is matched [emphasis theirs].

Cuesta College
Physics 8A Spring 2007 sections 4909, 4910, 4911
(N = 26)
Percentage of favorable:unfavorable responses
        Overall Indep.  Coher.  Concept Real.   Math    Effort
Initial 54:25   44:25   39:34   52:26   67:10   53:21   65:12
Final   44:31   33:26   45:28   36:37   57:13   41:29   45:35

The results between the unnamed two-year college and Cuesta College have four notable differences:

• The unnamed two-year college and Cuesta College Physics 8A students had comparable favorable initial attitudes towards independence (i.e., "takes responsibility for constructing own understanding," versus "takes what is given by authorities (teacher, text) without evaluation"), but Cuesta College students have markedly lower final favorable attitudes in independence.
  
• Cuesta College Physics 8A students had higher favorable initial attitudes towards concepts (i.e., "stresses understanding of the underlying ideas and concepts" as opposed to "focuses on memorizing and using formulas"), but these dropped down to levels comparable to the two-year college in the Redish, Saul, and Steinberg study.
  
• While the unnamed two-year college and Cuesta College students had comparable favorable initial attitudes in regards to math link ("considers mathematics as a convenient way of representing physical phenomena" as opposed to "views the physics and the math as independent with little relationship between them"), Cuesta College Physics 8A students show a strong negative shift, towards lower favorable final math link attitudes by the end of the semester (while the two-year college student math link attitudes remain constant).
  
• The fourth difference was the less favorable initial and final attitudes of Cuesta College Physics 8A students towards effort (i.e., "makes the effort to use information available and tries to make sense of it" versus "does not attempt to use available information effectively") compared to the sample two-year college (but both have comparable downwards shifts in effort).

These results are quite disheartening. However, the spin on this, apparently, is not just that there are negative shifts in attitudes, but that there are noticeable shifts at all for the first semester of an introductory physics course. The optimistic implication is that the impact of this course can eventually, hopefully be made to cause positive shifts, as Redish, Saul, and Steinberg discuss that:

"In all cases, the result of instruction on the overall [MPEX] survey was an increase in unfavorable responses and a decrease in favorable responses. Thus, instruction produced an average deterioration rather than an improvement of student expectations... The failure to begin to move students from a binary view of learning to a more constructivist set of attitudes in the first term of university physics is most unfortunate. The start of college is a striking change for most students. This change of context gives instructors the valuable opportunity to redefine the social contract between students and teachers. This redefinition offers an opportunity to change expectations... If students experience a series of science courses that do not require deeper understanding and a growth of sophistication, they will be much more reluctant to put in the time and effort to change in a later course."

Compilation of earlier semester data for Physics 8A and 8B students at Cuesta College, and data for subsequent semesters of Physics 5A (college physics, algebra-based) students at Cuesta College will be posted in the future.

E. F. Redish, J. M. Saul, and R. N. Steinberg, "Student Expectations in Introductory Physics," American Journal of Physics, 1998, 66(3).

Education research: static science attitudes in astronomy

In a recent article (2007) in the Astronomy Education Review, C. A. Garland and D. L. Ratay use current science articles as a primary resource in an introductory astronomy course (with a conventional textbook as an auxiliary resource) at Castleton State College in Castleton, VT. While there are some modest gains in learning and self-confidence, as measured by the Astronomy Diagnostic Test (Hufnagel, 2002); more interesting is their results from the Survey of Attitudes Towards Astronomy (Zeilik and Morris, 2003) which found that:

...Students' opinion of their own ability to do science was unchanged... Our demographic questions revealed that 40% of the class had previously taken one college-level science class or none at all. A mere 56 class meetings [in one semester] with these students is not enough to change their perception of how well they do science--there simply is not enough time...
--Garland and Ratay (2007)

Garland and Ratay observe that that this is consistent with the original SATA study by Zeilik and Morris (2003). However, it should be noted that this contrasts somewhat with recent findings at Cuesta College in San Luis Obispo, CA (Len, 2006), where electronic response systems ("clickers") were extensively used to generate collaborative interaction in class. When clickers were used with participation-only credit (regardless of whether responses were correct or not), students reported themselves as either self-testers (responding without talking to, or listening to other students); or as collaborators (responding after talking to, or listening to other students). The Cuesta College study also used the ADT and SATA to measure student self-confidence and attitude gains, and found that:

Self-testers reported a higher pretest proficiency in science and maintained positive attitudes toward science, in contrast to collaborators, who reported a lower pretest proficiency in science and subsequently experienced a negative shift in their attitudes toward science. The implication is that this one-semester astronomy course had a great impact on the formation of nascent science and learning attitudes of students with little or no background in science [emphasis added]... It is plausible that self-testers (with more exposure to learning science than collaborators) would have well-formed positive attitudes toward astronomy. In contrast, students with less exposure to science would be less comfortable answering on their own and would collaborate more before answering.
--Len (2006)

Garland and Ratay suggest that students' attitudes be surveyed as they take science courses in subsequent semesters, which is similar to the current effort (2006 onwards) at Cuesta College where students are specifically asked for the extent of their past science courses, and projected future science courses, in order to determine whether students' science attitudes do shift significantly (higher?) over time.

Garland, C. A., & Ratay, D. L. 2007 "Using Literacy Techniques to Teach Astronomy to Non-Science Majors," Astronomy Education Review, 6(1).

Hufnagel, B. 2002, "Development of the Astronomy Diagnostic Test," Astronomy Education Review, 1(1), 47.

Len, P. M. 2006, "Different Reward Structures to Motivate Student Interaction with Electronic Response Systems in Astronomy," Astronomy Education Review, 5(2).

Zeilik, M. & Morris, V. J. 2003, "An Examination of Misconceptions in an Astronomy Course for Science, Mathematics, and Engineering Majors," Astronomy Education Review, 2(1), 101.

Schrödinger's lolcat

Schrodinger's lolcat
Originally uploaded by dantekgeek, from original photo by Keven Steele

Physics 8C learning goal QX.x

This one lives... Close the box and try again. Or don't open the box, and keep 'em guessing.

Astronomy final exam question: novae

Astronomy 10 Final Exam, Spring Semester 2007
Cuesta College, San Luis Obispo, CA

Astronomy 10 learning goal Q10.3

[15 points.] Describe and explain what causes repeated nova explosions.

Solution and grading rubric:

• p = 15/15: Correct.
  A giant or supergiant overflows its Roche lobe, depositing hydrogen onto a white dwarf companion, which by itself cannot have fusion. When enough hydrogen gathers onto the surface of the white dwarf, then it will become dense enough and hot enough to undergo fusion. The process then repeats itself, as long as there is hydrogen still being taken from the giant/supergiant.
  
• r = 12/15:
  Nearly correct (explanation weak, unclear or only nearly complete); includes extraneous/tangential information; or has minor errors.
  
• t = 9/15:
  Contains right ideas, but discussion is unclear/incomplete or contains major errors. At least recognizes that hydrogen is transferred between two stars in a close-pair ("mass-exchanging") system.
  
• v = 6/15:
  Limited relevant discussion of supporting evidence of at least some merit, but in an inconsistent or unclear manner.
  
• x = 3/15:
  Implementation/application of ideas, but credit given for effort rather than merit.
  
• y = 1.5/15:
  Irrelevant discussion/effectively blank.
  
• z = 0/15:
  Blank.
  

Grading distribution:
Section 4136
p: 18 students
r: 0 students
t: 3 students
v: 3 students
x: 9 students
y: 0 students
z: 1 student

A portion of a "p" response (from T. B.):