Workshop: "Rubric Sorting Astronomy Essays" (Cosmos in the Classroom 2013)

Workshop presented at the Astronomical Society of the Pacific Cosmos in the Classroom: A Hands-on Symposium on Teaching Introductory Astronomy and Related Science, July 22, 2013, 4:30-5:30 PM, Student Union Almaden Room, San José State University, San Jose, CA.

Abstract: Student essays on introductory astronomy exams can be consistently and efficiently graded by a single instructor, or by multiple graders for a large class. This is done by constructing a robust outcome rubric while sorting exams into separate stacks, then checking each stack for consistency. Certain online resources readily provide primary source prompts for writing astronomy exam essay questions.

Outcomes: Workshop participants will learn how to grade sample student essays by sorting them into piles based on a rubric system, how to write robust rubrics, and how to find resources to write essay questions that lead to effective rubrics.

(This presentation, sample essay questions, rubrics and more information is posted at: tinyurl.com/rubricsorting.)

Me, and some words about myself.

The goals of this workshop are to help you solve two major assessment dilemmas. Grading large numbers of astronomy exam essay questions? We'll have a hands-on demonstration of how to systematically tackle this.

Drawing a blank in writing authentic assessment essay questions? We'll also share resources to help you with this.

First, you are now going to be graduate student readers, and will be grading some student exams. Divide yourselves up into groups of three (or two, or four), and then have someone from your group pick up a stack of actual student exams.

Briefly discuss a correct response for the exam questions, and then begin a coarse sort of the exams into "good," "fair," "poor," and blank stacks. The "good" stack should not necessarily be a perfect stack, as long as it includes responses better than "fair." Feel free to move exams around between stacks during this process, as this is a tentative, raw attempt at sorting. Don't worry about conventional "ABCDF" letter grades during this coarse sorting process.

Next, for each coarse-sorted stack, go through and distinguish two levels of student understanding--e.g. for the "good" stack, sort exams into "correct" and "nearly correct" stacks; for the "fair" stack sort exams into "right ideas" and "some merit" stacks; and for the "poor" stack sort exams into "some effort" and "irrelevant discussion" stacks. More letters can be used for additional granularity as necessary (such as "pqRstuvwxyz"), but six different stacks (disregarding the blank stack) should be adequate. Again, don't worry about conventional "ABCDF" letter grades during this fine sort process.

Briefly go through each stack as a check; you may find a few exams that need to be bumped up or down to an adjacent stack. Once you are satisfied with your sorting, then you can write-in the rubric letters on each exam. If you have more than one essay question on an exam, keeping these same stacks when sorting for the next question may be useful, as there could be some correlation between how students replied to different questions on the same exam.

Note the typography of the letters used, which prevent students from modifying a rubric letter to their advantage. The upper-case "R" is used instead of a lower-case "r" to prevent it being made into a "p" to gain more points. Likewise an italicized lower-case "x" is used to prevent it being made into a lower-case "t" to gain more points, or being made from a lower-case "y." This leaves only disadvantageous ways to modify these letters:

• lower-case "t" to italicized lower-case "x."
• lower-case "v" to italicized lower-case "x."
• lower-case "v" to lower-case "y."
• lower-case "v" to lower-case "z."

Then with the exams still sorted into their final stacks, write down a concise description of what distinguishes the categories from each other, such that students will clearly understand what they did (or didn't do) by referring to the rubric definitions, and also would assist in refreshing your memory in order to comparably grade late exams or regrade miscategorized exams.

A rubric template is given below as an example, feel free to reuse/adapt/recycle for your own purposes.

• p: Correct.
• R: Nearly correct (explanation weak, unclear or only nearly complete); includes extraneous/tangential information; or has minor errors.
• t: Contains right ideas, but discussion is unclear/incomplete or contains major errors.
• v: Limited relevant discussion of supporting evidence of at least some merit, but in an inconsistent or unclear manner.
• x: Implementation/application of ideas, but credit given for effort rather than merit.
• y: Irrelevant discussion/effectively blank.
• z: Blank.

Finally, assign points (or "ABCDF" letter grade equivalents) to these rubric categories.

Second, resources for writing authentic essay questions for astronomy exams, and two types of questions that can be extracted from these resources.

Two particularly rich sources for writing essay questions are Google Books, and Yahoo! Answers.

Google Books facilitates searching for phrases such as "watch the crescent moon rising" in classic and contemporary novels, which often contain astronomy-related errors and misconceptions. These can be readily adapted into essay questions that ask students to find and explain these errors and misconceptions. This and more examples below:

"On the twilight horizon, a crescent moon was rising. I watched it hook on to a branch right above me, and dangle there, like a new sickle blade gleaming in the darkening sky."
--Minfong Ho, The Clay Marble, Marshall Cavendish, 1992, p. 71.

"It was a rich warm golden evening early in that most beautiful appearance of nature on one side of the heaven, the sun sinking down to rest in a glory of mellow light and gorgeous colour, and on the other, the pure pearly crescent moon rising above the tree tops..."
--Henry F. Chorley, "Helen, A Sketch," The Honey-moon by the Countess of Blessington and Other Tales, Vols. 1-2, E.L. Carey and A. Hart, 1837, p. 71.

"I awake early the next morning with a mosquito whining in my ear... I go outside the tent and see a thin crescent moon rising above the lake. I would like to stay outside and watch the breaking of the day, but the mosquitoes drive me back."
--Max Finkelstein and James Stone, Paddling the Boreal Forest: Rediscovering A.P. Low, Dundum, 2004, p. 143.

"After dinner, the men assembled in the centre of the camp around a large fire... The crescent moon crept slowly behind the clouds in the east. When it was fully exposed, the tabla player changed the tune to a fast beat. The men stood up and watched the crescent rising in the sky."
--Shafik Benjamin, Kismet in the Sand, AuthorHouse, 2009, p. 5.

In contrast, a question can be posted on Yahoo! Answers to solicit responses. These can be adapted into essay questions where students analyze the correctness or completeness of selected answers. This and more examples below:

P-dog: ...If Venus is above the east horizon at sunrise (so it is a morning star), can it be above the west horizon at sunset (so it can be an evening star) later on that day?
aladdinwa: No, it cannot be both the morning and evening star on the same day... When Venus rises before the sun, you can see it before the sun rises and it is the morning star and it disappears below the horizon while the sun is still in the sky...
--Adapted from http://answers.yahoo.com/question/index?qid=20120511191408AARz0Af.

P-dog: Could a [planet more massive than Earth around another sun-like star] support life...? Would [it]...be too hot, or too cold, or could [it] have moderate temperatures?
Quadrillian: ...A giant rocky planet could easily exist outside the orbit of Mars and...be habitable [with moderate temperatures.]
--Adapted from http://answers.yahoo.com/question/index?qid=20111217134914AA3qcNa.

P-dog: Can a star be cooler and more luminous than a hotter, less luminous star?
green meklar: Yes. A star can be both cooler and more luminous, if it is larger.
--Adapted from http://answers.yahoo.com/question/index?qid=20130423004321AAlfY6k.

P-dog: Would an old star cluster or a new star cluster have supergiants and white dwarfs?
tham153: It would certainly be very unlikely for any cluster to have supergiants and white dwarfs...
--Adapted from http://answers.yahoo.com/question/index?qid=20130419183134AAKacCx.

In the time remaining we can address comments and questions from workshop participants.

Additional discussion on this workshop can be continued afterwards at "office hours" later today, and/or online as well.

Earlier blog posts:

Sample astronomy essay questions, rubrics, and student responses.

Education research: overcoming initial problem-solving block.

Presentation: lunar motions and cycles

Mike Myers as Dr. Evil in Austin Power: The Spy Who Shagged Me (New Line Cinema, 1999) with a model of the moon and Earth. We'll come back to this clip later at the end of this presentation.

So far we've covered the motions and cycles of the stars and sun in the sky, here we'll discuss the motions and cycles of the moon (but not eclipses, which will be covered in the next presentation).

First, let's review the boring but necessary terminology from the online reading assignment, and then review with a picto-quiz.

Make sure you can distinguish between very similar terms:

• full moon and new moon (well, this should be self-explanatory).
• waxing phases and waning phases (waxing phases get more full over time, waning phases get less full over time).
• crescent phases and gibbous phases (crescent phases are less than half full, gibbous phases are more than half full)
• first quarter and third quarter moon (both of these are exactly "half-moons," but the first quarter moon is a waxing phase that occurs one-quarter of the way through its cycle of moon phases, while the third quarter moon is a waning phase that occurs three-quarters of the way through).

Let's do a picto-quiz--you'll be shown a picture or movie clip of a certain moon phase, and then be prompted with possible responses. At that point, if you know the correct answer, shout it out--because the loudest answer is the correct answer...

Which phase is this--full, or new? (Yes, this one is mean to be easy.)

Which phase is this? (Look carefully at the logo--Twilight: New Moon is not a new moon!) Crescent, quarter or gibbous? Waxing or waning?

Crescent, quarter or gibbous? Waxing or waning?

Crescent, quarter or gibbous? First or third?

Crescent, quarter or gibbous? Waxing or waning?

Crescent, quarter or gibbous? Waxing or waning?

Now let's watch a time-lapse movie of the moon going through its phases. Note that the terminator--the border between the lit and dark sides of the moon--always sweeps across the moon from right to left. When a given phase happens, you all say "now." Synchronize! (Hat tip to Edward E. Prather et al., Center for Astronomy Education, University of Arizona for this activity.) You know that person that keeps saying "meow" instead of "now?" Don't be that person.

Second, let's get to our Earth-moon diagram model of phases.

Consider the origin of the word "month." A "moonth" is the time for the moon go through its phases, which is approximately four weeks, or...a month.

The different phases are caused by the moon revolving around Earth, and by convention the "moonth" starts with new moon in this simplified diagram (not to scale), with the sun far off to the right.

This is another synchronizing activity, so when a given phase happens, you all say "now." Synchronize!

Simultaneously, while the moon revolves around Earth, Earth is also rotating on its axis. This simplified diagram will have 12 hours of day an 12 hours of night, and the observer on Earth (looking down on the north pole) by convention will start at 12 PM (noon). We'll only consider eight key times of the day (followed by 3 PM, 6 PM (sunset), 9 PM, 12 AM (midnight), 3 AM, 6 AM (sunrise), 9 AM).

One last synchronizing activity--when a given time of day occurs, you all say "now." Keep in mind that the rotation of Earth and the revolution of the moon occur at the same time, and this is what makes the moon rise and set at different times of day, depending on its phase. We'll practice using this Earth-moon diagram on an in-class activity to determine these patterns.

Back to Dr. Evil in our opening movie clip. What are some things that you notice?

Astronomy in-class activity: SETI and BETI

Astronomy 210 In-class activity 24 v.13.06.11, fall semester 2013
Cuesta College, San Luis Obispo, CA

Students find their assigned groups of three to four students, and work cooperatively on an in-class activity worksheet to discuss how messages are anticoded, and experience practical issues with incommensurability barrier by attempting to analyze content in a fictitious message from an extraterrestrial advanced technological civilization.

Start out with going over the original Arecibo message (and presumptive crop circle reply in Chilbolton, England, 2001) in a whole-class discussion. Point out to or ask students about the key features of these messages: counting (binary); height and population of inhabitants; schematic representation of planetary systems and communication devices (radio dishes and supposed crop-crushing apparatus).


With these clues, students then start working in their groups.



This is a truncated version of a much longer in-class activity which also involves counting systems and arithmetic operations used in anticoded messages.

Astronomy in-class activity: compact objects with close-transfer binary companion stars

Astronomy 210 In-class activity 19 v.13.06.11, fall semester 2013
Cuesta College, San Luis Obispo, CA

Students find their assigned groups of three to four students, and work cooperatively on an in-class activity worksheet on the features of compact objects (white dwarfs, neutron stars, and black holes) with close-transfer binary companion stars.

Astronomy in-class activity: stellar evolution stages

Astronomy 210 In-class activity 19 v.13.06.11, fall semester 2013
Cuesta College, San Luis Obispo, CA

Students find their assigned groups of three to four students, and work cooperatively on an in-class activity worksheet on the different evolutionary tracks of different mass stars.