## 20080123

### Education research: feedback on clickers (Cuesta College, Fall Semester 2007)

During Fall semester 2007, Cuesta College students taking Physics 5A (college physics, algebra-based, mandatory adjunct laboratory) at Cuesta College, San Luis Obispo, CA used numerical keypad clickers (Classroom Performance System, einstruction.com) to enter homework and to engage in peer-interaction discussion questions during lecture.

During the last week of instruction, students were given the opportunity to evaluate the instructional components of the course, and the use of clickers in an online "Learning Resource Survey" hosted by SurveyMonkey.com. Questions from section II are adapted from the Student Assessment of Learning Gains (SALG) survey (developed by Elaine Seymour, Wisconsin Center for Education Research, University of Wisconsin-Madison), and questions from section III (III.1, III.3, III.5, and III.7) were adapted from a "Clicker Attitude Survey" (N. W. Reay, Lei Bao, and Pengfei Li, Physics Education Research Group, Ohio State University).

These are the complete survey results, with some preliminary commentary. No statistical analysis was done (this was the first and only administration of this instrument), but will be forthcoming after more data has been compiled from future semesters. Values for the mean and standard deviations are given next to the modal response category for each question. Note that the order of questions within sections II and III were randomly scrambled for each student.
`Learning Resource SurveyCuesta CollegePhysics 5A Fall Semester 2007 sections 0906, 0907(N = 35)I. In order to receive credit for completing this survey,    first enter your four digit ID number below:   ____II. How much did each of the following aspects of the class help     your learning?II.1 Lecture by instructor.1. Strongly disagree      0 : 2. Disagree               4 : ****3. Neutral               11 : ***********4. Agree                 15 : ***************   [3.6 +/- 0.9]5. Strongly agree         5 : *****     II.2 Doing assigned homework, to be entered using clickers. 1. Strongly disagree      1 : *2. Disagree               2 : **3. Neutral                7 : *******4. Agree                 19 : *******************   [3.7 +/- 0.9]5. Strongly agree         5 : *****II.3 Doing unassigned homework.1. Strongly disagree      1 : *2. Disagree               6 : ******3. Neutral               14 : **************4. Agree                 11 : ***********   [3.3 +/- 1.0]5. Strongly agree         3 : ***II.4 Using clickers to participate in class. 1. Strongly disagree      0 : 2. Disagree               2 : **3. Neutral                8 : ********4. Agree                 12 : ************   [4.0 +/- 0.9]5. Strongly agree        13 : *************II.5 Reading the textbook.     1. Strongly disagree      0 : 2. Disagree               8 : ********3. Neutral               10 : **********   [3.4 +/- 1.0]4. Agree                 12 : ************5. Strongly agree         5 : *****II.6 Demonstrations/videos in class.      1. Strongly disagree      0 : 2. Disagree               1 : *3. Neutral                7 : *******4. Agree                 18 : ******************   [4.0 +/- 0.8]5. Strongly agree         9 : *********II.7 Interacting with other students during class.   1. Strongly disagree      3 : ***2. Disagree               7 : *******3. Neutral                9 : *********   [3.2 +/- 1.2]4. Agree                 12 : ************5. Strongly agree         4 : ****II.8 Interacting with other students outside of class.1. Strongly disagree      7 : *******2. Disagree               3 : ***3. Neutral                7 : *******   [3.3 +/- 1.5]4. Agree                  9 : *********5. Strongly agree         9 : *********`
Students rated the usefulness of learning by using clickers in class (II.4), more so than any other instructional mode except for demonstrations/videos in class (II.6). Students rated doing assigned homework that was entered using clickers third highest in terms of learning usefulness, more so than lecturing by the instructor (II.1), reading the textbook (II.5), doing unassigned (but suggested) homework (II.3), and interacting with students outside (II.8) and during (II.7) class.
`III. Answer the following statements which may or may not describe your beliefs about the use of clickers in this class.III.1 I like using clickers. 1. Strongly disagree      0 : 2. Disagree               1 : *3. Neutral               12 : ************4. Agree                 15 : ***************   [3.8 +/- 0.8]5. Strongly agree         7 : *******III.2 Clickers helped me understand lectures better.   1. Strongly disagree      0 : 2. Disagree               5 : *****3. Neutral               11 : ***********4. Agree                 11 : ***********   [3.6 +/- 1.0]5. Strongly agree         8 : ********III.3 I would recommend using clickers in future semesters of Physics 5A. 1. Strongly disagree      0 : 2. Disagree               3 : ***3. Neutral                9 : *********4. Agree                 14 : **************   [3.8 +/- 0.9]5. Strongly agree         9 : *********III.4 I will avoid other classes using clickers in future semesters. 1. Strongly disagree     13 : *************2. Disagree              15 : ***************   [1.9 +/- 0.8]3. Neutral                6 : ******4. Agree                  1 : *5. Strongly agree         0 : III.5 Clickers were a positive experience.    1. Strongly disagree      0 : 2. Disagree               1 : *3. Neutral               13 : *************4. Agree                 15 : ***************   [3.7 +/- 0.8]5. Strongly agree         5 : *****III.6 Too much time in class was spent using clickers.   1. Strongly disagree      5 : *****2. Disagree              21 : *********************   [2.2 +/- 0.9]3. Neutral                6 : ******4. Agree                  2 : **5. Strongly agree         1 : *III.7 Too many clicker questions were asked. 1. Strongly disagree     10 : **********2. Disagree              16 : ****************   [2.0 +/- 0.7]3. Neutral                9 : *********4. Agree                  0 : 5. Strongly agree         0 :      III.8 Clickers should be used to collect assigned homework. 1. Strongly disagree      3 : ***2. Disagree               3 : ***3. Neutral                8 : ********4. Agree                 15 : ***************   [3.5 +/- 1.1]5. Strongly agree         6 : ******     III.9 Using clickers was difficult. 1. Strongly disagree     15 : ***************2. Disagree              16 : ****************   [1.7 +/- 0.8]3. Neutral                3 : ***4. Agree                  1 : *5. Strongly agree         0 :`
Overall, the responses in section III are positive towards the use of clickers. Although students collectively approved of the use of clickers to collect homework, a vocal minority expressed their comments below.
`IV. (Optional.) Please type in any comments you may have regarding the use of clickers in Physics 5A.`
The following are all of the student responses to this question, verbatim and unedited.
`"These are comments involving class overall: It would've been helpful if the problems done in class and assigned had keys to compare our answers to.. even if not turned in for points.. because when we're trying to grasp a concept and learn the correct way to solve a problem, it's nice to know if we're doin it the right way @ that moment, and not way later on. There's a lot of problems to learn so it's impossible to ask about every single one during class. The clickers made class feel kind of rushed when we were trying to learn something new right away, which is hard to learn if it's being rushed during class w/time limits and also while jotting down notes, listening to the instructor, and coming up w/our own analysis of how to approach a problem. Also, the quizzes were REally rushed and even if we all studied hard, it still felt like not enough time. A lot of the students and I went to the tutoring sessions all semester and still couldn't do as well on the quizzes. And WE TRIED to do what we could to be quicker and accurate!""At first, I was mostly deterred by the additional cost incurred. However, the ease of homework entry and the immediate assessment of it is truly of benefit, especially in such a calculation based class. The fact that the instructor could, during class, ask a clicker question and assess the class' general understanding level actually did benefit classroom experience and learning process. However, the explicit use of such an object would incline me to desire an additional half an hour be added per hour class. I feel that in such a subject as introductory physics (non-calculus), the relationship to real life experience is more than important, it is essential. When I sit at the bus stop and cars drive by, the Doppler Effect comes to mind. The more emphasis on experiential physics, the more the student lives the subject, rather than understands it. This is the ONLY subject when this is possible, and the experience is transcendent, rather than educational.""It was a good idea.""Overall, I liked using the clickers. Specifically I thought they were very useful and effective in class when we used them toreinforce a new lesson. We would learn a new concept and then a few clicker questions would be presented to see if we understood that concept. That was by far the best use for the clickers. The were good for Homework as well, but at first I was little annoyed that i had to register them, pay extra for them, etc. But I would definitely recommend them for future Physics 5A class.""there were a few times when time wasn't called so i missed some points.  Just remember to stop time and announce 5sec.""Clickers were definitely helpful.""I would say that for the clickers to be more effective in teaching this class, a longer class period would be needed. I could see the clickers being great tools to finding out what needs to be reviewed from homework or concepts that the majority of the class has missed but with the time in class was not long enough to allow the teacher to utilize this. In my opinion I believe that the clickers where unnecessary for such a short class.""The fact that we just entered our answeres and were never told how to completly solve the problems that the greater part of class failed to answer was irritating. Also because we were entering answers not work we could never recieve the answer via solutions manual or the back of the book to figure out how to do the difficultproblems on our own.""P dogg! good job this semester. thanks for making me laugh. you should use clickers again. They were motivating to come to class."`
(N.b. "P-dog(g)" is the students' nickname for the instructor.)
`"Didn't like them being used to answer homework questions. You either got them right or you got them wrong. I found myself getting little to no credit for homework I spent over a hour on. The effort you put in goes un-rewarded.""The clickers worked well for in class participation, but they were a hassle for collecting homework in my opinion.  Have a great day!!""It would also be helpful to maybe go through the assigned homework briefly after the answer have been submitted.  I don't think it is necessiary to go through the whole problem just maybe what equation was used.""Clickers should not be used to collect homework because a lot of the time I was able to complete the problem but because of a calculator error I received the final wrong number. So it was at times unfair. We the students spent the time doing the homework but some were never able to get the points they deserved. Also I found that the problems you did in class were usually much easierthan the homework. It can very discouraging when you feel totally overwhelmed when you sit down to do your homework. Besides that it was good to use clickers for participation points I thought that was very helpful.""While the clickers were a new and engaging way to learn, I did not feel they were worth assigning even problems in the book that I could not check the answers to myself while doing the homework. This lead to a cycle of not knowing whether or not I'm doing the problems right and having to wait until the next class session to get the right answers. All in all, I'd say lecture clickerquestions are great, but the clicker homework questions for credit kind of made the course more difficult.""Maybe an explanation of solution in detail for the homework questions that are due the same day after the clicker questions are answered for problems that the class thought were especially difficult.""I really liked using clickers.  It made class fun, and it was helpful getting instant feedback about the class's progress and understanding of the material.  I really wish we could use clickers in my other science-based classes (like Chemistry.)"`
The common threads in these student comments were (1) using clickers during lecture to discuss concepts and problem-solving strategies was received positively; and (2) that assigning even-numbered problems from the textbook (which have no solutions), to be entered using clickers was rated negatively, apparently due to the lack of feedback and guidance, and the arbitrary binary nature of having answers graded correctly or incorrectly. To address this concern, in subsequent semesters assigned homework problems could instead be made conceptual, proportional or ratio analysis extensions to the odd-numbered problems from the textbook (which would have solutions such that students can check to get some feedback before moving on).

Previous posts:

#### 1 comment:

Anonymous said...

Hey P-dog,

Once again you make us old school instructors look bad! Seriously, a very interesting survey and I think you've demonstrated again that proper use of the clicker is worth the extra time and cost. I do agree with the student view that it is best in a longer class. Anyway, I'll try again this semester and see if I get it right!

Signed

geonerd (can you guess?)