An electronic immediate feedback assessment technique for use in Team-Based Learning

Gary D. Theilman, Pharm.D.                                                                                                          University of Mississippi School of Pharmacy

An electronic immediate feedback assessment technique for use in Team-based Learning

A “Preventive Medicine and Public Health” course in the spring of the third-professional year has been taught for the past 5 years using the Team-based Learning process.

An overview of the Team-based learning process.   The project was developed for use during the “Group Test” portion.           Source:  http://www.med.wright.edu/aa/facdev/tbl/Introduction

An overview of the Team-based learning process.   The project was developed for use during the “Group Test” portion.          

Source:  http://www.med.wright.edu/aa/facdev/tbl/Introduction

The students begin the session by completing a short (10 question) multiple-choice Readiness Assurance test on the pre-class reading.  Immediately after taking the Individual Test, students break up into teams and re-take the same quiz as the Group Test.  While they are still not allowed to use books or notes, they may now work together to come to a “consensus” answer to each question.

Traditionally, the students have used Immediate Feedback Assessment Technique (IF-AT) scratch-off cards during the Group Test.  These cards are similar to instant lottery tickets.  The test has been set up so that the correct answer for each question corresponds to the location of a printed star that is under one of the scratch-off squares.

  

An IF-AT card showing an incorrect guess for question 2.   If the star is not under the first box scratched off, the team makes a second choice.    They continue to make guesses until they discover the correct answer.   Each incorrect choice costs them points. Naturally, the test must be designed so that the correct answer corresponds to the pre-printed location of the star. Vendors provide different versions of the scratch-off cards so that students can’t just memorize the correct answers from previous weeks. http://www.epsteineducation.com/home/about/

An IF-AT card showing an incorrect guess for question 2.   If the star is not under the first box scratched off, the team makes a second choice.    They continue to make guesses until they discover the correct answer.   Each incorrect choice costs them points. Naturally, the test must be designed so that the correct answer corresponds to the pre-printed location of the star. Vendors provide different versions of the scratch-off cards so that students can’t just memorize the correct answers from previous weeks. http://www.epsteineducation.com/home/about/

By the end of the test, the students have found the correct answers for all the questions.  The instructor collects the cards and assigns a test score based on how many boxes were scratched-off.

We had used the IF-AT cards for several years.  However, we had reached a point where we had run out and would need to obtain more.

At that time, the minimum order for the cards is a pack of 500 (about $85).     These 500 cards are all identical and the stars are in the same locations on each card.   To prevent the students from memorizing the location of the stars from week-to-week, a different card layout must be used each week.   This may necessitate purchasing 4-5 different packs of 500 cards and then rotating which card set is used each week.     Our class had only 10 teams.   For the 9 weeks we had the team tests, we would be using 90 cards.   However, we may need to purchase 2000-2500 cards in order to have 4-5 different layouts available.

Another problem is trying to force the test choices to conform to the pre-printed locations of the stars on the card.   For example, in a question which has a logical order of item choices like

What is the usual value of a year of life used for insurance company estimates?

A.  $25,000

B.  $50,000

C.  $75,000

D.  $100,000

One must either find an item number on the card where the pre-printed answer is “B” or must rearrange the choices to match where the star is printed.  A similar situation occurs with questions where the correct answer is

D.  All of the above

One must find an item on the card where the star is under “D”.   If there are no more “D is correct” items on the preprinted card, the question must be modified.

The TBL model calls for the instructor to assess which questions were giving the class trouble and spend a few minutes reviewing the concepts associated with the question (the Instructor Feedback stage).   The simplest solution would be to collect the cards from the teams and glance over them quickly to see which questions had multiple scratches.   This is not difficult with small classes, but one year we had a class with 42 teams.   “Glancing quickly” through 42 cards to try to determine which questions gave students problems takes a bit more effort.

The objectives of this project were to develop a process that would

  • Allow the instructor to design the test without having to force the answers to conform with the pre-printed IF-AT cards.
  • Avoid the need to purchase hundreds of cards that might never be used.
  • Better assess the level of student difficulty with each question in order to tailor the “Instructor Feedback” portion to better meet student needs.

A web-based application was created that allowed students to electronically ‘scratch-off’ choices in the same way they would with a physical IF-AT card.   

Before the class, the instructor would create a test using whatever questions and whatever item order was desired.   The correct answers to each question were entered into an online database.

 

 

 

 

 

 

 

 

Another webpage was written to generate slips of paper with a shortened URL and a QR code.   These slips of paper were distributed to the student teams at the beginning of the team test.

The students could either use the shortened URL or could scan the QR code to open a webpage that displayed the scratch-off card.

 

The page was designed to display correctly on laptop screens, tablets or smartphones.   Tapping (or clicking) on a choice would display an animation of the choice “turning over” to reveal either an “X” or a “Checkmark” on the other side.  Sliding (or clicking) the edge of screen would move from question-to-question.

 

Students discussing which answer to choose during the “team test”. The team-based learning model lends itself well to the use of lecture halls.   Students are assigned to sit with their teammates.   During the team portions of the class, the students merely turn in their seats to interact with each other.   The animation of the choice “turning over” was set to be slow enough that there is a bit of suspense as the student waited to see if they got a “check” or an “X”.

Students discussing which answer to choose during the “team test”. The team-based learning model lends itself well to the use of lecture halls.   Students are assigned to sit with their teammates.   During the team portions of the class, the students merely turn in their seats to interact with each other.  

The animation of the choice “turning over” was set to be slow enough that there is a bit of suspense as the student waited to see if they got a “check” or an “X”.

Another team is accessing the scratch-off card using a cell phone.  The quiz questions are on the pink pages.

Another team is accessing the scratch-off card using a cell phone.  The quiz questions are on the pink pages.

A closer look at how the page appears on a MacBook.   The slip of paper with the QR code is on the desk.