Gold Apple Software Web Development by Geoff Appleby

QuackBack Peer Review

Quackback is on online system to faciliate the implementation of a peer-evaluation methods within universtiy classes, led by Professor Patricia Lasserre of The University of British Columbia Okanagan.  The project started with a student development group in early 2009 building the first version of the system, which was then used within an introductory computer science class the following fall.  I participated in a second student group in 2010 implementing additional features based on instructor and student feedback.  Both student groups operated as part of the Comptuer Science 319: Software Engineering Project course.

Group Learning and Peer Evaluation

Students are divided into small groups and work together on in-class assignments and other projects through the duration of the course.  Groups are chosen to place students of varying skills within a single group with the goal of providing peer assistance to less experienced students.

At several times through the duration of the course, students are required to complete evaluations of their team members.  These evaluations focus on measures such as how actively they participate in group work, how much they attempt to help other team members, and whether they devote comparable amounts of time (e.g. showing up to classes, working on group projects outside of class).  The resulting grades from these questionnaires are combined with the students other marks from individual and group work to compose their final course grade.

During Dr. Lasserre's initial implementation of this method evaluations were completed in class on printed forms.  Challenges encountered included the impact on instruction time needed to facilitate students completing the questionnaires, concern of bais being introduced by students not being able to complete the questionnaires in a private setting, and the substantial amount of time required by the professor to enter student responses and calculate grades within a spreadsheet.

Quackback allows students to complete questionnaires on their own time, and aggregates the data in easily reviewable reports for the instructor, greatly reducing the time required.

Implemented Features

Grading Formula

The initial version of the system was implemented based on an initial questionnaire used by Dr. Lasserre, and had a hard-coded evaluation function to determine grades.  One of the improvements I undertook was the implementation of a parser and evaluator in order to allow instructors to provide a customizable evaluation function for each questionnaire.  An important factor was to ensure that a user could not input a malformed function, and provide helpful error messages to indicate to a user if their function will not work.

Improved Reports

One significant issue discovered through the use of the system in fall 2009 was the difficulty in determining which students had not completed a questionnaire, and resulted in the creation of a report to provide such information.  With this data the system can be also be configured to penalize students who do not complete a questionnaire, and the instructor can optionally override such penalization for individual students in extenuating circumstances.

Some students expressed concern that a disgruntled team member may negatively affect their grade.  The result was the creation of reports to view a students given and recieved rankings, and optionally remove a students rankings from affecting their team members' computed grade - a task that would previously require manual intervention and computation by the instructor.

Due to the large amount of data that results (a class of 100 students, in groups of 5, completing a 5 value questionnaire, would create 10000 data points) the database structure required improvements to support these reports, and several iterations of the SQL queries were required to ensure that reports were generated in a reasonable amount of time and would scale to larger datasets without issue.

Deployment

Phing was used to run unit tests, generate code coverage reports, generate HTML documentation from PHPDoc, and provide build archives from source control.

As a requirement of the course in which the project was being developed, unit test line coverage was improved substantially with numerous small bugs discovered and remedied in the course of these efforts.

WCCCE 2010

During the Western Canadian Conference on Computing Education 2010 I co-presented the application and results of its implementation, receiving positive interest from many attendees.

Class Overview
Questionnaire Creator
Student Form
Questionnaire Results
Student Results
Responses Submitted by a Student
Responses Received by a Student
Missing Responses
Class Statistics
January, 2010 - April, 2010