Attendance: S. Tulin, J. O’Brien, S. Lotterer, Y. Widenor, M. Djuth, M. Gallimore, D. Drew, C. Seymour, S. Clark, M. Wood, and M. F. Astiz
The meeting was temporarily relocated to a classroom on the first floor of Old Main due to the scheduling conflict with the interview with the Vice President of Student Affairs. No one was hurt in the transition from the President’s Board Room.
The main topic of the meeting was the discussion of finding an alternative to Scantron.
Scott Clark began the discussion by outlining the issue. Presently, the college rents a Scantron reader and pays a license for the analysis software. Scott’s office processes the Scantron forms and returns the grades and analysis to the instructor.
The rental contract is up for renewal in Spring 2020. The current cost os about $5,000 to rent the machine, $2,000-3,000 for a maintenance contract, and a few hundred for the software. He suspects that the prices will increase with the new contract since the current machine is old and will most likely be replaced with a new model and the software will be upgraded to Windows 10 compatible. When he puts together his office’s budget, he will have to justify the cost.
He wants to try an open-source alternative in place of Scantron. The advantages are
- software is free (open source).
- no need for Scantron forms. The new system scans in the answer sheets with our networked copiers. The software uses image recognition to find the bubbles. The Scantron forms cost 12 cents each and are purchased from the print shop.
- test sheets can be customized by the instructor with the number of questions (up to 150 per page). Logos and graphics can also be added. There are templates in Microsoft Word.
- Software provides analysis of results and exports to Microsoft Excel.
- Open source system has not been tested with needs of the instructors.
- Learning a new system.
Scott is looking for volunteers to test the open source alternative before Spring 2020. If the faculty are satisfied, he will implement it. If they are not, he will rule to keep the Scantrons. With either system, his office will continue to process for the instructors.
The meeting was opened up for a discussion.
Tulin asked about the specifics of the budget and costs. For ITS, it is about $8,000-10,000 per year. Scott said that Scantron will most likely up the rates by justifying a new machine and new software. Scantron sheets are 12 cents each compared with 1-2cents for printer paper.
Aside: half of the printer paper in the Old Main printer is not actually printed. It is taken out of the drawers before printing.
Astiz said that online D2L quizzes and tests are too difficult to work with and are not an alternative.
Gallimore suggested a trivia night to try to break the open source alternative.
O’Brien suggested that BIO and ABEC would be willing to try the open source, possibly starting this semester.
Astiz asked about training. Scott said that his office will continue to process the answer sheets. His office will also train instructors in using the alternative system.
Scott was asked why student response analysis was no longer offers like it was in the past. He said he was never aware of such an analysis. Tulin said that BIO has a macro from am emeritus professor that may be what the question is referring to. That macro is becoming antiquated.
Some instructors were not able to attend the meeting. Here are their emails:
I’m not comfortable giving an online exam at this point and if you can find alternatives, I’m willing cooperate with alternatives.
I have seen a number of changes to Scantron grading technology, and always designed to save money. I would like ACAC to also consider usage for those of us who still use this mode of grading for some class assignments. I don’t really care about any particular technology so long as I can get multiple-choice exams graded with at least a final weighted score for each student.
I don’t think the college wants to hire TAs to do grading by hand?
Unfortunately I won’t be able to be at ACAC for this meeting and the discussion on the scantrons. I would however like you to communicate my thoughts to the group and I have included my department chair and dean on this email as the decision to eliminate the scantron unit would have an adverse effect on my job performance.
As one of the consistent users of scantrons for my evaluations, I can’t stress the utility of this machine enough for my work. I routinely use the scantron to not only grade but to provide a statistical analylsis of each test and test question. Over the years this has allowed me to evaluate every exam I give and within that, each question I write. I use this to rewrite questions for improved effectiveness, eliminate poorly written questions and overall to develop and assess the learning objectives in each of my classes. To perform such an analysis by hand would simply be too onerous and would ultimately eliminate this important facet of my teaching.
I found the initial shift some years ago at Canisius away from formalized reports from the scantron quite disruptive. In response, I had to spend significant time creating spreadsheets and appropriate formulas that would allow me to perform some basic psychometric evaluations of my exams and exam questions that had been lost. As such, if the scantron unit was eliminated now it would significantly and negatively affect my pedagogy and the assessment of my student’s performance. I know another faculty member in my department who is also a long time user who would be adversely affected as well. This is not to say that alternatives could not be employed, as long as they maintained an output that delineated each student’s performance and the performance on each individual question. The imperative point in an alternative device would be the ability to provide a statistical analysis for the class on each in-class exam, as well as a statistical assessment on each student and each question.
Thank you for asking for input and for taking my own back to the larger group.
I know you received this email from Karl today as well. Before a decision is made to discontinue the use of scantrons or to use an alternative system, the three deans would like to have a conversation about this. Since this would impact faculty, we want to make sure there is broader faculty input than just the ACAC reps and there is clear communication with the faculty.
Gallimore announced the following:
- D2L has a built-in video tool called Video Note. It is useful for short videos.
- Podcasting has started in OM402. Two ABEC students have already started recording.
- Volunteers are needed to test alternatives to D2L. The D2L contract is not renewed for another 4 years. However, course management systems have a tendency to price gouging. If the College has tested alternatives, the college is in a stronger negotiating position to push back.
Next Meeting – April 3, 2019
- technology and computing related issues in the policy revisions
- ACAC award nominations
- ACAC relevance