Concrete Suggestions for Measuring Teaching Quality in Practice-Oriented Computer Science/Information Technology streams

Last updated on 16th September 2014

At the outset, I request the kind indulgence of readers for blowing my own bugle a little, with the intention of substantiating my case.

Yesterday I received an email from a recently passed out M.Tech. (Computer Science) student (I don't know if exam results have been declared yet but the student is a high-flyer and will, in all probability, pass). He thanked me for my C++ classes which he felt played a significant role in him clearing a job interview and getting a job offer which he intends to take up.

--- Start passed out student message extract ----

I must thank you for getting this offer because my interview was solely based on C++ programming and very little of algorithms. Even the latter was with respect to data structures that are part of the C++ STL. Your C++ classes, and at a later stage your slides were of great help to understand the basics of C++. Therefore, I felt this urge within to inform you about this.

--- End passed out student message extract ----

I would have taught him the C++ course when he was in Ist M.Sc. (Maths) in academic year 2009-10. The department is a Mathematics & Computer Science department. Many of the two year M.Sc. (Maths) program students continue on to do the two year M.Tech. (Computer Science) program. [BTW in March 2012 I parted ways with the above-mentioned department and educational institution.]

Now I would not like to comment much on why the tech. interview (as per the student's description) did not cover algorithms in some depth. IMHO, ideally there should have been a balance of questions on both algorithms and the particular programming language skills they are interested in, which was C++ in this case.

However, the reality of the situation is that the practical programming skills imparted to the student via the lab. course was instrumental in him clearing the job interview and landing the job. So the teacher concerned would seem to have done a good job and would earn the appreciation of academic administrators.

But the way things are run in UGC/AICTE regulated educational institutions in India, with accreditation agencies like NAAC being powerful forces, good teaching quality like in the above case, are not recorded in any fashion in their evaluation metrics. If I am not mistaken, the teaching quality assessment according to UGC/ACITE norms is supposed to be or can be a self-assessment! Naturally all teachers will be assessed as good or excellent teachers then :-) (unless there are many theory exam failures).

The key measure that is recorded is the research publication record. NAAC (the key national accreditation agency for UGC institutions, I believe) seems to be focused mainly on the research publication record of institutions it accredits and grades. So academic administrators have become obsessed with research publication record to ensure that the institutions they administer get a good NAAC rating. This results in a scenario where academic administrators put immense pressure on teachers to publish research papers, and almost ignore promoting excellence in teaching quality.

Now, in my special case, I had retired from commercial work in Sept. 2002, and was offering honorary service of teaching lab. courses in a 'deemed' university. I was utterly disinterested in an academic career or in academic research (my designation during my stint was Honorary Staff, Honorary Faculty and finally Visiting Faculty). I was only interested in my spiritual career and part of that involved serving students by teaching them what they needed to know using my knowledge base of 18 years international software industry practitioner experience and also studying new knowledge areas required to teach specific lab. courses allotted to me. So my focus was on excellence in teaching lab. courses. I have had the satisfaction of teaching them to the best of my ability, given the circumstances, and receiving the love and gratitude of the students in return. From a spiritual and ex-software industry practitioner/professional career point of view I consider it to be a very successful stint (9 years) of spiritual and industry-quality service.

But from an Indian academic system point of view my question is, will the typical Indian CS/IT academic focus on excellence on teaching lab. courses like the C++ course I taught? IMHO, the clear answer is No (there may be some exceptions, of course - I am talking about the typical Indian CS/IT academic). As his/her bosses are mainly bothered about NAAC grading and UGC/AICTE norms for which the only thing that really matters is research publication record.

Further, lab. courses are looked down upon as unimportant courses which can be handled by junior staff like Teaching Assistants or Research Assistants (Research Scholars). The evaluation of lab. courses are very lenient. Rarely is a student failed and many students get the top grade. Theory courses have paper examinations with external evaluation. That is considered far more important in the (UGC/AICTE) Indian academic system. Theory courses have more credit than lab. courses. [If UGC/ACITE publish statistics about failures in CS/IT lab. courses in the institutions they regulate, I am quite sure it will be a miniscule number. However, the failures in CS/IT theory courses will be, I am quite sure, a not insignificant number, in comparison.]

This is very strange for a very practice oriented field like Computer Science/Information Technology. As the above freshly passed out student interview example clearly shows, the vital skill that is assessed during typical CS/IT job interviews is the practical skill. Of course, theory is important too and ideally there should be a balance between theory and practice. But, IMHO, an administration policy where the teaching of the practice of software development (lab. courses) is relegated to junior teachers and considered unimportant, is a disastrous education policy for a practice-oriented field like Computer Science/Information Technology.

What can we do to rectify the situation? Teaching Quality must be measured even if the measures may have some flaws in it (like the flaws in measuring research contributions). Over time the flaws in measuring teaching quality could be addressed/controlled. I have a few concrete suggestions for measuring teaching quality of lab. courses in Indian (UGC/AICTE) CS/IT academia:

1) Teachers who create course material for a course (as against using course material from other sources) should have their course material reviewed by other teachers just like research papers are reviewed. Appropriate credit should be given to teachers whose course material gets a good review. The review may include student assignment submissions including source code, so that reviewers get an idea of how students are benefiting from the course.

2) Lab. courses should be evaluated as strictly as theory courses. It should have external examiners like theory courses. There should be no hesitation in failing students who fare poorly in assignment submissions and exams. Administrators should examine lab. course result patterns and compare it with theory course result patterns. Very high success rates in lab. courses contrasted by quite different success rates in theory courses should ring alarm bells and invite investigation. If the teachers concerned are found to have done an overly lenient job of evaluation they should be penalized.

Additional suggestions to improve the teaching of the practice of software development, not related to measuring teaching quality, are:

a) Lab. course credits should be at least equal to theory course credits. Further, the number of lab. courses should be roughly equal to the number of theory courses. [I believe, the typical current ratio is 4 to 5 theory courses and 2 lab. courses in a semester, with theory courses having 3 credits and lab. courses having 2 credits.]

b) The practice of relegating lab. courses to junior teachers must be abandoned. Given the vital importance of practice in CS/IT, senior teachers should teach lab. courses.

---

Initially I had suggested student feedback as one measure of teaching quality, as follows:

*) Student feedback about the course should be collected. Industry training institutes heavily use participant feedback to get an idea of the quality of teaching and suggestions for improvement. Academia can learn from them. Yes, there may be some serious concerns like students using it as a weapon against the teacher, and the teacher targeting classes where he/she receives bad feedback. But there surely must be ways to mitigate such issues. Students are the key customers/stakeholders, usually paying customers, of the education system. Teachers in academia must learn to respect this fact and learn to accept criticism from students about their teaching, just like teachers in industry training institutes respect (or are forced to respect) criticism from participants/students of their courses.

--- end suggestion ---

I received vehement criticism from a US CS academic (over two separate mails which he allowed me to share with others). I have shared below the key parts of the mail exchange:

The US CS academic wrote:
It may not have occurred to you but some students punish good teachers for being demanding and for not being soft at grading. I have seen the most outrageous *anonymous* student feedback for a professor who caught many cheating. The easiest way for a professor to get a good rating from anonymous feedback is to lower standards and give higher grades.

My response was: Oh Lord! While I thought that certainly there were possibilities of students using the anonymous feedback as a weapon against the teacher, I did not know that they actually do go that far. You have the experience of such feedback systems in academia - I don't (though I do have industry feedback experience, which I think is a very different ball game). I am quite shocked to know of this. Hmm.

The US CS academic wrote back:
Note that anonymous accusations traditionally are absolutely disregarded or in cases even illegal under Western law. Except, it seems for professors. Disgusting.

My response (slightly edited to give more clarity) was: Thank you so much for bringing out this aspect so clearly. I think an academic who gets targeted via such anonymous feedback/accusations would go through horrifying emotional torture. He/She would feel there has been no justice, no chance for him/her to prove his/her case.  Hmm.

I really need to pause, think hard, and ensure that my blog post does reflect this horrifying abuse possibility. And then explore whether there is a "safe" way to use student-customer feedback as one measure of teaching quality. As of now, I am quite clueless about suggestions on how to prevent abuse of student feedback like the cases you mentioned and how to address/prevent/resolve the 'disgusting' nature of anonymous feedback accusations being considered as a negative measure of teaching quality.

Thanks again for the reality-bite.

--- end mail exchange extracts ---

After learning about the US academic's experience with abuse of anonymous student feedback to punish good teachers I realize that my earlier view that "there surely must be ways to mitigate such issues" may be wrong. I am not so sure any more about ways to prevent or even mitigate such issues in the Indian UGC/AICTE regulated institutions context. [For the 'cheating' example shared above one could ignore feedback from such students but they could influence their student-colleague-friends to give bad feedback.] My experience of teaching in academia is limited to an educational institution with a focus on human and spiritual values where the students were exceptionally well behaved and had tremendous respect for their teachers. Perhaps there is a very real possibility of such abuse of anonymous student feedback to punish good teachers in some other Indian educational institutions.

Direct (non-anonymous) student feedback used as a measure of teaching quality, when negative, has the possibility of seriously damaging the human relationship between student and teacher. So I don't think that can be considered.

Therefore I withdraw the suggestion of using student feedback as a measure of teaching quality.

========================================================

In September 2014 I made a small academic-type article based mainly on the above contents of this post and tried to submit it to a pre-print archive which did not accept the submission stating:

This repository is only for self-contained research results. Your attempted submission does not appear to be a substantive research document. All articles submitted to [pre-print archive name blinded] must be in a format appropriate for publication in a conventional journal. Please find a more appropriate forum.

--- end pre-print archive mail extract ---

I responded to them as follows:

Noted your response. I guess the article is more of a viewpoint/opinion piece.

Thanks.

--- end main extract of my response ---

I have put up the article titled, Concrete Suggestions for Improving Teaching Quality in Computer Science/Information Technology Lab. Courses, here in case you want to have a look.