Search This Blog

Thursday, May 25, 2017

HC insists on CBSE Moderation of Marks

CBSE has been increasing marks obtained by students for the last several years. People interested in knowing the details of this must visit this blog. Officially the claim was that "moderation" was to ensure that students don't suffer because of an ambiguous or poorly designed question, an unusually tough paper, etc., and also the marks distribution across the years should be comparable. In reality, there is no evidence that CBSE has ever produced that the increase in marks were related to any of these. People have been quoted as saying that marks were increased to ensure that Delhi students have a  fair chance of admission to colleges of Delhi University. So you could have attempted questions worth 70 marks, but you may actually receive 80 marks.

All this was accepted by everyone in Delhi as long as CBSE was inflating more than others. And usually, CBSE was inflating marks of Delhi students more than marks of outside Delhi students. To enable this difference, CBSE would actually prepare a different paper for Delhi and a different one for the rest of India, and could claim that Delhi students will have larger increase since Delhi paper was more difficult. No evidence of difficult was ever deemed necessary (since the goal appeared to be admission to DU).

Last year, we suddenly had a large number of students from Tamilnadu board take admission in SRCC, one of the premier colleges of DU. It was in national media, and it was felt by many that from this year onward, students from TN board (and other boards with large number of high scoring students) would seek admission in large numbers. The situation is explained nicely by this blog.

CBSE had two options: One, to continue playing the game and inflate marks even more to compete with some of the worst boards of the country, and reduce its own reputation and quality in the bargain. Two, to admit defeat, cry in front of MHRD that others are beating them in their own game, and try to put pressure on all boards to not inflate marks. The third option of going back to being an honest board, and let the universities worry about the fairness in their respective admission processes, perhaps did not occur to the leadership.

So a meeting was called of all the boards and most of the boards decided that there will be no moderation from this year. Here in lies the difficulty. CBSE has never admitted to increasing marks arbitrarily (except grace marks for increasing the pass percentage). Indeed, no board has publicly admitted to this. Everyone claims that what they do is moderation. Not, it is obvious that moderation is the good aspect of conducting an exam, and must continue. And what was going on for last 10-15 years was not moderation but arbitrary increase of marks.

If officially, there was no arbitrary increase of marks, then how can a board say that we are stopping such a practice. The boards could not publicly admit that they were falsifying the marks all these years, and this practice of falsification will now stop. So they used euphemism and said that they will stop moderation. We were all supposed to interpret it as there will be no falsification of marks, and we did. However, this euphemism did not pass the High Court test.

A parent took the matter to the Delhi High Court, and Delhi HC correctly ruled that moderation is an important and necessary part of conducting a large public exam and without this, there may be serious repercussions on the students. Note that Delhi HC has not said that falsification of marks must continue this year. It has only said that moderation must continue this year. The final verdict will come later, but the chances are that the court will insist that moderation continues even in future. (The court even quotes Supreme Court on this.) So the ruling changes nothing for CBSE, and CBSE could have declared the result saying that they did consider moderation after the HC ruling but there was no need.

But I have a feeling that CBSE would continue its policy of falsification of marks this year too. They have been pretty desperate to find ways to ensure that a large number of DU seats are taken by CBSE students. They even wrote to Delhi University Vice Chancellor to give appropriate weight to CBSE marks in the admission process, since other boards were inflating the marks. This was bound to be rejected by DU, and they did. After the result of TN Board was declared the desperation in CBSE only grew, and the HC interim ruling has come as a huge relief to CBSE. Now, they can continue to falsify marks to ensure a greater number of CBSE students in DU.

I find some of the reactions to the Court ruling very interesting. The foremost is that the courts have saved student careers, because without the increased marks, students would have faced serious problems. And why would they face problems. Because other boards increase marks arbitrarily. OK. So let us accept that other boards do it. What is the solution. Is a race to the bottom a solution to this problem. Should people be shouting at CBSE and insisting that they falsify marks. Shouldn't the job of a board be to show you the mirror, give you an honest feedback. Falsification of marks can only lead to (and already leading) erosion in quality of education. If marks are falsified by other boards, we must go to the universities and tell them not to use falsified marks. May be have standardized tests, or their own admission tests, or something else. But how many people will write to DU versus how many people will write to CBSE. Forcing CBSE to cheat is easier than forcing DU to do the right thing. Everyone in India understands that asking someone to do wrong is easier than forcing someone to do right. And hence everyone is after CBSE and not DU.

Let us restate this argument. I have seen students getting undeserved marks for the last 15 years. So I planned my future education and career based on the assumption that I too will enjoy undeserved marks. If I was told one year ago that undeserved marks will be removed, I would have planned something over this one year. So CBSE must roll back the decision of not giving undeserved marks.

This is dangerously similar to the following. Over the last 15 years, I have seen lots of people getting cash which is undeclared to income tax. So I planned my future finances based on the assumption that I too will enjoy my ill gotten wealth. If I was told one year ago that all cash will be demonetized, I would have planned something over this one year. So RBI must roll back the decision to demonetize.

Just imagine the chaos that would have happened if courts had agreed to this argument last November.

And of course the latest news is that CBSE might go to Supreme Court.  This is stupid. The HC has only said that you must have moderation. It has not said that you must have falsification of marks. Falsification was never an announced policy anyway. So just quietly stop falsification, take the high moral ground, and force people to complain to Delhi University. The problem is of DU. They are the ones accepting falsified marks. Why should CBSE care if its competitors are corrupt or not. Why should CBSE care that the corrupt practices of its competitors are hurting its own customers. As a government board, it can just start doing the right thing and not do corrupt things itself to counter the corrupt practices of others.

Thursday, April 20, 2017

Merit-based admissions

People who do well in JEE Advanced will mostly tell you that merit is a one-dimensional beast, never mind that it consists of performance in three heterogeneous disciplines, mapped into a single number by the simple addition of three numbers without checking if these numbers are comparable or represent something which can be simply added. It is further believed that admission to higher education is a reward for performing well in an exam under the conditions of extreme pressures. And hence the only justified admission policy is to admit students in the decreasing order of merit or performance in that exam (modulo any laws passed by the parliament, but of course, not before they have pointed out that merit has been violated by those laws).

Should a university consider admission as a reward for performance. Or should it be something else.

Well, it has to be something else. In fact, no university will couch its admission policy in terms of rewarding past performance. University goals are varied: Who is likely to do well in our programs? or Who is likely to do well even beyond our program and be successful and enhance our reputation?

If university goals are somewhat in line with what I have mentioned above, then it is obvious that the admission through a single test is not ideal. No one can perhaps argue that performing in a particular test can both be an indicator of future performance in Computer Science, as well as an indicator of future performance in Economics. No one can argue that India has the best admission process since this is the only major country in the world that does not take language abilities into account for admission. No one can argue that the ability to answer MCQs is a strong indicator of the ability to write long notes necessary in any studies.

The reason for following a single exam is that it provides an objective, transparent mechanism while simultaneously measuring one variant of irrelevant merit. In other words, the most important outcome of JEE based admission has been that we have kept out pressure from the rich and influential as well as won most court cases. And by no means, this is a small achievement. In India, one may genuinely consider keeping pressure out and winning court cases as more important property of admission process than getting students aligned with the university goals. However, over a period of time, people doing well in JEE have started believing that the prime purpose of JEE is to evaluate merit, and that there can only be one definition of merit, which is the JEE rank.

Besides the obvious inadequacy of a single number predicting success of all programs in all disciplines in all institutes, there are other problems as well.

If a university asks the question who is likely to do well in our programs, and let us assume (despite absence of any data) that performance in PCM has a strong positive correlation with performance in Mechanical Engineering. What if we were compare the performance of people who have 80% in JEE after intense coaching, whose parents were rich and provided a good study environment whether at home or in a hostel, with the performance of people who have 75% in JEE without any coaching, and who come from modest backgrounds, and are the first generation learners. When both these groups compete, who is likely to do better. Most universities would have data to show that latter group performs better, and hence they would actually create admission policies to offer more admissions to the latter group.

When we point this out, an immediate reaction is, but how do you evaluate deprivation, and what has been the impact of that deprivation, and how much extra "credit" should be awarded to compensate for that deprivation so that we can compare the merit of the two. Now, in this question, there is an inherent belief that JEE score is a strong indicator of future performance of every sort. Gold standard of transparency and honesty (?) has been converted to Gold standard of merit. Do you ever question why people with same total in JEE score are ranked differently based on marks in some subject or the other. Does that imply different merit. Isn't it true that luck plays a part. If you had a headache that day, your merit may be way down. That any exam performance is only an indicator within a significant band is conveniently forgotten. But when it comes to any affirmative action, we want exact data, why 5% and not 4%. Why not ask data about JEE also.

Let us do another thought experiment. Again, let us assume that PCM performance is an indicator of future performance in all programs. Consider two possible admission decisions. One, take the top 100. Two, take the top 90. Study whether there is sufficient diversity in the class. If certain backgrounds are missing, let us fill the last 10 seats through them. Now, there is a belief (and educationists may even have data, I am not one) that diversity is good for education. That diverse inputs will cause more innovative projects. That having diversity in your peer group in college will prepare you for a much more diverse workplace that you are likely to face. So both performance within the university and success beyond the university is likely to be enhanced if the class is more diverse. Because of these reasons, universities encourage student exchange programs besides having diversified student admissions to begin with.

Let us for the sake of argument assume that there is data to show that diversity helps. What should a university do? Should you admit 100 students whose average performance will be 60, or should you admit 90 students whose average performance will be 70 (helped by diversity) and 10 students whose performance would perhaps be only 50. There is certainly a plausible argument for not admitting strictly through merit (if at all merit can be determined by a single number). But ask someone who was 91st in our imaginary list. Will he agree. If you assume that admission is a reward for performance in a particular test, you will not like what the university is doing.

Universities may have other goals. For example, should we admit those who are the smartest and will benefit very little from university education, or should we admit those who are likely to gain a lot more from university education. So on a scale from 1 to 10, should we admit students at 9 who can be helped to reach 9.5, or should we admit students at 7 and take them to 8.5. In other words, should "merit" be the only criteria for admission or should we also look at what will benefit the society most. This question is particularly important when the society is funding the cost of that education. Again, I know what the "merit" crowd will say, but I will only suggest that most universities around the world will admit both kinds of students in some proportion. (We also are mandated by law to do affirmative action, which means admitting students at lower "merit" but they are likely to benefit the most from our education.)

A lot of universities have a stated goal of helping the society that nurtures them. One way to do this is to prepare entrepreneurs who will create jobs in that society. Depending on the geographical location (and particularly so, if the location does not attract a lot of investments), universities may prefer to have some weight to the nearness criteria, since students belonging to that society are more likely to stay there beyond graduation. A lot of universities will, therefore, admit more students from within the state even when that magic number representing merit is lower for these students.

In summary, the insistence on admitting based on a single number is misplaced. As long as the single number is considered a proxy for transparent/honest process, the insistence has a logic, but when this number starts getting treated as a proxy for merit, there is a problem. On the other hand, it is necessary for public funded universities to not only admit the best, but also give an appearance of admitting the best. As long as public universities like IITs have a transparent/honest process, they should have flexibility in both defining the merit (as combination of multiple parameters, perhaps), as well as bypassing merit to achieve the multi-objectives of  the university.

Sunday, April 16, 2017

Improving the gender ratio in IITs

Recently, there have been media reports on a decision taken by Joint Admissions Board (JAB) of IITs where by IITs would create some extra seats for women so as to improve the gender ratio in the under-graduate programs. Two of the reports are here and here. Earlier, JAB had asked a committee headed by Director of IIT Mandi, Prof. Timothy Gonsalves, to look into the ways of improving gender balance in IITs. This decision is apparently one of the recommendations of the committee.

Though the details are sketchy, it seems that there is a goal of having at least 20% women in the under-graduate class in stages. For 2018 admissions, the goal has been set as 14% which will increase by 1% every year to reach 20% in 7 years. In recent years, number of girls admitted to IITs number around 9%. To achieve a 14% ratio in 2018, they will have to increase the number of seats by 6%, and all these 6% will be filled exclusively by women candidates.

A little over a year ago, I had suggested in this blog that we must do some research into why women are not getting selected in larger numbers despite their performing extremely well in Physics, Chemistry and Mathematics in class 12th and also they perform well after they get selected in engineering. My own contention is that there is an inherent bias in the society which restricts the coaching options for women. The fraction of women in Kota coaching classes is very small, for example. Even in larger cities, the fraction of women in JEE coaching is less than the fraction of women in science sections in schools. Sometimes, it could be lack of willingness to pay high amounts for a girl child. Or it could be the concern for their safety.

I am not sure what data the committee looked at, but apparently they did find that part of the reason for lower number of women is societal bias, and hence that bias needed to be compensated by some mechanism. And what other mechanism do we know of in this country but reservation.

Is reservation the best mechanism to achieve gender balance?

The obvious shortcoming of reservation or a quota system is that its benefits are not directed towards the disadvantaged class but a larger class. The additional seats may not all go to those women whose parents have refused to send them to outstation coaching, or even to a more expensive coaching within the city. At least some of the seats will be taken up by women who actually go for expensive or outstation coaching.

But note that this is the problem of reservation based systems in general. Aren't OBC reservations benefiting students whose both parents are well educated and can hardly be called "Educationally Backward." But in India, we always argue that if we don't use simple proxies for disadvantages faced by people, the whole system will be gamed by rich and influential. In case of simple proxies (like caste for socially and educationally backwardness), some non-deserving people may sneak in, but it helps those who need such help. Something similar is likely to happen with women reservation as well. A few non-deserving women will get admission, but overall, it will help compensate the societal bias to some extent.

I think something more interesting may happen here. Once the parents know that getting admission to IITs is somewhat easier for women, they may actually be more willing to get them coached. Currently, one of the reasons for not investing in their coaching is that the chances of success are so low, and the expected return on investment is consequently low. They are willing to invest in sons' coaching even with that lower expected return, but will invest in daughters' coaching if the expected return is higher. As a result, a greater percent of women may succeed in the admissions process on their own. My gut feeling is that the percent of women in the normal process will keep increasing and they will need only 5-6 % supernumerary seats even as the goal improves from 14% to 16% and all the way to 20%. And because of this hope, I am positive about the reservation.

Are there other methods that they could have used to increase women admission?

Absolutely. JNU has had the scheme of deprivation points in their admission process. Under this scheme, they would add a few points to the other pieces of evaluation based on some criteria of background of the candidates. One of the criteria is gender, and a small benefit accrues to female candidates.

IITs could do something similar. They could increase the marks of every women candidate by some small number in a way that in the top 10,000 ranks, there are exactly 1400 women. And now women have ranks based on this new marks. One could a priori decide what is the maximum number of marks to be added, and if to ensure that there will be 1400 women out of 10,000 ranks require a higher number of marks to be added, then we will still stick to the maximum marks. (And, of course, we would know how many of them were in top 10,000 before these extra marks, and how many have been added, and create that many more seats to satisfy the current policy of not reducing the number of seats for categories not part of new reservation.)

Of course, it is easy to expand this mechanism to implement all sorts of reservations, and we will have data on exactly how much difference there is between various categories. So we could increase the marks of all SC students in a way that there are exactly 1500 of them in the top 10,000 (subject to the maximum number of marks, note that even now there is a limit on how much lower we will go in the merit list).

This mechanism is very useful when you have very small reservations, for example, in case of Physically challenged students. A PH-ST student is competing for a 0.2% quota (3% of 7.5%), which means that in a large number of programs, there will be 0 reserved seats in any given year. But if you add enough marks to their score that they represent 3% in 10,000, they will be able to seek admission to any seat that they deserve at their performance level.

This mechanism will be useful if we want to compensate for any other bias or discrimination or deprivation that candidates have faced.

Of course, what we are arguing now is that a 15% reservation over 10,000 seats means that the reservation is overall and not in each program. This would mean that they may get slightly less than 15% in some programs and slightly more than 15% in some programs. We only need to make sure that the distribution of marks are such that it won't lead to very high or very low presence in the popular programs (which I suspect will not happen). I don't know how courts will look at it, but it is worth trying.

Can we improve gender balance without any affirmative action?

That would be the least controversial and best method, in general. But that would require a lot of research, and we normally want to solve the problem without doing research. For example, if the hypothesis that lower women representation is due to societal bias and consequent lack of investment in their coaching turns out to have some merit, then perhaps we need to have the entrance exam (or at least some components of it) which are not impacted by such high pressure coaching. Small amount of coaching would be enough. One way to do that is to have speed tests, I am told, instead of very difficult to remember tricks. On top of that, something that government has already asked IITs to do, we can get study material prepared by IITs. And, of course, we are also seeing development of apps where by a candidate can practice for speed tests and get feedback sitting at home, all at a very low cost.

Of course, if the research shows some other reasons behind gender imbalance, we will need to tackle that properly.

Will this lead to more demands of diversifying student population?

Tamil Nadu has a little over 5% population of India, but it does not send 5% students to IITs. Muslims have about 15% population in India, but the fraction of Muslim students in IITs is much smaller. Wouldn't there be demands for increasing their representation.

Of course, there will be. But note one thing. The system proposed is saying that if 50% population does not have even 20% representation then there is something wrong somewhere, and we need to do something about it. So, the goal is not to ensure representation aligned to population fraction. Also, this is the population which seems to be doing much better in pretty much every exam in the country, except JEE advanced. If there are other groups which meet these criteria, that too can be studied.

Summary

It is a difficult decision. But one that I think could lead to attracting better talent by IITs. I am hoping that the "quota" part will remain very small and will eventually go away, and that IITs will implement other ways to attract talent, including changes to JEE.

Added on April 22, 2017:

Prof. Timothy Gonsalves, the Chairman of the committee on improving gender imbalance in IITs has made a posting on his FaceBook wall giving a summary of what went behind the report. I strongly encourage everyone to read that.
 An excerpt from the same:
--------
Would admitting girls with slightly lower ranks compromise on quality at IIT?
 A study in IIT-Delhi looked at the final CGPA of male vs. female students. Over a period of 13 years (2003-2015), females outperformed males consistently by an average of 1 grade point, despite having lower JEE ranks! This amazing finding gels with our experience as teachers in other IITs also. It is an indication that this cohort of young women is extraordinarily talented and highly trained despite the disadvantages of growing up as girls in India.
--------
I am told that the difference in grade between girls and boys of similar JEE ranks at IIT Delhi is a whopping 1.5. It proves beyond a shadow of doubt that giving some push to women whether through quota or bonus marks, or whatever, will actually admit better students to IITs.