The first PISA results for India: The end of the beginning

The PISA 2009+ results are the end of the beginning. For the last decade there has been a debate. Some argued the levels of learning inside Indian elementary schools (primary and upper primary) are a national scandal and a threat to the future of India’s society, polity, and economy. Others appeared to believe that the main, if not only, problem with Indian schools was that not enough children attend them and that with more money and more of the same, all would be well. The last five years saw a relentless accumulation of evidence about the crisis of learning. The establishment has tried to deny, deflect, and dismiss the evidence on learning. Eventually the Government of India agreed to participate in the PISA (Programme for International Student Assessment) – but only for two states, Tamil Nadu and Himachal Pradesh – and both sides agreed PISA was the litmus test. The PISA 2009+ results, which are both official and are beyond gain-saying are unspeakably bad. They confirm the worst of what anyone has been saying about the levels of learning in India elementary education.

  • In reading of the 74 regions participating in PISA 2009 or 2009+ these two states beat out only Kyrgyzstan.
  • In mathematics of the 74 regions participating the two states finished again, second and third to last, again beating only Kyrgyzstan.
  • In science the results were even worse, Himachal Pradesh came in dead last, behind Kyrgyzstan, while Tamil Nadu inched ahead to finish 72nd of 74.

But just coming in last (if we can dismiss as a relevant comparator for India a tiny Central Asian state) does not convey the enormity of how bad these results were, as not only was India last, it was far, far, behind its aspirations, both at the bottom and at the top levels of performance.

PISA expresses the levels of performance in two ways, an overall index number and the fraction of students achieving various “levels” of achievement. The PISA index numbers for each subject are scaled so that the typical OECD student is at 500 and the standard deviation across OECD students is 100. The testing of thousands of students allows the results to present not only the average but also the worst (5th percentile) and best (95th percentile) students do in each country/region. PISA also classifies student performance into “levels” that represent different degrees of mastery of the material.

Table 1 compares India’s performance to three groups of countries. The economic superstars have successfully completed the transition from poor to rich economies in just two generations – Singapore, Hong Kong, Korea (China’s only results are just for the city of Shanghai, which are the highest scores of any region tested, but this is too a typical to really be comparable) and India aspires to their sustained success economically. The current super powers are represented by the USA and the OECD average reflects India’s aspirations as a superpower. The rising powers are represented by the BRIC countries of Russia and Brazil which reflect the rise of the emerging markets.

Compared to the economic superstars India is almost unfathomably far behind. The TN/HP average 15 year old is over 200 points behind. If a typical grade gain is 40 points a year Indian eighth graders are at the level of Korea third graders in their mathematics mastery. In fact the average TN/HP child is 40 to 50 points behind the worst students in the economic superstars. Equally worrisome is that the best performers in TN/HP – the top 5 percent who India will need in science and technology to complete globally – were almost 100 points behind the average child in Singapore and 83 points behind the average Korean – and a staggering 250 points behind the best in the best.

As the current superpowers are behind the East Asian economic superstars in learning performance the distance to India is not quite as far, but still the average TN/HP child is right at the level of the worst OECD or American students (only 1.5 or 7.5 points ahead). Indians often deride America’s schools but the average child placed in an American school would be among the weakest students. Indians might have believed, with President Obama, that American schools were under threat from India but the best TN/HP students are 24 points behind the average American 15 year old.

Even among other “developing” nations that make up the BRICs India lags – from Russia by almost as much as the USA and only for Brazil, which like the rest of Latin America is infamous for lagging education performance does India even come close – and then not even that close.

To put these results in perspective, in the USA there has been huge and continuous concern that has caused seismic shifts in the discourse about education driven, in part, by the fact that the USA is lagging the economic superstars like Korea. But the average US 15 year old is 59 points behind Koreans. TN/HP students are 41.5 points behind Brazil, and twice as far behind Russia (123.5 points) as the US is Korea, and almost four times further behind Singapore (217.5 vs 59) that the US is behind Korea. Yet so far this disastrous performance has yet to occasion a ripple in the education establishment.

Table 1: Comparing Indian (Tamil Nadu and Himachal Pradesh) students mastery of mathematics to economic superstars, current superpowers, and rising superpowers
Country/Region 5th mean 95th HP+TN average to comparator average HP+TN average to comparator 5th percentile HP+TN best (95th) to comparator’s average HP+TN best (95th) to comparator’s 95th
Points TN/HP is behind (-)/ahead(+)
Economic Superstars
Singapore 383 562 725 -217.5 -38.5 -99 -262
Hong Kong 390 555 703 -210.5 -45.5 -92 -240
Korea 397 546 689 -201.5 -52.5 -83 -226
Current Superpower
OECD avg. 343 496 643 -151.5 1.5 -33 -180
USA 337 487 637 -142.5 7.5 -24 -174
Rising Superpowers
Russia 329 468 609 -123.5 15.5 -5 -146
Brazil 261 386 531 -41.5 83.5 77 -68
Indian States
Tamil Nadu 241 351 468
Himachal Pradesh 223 338 458
Average of TN and HP 232 344.5 463
Source: PISA 2009 Plus Results, Table B.3.1 for first three columns and author’s calculations.

I have emphasised Mathematics because many believed math was an Indian strong suit. The results for reading and science are similarly bad. Table 2 shows science results in a different format, which shows the proportion of children in various categories of performance. There are three points:

  1. “Below level 1″ doesn’t even have a description as it implies  that so little proficiency is demonstrated it is impossible to
    distinguish from not knowing anything at all. In the USA, even with its socio-economic and racial inequalities and language inequalities and its failing inner city schools, only 4.2 percent are in this category. In HP 57.9 percent of 15 year olds in school cannot be distinguished from not having learned any science at all and in TN 43.6 percent all in this category – ten times as many as the USA.
  2. PISA considers “level 2″ as the minimum level that provides the science competencies that will enable them to participate actively in life situations related to science and technology. Since more than 80 percent of students in both HP and TN
    are level 1 or below this most students in these states have reached age 15 ill-equipped for the century they will face.
  3. While a thin elite that competes for the few highly selective technical institutes are globally competitive, this is a tiny fraction of the population. The estimate of the fraction of TN or HP students at level 6 in science proficiency was zero. Their estimate of the fraction at level 5: also zero. Of course this does not mean there are not such students in these states, of course there are, just that from the samples available in the study the best estimate was so small as to be indistinguishable from zero.
Table 2: Comparison of science proficiency in Tamil Nadu
and Himachal Pradesh to India’s aspirations
Country/Region Below level 1 Level 1 1 Level 5 5 Level 6 6
Singapore 2.8 8.7 15.3 4.6
Hong Kong 1.4 5.2 14.2 2
Korea 1.1 5.2 10.5 1.1
OECD avg. 5 13 7.4 1.2
USA 4.2 13.9 7.9 1.3
Russia 5.5 16.5 3.9 0.4
Brazil 19.7 34.5 0.6 0
Tamil Nadu 43.6 40.9 0a 0a
Himachal Pradesh 57.9 30.9 0a 0a
Source: PISA 2009 Plus Results. Description of levels Table 3.2, percentages Table B.3.4.

1) At Level 1, students have such a limited scientific knowledge that it can only be applied to a few, familiar situations. They can present scientific explanations that are obvious and follow explicitly from given evidence.

5) At Level 5, students can identify the scientific components of many complex life situations, apply both scientific concepts and knowledge about science to these situations, and can compare, select and evaluate appropriate scientific evidence for responding to life situations. Students at this level can use well-developed inquiry abilities, link knowledge appropriately and bring critical insights to situations. They can construct explanations based on evidence and arguments based on their critical analysis.

6) At Level 6, students can consistently identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations. They can link different information sources and explanations and use evidence from those sources to justify decisions. They clearly and consistently demonstrate advanced scientific thinking and reasoning, and they demonstrate willingness to use their scientific understanding in support of solutions to unfamiliar scientific and technological situations. Students at this level can use scientific knowledge and develop arguments in support of recommendations and decisions that centre on personal, social or global situations.

a) In Table B.3.4 these are reported as blank but the estimated percentages in below 1 to level 4 sum to exactly 100 percent. Obviously this not imply that there are exactly zero students in all of these two states meeting these levels but that with the sample sizes assess students of 1616 in HP and 3210 in TN there was insufficient information to create a non-zero estimate.

These results on PISA 2009+, while tragic for what they imply for Indian youth and perhaps shocking to newcomers to this subject, come as no surprise to those who have been working on basic education in India:

  • Das and Zajonc (2008) used results from Orissa and Rajasthan to create indices on mathematics performance similar to those of TIMSS (Trends in Mathematics and Science Study) and found these states near the bottom of the global rankings.
  • Educational Initiatives carried out an 18 state study using sophisticated testing instruments and found levels of performance on TIMSS comparable items that were stunningly lower. For instance on the open ended question “Write a fraction larger than 2/7″ less than 30 percent of Indian students in standard 8 could answer correctly compared to more than 70 percent internationally.
  • The APRest study led by Karthik Muralidharan and Venkatesh Sundararaman in rural AP asked the same questions of students in grades 2 to 5 and found very slow rates of learning progress.
  • The results year after year from the ASER [2010 2009] study supported by Pratham find that significant fractions of students in Standard 8 cannot master even Standard 2 curricular basics. In rural areas nationwide a third of children in grade 8 could not do a simple division problem and almost 20 percent could not read a level 2 text. The 2011 results, due out in a few weeks will show continued stagnation or even retrogress in learning.
  • Numerous studies by MIT’s JPAL, World Bank, NCAER/University of Maryland and other researchers found levels of performance that were shockingly low compared to curricular expectations.

These PISA 2009+ results are the end of the beginning. The debate is over. No one can still deny there is a deep crisis in the ability of the existing education system to produce child learning. India’s education system is undermining India’s legitimate aspirations to be at the global forefront as a prosperous economy, as a global great power, as an emulated polity, and as a fair and just society. As the beginning ends, the question now is: what is to be done?

4 comments to The first PISA results for India: The end of the beginning

  • abi

    Indians are low in iq(avg iq of indians is 81, is the reason behind these low scores.One cannot improve the iq by giving education.So there is no meaning in investing in education in india.Students of india dont have the capacity to absorb quality education.

  • Saroj Deb

    IQ is not only the factor. More prevalent, I think, is our coaching system which has turn out to be the most unscientific at the behest of inept politician and unscrupulous businessmen. Once upon a time Indian students in foreign universities were used to be earmarked to take the top position in class.
    Remember there had been a proposal in our parliament to include Astrology in syllabus. And I also think no state education minister is aware of the fact or interested in knowing the fact(Result of PISA).

  • Shan95

    75% of the Indian population is getting quota.

    Any group that gets quota is by definition low genetic IQ

    In Tamil Nadu, for this test, 60% were from govt schools and 40% from private schools.

  • Shan95

    Calculating Average Indian IQ from PISA

    TN raw math PISA score = 351
    TN implied IQ = 100 – 1.5 x 15 = 78

    HP raw math score = 338
    HP implied IQ = 100 – 1.62 x 15 = 76

    Indian Avg IQ based on raw IQ = 77

    Next step is to remove the bias caused by the PISA sample having
    75% bilingual kids

    TN mono-lingual = 378
    Implied IQ = 500 – 1.22 x 15 = 82

    HP mono-lingual = 401
    Implied IQ = 500 – 15 = 85

    Next there is a 40 point difference between scores for ‘Village’ and scores for ‘Large city’
    In HP and TN, The village category is over-represented by a factor of 4
    Even worse, in HP, City and Large City are entirely removed from the survey sample

    So adding an urban correction of 20 ( half the village-large city difference )

    TN semi-urbanised mono-lingual = 378 + 20 = 398
    Implied IQ = 85

    HP semi-urbanised mono-lingual = 401 + 20 = 421
    Implied IQ = 100 – 0.79 x 15 = 88

    Current Indian IQ = 86

    Next we look to the future as malnutrition is removed

    The only Indian kids who go to govt school is for the mid-day meal,

    If they are not starving they go to private school

    Private schools score 45 more than govt schools and thats the future as poverty reduces

    HP – future – semi-urbanised- mono-lingual = 401 + 20 + 45 = 466

    Implied IQ = 95

    TN – future – semi-urbanised – mono-lingual = 378 + 20 + 45 = 443

    Implied IQ = 91

    Future Indian IQ = 93

    Given the huge bias in sampling towards over-representing the lower end IQ,
    by the poverty pimp NGOs, I am certain that none of the CBSE or Cambridge schools
    that serve the top 15% are included in the survey

    And they have an entirely different IQ profile and cause an IQ bulge at the top end

Leave a Reply




You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>