Measuring education: learning curves and black boxes
The Economic Intelligence Unit has just published The Learning Curve, which will certainly attract much attention. The report looks at international evidence on educational performance – or, I should say, on school system performance, since it concentrates wholly on schools, a familiar if frustrating pattern for those of us concerned with lifelong learning. Having had my minor beef about that, I want to say that overall the report marks a big step forward in how we ought to approach the measurement of educational performance.
My first big cheer is for the report’s explicit declaration of humility in the face of complexity. Those involved have undeniably a very high level of technical sophistication. There’s no doubt they can generate enough regressions to fill many volumes, but they acknowledge that getting clear correlations that make sense, let alone causal relations, is often beyond them. The report recognises that the black box of education remains largely impervious even to the heaviest of methodological firepower. “We really don’t understand …the complex mix of inputs – family, community and learning – that lead to skills and temperaments.”
For me, this declaration of the need for humility is a major advance. It should open the way to a more rounded approach, which seeks a variety of kinds of evidence, and which broadens the scope of educational research. Too many studies use overelaborate methodologies to give spuriously precise results. (The report does in the end come up with a single global ordering of education systems; it’s not being too cynical if I say that this may have been partly for the undeniable power such tables have to gain media coverage…)
I should declare an interest here. I’ve worked for two spells of 4 years each at the OECD’s Centre for Educational Research and Innovation. OECD specialises in comparative analysis, and does a pretty good job in both building and using educational statistics for this purpose. I’m proud of much of the work that it does. But these statistics can be misused, internally and externally. A typical example is the way the figures for educational expenditure are taken and used as political cudgels, without any examination of how expenditure is used.
So a second big cheer is that the Learning Curve puts due emphasis on outcomes from education rather than inputs to it. The outcomes may be cognitive skills, but also wider outcomes such as employment or health. This is where our real focus should be, not on how much is spent (relevant though that is). At CERI we launched a project focussed exactly on the social outcomes of learning which still continues (sadly not referenced in the LC report). It was a huge challenge to balance the measurable with the meaningful, given the complexity of the links between learning and social outcomes such as better health or more active citizenship.
A third cheer is the way the report stresses the long-term nature of change in education. There will rarely be quick results from educational reform, so it makes sense to have longer-term strategy – politicians please note.
And this leads us to the double connection from the Learning Curve to the Paula Principle. It’s not an immediately obvious one, since the report mentions gender only in passing (not surprisingly in such a broad overview). First, the PP is all about the connections (or the lack of connection) between educational performance and outcomes. It asks how and why girls’ and women’s superior educational performance doesn’t show up as clearly as it should in the workplace. Simplistic messages about higher skills leading to greater productivity and income (for individuals and countries) do not hold up well in the face of the evidence. This is exactly the message from the Learning Curve.
Secondly, the Learning Curve rightly suggests that we need long time horizons: to plan change, to implement it and to see the results. One of my themes in the PP is that we need to have a realistic idea about the lags which occur between one change and another – and to decide what is an acceptable lag and an excessive one. Girls have been outperforming boys now for over 20 years in schools, and for 12-15 years in higher education. We see some effects from this, in initial salaries. But we don’t see it sustained into the mid-careers of women and men, and beyond.
I very much hope that the Learning Curve people will pick up on the PP. They will surely have gender as a focus since they deal with countries where ensuring the schooling of girls is still a major issue. But I hope they will include gender as a central dimension when it comes to measuring the outcomes of education, not just on girls’ participation in school and college. If they do, it will call for another basic shift, towards the black box of the workplace.