In 2010, the HEA produced a report in which they concluded that ‘prior educational achievement’ was strongly correlated with progression rates at third level. This effect was found to be strongest at Level 7 but was still significant at Level 8. Very few academics would be surprised by this.
Yet, a number of very reliable people with whom I have spoken in recent years have said to me (based on sound data analyses) that CAO points are not an especially good predictor of third level performance. The number typically quoted is that CAO points account for about 20% of the variation in academic performance at third level.
At first glance these two findings seem contradictory; but that is not necessarily the case.
Suppose you take a particular programme with a particular median CAO points value and do a scatter plot of, say, First Year precision mark of individual students versus their CAO points. When you do a linear regression, you’ll probably find a positive slope but a pretty low correlation coefficient.
Now suppose you repeat the exercise a number of years later where the median points value is substantially different. Now when you do the scatter plot and regression you’ll find a similarly low correlation coefficient but it is quite possible, likely even, that the new precision mark data will not overlap with the original set of data. Most academics would suggest that if the median points value in the second study were substantially lower, the scatter plot would be shifted left and down as shown schematically below (ignore the numbers!).
Of course a big factor in all of this is the fact that it is by no means certain that third level standards are constant. I tend to think that standards are constantly being adjusted slightly (and teaching methods being adapted) to meet the capabilities of the incoming students. Thus, any correlations you might expect to see tend to be ‘dampened out’ somewhat.
But it would be a nice to see a real analysis, covering a number of years and a broad range of median points, in the manner of the graph below.