Do CAO points predict third level performance?

In 2010, the HEA produced a report in which they concluded that ‘prior educational achievement’ was strongly correlated with progression rates at third level. This effect was found to be strongest at Level 7 but was still significant at Level 8. Very few academics would be surprised by this.

Yet, a number of very reliable people with whom I have spoken in recent years have said to me (based on sound data analyses) that CAO points are not an especially  good predictor of third level performance. The number typically quoted is that CAO points account for about 20% of the variation in academic performance at third level.

At first glance these two findings seem contradictory; but that is not necessarily the case.

Suppose you take a particular programme with a particular median CAO points value and do a scatter plot of, say, First Year precision mark of individual students versus their CAO points. When you do a linear regression, you’ll probably find a positive slope but a pretty low correlation coefficient.

Now suppose you repeat the exercise a number of years later where the median points value is substantially different. Now when you do the scatter plot and regression you’ll find a similarly low correlation coefficient but it is quite possible, likely even, that the new precision mark data will not overlap with the original set of data. Most academics would suggest that if the median points value in the second study were substantially lower, the scatter plot would be shifted left and down as shown schematically below (ignore the numbers!).

Of course a big factor in all of this is the fact that it is by no means certain that third level standards are constant. I tend to think that standards are constantly being adjusted slightly (and teaching methods being adapted) to meet the capabilities of the incoming students. Thus, any correlations you might expect to see tend to be ‘dampened out’ somewhat.

But it would be a nice to see a real analysis, covering a number of years and a broad range of median points, in the manner of the graph below.

CAo blog

Advertisements

About Greg Foley

A lecturer in Biotechnology in Dublin City University for more than 25 years. Trained as a Chemical Engineer in UCD (BE and PhD) and Cornell (MS). Does research on analysis and design of membrane filtration systems.
This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Do CAO points predict third level performance?

  1. Pingback: Ninth Level Ireland » Blog Archive » Do CAO points predict third level performance?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s