Climber Performance Rating (CPR)
A new approach to rating and ranking in the climbing world.
This article covers the following topics:
- Why have a climber rating system?
- Goals and challenges
- A model for climbing ability
- Implementation of CPR
Many climbers love to get feedback on how they are progressing, and how they compare to their mates and other climbers in their region or around the world. And even though competition climbing is becoming increasingly mainstream and will be in the 2020 Olympics, there is no standardized approach to rating and ranking for outdoor climbing. Some climbing websites have ranking systems which seem fairly arbitrary and provide no statistical or scientific evidence for their ranking system and thus only provide very limited value to measure and monitor personal progress.
theCrag team thinks a rating system can be more than just a way to compare yourself to others, we also want it to be a way to help you gauge your own progress, to inspire you to climb harder if that is your goal, and help find your weaknesses and strengths. A great rating system should work for climbers of all abilities from elite climbers, to newcomers and to regular weekend climbers.
For new climbers it should be a fun and easy way to track your progress, provide encouragement and compare yourself to your mates. For elite climbers a rating system can be serious business and can be directly linked to fame and fortune and for the scientific community it is a basis for structured research in a steadily growing field.
The figure above shows the CPR timeline for one of the site's longest standing members, Lee Cujes. Please read the CPR timeline explained to learn how to interpret this timeline and hopefully you will be as excited as us about this development.
The aim of the CPR is to provide a system which reflects as accurately as possible your current climbing ability, and can provide some predictions of your ability in your unknown areas. The rating system has to be heavily grounded in real world statistics and on a sound theoretical model for climbing ability and how it relates to grades.
Not everyone records their ascents (ticks) in the same way. Many climbers just record their successful ticks and not all their attempts. Some only record the most 'interesting' or memorable ascents. Also every ascent is different, some are red-point, some are onsight, some are seconded etc, but the system should use everything it has as a signal where possible. So the first challenge is using as much information provided but not ever penalizing someone for missing or incomplete data or because they are logging, or not logging, try's and failed attempts. The system obviously can't read people's minds, and also can only see their tick history and is not privy to how hard they are training, or if they have been injured or ate too much pizza. And at the end of the day it also relies on honest logging of data.
After quite extensive statistical analysis of a million ticks on theCrag, talking to a range of climbers at all levels and other experts in the field, four main observations were identified and validated as the foundation of CPR, the new climbing rating system.
Power factor between grades
Every grade is harder than the one before it, but what does this really mean? It's not as simple as each grade just being a bit more difficult than the last. Fundamentally each grade is actually an order of magnitude harder than the last grade, in other words the climbing gets exponentially harder as you go up the grades. When you look at large numbers of ticks you see a clear pattern: for a given climber who has just ticked their first climb at a new grade, on average they needed to climb P routes of the grade below it to improve their ability.
So for every unique route you ascend the system gives you roughly P^g points - g being the grade in our internal fine grained grading system, applying some adjustments which are outlined below, then add them up to produce your final score. Then the system takes the logarithm of it in base P and translates it back to a meaningful grade. This is called your 'performance rating' (CPR).
In consultation with elite climbers, trainers and statistical analysis we have choosen a grade factor P of 5 under the Ewbanks grading system. This means, for ranking purposes, 5 ascents of a particular grade is equivalent to one ascent of the next grade.
Solidifying a new grade
We distinguish two aspects here, the CPR and your grade in the sense of climbing ability. If you have just ticked your first route at a new grade X, most climbers would not consider you 'a grade X climber'. There is too much variability, it could be a soft route, it could be a fluke, etc. But if you have done a few routes then there is clear consensus you are climbing at that grade. Therefore the system applies a small penalty so that you need around three ticks at a particular grade for your personal climbing ability, your personal grade to be at that same grade. However, in terms of CPR every ascent counts and each tick at whatever grade adds to it. This number 3 is not based on any underlying fundamental principle but instead on the consensus of the climbing community.
Grade shifts for tick types
Some ascents are a clean red-point, some are onsight, some are top rope etc. How can all of this information be used and normalised in some way so it can be compared apples to apples? Fundamentally, the different tick types are easier or harder because of the additional work you need to do while climbing. Route finding makes it harder, so an onsight is more difficult and so worth more than a red-point. Pink-pointing a trad route is not as hard as placing the gear, so it is easier than red pointing and even more so than onsighting it. If you had two routes with identical technical difficulty but one was sport and the other traditionally protected, then pink pointing either should be identical in score, but onsighting the trad route would be harder and so worth more.
We at theCrag looked at all the ticks and found clear grade offsets for each tick type which are dependent on whether it is sport, boulder or trad, etc. For instance in sport climbing you can typically red-point 2.1 grades harder than your hardest on-sight or the difference between a pink-point and red-point in trad climbing is about 0.8 grades (see Figure 2 for details). Based on that statistical analysis a tick shift was worked out that applies for every type of tick across every type of climbing. This shift is then applied as offset to your ticks to calculate your CPR. Note when we say 2.1 grades, this is using the Australian Ewbank grades - but the same applies universally regardless of grade system, however each system has its own relative 'widths' of each grade which we have adjusted for.
We have done extensive statistical analysis to come up with the tick shifts. See Calculating tick shift article.
These graphs shows how ascents of different tick type are valued for your CPR. As an example a sport pink point ascent of a 6c+ (FR) is equivalent to a red point ascent of a 6c, a flash of a 6b+ and an onsight of a 6b. The coloured boxes around the grade table reflect these shifts and show by how many grades each respective tick type is shifted for the calculation of your overall CPR.
Climbing ability decays over time
If you stop climbing, your ability goes down. It's fairly obvious! But maybe you are cross training and if you climbed you'd actually do better, or maybe you are injured and can't climb at all so your ability should drop considerably. There are so many unknowns for this factor, and it is certain that it is more complex than just an arbitrary cut off by date or period which many other rating systems have introduced.
The first validation approach we tried was to look for climbers who had taken a break and then came back and worked out how much ability they had lost, but this is by its very nature an incomplete data set, and even with a more complete data set is not trustworthy because we lack visibility into the underlying reasons for the break.
The second approach was to look at the much larger group of climbers who still climb regularly and see how much they are progressing or not progressing. If your climbing ability is increasing every time you climb, but your ability is also decaying at the same rate, then you are in a rough equilibrium and have 'plateaued' and thus a decay rate can be inferred from this. The reality is, that many climbers improve to a certain level fairly quickly and then plateau, providing a lot of data to work from. The decay rate identified and applied by the system to your CPR is 0.7 Ewbank per year.
Having performed this extensive analysis and being convinced that CPR adds value for individual climbers and the climbing community, theCrag has decided to re-introduce a climber rating and ranking system. Having a strong mathematical and statistical basis theCrag team is truly excited about this fresh look into climber ratings and is confident that climber rating and rankings can be moved away from an arbitrary to a scientifically and statistically sound approach.
The realization of the theoretical approach outlined above required a few more practical considerations and features in order to make CPR useful for every climber. theCrag chose the following approach in its implementation:
Three separate CPRs for bouldering (BPR), sport climbing (SPR) and trad climbing (TPR) are implemented, as we found large significant differences between these different styles of climbing, and it is misleading to combine these into one rating.
Routes ticked from any grade system count against your CPR. theCrag currently understands over 20 grading systems from all over the world and converts them into an internal grading system for statistical analysis and calculations.
As a climber you get a CPR for each style of climbing from your first tick onwards in order to monitor your personal progress from the start. Then every tick of applicable style counts.
On your profile page you will still see your classification by hardest and onsight grade
For repeats of a route only the tick with the highest score is chosen. After that the shifts and decay are applied. This means that repeating a route will typically not result in an increase of your CPR.
For better understanding, each of your CPRs is mapped back from a theCrag- specific internal value to a grade in your local grading system.
Community consensus grades are used to calculate the value of each tick against your CPRs for any grading system. The use of consensus in contrast to opinion reduces - but not eliminates - the error based on hard or soft grades for individual routes.
theCrag provides you with an all time CPR and time based CPR, allowing you to monitor your CPR and rank over all time (what was your best shape ever) as well as to judge your current climbing ability.
Due to its hierarchical nature, theCrag allows you to rank your CPRs per crag, area country or worldwide and any other parameter such as gender or age. This allows you to compare yourself with your mates at your favourite crag as well as to create global or contest based rankings.
If a climber has a rating then they have a ranking. As well as a global ranking you can see rankings at crag level and create your own ranking filters.
We have developed the underlying rating technology to be flexible. For example we can create a performance profile for a whole crag.
For bloggers and climbers with personal pages we have made everything CPR embeddable. For example the graphs in this article are using embedable components.
theCrag-team believes that by providing the CPR to the climbing community we are not only allowing climbers to quantify their individual progress over time and to compare themselves to their mates or fellow competitors on a sound basis; We also think that the CPR will open many more possibilities such as supporting scientific research, identifying wrong grades of selected routes, comparing the grading of different climbing areas on a statistical basis or evaluating your grade shift from indoor to outdoor climbing. You could even think of identifying the best slab climber on granite or answering many more questions you might have. We encourage you to send us your feedback and ideas. Contribute to new features with and around CPR and help shaping the future of climbing and the scientific research about it.
As a climber you get all these capabilities for free on theCrag. It is sufficient to join now and start ticking your ascents. Your profile page gives you access to your CPRs and basic statistical analysis.