How do stars work?
thecrag.com uses a star system for routes in their publications.
Stars are community rated and are derived from user feedback.
This document covers the following topics:
- Stars calculation
- User grade contribution
- Publisher grade contribution
- Quality Ratings
- Quality Score
1. Stars calculation
Route stars are assigned by the system according to the following rules:
- If there are no ascents logged for the climb then the stars come from the average of the stars in the user and publisher grade contributions (these can be found on the route page).
- If there are more than 20 ascents logged with quality ratings then the stars come entirely from the user quality ratings.
- Otherwise stars come from the weighted average quality ratings, user contributions and publisher contributions; where a user contribution is weighted twice as much as a quality rating and a publisher rating is weighted five times as much as a quality rating.
The implications of these rules are:
- If a route has no stars then this does not mean that the route is not worthy of stars. It could just mean there have not been enough ascents to assess the stars.
- Once a route has had sufficient ascents then no individual will be able to set the stars. This is a true community rating.
2. User grade contribution
When a user makes a grade contribution they may include stars as textual stars. For example their contribution may be "5.12**", in which case the system will work out the user thinks it is a 2 star route. Note that when inputting stars, they are simply added to the end of the grade as plain text.
You may make your own grade contribution when you:
- Create a new route; or
- Update an existing route.
Note that you can update your grade contribution at any time. Also there may be multiple user grade contributions for a particular route. User grade contributions are shown on the route page.
If you are the only person to have entered stars as part of a grade contribution and there are no ascents then the route will adopt your star rating. In otherwords when you first create a new area all the stars will be what you create, and as people start to log ascents these stars will be adjusted to the genuine community sentiment.
3. Publisher grade contribution
You may cite the grade and stars from a particular publication. It's our policy to only allow citing of grades from a particular publication if the publisher has expressed an interest to do so. When you create or update a climb you can cite the publication by clicking the 'Cite Publication' button. You will be asked to choose from a list of configured publications. If the publication is not there then you may not cite it.
Publisher contributions are listed along side of user contributions in the route page.
Publisher contributions for stars are given more weight because they are published documents.
If you are a publisher and wish to make your guidebooks citable on thecrag.com then contact us. The publication will also be promoted in the index.
4. Quality Ratings
When a user logs an ascent they are given the option to rate the route with one of the following:
- Don't Bother
- Very Good
- Mega Classic
Combining the ratings from multiple ascents you can get an overall feel for what the rock climbing community think of the route. For example the graph below shows combined quality ratings from for routes (from left to right: 245 ascents from a route, 139 ascents from a route, 155 ascents from a route and 127 ascents from a zero star route.
In order to work out when a route changes from say a two star to a three star route we quantify these graphs into a single number called Quality Score (see below) and define thresholds for one, two and three star routes.
5. Quality Score
The Quality Score is a single number out of 100 to quantify all the user quality ratings. To calculate the Quality Score we add up each of the quality ratings in a 0-6 scale (Crap being 0 and Mega Classic being 6) then divide by the number of quality ratings and normalise to 100.
For example if the users have made three quality ratings of Good, Good and Very Good then this would be 100*(3+3+4)/6*3 = 56 Quality Score. As more user rate a route the more reliable the Quality Score becomes. If there is less than three quality ratings we do not calculate a Quality Score.
The final thing you need to know is how we determined the Quality Score thresholds for one, two and tree star climbs. To do this we calculated the average Quality Score from the one, two and three star climbs from 200 published guidebooks (about 100,000 routes). This came up with the following thresholds:
- maps to a Quality Score of 72.
- maps to a Quality Score of 62.
- maps to a Quality Score of 55.
In order to know which is the better route within a particular star group we also show the Quality Score in the tooltips as you hover the mouse over the stars. For example hover the mouse over the following stars .