Using Sonar Quality Gates
SonarQube and SonarCloud require the use of Quality Gates to ensure code quality is maintained. This page contains descriptions for the variety of recommended Quality Gates that we judge our software by.
The need for quality gate levels
There are several levels of achievement when judging high quality software. Each piece of software has a different lifespan, and software with a longer lifespan tends to lose quality over time as software becomes more challenging to maintain. You can have software that has changed hands, and while quality standards may not have been high in the past, you want to ensure much higher standards on newly developed code.
Sonar provides a number of metrics and operators to describe code quality, but one must implement their own Quality Gates to get effective use out of the functionality. The more opinionated the Quality Gates are, the more effective they can be at maintaining high code quality standards.
You want software engineers to be incentivized to achieve high quality standards, while being flexible enough to not block momentum. If Quality Gates are not used to prevent code changes of a lesser standard, they can easily become blockers if there isn’t a prescriptive and opinionated way to handle them and describe what they mean.
Sonar Quality Gates
Platinum Quality Gate
Only applications and codebases of the highest quality are capable of passing the Platinum Quality Gate.
Platinum apps and their developers demonstrate a true passion for excellence. They should be recognized and rewarded as the metrics required for platinum status are no small feat for developers to achieve. That is why so few apps will be capable of achieving platinum status.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | Yes | is greater than | 0 | |
| Coverage | No | is less than | 95% | 90% |
| Coverage on New Code | Always | is less than | 95% | |
| Critical Issues | Yes | is greater than | 0 | |
| Duplicated Lines (%) | No | is greater than | 0% | 1% |
| Duplicated Lines on New Code (%) | Always | is greater than | 0% | |
| Maintainability Rating | Never | is worse than | A | |
| Maintainability Rating on New Code | Always | is worse than | A | |
| Major Issues | No | is greater than | 5 | 10 |
| New Major Issues | Always | is greater than | 0 | 5 |
| Reliability Rating | Never | is worse than | A | |
| Reliability Rating on New Code | Always | is worse than | A | |
| Security Rating | Never | is worse than | A | |
| Security Rating on New Code | Always | is worse than | A |
Gold Quality Gate
The Gold level is reserved for the applications and codebases that take the next leap above the industry standard.
Developers of Gold apps are reaching for excellence and demonstrating a very high level of code quality. They have a very real opportunity to take the next step to excellence.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | Yes | is greater than | 0 | |
| Coverage | No | is less than | 90% | 85% |
| Coverage on New Code | Always | is less than | 90% | |
| Critical Issues | Yes | is greater than | 0 | |
| Duplicated Lines (%) | No | is greater than | 0% | 3% |
| Duplicated Lines on New Code (%) | Always | is greater than | 0% | 1% |
| Maintainability Rating | Never | is worse than | A | |
| Maintainability Rating on New Code | Always | is worse than | A | |
| Reliability Rating | Never | is worse than | A | |
| Reliability Rating on New Code | Always | is worse than | A | |
| Security Rating | Never | is worse than | A | |
| Security Rating on New Code | Always | is worse than | A |
Silver Quality Gate
Silver status represents the industry standard and should be the baseline that every application and codebase aim to achieve.
Most apps should fall into this category. When quality gates below this are applicable to more apps, there should be unrest. While it is acceptable for an app to remain indefinitely at Silver status, it is totally unacceptable for an app to remain at any lower level.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | Yes | is greater than | 0 | |
| Coverage | No | is less than | 85% | 80% |
| Coverage on New Code | Always | is less than | 85% | |
| Critical Issues | No | is greater than | 5 | 10 |
| Duplicated Lines (%) | No | is greater than | 1% | 5% |
| Duplicated Lines on New Code (%) | Always | is greater than | 0% | 3% |
| Maintainability Rating | Never | is worse than | A | B |
| Maintainability Rating on New Code | Always | is worse than | A | |
| New Critical Issues | Always | is greater than | 0 | |
| Reliability Rating | Never | is worse than | A | B |
| Reliability Rating on New Code | Always | is worse than | A | |
| Security Rating | Never | is worse than | A | |
| Security Rating on New Code | Always | is worse than | A |
Bronze Quality Gate
Applications and codebases achieving Bronze status are on the precipice of industry standard acceptability, but falling short in some key areas.
Developers of Bronze apps should not remain bronze for long, as the industry standard is well within reach and only complacence can keep them from it.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | Yes | is greater than | 0 | |
| Coverage | No | is less than | 80% | 70% |
| Coverage on New Code | Always | is less than | 80% | |
| Critical Issues | No | is greater than | 15 | 30 |
| Duplicated Lines (%) | No | is greater than | 5% | 10% |
| Duplicated Lines on New Code (%) | Always | is greater than | 0% | 5% |
| Maintainability Rating | Never | is worse than | B | C |
| Maintainability Rating on New Code | Always | is worse than | A | |
| New Critical Issues | Always | is greater than | 1 | 5 |
| Reliability Rating | Never | is worse than | B | C |
| Reliability Rating on New Code | Always | is worse than | A | |
| Security Rating | Never | is worse than | A | B |
| Security Rating on New Code | Always | is worse than | A |
Iron Quality Gate
The Iron level is for applications and codebases that have a concerning lack of code quality, but preventions of it getting any worse.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | No | is greater than | 0 | 5 |
| Coverage | No | is less than | 70% | 60% |
| Coverage on New Code | Always | is less than | 80% | 70% |
| Critical Issues | No | is greater than | 20 | 40 |
| Duplicated Lines (%) | No | is greater than | 10% | 20% |
| Duplicated Lines on New Code (%) | Always | is greater than | 0% | 10% |
| Maintainability Rating | Never | is worse than | B | C |
| Maintainability Rating on New Code | Always | is worse than | A | B |
| New Blocker Issues | Always | is greater than | 0 | |
| New Critical Issues | Always | is greater than | 1 | 10 |
| Reliability Rating | Never | is worse than | B | C |
| Reliability Rating on New Code | Always | is worse than | A | B |
| Security Rating | Never | is worse than | A | B |
| Security Rating on New Code | Always | is worse than | A | B |
Stone Quality Gate
Stone status is for applications and codebases that have a seriously alarming lack of code quality. The are preventions of it getting any worse, but the situation needs immediate attention.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | No | is greater than | 0 | 10 |
| Coverage | No | is less than | 60% | 50% |
| Coverage on New Code | Always | is less than | 70% | 60% |
| Critical Issues | No | is greater than | 30 | 50 |
| Duplicated Lines (%) | No | is greater than | 20% | 30% |
| Duplicated Lines on New Code (%) | Always | is greater than | 10% | 20% |
| Maintainability Rating | Never | is worse than | C | D |
| Maintainability Rating on New Code | Always | is worse than | B | C |
| New Blocker Issues | Always | is greater than | 0 | 5 |
| New Critical Issues | Always | is greater than | 5 | 15 |
| Reliability Rating | Never | is worse than | C | D |
| Reliability Rating on New Code | Always | is worse than | B | C |
| Security Rating | Never | is worse than | B | C |
| Security Rating on New Code | Always | is worse than | B | C |
Wood Quality Gate
Software that is capable of spontaneous combustion at a second glance. The Wood status is no status at all really. The situation will either improve or be relegated to the bonfire of bad software.
| Metric | On New Code | Operator | Warning | Error |
|---|---|---|---|---|
| Blocker Issues | No | is greater than | 10 | 20 |
| Coverage | No | is less than | 50% | 40% |
| Coverage on New Code | Always | is less than | 60% | 50% |
| Critical Issues | No | is greater than | 40 | 60 |
| Duplicated Lines (%) | No | is greater than | 30% | 40% |
| Duplicated Lines on New Code (%) | Always | is greater than | 20% | 30% |
| Maintainability Rating | Never | is worse than | C | D |
| Maintainability Rating on New Code | Always | is worse than | C | D |
| New Blocker Issues | Always | is greater than | 0 | 10 |
| New Critical Issues | Always | is greater than | 10 | 20 |
| Reliability Rating | Never | is worse than | C | D |
| Reliability Rating on New Code | Always | is worse than | C | D |
| Security Rating | Never | is worse than | C | D |
| Security Rating on New Code | Always | is worse than | C | D |