The Code Review Tools Ecosystem
Nowadays, GitHub is the home of open source. The adoption of pull requests has been incredible. But what happens when we move away from open source? What is the most widely used code review tool? To get a better understanding, we took a deep dive into JetBrains Developer Ecosystem Survey 2020 data.
We analysed the data from 23589 respondents of the survey JetBrains Developer Ecosystem Survey 2020. This survey is very comprehensive. The PDF with the questionnaire alone is 168 pages (!!). Hats off to JetBrains for this huge accomplishment and providing the data for others to use.
The big three are dominant
As expected, the data suggests the majority of developers are using the big three (Bitbucket, GitHub, and GitLab) for code reviews. Our analysis shows that 90% of developers that use more than one code review tool are using both GitHub and GitLab. The latter has a slight edge. What else can we learn?
We focused on the following questions, related to employment status, job role and code reviews:
- What is your employment status?
- Which of the following best describes your job role regardless of the level of your position?
- In which kinds of activities are you involved?
- Which of the following tools do you regularly use?
- Which tools do you use in your company / organization for Code Review?
A look at the key statistics
From the 23589 respondents of the survey:
- 82% regularly use code hosts
- 59.5% have activities that include code reviews
- 12.2% regularly use code review tools
- 18.2% regularly use code reviews tools given than their activity includes code reviews
The big gap between people that have activities that include code reviews and those that regularly use code reviews helps us figure out which numbers we can use to deal with the untapped market of code reviews. The market will necessarily grow as more and more teams start both doing them and using specific tools to do so.
Who is doing code reviews?
Employment status
Not surprisingly, the vast majority of people in the survey are fully employed. Code reviews are performed by people that are fully employed at an even greater rate. This is also not surprising.
Check out the following table that details this information with the percentage of respondents per employment status and the percentage whose main activity includes code reviews.
Job role
We often hear that code reviews are performed by more senior developers. The data corroborates this intuition, as the percentage of senior developers doing code reviews is greater than those who aren’t doing them. The opposite occurs for junior developers. This indicates that not all junior developers are regularly involved in code reviews. The numbers are on the following table:
Code review tools usage
The following plot shows the ranking of code review tools per usage.
Note that it is possible for a developer to be using more than one tool.
Developers are doing code reviews with the code hosts interfaces of Bitbucket, GitHub, and GitLab. These three tools combined represent more than 50% of all tool usage. About 9% of the developers do code reviews without code review-specific tools or, of course, using custom tools.
The last thing we did was look into the developers that regularly use more than one code review tool.
We found out that 1 out of 4 developers that do regular code reviews use more than a code review tool.
Initially, we thought that these were developers that were using a major code host and a dedicated code review, for example using GitLab and Gerrit.
It turns out that 90% of those developers actually use both GitHub and GitLab.
Assuming that a lot of time spent in GitHub/GitLab is in the code review interface and considering that GitHub is the home of open source, one wonders if more of the teams that are doing code reviews are paying for GitLab than GitHub.