I analyzed 370 million files of 614k Github repos to find out which Continous Integration services developers use. I looked at public Github repos that were recorded in the GHTorrent dataset of 2015-01-29 and are not forks of other projects. The dataset is available at the repostruct Github Repository.
tldr: Travis has won the game but there are still alot of projects that don’t use CI at all.
Codeship doesn’t appear in this analysis even though it is a major player. This is because there is no config file I can look for in the repository.
CI Integration across Languages
- Pythonistas use CI less than their dynamic companions (only 20%). Perhaps this might be because Python is used alot for scriping where testing is not as important.
- Java developers use CI services in only 10% of projects. That is as bad as C programmers. For a language with the most mature tooling this is astonishing. Perhaps this is because they rely on custom builds with Jenkins or Teamcity.
- C# projects have not yet arrived the age of CI (less than 5%).
- Projects with native code like C++ or Objective-C don’t use CI as often, but surprisingly more often than Java. Approximately 15% of projects use CI. They are also more willing to use other CI services than Travis.
- The Go community is the most interesting. They use alot of services and Drone.io and Wercker start to gain traction among those projects. Go programmers are also exemplary in their usage of CI (40% of projects!)
- Travis is the #1 for Open Source CI integration and it seems unlikely that will change in the future. No one has even close the numbers Travis has across all languages.
- AppVeyor is well positioned with its Windows based builds and they might get significant market share in languages where platforms matter like C++, C and C#. These languages still have some room to grow because they don’t have a high CI usage yet (especially C#).
- Drone is mostly used by Go developers. No wonder as it is based on Docker and Go.
- Close to no one uses SensioLabs or Solano.
- Wercker and CircleCI are new kids on the block and therefore might be a bit underrepresented in this analysis. I think it will be tough for them to compete with Travis. They need to convince the users creating new repositories. Looking at the Go community where most projects are very new, we see that Travis alternatives have a better chance.
There is still room for more CI services, but given that not all Projects on Github are of serious nature, more than 50% of projects are unlikely to use CI.
My guess is that Travis will remain the big player for Open Source projects in the future followed by AppVeyor. The others will have a little share of the projects even though they have better features than Travis.
This is simply an educated guess how many Open Source repositories use which servies. I would be very happy if the people working at those Companies could confirm them.
Please note that if Projects don’t use a configuration file I cannot relate them to a CI Platform.
Conclusion: The data gathered is pretty accurate, perhaps not down to the number but the distribution ratio of CI integration services is correct.
I verified the results with results from Google Search and Github Code search.
The results differ, but the ratio stays the same (Probably because I didn’t get
the whole Github dataset).
If you scale Google Search with
1.5 and the Github search with
0.4 you get the same results.
Github has much more records for
appveyor.yml than I found.
Reasons might be:
- Github Code search includes forks
node_modulesfolder checked in)
|CI config file||repostruct||Google Search||Google Search (intitle)||Github (in path)|
- Google Search with
- Google Search with
- Github Code Search with