A group of student researchers set out to determine whether people are biased against code written by women, and discovered the opposite: Code written by women has a higher acceptance rate on the social coding site GitHub than code written by men.
But there’s one tremendous caveat; this is only the case when the individual’s gender is unknown to GitHub users.
“Our results suggest that although women on GitHub may be more competent overall, bias against them exists nonetheless,” researchers wrote.
GitHub is the largest open-source community, with over 12 million people using the platform to share, collaborate, and manage technical projects.
To figure out whether gender bias explicitly exists on the platform, researchers took a look at how often pull requests were accepted, and from which accounts. A pull request is a change a developer suggests to a project’s code or documentation on GitHub. Anyone can make a pull request, but the project’s owner must accept or “merge” it themselves.
After analyzing 3 million pull requests from almost 1 .5 million users with identifiable genders, researchers found that code written by women was merged more frequently than code written by men, with 78.6 percent and 74.6 percent, respectively. However, when a women’s gender was explicitly identifiable on GitHub, acceptance rates dropped significantly.
Because GitHub doesn’t ask people about their gender, researchers took user emails from the GHTorrent, cross-referenced them with Google+, and if a user had a profile with gender information provided to Google+, used it in the study.
In an attempt to explain the discrepancies between code acceptance and gender, researchers also analyzed a number of factors that could potentially contribute to the biased results. For instance, whether women were making more urgent pull requests or whether the changes they were making were smaller compared to men. Data proved the hypotheses false.
The study comes at a crucial moment in tech, where sexism and racial discrimination in the industry are at the forefront of conversation. Companies are doubling down on initiatives and investments to improve their culture (and public perception), hiring efforts, and diversity, while venture capitalists are encouraged to invest more in minority entrepreneurs.
Despite the data that shows women fare better at submitting code, gender makeup of tech companies is disproportionately men. At Google, just 18 percent of the technical workforce is female, while 22 percent of Apple’s tech employees are women.
GitHub itself has a rocky history with gender discrimination. In 2014, its CEO and co-founder resigned amid allegations of sexism and the company launched a third-party investigation into claims that a developer was harassed out of the company.
Along with the data that demonstrates the success of women’s pull requests, the study also listed six observations, and took issue with the claim of “meritocracy” within the industry.
1. Women are more likely to have pull requests accepted than men.
2. Women continue to have high acceptance rates as they gain experience.
3. Women’s pull requests are less likely to serve an immediate project need.
4. Women’s changes are larger.
5. Women’s acceptance rates are higher across programming languages.
6. Women have lower acceptance rates as outsiders when they are identifiable as women.
“While our big data study does not definitely prove that differences between gendered interactions are caused by bias among individuals, the trends observed in this paper are troubling,” the paper concluded. “The frequent refrain that open source is a pure meritocracy must be reexamined.”
