News - 19/09/2018

Building Open Source software in a large corporation

The world runs on data. However, without the dynamic, accessible and adaptable nature of OSS (Open Source Software) the pace of exploitation of data-rich fields would be painfully slow. Imagine a world of Data Science without Linux, Python, Anaconda or Tensorflow, just to cite some relevant examples of Open Source Software.

During the last few years, the trend of using OSS for data processing and analytics has been increasing up to the point where most solutions today are in fact OSS. The largest corporations, including Google, Microsoft or Facebook are actively contributing to this. Despite the success stories, adopting an open source philosophy requires challenging numerous established ideas, especially if we want to attract active supporters instead of users who are simply not providing knowledge to the project, or free riders.

We identified three fears that keep corporations from embracing OSS:

  • Fear of letting competitors benefit from their investments.
  • Corporate production roadmaps do not favor long-term development of necessary tools to create future products.
  • Not controlling the narrative. Legacy corporations are used building a perfect narrative and dislike open competition, despite their claims to the contrary.

OSS 101 for executives

  • Contributing to OSS always pays back. In a recent study from the Strategy Unit at Harvard Business School, Professor Frank Nagle states: “Paying employees to contribute to such [OSS] software boosts the company’s productivity from using the software by as much as 100 percent, when compared with free-riding competitors.” Active contributors produce more and better code.
  • On production roadmaps: Tools necessary to create applications that typically result in large projects that, today, are mostly OSS. What corporations develop are typically business applications built on top of that. The schedules and incentives are different in each case. OSS communities are meritocratic, while in corporate development legacy, hierarchy, budgets and schedules are what shape the products.
  • In OSS you can control the landing experience by creating introductions, tutorials, and documentation, but you cannot control the narrative. Users try it and decide if it suits their needs. All claims can be verified. To corporations this may sound like a nightmare, which it isn`t. In actual fact, transparency is one of the main reasons why OSS has already replaced the old style business model. Technical users are saturated with overhyped unverifiable narratives and seek transparency.

The Community

We used to think about OSS as a piece of stable, useful, well-documented code that is released continuously. This is the visible result in a success case. However, most OSS projects fail before that stage. The reasons are diverse, including creating a product nobody wants. However, a crucial reason is to fail to build a community. Bootstrapping a community is the hardest part and is a requirement for success. Founders must invest considerable resources to produce something usable enough and reliable enough to attract early adopters at a stage when there is little incentive to use the product. This is where large corporations have the power to support new endeavors. They can help by supporting branding, communication, infrastructure, part-time contributors and budget. Companies should envision OSS as an opportunity to invest in ambitious projects for their own benefit in exchange for an meagre fraction of what they invest in business solutions. This improves employee motivation and productivity. It also gives them the power to choose what they support or whether to cut their funding at no cost at any time.


As an example, BBVA released Jazz in December 2017 as OSS. Jazz was originally a highly efficient distributed data store with an R client (now in cran) developed for internal use at BBVA Data & Analytics. After its release as OSS, its main author decided to redesign the product pivoting it into an AI platform that addresses issues that are poorly, or not at all, supported on mainstream platforms.

Jazz has been steadily advancing through the first nine months of 2018. Refactoring is now complete, and a list of pending tasks is being addressed. Similarly ambitious projects had a two to four year “under the radar” period before they became relevant. By these standards, Jazz is still young. Time will tell if it becomes another abandoned project or a successful story of OSS grooming by a big corporation, or that it might one day tell its creators: This mission is too important for me to allow it to be jeopardized by you.((Hal in 2001: A Space Odyssey, 1968))