Homeless in Vancouver: Surprise! Adobe Flash wasn’t the buggiest and neither was 2015

The numbers are in. For all of 2015, there were some 6,412 distinct bugs and security vulnerabilities discovered  in software released by a total of 17 vendors.

This was a drop of 19.3 percent from 2014’s total of 7,946—the highest total since 1999 and—surprise, surprise—in 2015, only 4.9 percent of all flaws were in Flash Player!

Adobe’s Flash Player was only the third buggiest piece of software in 2015, with “only 315 vulnerabilities detected". This according to the Common Vulnerabilities and Exposures (CVE) website, one of the databases of record for information system bugs and security flaws.

The CVE’s Top 50 Products By Total Number Of “Distinct” Vulnerabilities in 2015 shows that the two buggiest software products of last year both belonged to Apple. The Mac OS X desktop operating system had the highest number of software vulnerabilities detected in an application: 384, followed by Apple’s mobile iOS operating system, which racked up 375.

Overall, more than 70 percent of the vulnerabilities were found in commercial software made by just three companies: Microsoft, Adobe, and Apple, which represented 17 percent of vendors. For its part, free open source software (FOSS) accounted for somewhere between 15.4 to 20.5 percent of vulnerabilities in software released by 58 percent of vendors.

Good year for bug collectors, especially Windows NTmologists

When CVE sorts by vendor, three companies jump way out of the pack: Microsoft (1,590), Adobe (1,504) and Apple (1,147); the trio accounts for 4,242, or a whopping 70.26 percent of the total 6,036 known software vulnerabilities in 2015.

Oracle, the maker of Java, was a far behind in the next position (341), followed by Google (317), *Mozilla (272), *Canonical (152), Novell (121), *Debian (111), *Linux (77), Cisco (84), *Fedoraproject (38), *XEN (34), *OpenSSL (34), *Wireshark (33), *MediaWiki (31) and *FfMPEG (29).

But CVE’s way of parsing the numbers seems misleading. Fully 10 of the 17 vendors are interlinked free open source software projects (everything I’ve marked with an asterix).

Canonical makes Ubuntu, which is built on Debian, and both are versions of the open source Linux operating system. To my mind, it makes as much sense to combine the FOSS project as it does the Microsoft products.

When combined,  free open source software comes in a close fourth after Apple, accounting for 932 software vulnerabilities. This doesn’t include Google, which uses FOSS to build its products (like Search, Chrome and Android) but always adds an amount of closed, proprietary code. If Google is included, then FOSS leaps into third place with 1,239 bugs to its credit.

What the numbers don't tell us

Looking at the 2015 CVE numbers, one could jump to the conclusion that proprietary commercial software is far buggier than open source—and that may be true—but, without context, all the numbers really tell us, at first glance, is that there were over 6,412 software flaws discovered in 2015.

If we could measure the flaws against the quantity of computer code written, I suspect we’d would see a direct correlation.

Every vendor on CVE’s list, commercial or open source, knows and probably tries to follow the same best practices for vetting computer code errors. It’s therefore not unreasonable to expect that all 17 vendors generated about the same number of software flaws per 1000-lines-of-code (IBM’s original “KLOC” measure of programming output—famously despised by Microsoft’s Steve Ballmer).

What CVE’s numbers might therefore be reflecting then, is that in 2015 Microsoft accounted for more lines of code than Apple, which, in turn was responsible for more than all the FOSS projects combined, and so on.

Consider what Information is Beautiful’s 2013 Codebases infographic and other sources tell us about code production:

• Windows 7 and Windows XP—each about 40 million lines of code.
• Windows Vista—about 50 million.
• Microsoft Office 2013 (a mere program)—about 45 million lines.
• Mac OS X 10.4 (“Tiger”)—85 million lines.
• Linux kernel (pre 4.2)—22 million lines.
• Debian 5.0 codebase—67 million lines.

My parity argument, however, all but founders against the anomalous estimates of Google’s monolithic code base. All Google Internet services—Search, Gmail, Maps, etc., were estimated in 2015 to represent some 2 billion lines of computer code—treated in-house like one big program, something like 40 times the size of any version of Microsoft Windows and possibly more than all the code of all the other 16 vendors combined!

It’s understood that Google employs some 25,000 software developers, while Microsoft says only that it employs about 53,895 “engineers” world wide. Apple has been estimated to have about 16,786 software engineers, based solely on Linkdin searches (?). And a whopping 4,171 software  developers contributed to the Linux kernel 3.15, according to the 2015 Linux Kernel Development report.

Google reportedly has its own version control system for juggling software code called Piper, which coordinates all the code work across the entire Google infrastructure and spans 10 different Google data centers.

Piper must be a pretty great system if it allows Google to potentially produce more code than almost everyone else put together and only 4.9 percent of all software vulnerabilities. Remember, Google’s total of 317, for all its software, is just two more security flaws than the 315 racked up by Adobe’s Flash Player, which is just one program for F-sakes.