The Octane benchmark suite was released in 2012 and was the evolution simple collection of V8 test cases. It soon became a very popular and commonly used benchmark tool all across the internet. It contained 17 different test to cater to a variety of workloads. It was used by developers to test and assess how well JavaScript ran on a browser, the JavaScript being an important part of the internet.

It was, therefore, crucial for further development of more efficient browsers. Now, in April 2017, Google is ending support for the popular benchmarking tool as it determined that the tool had become detrimental to web performance of a browser.

Google to end support for the browser benchmark Octane
Google to end support for the browser benchmark Octane

Developers used the test to make better browsers and the end products got better at it. Google said:

it began to notice that JavaScript optimizations which eked out higher Octane scores often had a detrimental effect on real-world scenarios.

This can be attributed to the fact that the Octane benchmark was developed for the 2012 internet. The internet has progressed significantly since then. Browsers have gotten more efficient and websites are made in different ways now.

Google states that Octane doesn’t important ways the modern version of the web works as it may not always work as the real world code does. This was worsened by the fact that developers cheated the test. For example, a bug in Octane allowed developers to attain a 15% increase in their score.

The test also didn’t work well with some codes that would have been better in a real world use case, thus forcing developers to write worse codes. Google is against benchmark tests as it says “unfortunately, similar issues exist in other static or synthetic benchmarks”. Benchmarking tools are helpful, as Google’s JavaScript engine team says, they aren’t always correct and worsen as they age.