Skip to content →

The effect of bug reporting systems on developer-to-tester ratio

During the planning and design phases of the life circle of a project, risk is accounted based on factors that introduce it. Most common factors, are the change in the requirements, developer turnover, specification breakdown, bad estimations & planning and reduced productivity.

In this article, I’m going to add one more factor, not so much discussed, mainly because it’s roots are deep inside the code and most of the times not visible in the initial phases of the software development. I call it dependency risk. Developers see it every day, but they can’t communicate it. Managers could potentially communicate it, but they never see it.

Dependency risk

Dependency risk, is a non-deterministic, non-accountable factor influencing software behavior and the development progress. Dependency risk is induced in a project, when the project depends on external resources which the company handling the project, can’t control. Examples of dependency risk inducers usually are external libraries-dependencies and APIs.

In the “world of the web”, one simply can’t stay away from the web’s interconnection. REST APIs are everywhere. Even if you don’t want to use them, a customer will demand them and as a project requirement, you will be required to include them in the project. Customer needs also demand the introduction of new features in a project and with time-to-market getting shorter and shorter decade after decade one can’t really completelly on building everything from scratch, thus dependencies usually have to be introduced in a project.

3rd party API changes

While companies such as Facebook deprecate API’s by alerting the users long before the deprecation actually happens, you can’t expect that from every 3rd party API you are going to use in your application. There will be cases where a 3rd party API will change it’s interface during your development phase. The best case scenario is that your developers will have to spend extra hours updating the code layer that handles communication with the API, testers will have to test it (again) and in general there will be a delay induced in your project’s completion estimation. The worst case scenario, is that the API will introduce changes that will conflict or even break down your project specifications and customer requirements.

It can happen. Somewhere around 2014, Facebook deprecated an endpoint of their API used by media monitoring companies to perform keyword searches in Facebook’s public data. Compared to applications that instantly stopped working and throw owners and businessmen in instant panic, me having to change my code mid-development sounds insignificant, but it’s something that can happen and influence your project’s development.

Libraries with bugs

The code of an application compared to the size of the libraries it depends on is insignificant. There are so many layers of libraries stacked on top of each other that your project depends on, that there’s something that will always break down during development.

Libraries get updated frequently and while the main functionality is well tested there will always be some feature not covered by testing and that feature might some time end up in your code due to customer requirements.

It gets worse. There’s multiple libraries in a project that have a common dependency denominator. While the underlining library is the same, the version dependencies might differ. This is known as dependency conflict. In Java, while conflicts can show up in the dependency hierarchies of modern build tools, it’s impossible to know if those dependency conflicts will break your code since they are only visible at runtime and only when the dependency in question is used.

This is a very bad thing. Imagine, coding against two specific versions of library A and B, that both depend on library C. The variability of A and B versions is not always possible since they might be proprietary or locked in by obsolete code that you’re not allowed to change or many other factors. You have to code two features. Feature one is being addressed first and interacts with library C methods that are common in A and B. Everything’s good, your feature is almost complete and you now have a good estimation of the pace of the project, good enough to even inform your higher ups about the development estimation (and managers believe me do like to know estimations ! ). Then you move on to work on Feature two. Feature two, depends on some methods of library C that are conflicting between A and B. As a result, while your code compiles, you realize in runtime that method you’re calling, doesn’t exist. Java developers will say here : “O not another “NoSuchMethodError”.  And then, good luck communicating to your managers why Feature one has to be developed from zero using another library.

Libraries that are not well-documented

There’s many cases where a developer creates something based on a library dependency, tests the main implementation but does not provide full test coverage for all side-features supported by his system. There’s many times that assumptions are made on how the system will perform and those assumptions are proven wrong when the users test the application.

How to reduce dependency risk

Most developer teams pick a library based on feature-related research. The developer team identifies the specifications based on the project requirements and picks a library that satisfies those requirements better. Managers support such decisions because it accelerates development and it introduces a good base to avoid reinventing the wheel. My experience so far with software development has taught me that sometimes you have to re-invent the wheel especially when there are indications that the existing wheel you are planning to use, is broken.

As you might notice, I specifically used the word indications because no software will ever display it’s problems until used. Searching for those indications I believe is much more important than evaluating dependencies based on their feature set. I have developed the following three evaluation tactics to make better decisions when picking dependencies.

  • The repository says more than the web page
  • The Issue tracker says even more than the repository
  • Stack overflow says it all

The webpage only talks about features

The webpage of a library – API – dependency has no info on the stability, bugs and backwards compatibility of the product. For that reason, decisions should not be made only by the info provided in the website of the product.

The issue tracker is the official image of the product

Determining if a library has a nature of breaking backwards compatibility too often

To determine if a library is being developed without much consideration for backwards compatibility, we have to map compatibility issues to user posts in services such as stackoverflow. Take Java for example. What happens when a library update breaks compatibility ?

1)Missing methods that are translated into NoSuchMethodException and NoSuchMethodError

2)Missing classes that are translated into ClassNotFoundException

Once such errors start appearing in the code, developers tend to post their problems online. This is valuable information to evaluate a dependency by performing the search queries:

Q1: text: <library name>

Q2: text: <library name> AND <insert exception here>

compatibilityBreakingRatio = count(Q2)/count(Q1)

By normalizing with the trending factor & popularity of a library, you can perform relative comparisons between libraries when having to chose.

Published in Software Development


Leave a Reply

Your email address will not be published. Required fields are marked *