Tag: DevOps

Unit testing: climbing the Uphill road

This article is not about discussing the basics of unit testing or simplifying the concepts, nor it is about merits of the practice. Unit testing methods have evolved quite a bit, with many books and online content. However, the real world practice of Unit testing can be an uphill task in several contexts, due to different reasons, even after how the ecosystem around Unit testing has matured quite a bit.

Unit testing is both an Architectural as well as DevOps concern. Tactically both these departments strive to get the software development operations to effectively incorporate Unit testing.

What are some of the challenges ?

  1. Brownfield vs. Greenfield : Unit testing is easier to adopt when you are starting a new application. The reason being an application needs to be written in a certain way, applying principles like S.O.L.I.D to be able to Unit test it effectively. There are a few key programming knowledge areas which need to be understood well before starting Unit testing correctly, like Interface driven programming and Dependency Injection. The industry is full of so called ‘Brownfield’ applications or simply existing production applications which may not have been written using some of the best practices. These Brownfield applications can become major barriers of entry into Unit testing. Software Engineers who are involved day to day supporting these applications often find it difficult to introduce Unit testing into them because they need to be refactored before doing so.
  2. Time : Unit testing applications can be time consuming, requiring additional development time to be factored in. Often this upfront time is seen as additional cost to development without realizing that, there could be long term benefits like reduced QA footprint and automated regression tests. Hence a technical leadership which creates a culture of Unit testing by coaching the stakeholders on benefits of Unit testing has to be fostered by organizations.
  3. Training: To Unit test effectively, software engineers need to have knowledge of Unit testing basics like creating Mocks, Code Coverage and the AAA pattern ( also known as Arrange, Act and Assert ) . The lack of this knowledge itself can be a barrier even if the first two barriers do not exist. Hence training on Unit testing fundamentals and the available Unit testing frameworks needs to happen.
The Uphill task of Unit testing

The industry is full of so called ‘Brownfield’ applications or simply existing production applications which may not have been written using some of the best practices. These Brownfield applications can become a major barriers of entry into Unit testing. Software Engineers who are involved day to day supporting these applications often find it difficult to introduce Unit testing into them because they need to be refactored before doing so.

All the above challenges need to be overcome to create an atmosphere and mechanism of regular Unit testing.

How do we overcome the challenges ?

  1. A brownfield application which does not lend itself to Unit testing through it’s design will have be refactored first – piecemeal by piecemeal to facilitate the testing. Refactoring can include anywhere from making the code more object oriented, removing deep coupling, adding dependency injection and creating interfaces. Brand new code within the project can be written keeping Unit testing in mind. The more monolithic and deep coupled the code, more refactoring has to be done. It is best to chew at it slowly – over a period of time code will improve. Scrapping a brownfield completely to start a new greenfield application is not always an easy decision. Hence refactoring existing code for improvement one step at a time, can be more cost effective and beneficial.
  2. ‘Time’ is the one of the most negotiated element during software development. Most stakeholders like Product Owner and end clients will want the application deployed ‘yesterday’. In such a situation anything that adds more time to development can mean negotiation, conflict and stress. It is very normal for software engineers to give up on Unit testing when time is a constraint. A strong technical leadership committed to building consensus over Unit testing is critical to overcoming the time constraint- time spent upfront will save time later. The most significant benefits are as stated earlier Reduced QA time, Automatic regression and Better product quality.
  3. Unit testing does not come naturally as part of software development. It is still a habit which has to be developed and cultivated- training is also required to understand techniques of Unit testing. Lack of knowledge itself can inhibit Unit testing to happen even if the above two situations don’t exist. Often the difference between Unit testing and Integration testing are not understood. A culture of training and learning Unit testing has to be created. Team members who are ahead in Unit testing can be appointed as ‘coaches’ in Unit testing. Unit testing demos, presentations and discussions must be organized regularly to create the energy and enthusiasm around it. Technical leaders must ‘call out’ above and beyond Unit testing efforts; Incentivize or reward Unit testing efforts. An overarching policy of ‘incremental’ code coverage can be set, gradually increasing percentage code coverage. As needed ‘time’ must be allocated for the purpose of unit testing within the development cycles with the help of project managers.

A 100% code coverage may or may not be necessary based on the type of code or type of project. It’s a great goal to have, however it’s more important to test code for where rules, algorithms, calculations and complex logic need to be tested thoroughly. If time is a constraint decide where there is more value in Unit testing, and Unit test that part of the application.

An overarching policy of ‘incremental’ code coverage can be set, gradually increasing percentage code coverage. This way as the team matures into Unit testing, code coverage requirements are increased. Team members who are ahead in Unit testing can be appointed as ‘coaches’ in Unit testing. Unit testing demos, presentations and discussions must be organized regularly to create the energy and enthusiasm around it. Technical leaders must ‘call out’ above and beyond Unit testing efforts; Incentivize or reward Unit testing efforts.

A culture change like Unit testing can only happen when there are strong technical leads committed to the quality of the software, willing to go above and beyond to achieve that excellence. They have to be leaders willing to see beyond self-interest and work towards consistently evangelizing the practice.

Collaborating teams build systems that collaborate

Information systems of a company are a reflection of the communication patterns within the organization. The more teams communicate with one other cohesively, these systems will connect and grow together more as a result. In today’s world of digital transformation and data intelligence, it is imperative that multi function organizations communicate and collaborate, so systems built by the development staff will bring the necessary business intelligence and scale. This cohesive intelligence is critical to further growth. It is very common in multi function organizations for ‘Silos’ to develop over a period of time. These ‘Silos’ focus successfully on one aspect of the business, however they fail to connect with other systems in the workflow such that it reflects the customer journey. This ‘disconnect’ can create many black-boxes producing their own data, which as a whole brings minimal value in the long run.

It is very interesting that many popular methodologies and architectural patterns have evolved which fundamentally solve this problem of communication. Let’s take DevOps for example. DevOps is designed to have better collaboration between Development teams and Network operations. Although they are independent in function, lack of communication between them results in surprises, errors and continuous finger pointing. This can be very expensive operationally leading to client dissatisfaction. Today DevOps has had great success which leads to test, buildĀ and deployment automation. This in turn increases speed-to-market and system reliability.

In today’s world of digital transformation and data intelligence, it is imperative that multi function organizations communicate and collaborate, so systems built by the development staff will bring the necessary business intelligence and scale as well.

Agile development methodologies solve the collaboration problems between all disciplines of an organization. The SCRUM methodology with it’s different ‘ceremonies’ brings all relevant departments together in one room regularly. From Sprint Planning to Sprint Retrospective, different departments like products, development, QA, and user experience come together to discuss topics from requirements to ‘lessons learnt’. This collective, methodical way of communication leads to better product definition, risk mitigation, estimation and release planning. Everyone is in a continuous improvement and feedback loop, improving deliveries significantly.

collaboration

Let’s take Micro-services architecture, for example. This style of architecture was embraced by many companies in an effort to have small teams become autonomous by fostering a loosely coupled architecture connected via independent services. Again, at a high level team structures and their communication patterns dictate how the Enterprise architecture shapes. Netflix adopted the Micro-services architecture to increase speed-to-market which led to a social culture of small autonomous teams collaborating together.

Companies don’t just build products and offer services , they build professional communities and offer personal growth.

Any organization which looks to build information systems that work together, while they are independent units, has to build a communication culture and team structure which can facilitate such systems. Cohesive systems bring business intelligence while loose coupling allows them to scale. Cohesion comes from continuous collaboration and loose coupling comes from small autonomous teams. Adopt methodologies and architectural patterns which bring teams and people together. Companies don’t just build products and offer services , they build professional communities and offer personal growth.