Cloud Deployment Techniques: Which is Best, Waterfall or Agile?

By Mark Gray, Senior Consultant with Navigator Management Partners

Most programmers hate documentation. More specifically, a large percentage of programmers are averse to thoroughly documenting their own code during its design and development. This maxim doesn’t necessarily extend to the poor souls responsible for testing or maintaining that software in the future. Ironically, the same programmer can be of two minds, depending on which hat they happen to be wearing at the time.

This is a simple understanding that I have had for a long time, since I myself was an application developer. Today, in my role as a Project Manager, overseeing the full lifecycle of development projects, systems implementations, and various other crossroads of the business and IT landscape, I have what I feel is a fairly good perspective on the subject from multiple points of view. As I have exposure to more and more toolsets in my daily work, I have had the pleasure of experiencing “self documenting systems”, the focus of this article.

In a traditional waterfall approach to development or systems implementation, there are rigid rules to the game. Each layer of the onion must be peeled back to reveal the next, and each set of deliverables must be completed in succession and approved to allow the passage of a gate, phase, or milestone. This has two significant effects on a project if done correctly. First, it ensures a clearly defined objective is met to the specifications of the stakeholders’ requirements and, secondly, it can take a significant amount of time and energy. Given that, in many situations, all of the answers (or even questions to ask) are not readily apparent. Add the inevitable time and budget constraints, and waterfall does not always make the most sense. Enter Agile and all of its glory to save the day…sort of.

An agile methodology shifts the paradigm. Live with greater ambiguity regarding the final outcome, focus on short-term objectives, and implement a constant cycle of working a backlog of user story-defined needs to create something quickly and get feedback. Lather, rinse, repeat. If done correctly, Agile works exceptionally well in the world of product creation, especially software development. It does not fit as neatly in the world of system implementation, where a great deal of process complexity can be involved, and entrenched business function often already exists. However, because of the potential difficulty of using traditional waterfall to deliver quickly and efficiently, many leaders wind up pushing toward a somewhat hybrid model of waterfall and agile at the end of the day. The proliferation of more cloud based systems, in theory at least, standardizes a broad segment of the process element for scalability, and eliminates “customization” in favor of “configuration”. This causes a stronger push towards the agile end of the spectrum – especially with the “self documenting system”.

The theory with a self documenting system is really that the product is what it is – the design is baked into the effort to configure the system itself in an agile manner. This sounds good and is not wholly invalid, but there are several points of caution here. The developer at the beginning of our article is happy to build something without documentation to save time and energy, but much less so once they are forced to test or maintain it. The same is true of our cloud-based systems. Based on my experiences and discussions with fellow practitioners, I have some basic concepts to avoid turning “quick wins” into trouble on the horizon.

Remember what we’re here for. Even in an agile framework, the goal is not to boil the ocean. There needs to be a good concept of what the end game is, although some of the finer detail will be defined iteratively. In fact, some would argue that having a strong foundational scope and set of requirements is more important in an agile project; otherwise, there is no way of understanding when the objectives are met and there is little hope to keep timeline and budget reined in. There is very little tolerance in executive suites to the idea of constantly building something and spending money all the while. Being able to clearly articulate the boundaries of the project will allow you to better understand change management needs, plan the timeline and resource demands, and have a set of goalposts to measure success by prior to user acceptance testing (UAT). Which brings us to the next topic…testing.

Requirements matter – a lot. While creation of process flows to the nth level might not be needed with a system that is largely off-the-shelf, good requirements are a must for several reasons. First, they help with the forming of our aforementioned scope. Secondly, the process of soliciting requirements from the client will draw out the concepts from which we will build our stories. Good requirements gathering, if done cross-functionally, can also foster a synergistic outcome – the sum is greater than the parts and it can be eye-opening how often department A had no idea department B did something important. Finally, trying to test against a configured system with no documentation is impossible to do well – requirements allow for traceability, which leads to a lower cost of quality. This is key in one other area and is often missed or minimized early in project – integrations and interfaces.

It is a rare situation when an organization has a single piece of software or platform from one vendor, with no interaction to other systems or sources of data. Even giant, international corporations with multi-million dollar ERP implementations will have some level of connectivity to other tools and processes – probably to a high degree. These will also almost invariably have a higher level of “customization” than the new cloud tool. Companies which have grown through mergers and acquisitions over many years have some of the most complex and convoluted situations of all. The reality is that, in many cases, documentation of what already exists is incomplete or non-existent. Therefore, there needs to be an understanding of the unique set of circumstances in order to determine where a component of design should be written in stone and documented in order to ensure proper build through deployment and support can happen. This can take a lot of time and energy, and often requires access to people without availability or even knowledge that no longer exists within the organization. In this space, each implementation will have some uniqueness to it. Understanding that as early as possible will illuminate risks and opportunities when they can have the greatest impact.

In conclusion, there is not a one-size-fits-all approach to systems implementation and the term “self documenting” is a bit of a misnomer. A combination of waterfall and agile techniques and mindset can and should be used in most cloud deployments to streamline processes while maintaining rigor and an eye towards quality. Putting in the work upfront to identify scope and clearly define requirements will enable better understanding of the project needs, allow for cleaner build of complex interface points, make solid, traceable testing possible, and enable change management and downstream support to be successful. Remember the developer with the two hats and find the balance in your particular project for yourself and the client.