“Quality is everyone’s responsibility” – W. Edwards Deming
Being someone who started their software career in the field of testing and later moved on to lead groups to implement automation within testing groups, I have a strong belief in the value of tested software. That being said, as we have shifted into the world of agility, the idea of “tossing features” over the wall is an antiquated and less productive way of thinking. Software testers, QA or whatever you want to call that domain do NOT assure software quality in isolation. I have been thinking a lot about this lately as we are examining test automation and it has been primarily at a GUI perspective and it makes me wonder if we are fascinated with the tools or actually have a solid test strategy.
When I began my journey towards becoming more agile, I came across a phrase very early on that has stuck with me and helped not only shape my ideas of software quality but also discard some of my previous teaching as a quality domain professional. The phrase is the title of this blog post … “baking in quality”.
I suppose this resonated with me initially based on the bad experiences that all testers face in their career where they are provided features late, largely untested and often in compromised product schedules.The idea that as a team we would ensure quality appealed to me as it made it a shared responsibility of the team as a whole and not the sole domain of the angry tester telling the developers why their code is full of bugs.
In direct response to this when I began building software teams, I routinely asked the question “how many testers are on this team”? Commonly, I got the answer that only counted the people that had expertise in the testing domain. To which my reply was always, “I see N number of testers” (replace N with the total team number). It took a while but the phrase “everyone is a tester” is something I hear routinely although I do not think it is at maximum effectiveness just yet.
Adding the Right Measure of Ingredients
Mike Cohn in his book “Succeeding with Agile” talks a lot about the idea of testing and how automation is critical to the testing process as teams mature and become more effective. Almost every new team starts out with manual testing of some sort and the pains of this practice show very soon when the teams start hitting a consistent cadence. Without good quality practices many teams, including my own teams, begin emulating a “mini waterfall” method in which test domain experts spend a great deal of time front-loading manual test scripts and being delivered features late in the sprint in which they scramble to get “tested”. Even with the “everyone is a tester” belief ,the primary test expert is often overwhelmed by the wall of days of code development and scrambles to meet the commitments made by the team. Most testers are used to this “stuff rolls down hill” scenario and are often powerless to avoid it. But I think there is a better way to “bake in” the overall quality of a product. Will it be easy? For some, maybe as they are well on there way bt for others it will be a large hill to climb. The great thing about being truly agile is we evaluate small experiments and adapt and adjust as we learn.
Mike Cohn and Martin Fowler both talk about the testing pyramid. If you are well read in quality techniques or have a lot of experience in the test domain, you have likely heard this term. The traditional way I was taught to incorporate this philosophy into testing looked something like this:
This stemmed from the idea that testing at the user interaction level would yield the best results in terms of finding problems. This approach was a standard for a long time in many gated project methodologies. I made a career pivot in teaching others how to effectively use testing tools, many of which are targeted at the E2E (end to end) GUI level, to build scripts to do this. But given we are developing the system in an iterative fashion these days, this diagram no longer makes sense. (oh my shattered innocence)
Cohn and Fowler talk about the inversion of the pyramid focus in terms of agile development so that it looks something like this:
Part of the realization behind this is that with the system being iteratively built, the volatility of the interface (and therefore the GUI script fragility) can be slow and costly to refactor in the midst of building new features. Many teams work to mitigate this through adding a specialized role to focus on test automation, but this is again an increase in cost (as it adds a speciality person to the team) and promotes a decrease of team ownership of the quality of the product through relinquishing this responsibility to “the automation dude”. Some create a shared service that works across teams to take on this responsibility and typically the expanded scope of this person means they need to maintain multi product and possibly platform understanding and therefore defects may come later into the mix once the scripts are shored up. These folks are almost always focused on E2E regression tests and do to their spread scope, they are often just churning out scripts with little quality oversight to their own work. So this is where Cohn and Fowler flip the approach towards a “baking in quality” approach but one that uses significantly more code level testing and less GUI level E2E testing.
Their assertion is that if we are maintaining a solid base of unit tests (maybe even couple with gated code checkins) and upon that building what they call a strong “service” level of tests, which they define as “the points in which the application responds based on a set of input or inputs” that a large degree of code that is small, fast and repeatable is available to exercise the system regularly. This also alleviates working on redundant GUI level tests for validations and the focus can be for more “user journey/smoke tests/happy path” tests for E2E. This means that tests can be adjusted as a part of the overall code base for the most part and that the focused efforts of GUI level testing can be more targeted to perform an overall goal action.
So, what does this mean in practice?
It primarily means that if we want to make a shift towards “baking in quality” our strategy might be to focus on building in quality as we construct as opposed to trying to post build or bring in specialized people whom we entrust to build in this quality. It means that if we think that the quality of our products are important that we build in this discipline to how we work as a whole, not as a sub-process. It means we internalize the concept that “baking in quality” in how we build products, is what we do as a culture.
Before all of the automated tool vendors gather their armies and seek me out (as I have used a lot of these tools and like a lot of them), I do see a tremendous amount of value in both commercial and open source testing tools and frameworks. The thing I do see is the same thing I saw when I was brought into companies to help them with automated tests; either their testing practices were weak or they were amazed by a sales demo and were ready to automate everything based on the sales pitch, which always worked, in the hands of experts and in controlled environments, flawlessly.
But as I am often noted to say, “I think we may have our emphasis on the wrong syllable”. Maybe we should be doing as these agile pioneers suggest and working towards baking in this quality through solid coding practices as a complement to effective use of automated e2e GUI level testing. Maybe we’ll bake in more quality as a whole within our products.
I have spoken to a great deal of testers who have told me all of the amazing things that their automated GUI scripts have found in terms of defects and monies saved. I get it. GUI testing of systems has value. I agree. As a matter of fact I whole-heartedly agree, especially when it comes to becoming even more “dev-opsy” and you can provide operational deployment scripts which run smoke tests so that they can execute them as needed and feel confident about a deploy. So, don’t shoot the messenger here.
However, I do also believe that effective use of code level testing + guerilla and manual testing + GUI regression and smoke tests might be an effective recipe as well. Some of this can be supported with the challenge I put forth in my last blog post ( Just one thing … ) recently.
And will it slow down a team initially? Likely. But I think it can be accepted and will pay dividends in terms of more quality produced code and products overall in which extensions can be tested to effective ensure less “breakage” across a sea of code. And I believe once the practice becomes ingrained, the velocity of delivery will become consistent. It also allows you a terrific pairing mode opportunity for new hires to learn about code under construction.
So as usual, I am up to run an experiment to see if this works and bakes in more quality in products effectively. Are you? Have you already done this?