The cobbler’s children have no shoes.

“If we desert the inner self to focus on the energy outside, it leaves a vacancy inside”              – Trudy Vesotsky

There is a dilemma that every software group I have worked with has faced and I have never seen anyone actually “solve it” in the grand sense although I have seen some really good ideas in action. It’s the idea of investing back into the work you do as a software group so that you can leverage this investment in the ability to continue to “do the work” for others.

Hence the reason for my blog title. The cobbler does amazing work for all of the paying customers wanting to buy shoes but never finds the time to put shoes on his own children’s feet or teach them to be a cobbler themselves.

Where does this often stem from?

In my experience, this perpetuates inside an organization in several different ways:

  • Fear to apply capacity by the leadership for something in which direct end profit is not seen or justification seems hard to communicate upward to these leaders without technobabble.
  • Guilt, on the side of the workers, who feel like they will never be granted the ability to do this so they either complain or burn the candle at both ends to make it happen in shadows.
  • Lack of focus or understanding of need. As Stephen Covey talks about in his works, sometimes we are “too busy picking up rocks to as what kind of rock we are looking for”.
  • Sometimes it is just culturally engrained (which is the toughest). It is just generally accepted (and often with great grumbling) that anything without direct tangible end stakeholder benefit is just less important.

I remember when I was a young IT professional and I was sitting with an assistant director of the company who had tasked me to thoroughly understand how their IT organization worked and look at improvement opportunities.

I posed the question “I have observed that any project that is related to the business receives a budget, people and focus but I cannot find where the same approach is made when approaching upgrades of tools, servers or your technology stack and training, why is that”? There reply was immediate without hesitation, “those things are the cost of doing business and therefore below the line and as they are internal in nature are more flexible than customer delivery”.

I paused for a moment. “So, if they are the ‘COST’ of doing business, why are we paying for them on a revolving payment plan? What I mean is if the core goal and mission is to deliver modern and innovative solutions to end stakeholders, how can the justification of making the things upon which support your competitive advantage in the market be something that is addressed in a “stop and start” manner to be superseded by other items without completion”?

And here it came … the death blow. “This is the way it’s always been. We cannot just take time to improve internally and not deliver things to the customer”. Discussion over.

I walked away from this discussion feeling really sad (and as a young professional, deflated and experiencing an early disillusionment) as this company had so many bright, excited people who were inspired to do the work they did. They wanted to do interesting things and support the business through bringing potentials of innovation. They were, instead, given the communication that we keep the factory running. We only think about making improvements when a production shutdown is going on. What this meant for them over time was they saw themselves get further and further behind and the climb to get current made the effort, the learning curve and possibly the investment so great that they would never receive the support they needed to make it happen. So, they grumbled, they left, or they resigned themselves to work with what they had to get the work done.

I tell this cautionary story as in reading it you may see similarities in the company you are in now. If you are and you are a leader in this organization. Affect change. Even the smallest changes you can do as seamlessly as possible to keep some cadence of improvement going. Don’t let the hill you pile up for yourself make it impossible to climb.

So what can we do about this?

Again, I do not have “the answer” as my organization struggles with this like others do. We have at least made a commitment to our culture and ourselves to keep visibility on this and push for these things without fear. Sometimes we win and sometimes we don’t.

Please do not misunderstand me in discussing this. I am not talking about the “junior level developer fascinated by every technology under the sun” sort of change. Not that you are in a constant pivot. But that if you are going to be in the business of software, you have to ensure that your tools and approach allows you to grow at a reasonable pace with the industry”.

You do not want to find yourself in a situation where the iPhone has been out for a decade telling your team, “we need to learn how to develop for mobile”. You might get there but it’s not likely that you are going to be competitive here. You probably missed your window. The same comes from jumping on tech early just to “get there” with no plan on  cultivating the approach or refining the skills. I cannot tell you how many government agencies (and companies that contracted to them) I saw build a native iOS mobile or Android app just to see it wither on the vine, never get updated or enhanced and die on the vine. Mobile App storefronts are littered with these types of applications. They wanted to be “first” so they could take advantage early for the trend but in hiring out the work, they created a dependency on a end vendor to update this platform and typically at a premium, which often meant that unless the app was getting mass attention, slowly stopped being paid to update and support.

Those cautions aside, have some seasoned technology professionals (CTO, architects, senior developers) in your organization that watch technology trends and “sticking patterns” of languages. Understand your regional technology market so that you can hire for technology you can support more readily or develop a good “gig based economy” model for models that work in conjunction with teams you are training on a new technology. Don’t think you can ever stop learning and growing. And as a result of that, keep your toolsets as current as reasonably possible so that you are riding closer with software trends and not trying to just race to catch up.

If you are in the business of software, and this applies even if you are a software group within a non software company, cultivate and advocate to the business leaders as a whole being reasonable about the investment they make in technology as a core support of the business. Don’t be silent while budgets are drawn and plans are made and let the needs of the very thing you do not be heard. Be certain, the “cost” of doing business in building software is in fact a cost. Do not let it become something to do when there is time.

Get creative.

One thing we can try (and that we do here) is to adjust capacity of people to try and free up time for focus on doing these types of things. This can often be more palatable for a business. The idea of losing 2-3 people for 1-2 weeks to focus on this thing is something that “feels” reasonable on face value. It’s about the impact of a vacation by an employee. You will be amazed at what people who are motivated to leap, with some small pre familiarization, can get done when freed up to focus on a task. Just try and keep things small, targeted and end goal based. Even if the goal is research of a technology, have an end goal in mind like a “go/no go” and recommendations for investment.

Outline the tangible benefit for the effort. Think about what the investment will ultimately do for the business. And it does not have to be only about technology. Maybe it will allow you to recruit better in the market space, maybe it will allow you to retain staff as they continue to grow, maybe it is an early investment in the direction a given technology is headed. And sometimes, just sometimes; it’s to ensure the business does not make a large scale investment technically that it will be dependent on a recurring vendor cost to support, maintain and update or the marketspace indicates that leveraging of this technology is actually less effective than the vendor indicates. But stay focused to work on it. Don’t shelf work each time more work comes in. If you continue to do this two potential situations can result a) the ramp up time to get back to where you left off multiplies each time you walk away or b) the direction you were headed becomes obsolete and therefore require starting over or abandoning the effort (which may not be costly but it does have a negative impact on those who have done the work).

Don’t let yourself keep building shoes for others and never attending to your own family.

Take a moment and pick one of these kids and build them some shoes!

Baking in Quality

“Quality is everyone’s responsibility” – W. Edwards Deming

Being someone who started their software career in the field of testing and later moved on to lead groups to implement automation within testing groups, I have a strong belief in the value of tested software. That being said, as we have shifted into the world of agility, the idea of “tossing features” over the wall is an antiquated and less productive way of thinking. Software testers, QA or whatever you want to call that domain do NOT assure software quality in isolation. I have been thinking a lot about this lately as we are examining test automation and it has been primarily at a GUI perspective and it makes me wonder if we are fascinated with the tools or actually have a solid test strategy.

When I began my journey towards becoming more agile, I came across a phrase very early on that has stuck with me and helped not only shape my ideas of software quality but also discard some of my previous teaching as a quality domain professional. The phrase is the title of this blog post … “baking in quality”.

I suppose this resonated with me initially based on the bad experiences that all testers face in their career where they are provided features late, largely untested and often in compromised product  schedules.The idea that as a team we would ensure quality appealed to me as it made it a shared responsibility of the team as a whole and not the sole domain of the angry tester telling the developers why their code is full of bugs.

In direct response to this when I began building software teams, I routinely asked the question “how many testers are on this team”? Commonly, I got the answer that only counted the people that had expertise in the testing domain. To which my reply was always, “I see N number of testers” (replace N with the total team number). It took a while but the phrase “everyone is a tester” is something I hear routinely although I do not think it is at maximum effectiveness just yet.

Adding the Right Measure of Ingredients

Mike Cohn in his book “Succeeding with Agile” talks a lot about the idea of testing and how automation is critical to the testing process as teams mature and become more effective. Almost every new team starts out with manual testing of some sort and the pains of this practice show very soon when the teams start hitting a consistent cadence. Without good quality practices many teams, including my own teams, begin emulating a “mini waterfall” method in which test domain experts spend a great deal of time front-loading manual test scripts and being delivered features late in the sprint in which they scramble to get “tested”. Even with the “everyone is a tester” belief ,the primary test expert is often overwhelmed by the wall of days of code development and scrambles to meet the commitments made by the team. Most testers are used to this “stuff rolls down hill” scenario and are often powerless to avoid it. But I think there is a better way to “bake in” the overall quality of a product. Will it be easy? For some, maybe as they are well on there way bt for others it will be a large hill to climb. The great thing about being truly agile is we evaluate small experiments and adapt and adjust as we learn.

Mike Cohn and Martin Fowler both talk about the testing pyramid. If you are well read in quality techniques or have a lot of experience in the test domain, you have likely heard this term. The traditional way I was taught to incorporate this philosophy into testing looked something like this:

invertedPyramid

This stemmed from the idea that testing at the user interaction level would yield the best results in terms of finding problems. This approach was a standard for a long time in many gated project methodologies. I made a career pivot in teaching others how to effectively use testing tools, many of which are targeted at the E2E (end to end) GUI level, to build scripts to do this. But given we are developing the system in an iterative fashion these days, this diagram no longer makes sense. (oh my shattered innocence)

Cohn and Fowler talk about the inversion of the pyramid focus in terms of agile development so that it looks something like this:

test-pyramid

Part of the realization behind this is that with the system being iteratively built, the volatility of the interface (and therefore the GUI script fragility) can be slow and costly to refactor in the midst of building new features. Many teams work to mitigate this through adding a specialized role to focus on test automation, but this is again an increase in cost (as it adds a speciality person to the team) and promotes a decrease of team ownership of the quality of the product through relinquishing this responsibility to “the automation dude”. Some create a shared service that works across teams to take on this responsibility and typically the expanded scope of this person means they need to maintain multi product and possibly platform understanding and therefore defects may come later into the mix once the scripts are shored up. These folks are almost always focused on E2E regression tests and do to their spread scope, they are often just churning out scripts with little quality oversight to their own work. So this is where Cohn and Fowler flip the approach towards a “baking in quality” approach but one that uses significantly more code level testing and less GUI level E2E testing.

Their assertion is that if we are maintaining a solid base of unit tests (maybe even couple with gated code checkins) and upon that building what they call a strong “service” level of tests, which they define as “the points in which the application responds based on a set of input or inputs” that a large degree of code that is small, fast and repeatable is available to exercise the system regularly. This also alleviates working on redundant GUI level tests for validations and the focus can be for more “user journey/smoke tests/happy path” tests for E2E. This means that tests can be adjusted as a part of the overall code base for the most part and that the focused efforts of GUI level testing can be more targeted to perform an overall goal action.

So, what does this mean in practice?

It primarily means that if we want to make a shift towards “baking in quality” our strategy might be to focus on building in quality as we construct as opposed to trying to post build or bring in specialized people whom we entrust to build in this quality. It means that if we think that the quality of our products are important that we build in this discipline to how we work as a whole, not as a sub-process. It means we internalize the concept that “baking in quality” in how we build products, is what we do as a culture.

Before all of the automated tool vendors gather their armies and seek me out (as I have used a lot of these tools and like a lot of them), I do see a tremendous amount of value in both commercial and open source testing tools and frameworks. The thing I do see is the same thing I saw when I was brought into companies to help them with automated tests; either their testing practices were weak or they were amazed by a sales demo and were ready to automate everything based on the sales pitch, which always worked, in the hands of experts and in controlled environments, flawlessly.

But as I am often noted to say, “I think we may have our emphasis on the wrong syllable”. Maybe we should be doing as these agile pioneers suggest and working towards baking in this quality through solid coding practices as a complement to effective use of automated e2e GUI level testing. Maybe we’ll bake in more quality as a whole within our products.

I have spoken to a great deal of testers who have told me all of the amazing things that their automated GUI scripts have found in terms of defects and monies saved. I get it. GUI testing of systems has value. I agree. As a matter of fact I whole-heartedly agree, especially when it comes to becoming even more “dev-opsy” and you can provide operational deployment scripts which run smoke tests so that they can execute them as needed and feel confident about a deploy. So, don’t shoot the messenger here.

However, I do also believe that effective use of code level testing + guerilla and manual testing + GUI regression and smoke tests might be an effective recipe as well. Some of this can be supported with the challenge I put forth in my last blog post ( Just one thing … ) recently.

And will it slow down a team initially? Likely. But I think it can be accepted and will pay dividends in terms of more quality produced code and products overall in which extensions can be tested to effective ensure less “breakage” across a sea of code. And I believe once the practice becomes ingrained, the velocity of delivery will become consistent. It also allows you a terrific pairing mode opportunity for new hires to learn about code under construction.

So as usual, I am up to run an experiment to see if this works and bakes in more quality in products effectively. Are you? Have you already done this?

Just one thing …

Curly: Do you know what the secret of life is? [holds up one finger] This.

Mitch: Your finger? Curly: One thing. Just one thing. You stick to that and the rest don’t mean s***.

Mitch: But, what is the “one thing?”

Curly: That’s what you have to find out.

Jack Palance and Billy Crystal  from the movie “City Slickers”

I always found this scene amusing when I was a kid. The old ranch hand trying to teach a city guy the value of focus for clarity. Of course at the time I just thought it was funny. After all these years, I realize that Curly was teaching a true agile value. Focus.

Focus is that thing that often gives us clarity. It gives us the ability to think deep thoughts and really unravel an idea. It allows us to extrapolate from a big thing into discrete pieces upon which we can undertake. It applies to things in life as much as it does in software development. I have started to really appreciate this as I prepare for an upcoming house declutter and move.

Do I allow myself to always embrace this seemingly simple approach. Nope. I, like others, allow myself to get too “focused” (and I use this term loosely here as it really not focused at all) on too many things. Like those performers that are spinning plates on a skinny stick, I find myself focusing on keeping them all spinning sometimes. But, as I am want to do, which often results in a blog post, I find myself stopping, breathing, thinking, assessing and questioning my current chartered course. So here we are.

Focus is a core value in agility as it allows us to actually be more productive by allowing ourselves to work to accomplish “that one thing” as Curly put it.

Under scrum, we demonstrate this through the user story, the task decomposition in terms of work effort. We use focus in terms of daily risk mitigation through a focus meeting called the  stand-up. We focus on the features committed within the review and the team within the retrospective. If you look closely, focus is woven throughout

Kanban has WIP and the innate concept of “pulled work” which itself a way to focus and measures this through the idea of how effective our focus on delivery is through things such as cycle time.

But there is one dimension in which I often see less focus. In the process of delivery. What I mean by this is often I see scrum development teams use a “divide and conquer” approach in which teams members use their specialized skills to scatter and seek to deliver. They are in no way being less agile in this approach but I have always wondered what would happen if they allowed themselves to focus and iterate just like the larger product. There are examples of this. Just take a look at Woody Zuill’s mob programming or pair programming as outlined in Exteme Programming by Kent Beck.

My latest read, “Joy, Inc” talks about the use of pair programming as the standard way of how they hire and how they work. They see it as not only a way to create shared knowledge by this but also to ultimately be more productive.

So, I have posed this question many times to developers and often get the same response “well, development doesn’t quite work that way” to which I often reply “are you willing to try an experiment and either fail or be wrong”? *smiles*

My logic has always been that if a sprint backlog is supposedly composed of a top to bottom prioritized list of the highest business value, wouldn’t we want to work through the items (given we identify any dependencies and work to minimize those with the product owner) and focus to deliver each story in that manner during a sprint? Or even look at the absolute most complex item and focus on completion of that story and then work on the lower hanging less dense fruit?

I see 4 potentially significant benefits that could result from this:

  1. We get items in front of our team members specialized in quality quickly so they can assess features for compliance to acceptance, note any software issues or quality enhancements and they can become productive more quickly. They are reviewing and testing completed features, even if there are mocks behind some dependent needs of other stories. They also are able to learn a portion of the system under construction and build automation against it for regression more quickly if needed.
  2. If we for some unforeseen reason we are impacted to effectively deliver on all stories for our commitment, we can at the first responsible moment, assess remaining team bandwidth, current deliverable dependencies and make decisions so that we can have that conversation immediately with our product owner so we can remain transparent on the impediment and impact but emphasize our focus on the highest priorities or largest complexity first.
  3. We reinforce iterative development at the lowest level of development (by using refactoring and integration as part of the way we work) and can focus on ensuring that we meet our agreed definitions of done at each story level.
  4. We “should” end up with a set of features tested early which should allow us to as a team to swarm baking in quality for our product and preparing for our review at sprint’s end (given we did not over commit …)

So in my view this approach makes sense to me and I have worked on teams that worked this way and we remained productive and felt like we were accomplishing more towards our product goals. We effectively delivered features regularly for review for the stakeholders.

A lot of folks may be reading this and joining the chorus of previous conversations saying “yeah, sounds good but you don’t really understand” …

So I reply to you all … “are you willing to try an experiment and either fail or be wrong”?