Gallery

Interested in DevOps? Try XDE with XP practices

Culture, movement or practice whatever you call it, DevOps is beautiful. It ensures the collaboration and communication of software professionals who usually doesn’t think alike in most organisations – the Development team and the Operations team. DevOps practices advocates automation of the process of software delivery and infrastructure changes to align these two. Scalability, security and organised chaos promotes a distributed decision making culture, exactly what we need for being agile.

So which framework best suits us, while adopting this DevOps culture? In my biased opinion I feel it’s eXtreme Programming (XP). It’s brilliant practices/ideas are often borrowed by other frameworks including the concept of “User Stories”. Since most frameworks doesn’t specify these practices, most professionals include XP principles (e.g. TDD, pair programming) anyway. Reason why XP, as a methodology, is underrated and overshadowed by other popular frameworks quite heavily.

Xtreme Decoupled Engineering (XDE) framework compliments XP, by adding the concept of decoupled processes and making sure DevOps adoption doesn’t stay just a great idea but is also implemented to save the world 😉

Individuals and interactions OVER processes and tools

“Over” is often read as “Not” by a majority of Agile adopters, who finally starts to realise why agility is far better than many traditional techniques. Is it their mistake? Of course not. Agile Manifesto is quite philosophical and the inclusion of “over” just assumes that the reader won’t translate it to suit their needs. It’s either this or that, not both. If both then to what extent? No one have a definite answer, as it was meant to help evolution which is the beauty of the agile manifesto. But it does scare the organisation trying to transition as not all organisations are consultancies. Not everyone is working for a “client”. Sometimes stability is more important to measure few aspects of the transition. This stability and standardisation of processes is necessary to some extent, as long as it’s not blocking the product development.

No doubt, Individuals and Interactions are important, but it can’t work on it’s own without processes and practices to support the outcome. We need a basic level of standardised processes and practices to accompany this vision of adopting devops to become agile. In fact, most frameworks have vague undocumented processes which are not standardised for all teams. It is extremely unsettling for organisations who are paranoid to begin with. If documented, they are extremely ambiguous, hence often misinterpreted (ref. 1, 2, 3).

XDE complimenting XP – Closing the gaps

I have never seen XP work within immature teams because it needs a sense of responsibility and the urge to know WHY, which only comes from experience. If we ask an immature team to follow XP, they usually try Scrum instead and include TDD and pair programming to call it XP. Mostly because Scrum has a guide which they can refer back to. But there are some subtle differences between XP and Scrum, as documented by Mike Cohn.

When we realise most frameworks are ambiguous about implementing set processes, we often fill in the gaps to support the agile principles ourselves. But during this we may end up in a process which can do more harm than good. By leaving a gap we are letting our mind wander. Most professionals look for the processes first and then learn the principles behind it. Some never care to learn the principles at all, as they assume implementing a framework takes care of everything.

This blind faith and incomplete knowledge promotes half baked assumption of knowing what works and what doesn’t. First we should go by book and then we should focus on mastering it or even bending the rules. XDE tries to close these gaps by formalising the Definition of Done and support to DevOps mindset while advocating the best practices from XP.

Companies trying to adopt DevOps needs a framework which have a set of processes for all teams; is predictable yet highly customisable.

XDE provides that skeleton by defining the start and end of the development lifecycle within the bubble. “Done” for a product increment is defined to include End user feedback – Continuous Delivery plus at least some feedback from users before starting the next work item. It creates a transparent environment of keeping the road map visible at all time by focusing on the value to the end user.

To assure that the Bubble doesn’t start working on the next item in the backlog, XDE introduces One Rule (1R) which creates a process of working on one at a time and only focus on outcome not output.

One Rule (1R) makes sure we are focused on Outcome not Output.

Decouple Processes and Succeed

As we know XDE doesn’t proposes any practices on how the product is built but it recommends XP principles. XP principles with it’s test first approach suits the best but needs a robust stabilising skeleton which XDE provides – hence compliments each other. While the 1R team members work following the One Rule, if a team member is free doing nothing (as they are not allowed to work on anything else) they have no choice but to focus on the ongoing work.

  • “Are you free? Great, get the chair and let’s pair for the rest of the day.”
  • “Yoda is off sick today and we need to review the unit tests before he can start implementing the code tomorrow. Can you do it while you wait?”

Therefore, XDE helps organisation to adopt devops mindset smoothly with the least friction possible and XP assures that the quality of the delivery is spot on and ready for feedback. Try XDE along with XP to initiate the DevOps mindset and help your organisation is agile transformation. Focus on outcome not output.

Gallery

Test Automation is Dead, Period.

This article is aimed to all software test professionals who are working as or planning to pursue a career as Automation Test Engineer/SDET/SDITs in future. If the organisation we are working for is trying to be agile, these test professionals can make an enormous contribution by “directing” the development and not just being a silo of knowledge on a specialised department. So focus and read this very carefully as this will make you the legend you can be, before learning something which is dying or is already dead in agile development.

Simplest definition of test automation is provided in wiki –

Test automation is the use of special software to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

 

“Special Software” – this is your hint. An automation testing framework is exactly that, which is maintained side by side. In many cases, it has an entirely different code base; responsibility of which falls on the shoulders of specialist Automated Testers. I have been there myself. While working as a Software Developer in Test, I used to create frameworks from scratch and maintain. Doesn’t matter what programming language the application is written on, these frameworks can “test” every bloody functionality on the application under test, brilliant stuff. UI, API or Integration you name it, we used to have solutions for everything. Reason why I have enormous respect towards the Testing Community. Only problem was, we were testing AFTER the product was built. Here’s why it didn’t made sense.

Someone gave me a piece of advice years back – “Don’t call it wrong, call it different”. My reaction was neutral, even though I completely disagree, as the person had like 20 years of experience. In fact, I thought I learned something “valuable” that day from a very experienced professional. Few months later, both of us shared an awkward glance, which to some extent proved that I didn’t learned anything valuable and was simply fooled to believe so. That “calling different” attitude became an impediment on an ongoing project as it needed urgent rescue and we realised we should have called it “wrong” and made it right.. sad times.

 

My point is, we have to call it wrong when it is; unless you can never initiate the process of correcting it. Calling it “different” is bureaucratic encouragement to keep doing whatever we are doing and screwing the organisation we are working for, sometimes unconsciously. Let’s come to the subject which needs help, by not being called as different/wrong but being surgically removed – Test Automation. It might be right in a distant past, may be for some, which I have never seen producing real value so will trust and respect the past decisions.

Ask any software test professional “What is your test coverage like?”. You will usually get a reply with a number or percent. You can easily assume they have an Automated Testing Framework to check if the tests are passing, after a product is build/changed. These tests reduces the manual effort while regression testing (mostly functional tests) and saves valuable time before the release deadline. Isn’t it? Proud organisations share these metrics often by saying “we have X% test coverage which is way better than market standard of Y%”.

Now, here are some words/phrases mentioned in the above paragraph, which we should pay attention and understand why they smell.

Testing after the product has been build/changed

If we are testing after a product is built, we are –

Simply validating what it does, not what it should do.

If we don’t want a baby, we use a condom. We don’t have sex anyway and pray that she will not get pregnant; well unless you forget to buy one. Prevention is always advised over “dealing with the consequences” later on, doesn’t matter if it’s your sex life or delivering a software. Have protected sex and stop worrying about raising an “unwanted” baby or even worse consequences. The tests should be our condoms which we put in place before building a product to establish a predictable outcome. Try using this example at your work. If you get a mail from your HR then I am not responsible but feel free to blame it on me. Use weird analogies like this to make a point, people usually remember them for a long time.

Regression testing before the release deadline

Admit it, it’s the truth. In fact, I have seen many teams doing a ceremonial regression testing before a release date. They still does regression testing even if they have automated test coverage (god I hate the term) of 85-90%. Guess what, they still have missed critical bugs. When asked, the usual reply was “it was an edge case which wasn’t covered in our automated testing framework. We will include the test before the next production deploy, promise”.

So do we do regression testing in targeted areas, as in risk based testing? Yes we can, but then we might miss several other areas which “we think” is not risky. We can never win as there will always be bugs.

As long as those bugs are not failed acceptance criteria, they will be treated as learning rather than fails.

The trick to improve is to run regression test while doing continuous integration as part of the same code base. If it fails, fix and re-run as explained in the XP best practices. At least it is failing on your CI environment not in production. Regression testing should happen every time there is a change, doesn’t matter how small the change is. It should happen as early as possible and should never be treated as a ceremony.

Exploratory Testing, therefore, should be the only activity after a product is deployed to find edge cases and that’s why manual testing can never be dead.

Test automation, therefore, might reduce your manual effort but you are still managing and troubleshooting a separate framework and wasting enormous amount of time and effort to prove something which is already given. It is still not “defining” what the product should do.

Test Coverage fallacy

Quality assurance starts before the software is built. Fact is, it never assures the quality if you are writing tests after and it simply becomes an activity of executing the tests. So burn that test coverage report as it only proves that the work is done. It never says how it assures the quality.

Another aspect of “doing it right for wrong reasons” is unit testing in TDD. A good amount of software development professionals and higher management uses the % “test” coverage at unit level in their metrics to prove the hard work done to assure quality. You might know where I am going with this. If you are doing a Test Driven Development (TDD) you are NOT testing anything that matters to the end users, so stop adding the number of unit tests in your test coverage report.

TDD is for architecture and is a brilliant method to create a robust solution which supports the functionalities on top of it. The unit tests are merely the by-products of a good design and does not guarantee the complete functional behaviours. TDD is always recommended but don’t try to make it sound like it’s testing the acceptance criterion. In fact, a good programmer writes these unit tests to be confident in his own coding skills not because someone ask them to do it. In most cases, TDD is enforced to reduce technical debt by creating a culture of continuous refactoring.

Third aspect of the coverage fallacy is, coverage doesn’t mean it is always fully automated. It can be half manual and half automated. Coverage simply means we are confident about the % number of the functionalities and the rest are not tested. This creates a sense of false security. We should always test everything and we can only do it when we direct what we are building. If exploratory testing finds an edge case then we can add a direction to tackle the behaviour and not write to test to run for later. Regression testing after the changes are in place is considered already late. If you are an experienced testers you might read the code changes and figure out what areas have no affect by the change and decide to leave it, but you are still late.

Use your expertise BEFORE the development – Acceptance Guides not Tests

So what about you planned career as a Test Automation engineer? Well, no one said your skills are worthless, in fact you can use more of it.

We just have to shift left, a bit more left than we initially expected.

Not only we have to stop writing automated tests, we write them as guides before we decide to start working on the work item. So the programmers now have to implement the minimal code to pass your statement, hence preventing the defects for you. Better if you can help them in a pair programming session. Being said that, it doesn’t mean that we cannot apply the principles behind TDD. Enter Acceptance Test Driven Development (ATDD), Continuous Integration and Continuous Delivery.

Assuming that everyone have heard/read about these; if not there are a huge collection of online materials to read. In a nutshell, ATDD advocates that we should write acceptance tests before writing the code for the software we build. It defines what we should make with minimal codes to pass these tests, just like TDD but in a higher level which the end users are expecting to work.

Are these acceptance tests really tests though?

Yes and No. These are not tests but are called as one in my opinion (topic of a debate?). Enlighten me if you know. Instead these are expected results (or behaviours) which are used as guides to develop a piece of functionality. Therefore, we should really be calling them “Acceptance Guides” (just a thought). Reason why a lot of us use behaviours as proposed in BDD, as they kind of compliment each other. These can include non-functional tests as well.

 

Verdict

Therefore, to summarise, we are talking about removing regression tests completely after development while including them within the same code base as acceptance tests/guides (doesn’t matter what it’s called at this moment). Also, we need to remove unit tests as a data to include within a Test Coverage metrics. Behold, we have just proved Test Automation can be dead or is already dead.

Be Lean to stay agile.