In this episode Matthew and Federico sit to talk about some of the changes that are happening in the ASP.NET QA team. We have talked about the evolution of our team, and today we go over the idea of dividing the test team into functional disciplines and how it has worked so far.
100% of tester's time was dictated by a feature crew.
The 3 pillars of the team
Product Presence - test presence during product design and prototypes.
Product Development - core testing of new features as they are incorporated into the product.
Engineering Services - automation strategy, tools/infrastructure and regression testing.
Work flow of a feature crew
As a new feature is designed PP is engaged during early prototypes, verifying scenarios with customer expectations, first exploratory sessions.
When the feature is mature enough, PP hands of testing to PD group. These group of testers are normally assigned to cross discipline feature crews and perform the bulk of exploratory testing and any other test activities.
As the feature stabilized, ES comes in to help automate regression tests, making sure tests are robust and maintainable. At the end this group continues to run regression tests of the whole product.
Take away
Still an experimental model.
So far the gains seem to indicate that the overall capacity of the team is increased.
A potential next step is to recognize the value of "on-demand" testing and "constant" testing, and to focus a group of testers on each.
In this episode Federico and Matthew narrate their experience of how having unclear expectations can lead down a path that can eventually create a culture of treating QA as a burden. It is a series of event that we call the vicious cycle of QA. Federico has been on the team for almost eight years and today shares his observations of how the team got itself into an undesired place, what were the symptoms, and what our decisions created. Look for a future podcast to learn what actions we took to break this cycle and move forward.
In a nutshell: unclear expectations leads to blame which leads to more process which leads to bottle necks which leads to more unmet expectations.
Step 1: Unclear expectations
QA, PM and Dev do not have a clear understanding of what is the role of the testers. What can they expect from them, and more importantly, what are the limits of their responsibility.
Having unclear expectations means that every person on the team has a different idea of what are testers supposed to do. Even higher level managers.
Step 2: Fail and get blamed
Without having clear conditions of success, what may turn out is that each miss is considered a failure.
Since the responsibility is not clearly defined, it is easy to have the QA team be the focus of blame.
Step 3: Cover your back with processes.
The QA team may turn to create processes as a means to cover their backs and transfer responsibility to the rest of the team.
Many processes are created that gives security that as long as they are followed, the QA team can shield behind them.
Step 4: Push back when process doesn't fit the plan.
With so many processes and checklist across the board, the QA team may become the bottle neck of the development cycle.
Since processes have to be guarded, QA team starts to push back on new features due to lack of time to properly do testing.
Turf wars may begin between the several disciplines.
Repeat:
Push and pull eventually settles into a set of constraints that QA is unable to meet given the high cost of test process.
More unmet expectations ensue.
Culture of QA as a burden
Since the processes where created to cover ones back instead of doing smarter testing, the perceived value of QA is still low.
Eventually the perception and reputation of the QA team is that of a high cost and low value.
How do we break out of this cycle? How do we turn around the team and perception? This and more questions will be addressed in a future podcast
In the Episode Matthew sets down with Drew Miller, a fellow tester, and they talk about the Pair testing and how they are implementing in on the QA team.
News
Apologies for the lack of episodes on regular basis
In this episode, Matthew and Federico talk about how easy it is for testers to spend time in activities other than actual testing. They share their experience of being aware of the time spent not testing and how important is to keep a balance.
In this episode Matthew and Federico sit down with James Bach to talk about everything to do with testing. James ideas were very influential in shaping the test methodologies that our team practices, especially around the use of exploratory testing, and he shares his views and passion about becoming a test ninja with us.
How did James got started in the test discipline?
Exploratory testing
Exploratory testing as an approach instead of a technique.
Some of the misconceptions about exploratory testing.
Difference between exploratory testing and ad-hoc
Teach exploratory testing by creating a test culture around developing mental skills
How to know when testing is done?
How to evaluate good exploratory testing notes.
Using a screen recorder as a supplement to note taking.
Scripted tests
Strengths and weaknesses of scripted tests.
Automation as simple product checks.
Testing as a profession
How to get really good at testing?
Study and define the test activities until they are clear and no longer intuition.
Example: how to know if something is a bug
Management
Work to get rid of bureaucracy that stands in the way of the tester.
Good leads will sit with testers to guide and evaluate them.
Use sessions as a way to manage a team of exploratory testers.
In this episode Matthew and Federico sit with fellow team member Brad Wilson to talk about Unit Testing. Brad is a developer in the ASP.NET team, he has a lot of experience developing using Test Driven Development and is one of the creators of the xUnit testing framework.
What is Unit Testing?
What is the difference between Unit Testing and Test Driven Development (TDD)?
Unit Testing in practice
How does the MVC team uses unit testing?
How do you arrange your tests?
How do you name your tests?
What about using internal to be able to test something?
What if I inherited code that didn't had unit tests?
How can I convince my team to adopt unit tests as a development practice
Addressing common concerns about Unit Testing
It feels like a lot of work for little value.
Having to rewrite a lot of tests if something in the design changes is a pain.
Code ends up being a lot of very small methods and I don’t like that.
In this episode Federico and Matthew talk about how to apply a concept from lean manufacturing to rethink the processes of software development. Taking the real example of a “Regression Run” they discuss how to dissect the process to identify waste that the team can then drive to eliminate.
The process of a regression run and the waste associated with it
For our team, a regression run is the automated execution of tests that where written for features released in previous releases.
For every process, the question to answer is “what does the customer wants for this process”?
If we think of a regression run as a process, we can define that what the customer wants from it is to be able to install a new version of the product and have his/her applications continue to work as before.
The goal of a regression run is then to find breaking changes for the customer.
Waste is any activity that does not add value to the outcome of the process. In the context of a regression run it could be any activity that did not identify a breaking change for the customer.
Types of waste
Waiting. Time spent waiting for results, for a tool or for fixes.
Unnecessary analysis. Analyzing a failed test that was not caused by a breaking change.
Over analysis. Analyzing a test that ensures higher quality than necessary.
Time of analysis. Time spent between receiving a result and identifying the breaking change.
Unused creativity. Creativity of the people that could be used to improve the process.
Instead of thinking about identifying and enhancing the value-add activities, you could work on identifying and eliminating the non-value-add activities.
On this episode Matthew and Federico talk about how our team partitions the test bed by using priorities. They discuss why having an incremental automation test bed is important and how having a standarized meaning for priorities can help.
In this episode Federico and Matthew share some of their personal anecdotes regarding process overhead. By examining concrete examples, they discuss pitfalls and how to remove unnecessary processes.
Disclaimer: The opinions expressed herein are the personal opinions
of Matthew, Federico, and their guests and do not represent their employer's view
in any way.
Zune
iTunes
Other Options
