This show is an open discussion about what testing means. Federico and Matthew share their opinions around the meaning of "testing", "quality" and the role of a QA team. They don't always agree but each brings something unique to the table on topics so basic yet somewhat mysterious in the testing discipline.

News

• Release of .NET Framework 4 Beta 1
• Release of Visual Studio 2010 Beta 1

Topics of Discussion

• What is the role of QA?
• What is testing?
• What is quality?
• Should quality be measurable?
• What is risk?
• What is the difference between Dev and Test?

In this show Federico and Matthew talk about what exploratory testing (ET) is and how the ASP.NET QA team uses this methodology for testing.

News

• Dynamic Data release in CodePlex.com
• TechEd 2009
• Ajax Control Toolkit Refresh released.

Exploratory Testing

• Definition
• Differences between exploratory testing and scripted testing
• Differences between exploratory testing and ad-hoc testing
• The 4 principles of exploratory testing
  • Creativity
  • Goal oriented
  • Reproducibility
  • Adaptability
• How the ASP.NET QA team uses exploratory testing
• When is exploratory testing a good fit to test a product
• Brief example of an exploratory testing session
• What are the outcomes of an exploratory testing session
  • Scenarios tried
  • Questions
  • Assumptions
  • Defects
  • Future scenarios

In this show Federico and Matthew explore the question of "when too much automation is too much?" Based on their experiences in the ASP.NET QA team, they talk about the danger of automating for the sake of automation, the problems that it can create, and the value of context driven testing.

News

• New name and name website: Coding QA.
• .NET Micro Framework has been open sourced.

Automation for the sake of automation

• What is the goal of automation? To find bugs during development vs. to find regressions after development.
• What problems can arise from having too much automation? The tale of the ASP.NET QA "boat anchor".
• Context matters when deciding how to ensure a feature does not regress.
• What parameters can be used to decide the testing approach
  • Automation cost.
  • Time of delivery.
  • Resources available.
  • Risk of a regression.
  • Customer impact if a regression is introduced.
  • Quickness between when the regression is introduced and when it is discovered.
• Real examples from the team
  • Testing event handler generation for web controls in VS.
  • Testing picker dialogs for web controls in VS.

In this episode Matthew interviews Carl Dacosta, lead tester for the MVC 1.0 project, and talks about what it was like to develop and test a product with so many releases.

• MVC crossed the 100,000 downloads mark (currently at 140,000)
• MVC released under MS-PL license
• Interview with Carl Dacosta
  • Some history on Carl and how he got into test.
  • Role of Carl as a tester in the MVC crew.
  • What was the development and testing process of MVC to enable frequent releases.
  • Parts of the testing process that worked well for this kind of project, and parts that didn't work so well.
  • How to handle quality of releases when external partners are starting to rely on the framework.
  • What did Carl enjoy the most working on the MVC crew.
  • Testing the "unit testing" experience for developers working on MVC.