In this episode, Federico and Matthew talk about the benefits of having an automated test bed that can be easily transferred and run from any machine. They discuss some of the differences between a remote execution test bed and a portable test bed, and how the latter has worked well for the MVC feature crew.

News

• WebSiteSpark was released and is available for all web startups.

The portable test bed.

• How the remote execution test bed worked in the ASP.NET QA team.
  • Use internal test harness that can distribute tests on remote machines.
  • Enforces several design and execution paradigm on test authors.
• Problems and limitations that we faced with a remote execution test bed.
  • Difficult to share tests with other teams.
  • Difficult to react quickly to product changes.
  • Tendency for tests to be locked down to a specific execution environment.
• The concept of a "portable test bed" as a solution.
  • Test bed can be xcopy deployed to any target machine.
  • Low dependencies on external factors
• A layered automation infrastructure is preferable
  • Build automation solution that can move in a spectrum from high flexibility and high manual intervention to low manual intervention and potentially low flexibility.
  • If a high automated solution fails, tester can quickly and easily revert back to a more manual approach to achieve goal.

In this episode of CodingQA, Federico and Matthew share their experience of what an appropriate response is to critical bugs found during development. Using a real case from MVC, they discuss how the feature crew reacts to high impact bugs and how the problem is addressed for the future.

News:

• Microsoft AJAX can now be downloaded via CDN.

The appropriate response

• Summary of the MVC build verification process
  • Daily checkins
  • No continuous integration server.
  • Dev is responsible for running unit tests before checkin.
  • QA responsible for running functional tests with every build.
  • Functional tests are fully automated and run in less than 30 mins.
  • Test results are transmitted verbally.
• The ship stopper problem
  • At some point during the development cycle, QA makes more comprehensive runs.
  • During these runs a bug was found where a security exception is thrown when running on Medium trust and MVC is in the GAC.
• The response
  • First: Verify that you have discovered the extent of the problem. Sometimes a bug could be the tip of the iceberg, are there more bugs around the vicinity that we need to find?
  • Second: Investigate and understand how the bug got introduced. Was it there from the beginning? Did a subsequent checking broke the scenario? Is the problem in our code or in a collateral component?
  • Third: Place checks to prevent a similar problem from reoccurring.
• The learning
  • Always explore and app build with Medium trust (a project defaults to Full trust)
  • The QA team had been making runs with MVC in the bin instead of the GAC, going forward we will mix bin and GAC from the beginning.
• Do not focus on blaming people for a bug not found, focus on improving your process and your safety net.
• It is a misconception that the QA team is responsible for missing a bug. It is the whole team's responsibility to produce high quality software.

In this episode Matthew and Federico give their impressions regarding the agile manifesto. Drawing from experience in the ASP.NET QA team, the meaning of the agile values are discussed, with examples of how the MVC feature crew applies them and some pit falls.

News:

• ASP.NET 4.0 Ajax Preview 5
• Jim Wang to give a talk at DevConnections

The Agile Manifesto

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan