Coding QA http://codingqa.com "Thoughts on QA, HTTP Style" en Matthew Osborn matthew@osornm.com (Matthew Osborn) Liberated Syndication - libsyn.com podcasts@libsyn.com (Liberated Syndication) Tue, 02 Mar 2010 18:35:00 GMT 180 Coding QA is a weekly show about testing from the trenches. Federico and Matthew are members of the ASP.NET QA team, and in this podcast they share their views about everything that has to do with testing and their experience of being the front line testers of a web platform. ASP.NET, QA, Testing Matthew M. Osborn & Federico Silva Armas matthew@osbornm.com Matthew M. Osborn http://libsyn.com/podcasts/aspnetqa/images/AlbumCodingQA.jpg Coding QA http://codingqa.com Clean Episode 33 Team Structure http://www.codingqa.com/index.php?post_id=588510# In this episode Matthew and Federico sit to talk about some of the changes that are happening in the ASP.NET QA team. We have talked about the evolution of our team, and today we go over the idea of dividing the test team into functional disciplines and how it has worked so far.

News

Organization of the QA team

  • What was the problem that we identified?
    • Lack of specialization.
    • Lack of resources to deal with test debt.
    • Lack of resources to improve our efficacy.
    • 100% of tester's time was dictated by a feature crew.
  • The 3 pillars of the team
    • Product Presence - test presence during product design and prototypes.
    • Product Development - core testing of new features as they are incorporated into the product.
    • Engineering Services - automation strategy, tools/infrastructure and regression testing.
  • Work flow of a feature crew
    • As a new feature is designed PP is engaged during early prototypes, verifying scenarios with customer expectations, first exploratory sessions.
    • When the feature is mature enough, PP hands of testing to PD group. These group of testers are normally assigned to cross discipline feature crews and perform the bulk of exploratory testing and any other test activities.
    • As the feature stabilized, ES comes in to help automate regression tests, making sure tests are robust and maintainable. At the end this group continues to run regression tests of the whole product.
  • Take away
    • Still an experimental model.
    • So far the gains seem to indicate that the overall capacity of the team is increased.
    • A potential next step is to recognize the value of "on-demand" testing and "constant" testing, and to focus a group of testers on each.

 

]]> podcasts Tue, 2 Mar 2010 18:35:00 GMT http://www.codingqa.com/index.php?post_id=588510# 00:22:15 Matthew M. Osborn & Federico Silva Armas No Episode 32 Last Minute Changes http://www.codingqa.com/index.php?post_id=576142# In this episode Matthew and Federico again take from there experience of working on the MVC project and this time talk about last minute changes.

  • Why last minute changes happen?
    • High priority bug found late in the cycle.
    • Design change from customer feedback.
    • Stake holders disagree with a design decision.
  • How to tackle it?
    • Extend the date
    • Cut corners
    • Cut features
    • More resources
    • Be creative with the resources you have.
  • How to prevent them?
    • Build buffer into your planning.
    • Reach out to as many people as you can early and often.
    • Figure out the requirements and processes of your dependencies.
    • Build trust within your team.

 

 

]]> podcasts Fri, 29 Jan 2010 08:29:00 GMT http://www.codingqa.com/index.php?post_id=576142# 00:20:56 Matthew M. Osborn & Federico Silva Armas No Episode 31 The Vicious Cycle http://www.codingqa.com/index.php?post_id=570839# In this episode Federico and Matthew narrate their experience of how having unclear expectations can lead down a path that can eventually create a culture of treating QA as a burden. It is a series of event that we call the vicious cycle of QA. Federico has been on the team for almost eight years and today shares his observations of how the team got itself into an undesired place, what were the symptoms, and what our decisions created. Look for a future podcast to learn what actions we took to break this cycle and move forward.

News

Vicious cycle of QA

  • In a nutshell: unclear expectations leads to blame which leads to more process which leads to bottle necks which leads to more unmet expectations.
  • Step 1: Unclear expectations
    • QA, PM and Dev do not have a clear understanding of what is the role of the testers. What can they expect from them, and more importantly, what are the limits of their responsibility.
    • Having unclear expectations means that every person on the team has a different idea of what are testers supposed to do. Even higher level managers.
  • Step 2: Fail and get blamed
    • Without having clear conditions of success, what may turn out is that each miss is considered a failure.
    • Since the responsibility is not clearly defined, it is easy to have the QA team be the focus of blame.
  • Step 3: Cover your back with processes.
    • The QA team may turn to create processes as a means to cover their backs and transfer responsibility to the rest of the team.
    • Many processes are created that gives security that as long as they are followed, the QA team can shield behind them.
  • Step 4: Push back when process doesn't fit the plan.
    • With so many processes and checklist across the board, the QA team may become the bottle neck of the development cycle.
    • Since processes have to be guarded, QA team starts to push back on new features due to lack of time to properly do testing.
    • Turf wars may begin between the several disciplines.
  • Repeat:
    • Push and pull eventually settles into a set of constraints that QA is unable to meet given the high cost of test process.
    • More unmet expectations ensue.
  • Culture of QA as a burden
    • Since the processes where created to cover ones back instead of doing smarter testing, the perceived value of QA is still low.
    • Eventually the perception and reputation of the QA team is that of a high cost and low value.
  • How do we break out of this cycle? How do we turn around the team and perception? This and more questions will be addressed in a future podcast

 

]]> podcasts Fri, 15 Jan 2010 21:35:00 GMT http://www.codingqa.com/index.php?post_id=570839# 00:25:10 Matthew M. Osborn & Federico Silva Armas Clean Episode 30 Pair Testing http://www.codingqa.com/index.php?post_id=561862# In the Episode Matthew sets down with Drew Miller, a fellow tester, and they talk about the Pair testing and how they are implementing in on the QA team.

    News

    Introduction to Drew Miller

    • Drewâs work history
    • Fun facts about Drewâs interests

    Pair Programming/Testing

    • What is this pairing thing?
    • How Pair Programming fits into our teamâs methodologies
    • Justifying the cost of pair programming
    • Possible Issues when your new to pairing
    • Creating a pairing environment to help
    • Is pairing the end all of coding styles?

     

    ]]>     podcasts Sun, 20 Dec 2009 06:17:00 GMT http://www.codingqa.com/index.php?post_id=561862# 00:22:24 Matthew M. Osborn & Federico Silva Armas No     Episode 29 Time Not Testing http://www.codingqa.com/index.php?post_id=556282# In this episode, Matthew and Federico talk about how easy it is for testers to spend time in activities other than actual testing. They share their experience of being aware of the time spent not testing and how important is to keep a balance.

    News

    Managing your test time

    • What is test time?
      • Time spent designing tests
      • Time spent executing tests
      • Time spent evaluating results
      • Time spent investigating bugs
    • Other activities that take time away from testing
      • Meetings. Like sync-ups, status reporting, planning, training, triage, etc.
      • E-Mail.
      • Building relationships.
      • Hardware and software issues.
    • A day in the life of a tester. Matt's and Federico's day.
    • Some realizations
      • Building sofware is a social activity, mantaining relationships is very important.
      • There are many support activities that are crucial for the success of the project.
      • The importance is to make sure that you still have enough time to sit and test.
      • Try to balance test time and non-test time and make sure that you are happy with the ratio.
      • One easy way to increase your test results is to identify non-test time and to try to minimize it.
    • How much percentage of time are you dedicating to actual testing?
    • What do you think of managers that move away from actual testing?

       

      ]]>       podcasts Fri, 4 Dec 2009 08:09:00 GMT http://www.codingqa.com/index.php?post_id=556282# 00:21:49 Matthew M. Osborn & Federico Silva Armas No       Episode 28 Chatting with James Bach http://www.codingqa.com/index.php?post_id=550220# In this episode Matthew and Federico sit down with James Bach to talk about everything to do with testing. James ideas were very influential in shaping the test methodologies that our team practices, especially around the use of exploratory testing, and he shares his views and passion about becoming a test ninja with us.

      • How did James got started in the test discipline?
      • Exploratory testing
        • Exploratory testing as an approach instead of a technique.
        • Some of the misconceptions about exploratory testing.
        • Difference between exploratory testing and ad-hoc
        • Teach exploratory testing by creating a test culture around developing mental skills
        • How to know when testing is done?
        • How to evaluate good exploratory testing notes.
        • Using a screen recorder as a supplement to note taking.
      • Scripted tests
        • Strengths and weaknesses of scripted tests.
        • Automation as simple product checks.
      • Testing as a profession
        • How to get really good at testing?
        • Study and define the test activities until they are clear and no longer intuition.
        • Example: how to know if something is a bug
      • Management
        • Work to get rid of bureaucracy that stands in the way of the tester.
        • Good leads will sit with testers to guide and evaluate them.
        • Use sessions as a way to manage a team of exploratory testers.
      • Challenges of testing in Microsoft.

      Links and Plugs

      ]]>       podcasts Tue, 17 Nov 2009 06:59:00 GMT http://www.codingqa.com/index.php?post_id=550220# 01:06:50 Matthew M. Osborn & Federico Silva Armas No       Episode 27 Unit Testing With Brad Wilson http://www.codingqa.com/index.php?post_id=543842# In this episode Matthew and Federico sit with fellow team member Brad Wilson to talk about Unit Testing. Brad is a developer in the ASP.NET team, he has a lot of experience developing using Test Driven Development and is one of the creators of the xUnit testing framework.

      • What is Unit Testing?
      • What is the difference between Unit Testing and Test Driven Development (TDD)?
      • Unit Testing in practice
        • How does the MVC team uses unit testing?
        • How do you arrange your tests?
        • How do you name your tests?
        • What about using internal to be able to test something?
        • What if I inherited code that didn't had unit tests?
        • How can I convince my team to adopt unit tests as a development practice
      • Addressing common concerns about Unit Testing
        • It feels like a lot of work for little value.
        • Having to rewrite a lot of tests if something in the design changes is a pain.
        • Code ends up being a lot of very small methods and I donât like that.
      • Common pitfalls
        • Writing very big tests.
        • Testing to internal implementation.
      • Mocking
        • What does mocking mean and why is it useful?
        • What is your favorite mocking framework?
      • Resources

        ]]>         podcasts Sat, 31 Oct 2009 07:56:00 GMT http://www.codingqa.com/index.php?post_id=543842# 00:37:29 Matthew M. Osborn & Federico Silva Armas Clean         Episode 26 Reducing Waste http://www.codingqa.com/index.php?post_id=540864# In this episode Federico and Matthew talk about how to apply a concept from lean manufacturing to rethink the processes of software development. Taking the real example of a âRegression Runâ they discuss how to dissect the process to identify waste that the team can then drive to eliminate.

        News

        The process of a regression run and the waste associated with it

        • For our team, a regression run is the automated execution of tests that where written for features released in previous releases.
        • For every process, the question to answer is âwhat does the customer wants for this processâ?
        • If we think of a regression run as a process, we can define that what the customer wants from it is to be able to install a new version of the product and have his/her applications continue to work as before.
        • The goal of a regression run is then to find breaking changes for the customer.
        • Waste is any activity that does not add value to the outcome of the process. In the context of a regression run it could be any activity that did not identify a breaking change for the customer.
        • Types of waste
          • Waiting. Time spent waiting for results, for a tool or for fixes.
          • Unnecessary analysis. Analyzing a failed test that was not caused by a breaking change.
          • Over analysis. Analyzing a test that ensures higher quality than necessary.
          • Time of analysis. Time spent between receiving a result and identifying the breaking change.
          • Unused creativity. Creativity of the people that could be used to improve the process.

        Instead of thinking about identifying and enhancing the value-add activities, you could work on identifying and eliminating the non-value-add activities.

        ]]>         podcasts Fri, 23 Oct 2009 05:51:00 GMT http://www.codingqa.com/index.php?post_id=540864# 00:27:06 Matthew M. Osborn & Federico Silva Armas No         Episode 25 Test Priorities http://www.codingqa.com/index.php?post_id=539667# On this episode Matthew and Federico talk about how our team partitions the test bed by using priorities. They discuss why having an incremental automation test bed is important and how having a standarized meaning for priorities can help.

        News:

        Test case priorities

        • Grouping tests by a common priority bar is useful when dealing with a big test bed.
        • Pri 0
          • A small number of positive tests under one configuration.
          • Constitute 10-20% of a feature's tests.
          • Regression amounts to a ship stopper.
        • Pri 1
          • Coverage for 80% of the use cases of a feature under multiple configuration.
          • Constitue 80% of a feature's tests.
        • Pri 2
          • Edge or low impact scenarios.
          • Constitute 10-20% of a feature's tests.
        • Why do we need pri 0 functional tests if we already have unit tests?
        • Your guidelines are ambiguous, how do you apply them?

        ]]>         podcasts Tue, 20 Oct 2009 03:08:00 GMT http://www.codingqa.com/index.php?post_id=539667# 00:31:22 Matthew M. Osborn & Federico Silva Armas No         Episode 24 Process Overhead http://www.codingqa.com/index.php?post_id=535417# In this episode Federico and Matthew share some of their personal anecdotes regarding process overhead. By examining concrete examples, they discuss pitfalls and how to remove unnecessary processes.

        News:

        Process Overhead

        • The process of keeping internal specs and documentation up to date.
        • The process of triaging bug defects for a large project.
        • The process of checking code into the repository.
        • The process of maintaining a detailed scheduled of work.
        • Processes that provide value to the business vs. process that provide value to a person.
        • Reinvent a process vs. just throw it away.

        ]]>         podcasts Fri, 9 Oct 2009 06:01:00 GMT http://www.codingqa.com/index.php?post_id=535417# 00:27:27 Matthew M. Osborn & Federico Silva Armas Clean         Episode 23 Port-a-Test a portable test bed http://www.codingqa.com/index.php?post_id=530494# In this episode, Federico and Matthew talk about the benefits of having an automated test bed that can be easily transferred and run from any machine. They discuss some of the differences between a remote execution test bed and a portable test bed, and how the latter has worked well for the MVC feature crew.

        News

        • WebSiteSpark was released and is available for all web startups.

        The portable test bed.

        • How the remote execution test bed worked in the ASP.NET QA team.
          • Use internal test harness that can distribute tests on remote machines.
          • Enforces several design and execution paradigm on test authors.
        • Problems and limitations that we faced with a remote execution test bed.
          • Difficult to share tests with other teams.
          • Difficult to react quickly to product changes.
          • Tendency for tests to be locked down to a specific execution environment.
        • The concept of a "portable test bed" as a solution.
          • Test bed can be xcopy deployed to any target machine.
          • Low dependencies on external factors
        • A layered automation infrastructure is preferable
          • Build automation solution that can move in a spectrum from high flexibility and high manual intervention to low manual intervention and potentially low flexibility.
          • If a high automated solution fails, tester can quickly and easily revert back to a more manual approach to achieve goal.

          ]]>           podcasts Sat, 26 Sep 2009 05:51:00 GMT http://www.codingqa.com/index.php?post_id=530494# 00:27:24 Matthew M. Osborn & Federico Silva Armas No           Episode 22 Swatting Bugs http://www.codingqa.com/index.php?post_id=528172# In this episode of CodingQA, Federico and Matthew share their experience of what an appropriate response is to critical bugs found during development. Using a real case from MVC, they discuss how the feature crew reacts to high impact bugs and how the problem is addressed for the future.

          News:

          The appropriate response

          • Summary of the MVC build verification process
            • Daily checkins
            • No continuous integration server.
            • Dev is responsible for running unit tests before checkin.
            • QA responsible for running functional tests with every build.
            • Functional tests are fully automated and run in less than 30 mins.
            • Test results are transmitted verbally.
          • The ship stopper problem
            • At some point during the development cycle, QA makes more comprehensive runs.
            • During these runs a bug was found where a security exception is thrown when running on Medium trust and MVC is in the GAC.
          • The response
            • First: Verify that you have discovered the extent of the problem. Sometimes a bug could be the tip of the iceberg, are there more bugs around the vicinity that we need to find?
            • Second: Investigate and understand how the bug got introduced. Was it there from the beginning? Did a subsequent checking broke the scenario? Is the problem in our code or in a collateral component?
            • Third: Place checks to prevent a similar problem from reoccurring.
          • The learning
            • Always explore and app build with Medium trust (a project defaults to Full trust)
            • The QA team had been making runs with MVC in the bin instead of the GAC, going forward we will mix bin and GAC from the beginning.
          • Do not focus on blaming people for a bug not found, focus on improving your process and your safety net.
          • It is a misconception that the QA team is responsible for missing a bug. It is the whole team's responsibility to produce high quality software.

          ]]>           podcasts Sun, 20 Sep 2009 04:52:00 GMT http://www.codingqa.com/index.php?post_id=528172# 00:27:15 Matthew M. Osborn & Federico Silva Armas No           Episode 21 Agile Manifesto http://www.codingqa.com/index.php?post_id=525284# In this episode Matthew and Federico give their impressions regarding the agile manifesto. Drawing from experience in the ASP.NET QA team, the meaning of the agile values are discussed, with examples of how the MVC feature crew applies them and some pit falls.

          News:

          The Agile Manifesto

          • Individuals and interactions over processes and tools
          • Working software over comprehensive documentation
          • Customer collaboration over contract negotiation
          • Responding to change over following a plan

          ]]>           podcasts Sat, 12 Sep 2009 05:48:00 GMT http://www.codingqa.com/index.php?post_id=525284# 00:33:38 Matthew M. Osborn & Federico Silva Armas No           Episode 20 Test Contexts http://www.codingqa.com/index.php?post_id=519980# In this episode Matthew and Federico sit down to talk about test Contexts, and how our team uses this concept in automation. Positive and negative points are discussed as well as the challenges that the QA team faces when applying the concept in excess.

          • News
            • The ASP.NET QA team is hiring! If you are interested in working in our team please send us your resume.
            • Podcast is now accessible via Zune
          • Contexts
            • What are "contexts" or "variations"?
            • Some of the contexts that we use: TrustLevel, Impersonation, Browsers, ViewState, UNC, RemoteServer, SmartNav, etc.
            • Pros: breadth coverage, cheap for the tester, confidence of no points missed.
            • Cons: excessive results, skip evaluation of test value, maintainability cost is often disregarded.
            • What is the situation in our team? The challenge of maintaining a 10 year old product.

          ]]>           podcasts Fri, 28 Aug 2009 05:04:00 GMT http://www.codingqa.com/index.php?post_id=519980# 00:28:51 Matthew M. Osborn & Federico Silva Armas No           Episode 19 But QA is important too⦠http://www.codingqa.com/index.php?post_id=517447# In this episode Matthew and Federico continue to elaborate on the concept of treating the QA role as a âserviceâ and discuss some of the feedback and discussion that has carried on during the week with their co-workers.

          News

          • Federico has been confirmed as a speaker for Dev Connections!

          QA as a âserviceâ revisited

          • Feedback / Discussion
            • The Dev and PM roles also provide a service to the QA role.
            • A service is optional, QA is not optional.
            • QA provides lots of value why do you say we need to work on providing it?
          • Continued discussion of QA as a âserviceâ
            • What is the problem that treating QA as a âserviceâ solves?
              • Perceptions of the QA teamâs value.
              • Divisions between disciplines.
            • QA as a âServiceâ is just the beginning and should be part of a bigger picture.

          ]]>           podcasts Fri, 21 Aug 2009 06:16:00 GMT http://www.codingqa.com/index.php?post_id=517447# 00:18:33 Matthew M. Osborn & Federico Silva Armas No           Episode 18 Would you like fries with that? http://www.codingqa.com/index.php?post_id=515089# In this episode Matthew and Federico discuss what it means to treat the QA role as a âserviceâ rather than a ârequirementâ.

          News

          • Fede has a new blog about testing.
          • Fede to give a QA talk at Dev Connections

          QA as a service

          • Signs that development team sees QA as a burden.
          • Difference in QA cultures:
            • âsome else is responsible, they need to changeâ
            • âinfluence others by changing yourselfâ
          • Think of QA as a service provider and the rest of the development team as the consumer. Strive to give as much value.
          • Perception and reputation and why itâs important to provide value that is tangible to the rest of the team.
          • Change the QA response from âno, we canât do take that workâ, to âhow can we find something that worksâ.
          • Increase awareness across the development team about what, how and why the QA team does what it does.
          • First hand anecdotes from our team.

          ]]>           podcasts Fri, 14 Aug 2009 05:31:00 GMT http://www.codingqa.com/index.php?post_id=515089# 00:25:41 Matthew M. Osborn & Federico Silva Armas Clean           Episode 17 MVC 2 Preview 1 http://www.codingqa.com/index.php?post_id=512557# In this show Matthew and Federico share their experiences working as testers in the MVC feature crew. They go over how the team works, the development lifecycle from a QA point of view, the principles that the team adopted as well as an honest assessment of what didnât go so well.

          • News
            • MVC 2 Preview 1 released
            • What's new in MVC 2 (ScottGu)
              • Ability to easily partition and group functionality across an MVC application.
              • Ability to use Data Annotation attributes to validate input.
              • Ability to use Data Annotation attributes to drive the generation of UI in the Views (ala Dynamic Data)
              • Strongly typed UI-Helpers
          • Inside the MVC team
            • Lifecycle of a "feature": Idea -> Prototype -> Exploratory Testing -> Review -> Application Building -> Automation (Rinse & Repeat)
            • Challenges of testing an installer.
            • Development principles
              • QA is involved during design.
              • All feature check-ins are previously approved by Dev/Test/PM
              • All check-ins contain unit tests
              • QA drives the schedule
              • Weekly test plan.
            • What could had gone better
              • QA not working close enough with Devs.
              • A lot of non-MVC work caused a lot of distractions.
              • Big bug debt coming in from MVC 1.0
              • Ramp up time for new QA members took longer than expected.

          ]]>           podcasts Fri, 7 Aug 2009 05:45:00 GMT http://www.codingqa.com/index.php?post_id=512557# 00:32:43 Matthew M. Osborn & Federico Silva Armas No           Episode 16 "The Right Stuff" to be Successful http://www.codingqa.com/index.php?post_id=507236# In this episode Matthew and Federico sit down to talk about the personality, skills, and traits of a successful QA member. Join them as they each enumerate what they think makes up a great QA member. Tune in and learn what it takes to become a Mercury 7 astronauts, errrâ, a great QA member.

          News

          âThe Right Stuffâ

          • What are the types of testing
            • Application
            • Framework
          • What are the types of Testers
            • Automater
            • Coding QA (yes itâs a pun on the name)
          • Matthewâs Wish List
            • curiosity
            • awareness
            • assertiveness
            • humbleness (think skin)
          • Federicoâs Wish List
            • curiosity / creativity
            • attention to detail
            • awareness
            • technical skills
            • ability to judge customer value

          ]]>           podcasts Fri, 24 Jul 2009 01:44:00 GMT http://www.codingqa.com/index.php?post_id=507236# 00:26:31 Matthew M. Osborn & Federico Silva Armas No           Episode 15 Bug Review http://www.codingqa.com/index.php?post_id=504435# In this episode Matthew and Federico sit down to talk about some interesting bugs and lessons learned that our team has run across while testing ASP.NET. This is a different type of show that we are experimenting where the bugs take center stage.

          News

          Bugs Showcase

          • Sensitive data stored in ViewState.
          • How a test abstraction hide a security vulnerability bug.
          • How testing with slow connection revealed Ajax bugs.
          • Tale of a breaking change: How a bug that is shipped can be used as a feature by customers.
          • How we found some accessibility bugs by "being the customer".

           

          ]]>           podcasts Fri, 17 Jul 2009 06:10:00 GMT http://www.codingqa.com/index.php?post_id=504435# 00:32:15 Matthew M. Osborn & Federico Silva Armas No           Episode 14 Charting with Lakshmi Padala http://www.codingqa.com/index.php?post_id=501575# In this episode Matthew and Federico sit down with Lakshmi Padala from the SQL Reporting Services team to talk about how they tested the asp.net chart control and what some of the teams processes are. Join them as they learn about what its like to be a SDET in another business group at Microsoft.

          • News: The AJAX Control Toolkit has started using the Lightweight Test Automation framework for functional testing.
          • Introduction to Lakshmi and her choice of QA over DEV.
          • Overview of what the Chart Control is and what it does.
          • How did the cross team collaboration work with the ASP.NET team?
          • Overview of the how the chart control was tested. 
            • API level testing
            • Run time testing
            • Design time testing
          • How do you test a product that only produces an image? How does image comparison work?
          • Scope of the chart teamâs test bed. (number, speed, and type of tests)
          • Successful Model Based Test (MBT) implementation in the Reporting Services team
            • Advantages to MBT
            • Disadvantage to MBT
            • Work a rounds to be successful
          • Challenges using design time abstractions to test.
          • What is the life cycle of a feature/product in the Reporting Services team?
          • Trust level security concerns when testing the chart control.
          • Lakshmiâs favorite part about working at Microsoft.

          Links:

          ]]>           podcasts Fri, 10 Jul 2009 06:37:00 GMT http://www.codingqa.com/index.php?post_id=501575# 00:26:08 Matthew M. Osborn & Federico Silva Armas Charting with Lakshmi Padala No           Episode 13 Test Maintainability http://www.codingqa.com/index.php?post_id=499066# In this episode Matthew and Federico sit down to talk about test maintainability and several factors to keep in mind that can decrease the reaction time once a test fails.

          • The 3 dimensions of maintainability
            • Regressions (test fails due to a product bug). How easy can I identify the cause of failure?
            • Product Changes. How easy can I update tests due to new behavior?
            • Reusability. How easy can I use a test written in the past to test a new feature?
          • Some of the factors to keep in mind:
            • Unique failure.
            • Good logging.
            • Reproducibility/Robustness.
            • Atomic tests.
            • Result history.
            • Test recorders.
            • Abstractions vs. low level implementations.
            • Prediction.
            • Framework flexibility and extensibility.
          ]]>           podcasts Fri, 3 Jul 2009 18:40:00 GMT http://www.codingqa.com/index.php?post_id=499066# 00:34:59 Matthew M. Osborn & Federico Silva Armas Clean           Episode 12 Jim Wang on Microsoft AJAX http://www.codingqa.com/index.php?post_id=496207# In the twelfth installment of Coding QA Federico is still enjoying his vacation so Matthew sets down with his officemate Jim Wang to discuss the ins and outs of test the Microsoft AJAX libraries. Tune in as Matthew picks Jimâs brain on what it was like to test a client side framework.

          Interviewing Jim Wang

          • A little about Jim and his history at Microsoft.
          • What brought Jim to the QA team and his preconceptions about what it means to be QA team member.
          • Discussion around Jimâs responsibilities on the feature crew.
          • Do client side libraries have lots of unit tests written?
          • Browsers and what that means to the feature crew.
          • Handling hand off of code on a client side library.
          • Discuss on some of the challenges Jim faces on the Microsoft AJAX framework feature crew.
          • How the design is affected by the QA team.
          • What is Jimâs Favorite thing about testing the Microsoft AJAX framework and what is the thing he would most like to get ride of.
          ]]>           podcasts Fri, 26 Jun 2009 08:08:00 GMT http://www.codingqa.com/index.php?post_id=496207# 00:21:24 Matthew M. Osborn & Federico Silva Armas No           Episode 11 Interviewing Hong Li http://www.codingqa.com/index.php?post_id=493601# In the eleventh installment of Coding QA Federico is on vacation so Matthew sets down with his mentor, Hong Li, for an interview. Join Matthew as he learns a little about one of his fellow team members, servicing an released product, and working with team members that are offshore in China.

          Interview with Hong Li

          • A little history on Hong and how she came to the ASP.NET QA team.
          • Discussion on Hongâs role with the servicing team.
          • What is the difference between a hot fix and a QFE?
          • What is the life cycle of a servicing request?
          • What is the servicing team and who is on it?
          • How the offshore team plays into servicing the product.
          • How do you handle the two teams being on different sides of the world?
          • Benefits of having round-the-clock operations.
          • Is everything handled remotely with the offshore team?
          • What brought Hong to the QA discipline?
          • What is the biggest challenge Hong has faced when working at Microsoft.
          • What Hong likes most about working at Microsoft.
          ]]>           podcasts Fri, 19 Jun 2009 06:38:00 GMT http://www.codingqa.com/index.php?post_id=493601# 00:17:37 Matthew M. Osborn & Federico Silva Armas No           Episode 10 Project Structure http://www.codingqa.com/index.php?post_id=490794# In this show Matthew and Federico talk about how the ASP.NET team structures itâs sources and the theory behind it all. They talk about how the team handles the organization of all of itâs sources and what seems to work well along with what seems to cause hang ups.

          News

          Project Structure when developing a product

          • Keeping Developer and QA source separate
          • Branching sources and the the types of branches
          • Bring it all together in âMicrospeakâ (merging branches)
          • What seems to work well for the team.
          • What are some of the issues that the team faces

          Disclaimer: LTAF runner will be post soon, sorry for the delay!

          ]]>           podcasts Fri, 12 Jun 2009 06:09:00 GMT http://www.codingqa.com/index.php?post_id=490794# 00:23:47 Matthew M. Osborn & Federico Silva Armas No           Episode 9 Releasing a Product http://www.codingqa.com/index.php?post_id=488153# In this show Matthew and Federico talk about what the ASP.NET QA team does to prepare for a release. Taking into account the recent release of ASP.NET 4 Beta 1, theyâll go over several of the "exit criteria" that the product must meet before delivery to the public.

          News

          • Lightweight Test Automation Framework June Release

          Sign off on a release

          • Difference between Quality Gates and Exit Criteria
          • Accessibility. Mainly manual tests to validate feature conforms to accessibility standards.
          • Code Coverage. Metric used to identify problematic development areas.
          • Stress. Specialized automated long running tests to spot memory leaks and other stress problems.
          • Localization. Verification that the product can be correctly localized to several languages.
          • Test Passes. Functional verification of the product across many platform combinations.
          • Zero Active Bugs. Series of step down goals to drive feature crews to finish development.

           

          ]]>           podcasts Fri, 5 Jun 2009 05:18:00 GMT http://www.codingqa.com/index.php?post_id=488153# 00:34:25 Matthew M. Osborn & Federico Silva Armas Releasing a Product No           Episode 8 Testing, what is that? http://www.codingqa.com/index.php?post_id=485096# This show is an open discussion about what testing means. Federico and Matthew share their opinions around the meaning of "testing", "quality" and the role of a QA team. They don't always agree but each brings something unique to the table on topics so basic yet somewhat mysterious in the testing discipline.

          News

          Topics of Discussion

          • What is the role of QA?
          • What is testing?
          • What is quality?
          • Should quality be measurable?
          • What is risk?
          • What is the difference between Dev and Test?
          ]]>           podcasts Thu, 28 May 2009 16:43:00 GMT http://www.codingqa.com/index.php?post_id=485096# 00:35:17 Matthew M. Osborn & Federico Silva Armas No           Episode 7 A Tour Around Exploratory Testing http://www.codingqa.com/index.php?post_id=482513# In this show Federico and Matthew talk about what exploratory testing (ET) is and how the ASP.NET QA team uses this methodology for testing.

          News

          Exploratory Testing

          • Definition
          • Differences between exploratory testing and scripted testing
          • Differences between exploratory testing and ad-hoc testing
          • The 4 principles of exploratory testing
            • Creativity
            • Goal oriented
            • Reproducibility
            • Adaptability
          • How the ASP.NET QA team uses exploratory testing
          • When is exploratory testing a good fit to test a product
          • Brief example of an exploratory testing session
          • What are the outcomes of an exploratory testing session
            • Scenarios tried
            • Questions
            • Assumptions
            • Defects
            • Future scenarios
          ]]>           podcasts Fri, 22 May 2009 06:02:00 GMT http://www.codingqa.com/index.php?post_id=482513# Matthew M. Osborn & Federico Silva Armas No           Episode 6 http://www.codingqa.com/index.php?post_id=479060# In this show Federico and Matthew explore the question of "when too much automation is too much?" Based on their experiences in the ASP.NET QA team, they talk about the danger of automating for the sake of automation, the problems that it can create, and the value of context driven testing.

          News

          • New name and name website: Coding QA.
          • .NET Micro Framework has been open sourced.

          Automation for the sake of automation

          • What is the goal of automation? To find bugs during development vs. to find regressions after development.
          • What problems can arise from having too much automation? The tale of the ASP.NET QA "boat anchor".
          • Context matters when deciding how to ensure a feature does not regress.
          • What parameters can be used to decide the testing approach
            • Automation cost.
            • Time of delivery.
            • Resources available.
            • Risk of a regression.
            • Customer impact if a regression is introduced.
            • Quickness between when the regression is introduced and when it is discovered.
          • Real examples from the team
            • Testing event handler generation for web controls in VS.
            • Testing picker dialogs for web controls in VS.
          ]]>           podcasts Fri, 15 May 2009 02:38:00 GMT http://www.codingqa.com/index.php?post_id=479060# Matthew M. Osborn & Federico Silva Armas No           Episode 5 http://www.codingqa.com/index.php?post_id=475884# In this episode Matthew interviews Carl Dacosta, lead tester for the MVC 1.0 project, and talks about what it was like to develop and test a product with so many releases.

          • MVC crossed the 100,000 downloads mark (currently at 140,000)
          • MVC released under MS-PL license
          • Interview with Carl Dacosta
            • Some history on Carl and how he got into test.
            • Role of Carl as a tester in the MVC crew.
            • What was the development and testing process of MVC to enable frequent releases.
            • Parts of the testing process that worked well for this kind of project, and parts that didn't work so well.
            • How to handle quality of releases when external partners are starting to rely on the framework.
            • What did Carl enjoy the most working on the MVC crew.
            • Testing the "unit testing" experience for developers working on MVC.
          ]]>           podcasts Fri, 8 May 2009 06:30:00 GMT http://www.codingqa.com/index.php?post_id=475884# No           Episode 4 http://www.codingqa.com/index.php?post_id=461617# In the fourth installment of Coding QA Federico and Matthew discuss "The Good," "The Bad," and "The Ugly" of the history of the ASP.NET QA Team. Discussion ranges from the dark ages, to the renaissance, to the industrial revolution, to the "21st and half century".  Join them as they discuss the lessons learned and how the team has improved over time.

          • The team released the April update to the Lightweight Test Automation Framework.
          • "The Dark Ages" ASP.NET 1.0/2.0
            • During this time the team was very focused on automating everything.
            • (disadvantage) This approach is not good with a changing Spec.
            • (disadvantage) Bugs were being found to late in the process.
            • (advantage) Excellent automation coverage.
          • "The Renaissance" Atlas/Microsoft Ajax
            • The division moved to a feature crew model for development and QA.
            • (advantage) The team adopted a heavy unit testing practice. Increase in code coverage.
            • (disadvantage) The QA team still worked has it had before. It would automate everything is less time.
          • "The Industrial Revolution" ASP.NET 3.5
            • The QA team began to adopt the Feature crew model and started the process of agile testing with exploratory testing.
            • (advantage) Bugs were being found at the beginning of the process.
            • (disadvantage) The team still wrote a large amount of automation.
          • "21st and half century" ASP.NET 3.5 SP1/ASP.NET 4.0/vNext
            • The team has really started moving away from automating everything and is spending more time investigating user scenarios.
            • The team has formed its own unique blend of agile, scrum, and any other buzz word project management theory.
            • (advantage) Creating and automating sample applications provides feedback for real world customer scenarios.
            • (disadvantage) The team is young and is still learning the best practices.
            • (Struggle) Costing has become less of a clear cut process.
            • (Struggle) The is still a barrier between the QA and Dev teams.
            • (Struggle) No one wants the blame when a bug is found.

          Links from the show :

          ]]>           podcasts Fri, 24 Apr 2009 21:22:00 GMT http://www.codingqa.com/index.php?post_id=461617# Matthew M. Osborn & Federico Silva Armas No           Episode 3 http://www.codingqa.com/index.php?post_id=461615# In the third installment of Coding QA Federico and Matthew interview Mark Berryman, a test lead on the ASP.NET QA Team.  Join them as they learn the ins and outs of test management through Markâs eyes.

          • Release of the Mobile Browser Database by Live Dublin.
          • April refresh of the Lightweight Test Automation Framework.
          • Interview with Mark Berryman.
            • What is the difference between Test Manager and Test Lead.
            • A little history of Mark and how he came to join the ASP.NET QA team.
            • Responsibilities of a Test Lead / What Mark is working on.
            • What Mark looks for in a tester on his team.
            • Challenges Mark faced when he was an IC (individual Contributor).
            • What Mark would change on items he has tested in the past.
            • How Mark Manages risk on his team.
            • Who is Marks favorite person in the whole world.

          Links from the show :

          ]]>           podcasts Fri, 24 Apr 2009 21:21:00 GMT http://www.codingqa.com/index.php?post_id=461615# Matthew M. Osborn & Federico Silva Armas No           Episode 2 http://www.codingqa.com/index.php?post_id=461613# In the second installment of Coding QA Federico and Matthew discuss MIX 09 announcements and the ins and outs of supporting various browser versions on the ASP.NET QA team.

          • MIX 09 Conference announcements
            • MVC RTM 1.0
            • Silverlight 3.0
            • Web Application Installer
            • Web Application Gallery
          • What browsers the team tests on and how they decided which browsers to test.
          • What it means when the team says they support a browser.
          • How the team manages the risk of not testing on every possible browser.
          • The timeline for testing features on multiple browsers.
          • The infrastructure the team has to test on multiple browsers.
          • How the team keeps up with the fast growing browser market, such as the release of four major browsers in just a few months.

          Links from the show :

          ]]>           podcasts Fri, 24 Apr 2009 21:18:00 GMT http://www.codingqa.com/index.php?post_id=461613# No           Episode 1 http://www.codingqa.com/index.php?post_id=461458# In the first ever Coding QA Federico and Matthew discuss the motivation behind the community involvement from the QA team and the release of the QA teamâs Lightweight Test Automation Framework for ASP.NET and what it is.

          • Introductions to the podcasters, Federico and Matthew.
          • What is it that the ASP.NET QA team does.
          • The recent release of ASP.NET MVC RC2.
          • The recent release of the source code to the Lightweight Test Automation Framework.
          • The purpose of the release of the framework to codeplex.com and where it fits in the market.
          • Why the QA team chose to write their own automation framework.

          Links from the show :

          Disclaimer : This is our first try at creating, editing, and publishing a podcast.  Please bear with us as we iron out all the wrinkles in the process. Your feedback is greatly appreciated.
          ]]>           podcasts Fri, 24 Apr 2009 18:13:00 GMT http://www.codingqa.com/index.php?post_id=461458# No