segunda-feira, 1 de dezembro de 2014

Exploratory Testing - Positive and Negative points

Positive Points

  • Has a planning structure of testing ( called charters)
  • Find defects through exploring some pre-defined (planned) areas of testing
  • Do not rely in any formal documentation to be created.
  • So, it is useful when you do not have lots of documentation for each steps of the software or if this documentation changes often.
  • Easy to maintain and change
  • Minimize the time of test design (since it is design and execution at the same time)

Negative Points

  • Highly depends on the test experience
  • Easily misused
  • Can need to have a complementary test scripted to assure test coverage.

Study Cases  

Ad Hoc - Experienced tester & Error guessing

Project scenario: Application to validate promotions from telecom operator. (E.g. Mammy’s day), Development team: Outsourcing, Test Team: Outsourcing (not same company from dev team). Both were working in same client, in his own house.
Solution: It had been applied Ad Hoc tests added to error guessing technique by getting input from Product Manager (from marketing area), who informed problems found in previously; Testers used past experience and creativity to determine the scenarios may cause errors.
Key point: Testers experienced, even they hadn’t experience to the application the testers had a lot experience in software testing.

Checklist - Sanity tests 

Project scenario: Four outsourcing companies working on same software where all leads from each one had a weekly meeting and test team had to report sanity test result.
Solution: Testers weren’t able to execute tests in details following steps in same day that the build was make available to be tested. So, it has been created a checklist where test points were raised and test team used to execute exploratory tests based on that checklist. The test type was sanity (basic), just to identify high priority bugs on new build.
Key Point: Experienced testers in software testing and strong skill on that application.

Session Based Testing & Charter

Project scenario: Agile (Scrum), 1 tester, 12 developers and 1 Scrum master – All team was on the same location. The client was located into a different city (same country).
Challenge: The tester as the whole team received the stories at the same time, and joined the meeting with the PO (client side) to better understand the client scenario. Discuss some technological solutions and raise blocks (scenarios that were not covered by the documentation that needed to be clarified). While the development team was working on the code creation, the tester was planning the chart (created the missions, areas covered)
Solution: The use of chart proved to be very useful since it didn't relied on detailed step by step and allowed a “free time” to focus on learning how to use the product. During the chart execution, it helped to keep the focus and concentrate on the vulnerability areas. Also, the test do not got blocked by not developed yet functionality's, once it didn't not contained detailed step or dependencies between the steps
Bugs were found, value was added and the client was satisfied with the software quality.
Key Point: Experienced tester in software testing and agile methodologies.

Conclusion & recommendations

In a conclusion we can affirm following:

Get escaped bugs: There are some studies that are concerned in a linear way to executing tests mainly in a scripted tests, the point is when these tests are executed several times and in a several version it might become vicious testing and didn’t found effective bugs the result is exploratory tests are able to found escaped bugs.

 Fits in several project scenarios: It is a good practice to performing tests in an agile or not agile projects, in additional it fits in a different test type, it means exploratory testing can be used as system, component, regression, integration tests and so on.

 Used according to the test strategy: It can used in additional to test strategy mainly on saving time.

  Improve test’s effective when used as additional method on test phase: It is better applied when it is used as additional execution method that does why will assure the test coverage independent of development phase or test type. If exploratory testing is applied as complement of these tests the chance to find different bugs are improved.


[4] Oxford Advanced Learner's Dictionary of Current English, A S Hornby, OXFORDOxford University Press 2000