quarta-feira, 18 de março de 2015

Test Mirror: An test strategy using Scrum method in an outsourcing project

ABSTRACT


Although conduct software testing is extremely important to reaching the quality, tests are still not a common place among companies. Running software tests in agile development environment makes that situation even more complicated, since it has a quick development with little documentation. This scenario demands a complete adaptation of the testing processes. This work presents a case study applied in a multinational company during a project which was developed with the agile method named Scrum and some aspects of Feature Driven Development where the whole testing process was restructured on its many phases, offering a punctual and systematic way to coordinate outsourcing resources and conduct scrum team activities.

INTRODUCTION

Software testing improves the software quality and reduces the risks; using agile software development method the companies are able to deliver the product in a short time, therefore, the agile methods tend to use another ways for documentation, not formal documentations, it can result a lack on test coverage. In this specific project the agile method was adopted, more over reasons, it’s also applied to improve the tracking of requirements changes during the project.
The project leader used to set the priority on bug retests; it results in a bottleneck in test team.  Another point is the difficulty for the testers to understand and recheck bugs reported by development team (Feature Leaders and Architectures); they used to report as their preference describing coding for example.
Intermittent bugs, tester’s mistakes, bugs badly reported, imply a delay for development team fix. Each week the test team had a large number of bugs to be retested because the developers used put the bug in "Resolved" state in both following cases: when it is ready to be retested and when it hasn't enough information to be analyzed.
The test team didn’t have time to execute system tests in a current build; it was executed only “sanity test” to provide quickly feedback about software quality. Quality analysis based on sanity tests result, isn't a good practice because the proposal of the sanity test is to provide input to plan system tests.
Solving some of these troubles, a new test process has been created and applied during the software life cycle. The challenge was to define following:
·         tests roles;
·         the process to interaction of dev’s and tester’s;
·         following up outsourcing resources in Scrum;
This panel will describes the agile methods aspects and the software test importance in this specific project as well as presents project roles,  test team interaction in each Scrum phase, and the interaction among testers, outsourcing team and development team. In additional this case of study might be used as reference for another's similar projects.











segunda-feira, 2 de fevereiro de 2015

Overview - Lean Software Development

O conceito “Lean Production” surgiu com o lançamento do livro “The Machine That Changed the World”  em 1990 quando a Toyota era da metade do tamanho da GM (General Motors). Hoje em dia a Toyota está passando a GM como maior montadora do mundo e se posiciona no mercado como a empresa mais bem sucedida nos últimos cinquenta anos. O livro trata de um sistema de produção enxuta (Lean Production System) com conceitos que tornaram-se a base para esse sucesso duradouro.
            Esse modelo de produção enxuta (Lean production) vai além da indústria automotiva, os princípios do pensamento enxuto são universais e têm sido aplicados com sucesso em muitas áreas; na indústria de software esse modelo é chamado Lean Software Development, onde seus criadores Mary e Tom Poppendieck (defensores de agile e maiores autores de livros de Lean Software) apresentam essa abordagem voltada para o processo desenvolvimento de software, detalhado a seguir.
            O desenvolvimento de software lean é baseado em sete princípios, nos quais nem os próprios autores entram em um concenso. Em cada novo livro publicado os princípios são ligeiramente modificados, podendo ser novos princípios e/ou em alguns casos apenas alteração dos nomes e descrição dos detalhes. Embora a base do processo lean seja os sete princípios, (analogia ao número mágico: The Magic Number Seven), atualmente na página oficial de Tom e Mary são apresentados oito princípios. No entanto vou detalhar aqui apenas os sete princípios apresentados em seu primeiro livro: “Lean Software Development: An Agile Toolkit.
1.    Elimine desperdícios (Eliminate waste): O objetivo é entregar exatamente aquilo que o cliente quer, nem mais nem menos do que exatamente o que o cliente quer. Para isso são apresentados alguns pontos nos quais ensinam a identificar esses desperdícios (Learning to see waste).  Abaixo são mostrados esses pontos usando como exemplo as práticas de XP (Extreme Programming).
o   Funcionalidades extras: Desenvolva apenas as estórias de hoje;
o   Requisitos: Estórias são detalhadas somente na iteração atual (não detalhar estórias antes de entrarem no planejamento da iteração);
o   Passos extras: Codifique diretamente das estórias; esclareça as dúvidas diretamente com clientes (stakeholder);
o   Encontrar informação: Acomode todos do projeto na mesma sala, incluindo clientes (stakeholder);
o   Defeitos não encontrados por testes: Teste primeiro, ambos desenvolvedores e clientes testam;
o   Espere, inclua clientes (stakeholder): Faça pequenas entregas;
o   Troca de tarefas: Desenvolvedores trabalham diretamente com clientes(stakeholder)

2.    Crie conhecimento (Amplify learning/ Learn first/ Create Knowledge): Consiste em criar o máximo de conhecimento sobre o que está sendo desenvolvido, não só em suas fases iniciais, como é sugerido em cascata(requisitos definidos), mas o aprendizado deve ser constante, sendo ele  antes, durante e depois do desenvolvimento. Esse princípio visa trazer à tona todo conhecimento absorvido pelos integrantes do projeto, o conhecimento tácito deve se tornar explicíto e fazer parte da organização como um todo. O ponto principal por parte das empresas, seria  entender que aprender sobre o produto em desenvolvimento é importante mas codificar conhecimento para uso futuro é essencial.

3.    Adie decisões (Delay/Defer commitment): Adiar decisões importantes para o mais tarde possível no projeto. Com essa prática, toda decisão que tem impacto irreversível  no projeto é feita com muito mais maturidade, dentro do processo como um todo. Uma decisão de arquitetura por exemplo, pode mudar todo comportamento do sistema, quanto mais distante decisões importantes de arquiteturas forem tomadas mais assertivas elas serão. Isso não significa não ter comprometimento com o projeto, essas decisões devem ser tomadas por time e/ou pessoas experientes, com domínio tanto na tecnologia que está sendo usada quanto no produto e/ou área de negócio do sistema que está sendo desenvolvido; Esse time experiente e comprometido, saberá até quando manter uma decisão importante em aberto. Um mito conhecido é afirmar que seguir a risca um planejamento é ter comprometimento. O planejamento deve ser aberto a mudanças, pode-se criar um plano de alto nível e deixar os detalhes em aberto, para que as decisões importantes sejam tomadas na hora certa, isso é,  planejar cuidadosamente e o comprometer-se com moderação.

4.    Entregue rapidamente (Deliver fast):  Esse princípio significa fazer entregas o mais rápido possível, como exemplificado nessa frase do livro “Implementing Lean”: “The moral of this story is that we need to figure out how to deliver software so fast that our customers don’t have time to change their minds.” Nós precisamos entregar software tão rápido que nossos clientes não tenham tempo para mudar de opinião (mudanças durante o projeto). No entanto fazer entregas rápidas não significa baixa qualidade, é possível fazer rápidas entregas com excelente qualidade. Segundo os autores existem 2 maneiras de fazer entregas com qualidade, lenta com máximo de cuidado e atenção aos mínimos detalhes ou rápida onde as pessoas têm autonomia para aplicar melhorias em seus processos. Empresas com pensamento lean trabalham com padrões de qualidade, mas essas normas existem porque eles incorporam as melhores práticas em como fazer com excelência seus respectivos trabalhos.
 
5.    Dê poder ao time(Empower the team/ Respect People/ Energize Workers): Esse princípio nada mais é do que dar importância a quem está fazendo o trabalho em si. Essas são as melhores pessoas para sugerir melhorias que resulta no aumento de produtividade e excelência do time. Investir no conhecimento para o desenvolvimento profissional do time também é uma prática importante nesse princípio, quanto mais as pessoas obtiverem conhecimentos que agreguem valor ao processo, maior será o retorno pra empresa. Criar maturidade do time é confiar em sua capacidade de ser auto-gerenciável. Confiar na opinião do profissional que está na linha de frente do trabalho, é dar ferramentas para que ele próprio encontre maneiras de colocar em prática essas melhorias, pois não existe nenhum processo tão bom, que não possa ser melhorado.

6.    Inclua integridade (Build Integrity In/ quality in): Um mito existente é que teste é feito para encontrar defeitos, sendo que na verdade teste é para prevenir defeito. Quando o conceito de construir o software com integridade e qualidade desde o início é aplicado, testar entra em primeiro plano (test first). Hoje em dia, existem metodologias de desenvolvimento orientada a testes, como o TDD que tem como objetivo diminuir o número de defeitos inseridos durante a fase de desenvolvimento. Construir com qualidade desde o início não exclui a necessidade de testes, até mesmo em fases finais do desenvolvimento, mas simplesmente oferece meios de diminuir drasticamente os defeitos encontrados em fases finais do projeto. Desenvolver com foco total em qualidade é absorver a cultura de que tudo deve ser feito para prevenir defeitos e não para encontrá-los.

7.    Otimize o todo (See the Whole/ Optimize the Whole): Otimizar o todo é olhar para toda organização quando uma decisão de negócio for tomada. Essas decisões abrangem sistemas que serão desenvolvidos e/ou atualizados. Um exemplo real de um estudo de caso aconteceu na Fujitsu, que recebia muitas ligações da BMI em seu help desk, para manutenção e suporte às suas impressoras. Após um estudo interno foi descoberto que grande parte das ligações eram relacionadas a impressoras de modelos antigos e convenceram a BMI a trocar para novos modelos de impressoras. Após as novas impressoras serem colocadas em uso, o número de ligações caiu em 40%, seria um caso de sucesso se a Fujitsu tivesse se atentado ao fato de que a sua receita também  era composta  pela quantidade de ligações recebidas em seu help desk; Com a queda de ligações a receita foi  igualmente reduzida. Esse fato está relacionado ao mito de otimizar pela decomposição, onde os departamentos trabalham de maneira completamente independente, essa prática tende a tomada decisões como essa feita na Fujitsu, onde o lucro da área de vendas aumentou mas o rendimento que deixou de entrar pelo help desk teve impacto significativo na receita de toda compania.

Do meu ponto de vista, o  maior problema que vem acontecendo hoje em dia é que muitas pessoas confundem os conceitos de ágil com o seu  real significado. Já ouvi empresas falando que aplicam “técnicas ágeis” com paradgmias antigos e outras com bastante desleixo em relação a padrões e normas de qualidade. Como afirmado pelos autores, mas que ao meu ver se aplica todas as metodologias ágeis,  “Lean Software Development is very disciplined. It’s just that you look at discipline a bit differently.”  O Desenvolvimento de software lean (ágil) é sim bastante disciplinado, mas é apenas uma forma um pouco diferente de disciplina.




Referências
Ø  http://www.poppendieck.com/, último acesso: dez/2013
Ø  Poppendieck, Tom/ Mary , Implementing Lean Software Development, 1ª edição, Addison-Wesley Professional , Setembro 2006

Ø  Poppendieck, Tom/ Mary , Lean Software Development - An Agile Toolkit, 1ª edição, Addison-Wesley Professional , Maio 2003

segunda-feira, 1 de dezembro de 2014

Exploratory Testing - Positive and Negative points


Positive Points

  • Has a planning structure of testing ( called charters)
  • Find defects through exploring some pre-defined (planned) areas of testing
  • Do not rely in any formal documentation to be created.
  • So, it is useful when you do not have lots of documentation for each steps of the software or if this documentation changes often.
  • Easy to maintain and change
  • Minimize the time of test design (since it is design and execution at the same time)

Negative Points

  • Highly depends on the test experience
  • Easily misused
  • Can need to have a complementary test scripted to assure test coverage.

Study Cases  

Ad Hoc - Experienced tester & Error guessing

Project scenario: Application to validate promotions from telecom operator. (E.g. Mammy’s day), Development team: Outsourcing, Test Team: Outsourcing (not same company from dev team). Both were working in same client, in his own house.
Solution: It had been applied Ad Hoc tests added to error guessing technique by getting input from Product Manager (from marketing area), who informed problems found in previously; Testers used past experience and creativity to determine the scenarios may cause errors.
Key point: Testers experienced, even they hadn’t experience to the application the testers had a lot experience in software testing.

Checklist - Sanity tests 

Project scenario: Four outsourcing companies working on same software where all leads from each one had a weekly meeting and test team had to report sanity test result.
Solution: Testers weren’t able to execute tests in details following steps in same day that the build was make available to be tested. So, it has been created a checklist where test points were raised and test team used to execute exploratory tests based on that checklist. The test type was sanity (basic), just to identify high priority bugs on new build.
Key Point: Experienced testers in software testing and strong skill on that application.

Session Based Testing & Charter

Project scenario: Agile (Scrum), 1 tester, 12 developers and 1 Scrum master – All team was on the same location. The client was located into a different city (same country).
Challenge: The tester as the whole team received the stories at the same time, and joined the meeting with the PO (client side) to better understand the client scenario. Discuss some technological solutions and raise blocks (scenarios that were not covered by the documentation that needed to be clarified). While the development team was working on the code creation, the tester was planning the chart (created the missions, areas covered)
Solution: The use of chart proved to be very useful since it didn't relied on detailed step by step and allowed a “free time” to focus on learning how to use the product. During the chart execution, it helped to keep the focus and concentrate on the vulnerability areas. Also, the test do not got blocked by not developed yet functionality's, once it didn't not contained detailed step or dependencies between the steps
Bugs were found, value was added and the client was satisfied with the software quality.
Key Point: Experienced tester in software testing and agile methodologies.

Conclusion & recommendations

In a conclusion we can affirm following:

Get escaped bugs: There are some studies that are concerned in a linear way to executing tests mainly in a scripted tests, the point is when these tests are executed several times and in a several version it might become vicious testing and didn’t found effective bugs the result is exploratory tests are able to found escaped bugs.

 Fits in several project scenarios: It is a good practice to performing tests in an agile or not agile projects, in additional it fits in a different test type, it means exploratory testing can be used as system, component, regression, integration tests and so on.

 Used according to the test strategy: It can used in additional to test strategy mainly on saving time.

  Improve test’s effective when used as additional method on test phase: It is better applied when it is used as additional execution method that does why will assure the test coverage independent of development phase or test type. If exploratory testing is applied as complement of these tests the chance to find different bugs are improved.

References:





[4] Oxford Advanced Learner's Dictionary of Current English, A S Hornby, OXFORDOxford University Press 2000

sexta-feira, 14 de novembro de 2014

Exploratory test techniques

Ad Hoc

            Ad Hoc tests are executed without any plan or structure. Tester just open the application and going through, browsing according to his creative.  Bugs are found by accidentally and it is hard to reproduce, due to Ad Hoc doesn’t follow any structure. 
If you take a look only in the word "exploratory", it might be acceptable to say that the tester are performing a kind of exploratory testing, but defini tivelyit can not be classified as exploratory tests at all!! It happens because this test is performed without any plan, design or structure.

So, the question arises: Is there any scenario to apply Ad Hoc tests?
The answer is yes, there are!! It will depend of the strategy defined before testing. Following is some examples that this tests might be a good solution:

  •  Software’s stability: During code freeze phase or as part of regression tests, Ad Hoc might be a good practice to help on get software’s stability due to task force and browsing in the software without plan or pre-requirement.
  • Get knowledge about new software: When new person is joined to the team, in this case find a bug is "plus" of learning process.
  • Hands on by leader/manager: Test leaders/Manager tend to don’t test software in details due others tasks, Ad Hoc test could be applied to get feeling about the software which he is responsible.
Also can be exist others situations to apply Ad Hoc tests, once more, it will depend of the strategy defined. Testing software is a creative area, these examples are just to give you some ideas to define the best strategy for your project.

Checklist as guidance

An checklist with highlights about software to be tested, is created before performing tests. It means only main function without details.
Let’s suppose that it is software to use SMS function from your mobile device. There are options to send, receive, delete, save as draft, based on these main functions, an checklist is created. Take a look below:



Checkpoint
Test result
Send SMS


Receive SMS


Delete SMS


Save as Draft




The checklist is used as guidance, during exploratory testing. The tester can also use his creativity to explore each situation, see below.

Checkpoint
Examples of scenarios to be explored
Send SMS

concatenated, not concatenated, in roaming, not roaming and so on
Receive SMS

connected to phone, not connected, during a calling or not and so on
Delete SMS

delete saved SMS, not saved, in inbox, sent box and so on
Save as draft

when receive a call, when leave the screen and so on

Note that is important, the setup and clean environment for each checklist line, to garantee the traceability and bug reproducible.

Session Based Testing & Charter

Session based testing was created by James and his brother Jonathan Bach as a need to track the testing they were performing. Since Exploratory testing is a king of a ad hoc processing, not restricted to pre-defined test steps or test procedures with mission of  finding bugs without previous noticed and find them fast sessions-based testing were created.
Session based test management provide a way for the testers to make orderly reports and organize their work without obstruction the flexibility that makes exploratory testing useful.
As definition of what is a session, sessions are time-boxes within which the testing occurs. Each session has a charter (a little mission) and results in a session report.
Basically, each test session (charter) can be composed of the following sections (but not restricted to):

  • Mission statement: What you're going to test (Scenario)
  • Areas to be tested: The functionalities from the mission
  • Tester names
  • Date and time used for the session. That can be a previous planning of a minimum time and a maximum time.
  • Test Notes: Where the scripted part is created
  • Issues: questions or problems that not necessary is a bug
  • Bugs: The bugs found into the areas tested

segunda-feira, 3 de novembro de 2014

Exploratory Testing - What really are exploratory tests?

Despite the fact that the concepts of Exploratory Testing and Ad Hoc testing are similar, and sometimes defined as: something, different names, we need to establish the concepts for each one of them. It is a very useful thing when you talk to someone about exploratory testing subject, for example, and ask: “What exploratory testing is for you?” From the person answer you got the idea and you can start on what exploratory is or isn't.

For this article proposes, we can establish the following:
  • Exploratory Testing: were defined on the section above.
  • Ad hoc testing: If we get the Oxford dictionary [*] definition: “arranged or happening when necessary and not planned in advance”. In other words, translating into the test word, it can be defined as: an unplanned in advance testing but used when necessary to solve a problem. Prepared for a particular situation, as James Bach's definition.

So, Ad hoc testing can differ from Exploratory by the lack of previous planning, but it does not mean that it is a worthless technique of testing. If fact, we can think as ad hoc as a part of the exploratory testing, once that you are creating your test as long you are running it.
We dare to establish that: all ad hoc testing are exploratory testing, but not all exploratory testing are ad hoc.

Despite of what may seem, Ad hoc are very useful in some situation or scenarios. Such as:
  • Get to know the software: If you are going to a new project, or testing new software that you have never heard about, you might just what to navigate around without any scripts to know in what software you'll be working for.

  • Verify the Software stability:  After all the bug fix phase on software, it is supposed to be ready to go. So, you might what to just navigate around, and see if everything is ok, or just take a last look into the software that you help to build.

terça-feira, 28 de outubro de 2014

Exploratory Testing - Introduction

Abstract

What comes to your mind when your heard the word “exploratory testing”?
Some can say that this is a way of test software in a short period of time, others may think that this is a friendly way to run tests, once that it do not follow a formal documentation, or even there are some that might think that this is just a “walk through” on the software functionalities and then assure the product quality.
Besides that, there are other questions related, such as: How can I apply Exploratory Testing on my software? Who should run this tests ? Exploratory Testing really assure the quality of the product ?  How can I report the results ?
This speech has as objective clarify this and other questions related to exploratory testing as well as present some case studies where exploratory testing were applied in real projects with different objectives and software’s.

Introduction

Not different than others aspect, exploratory testing is one more thing that is not strictly defined yet. That’s why I am writing it, to explain the suitable concept about exploratory testing and help you to understand; although this can also been improved and changed further. =)
Some managers tend to not trust on exploratory method due to lack of formal documentation and in other hand, there are some managers that think the “poking around” on application is a exploratory method giving enough tests to guarantee product’s quality; Actually it is neither one nor the other, exploratory tests is just different way to executing tests but it doesn’t means that it is a messy, it is just different way to find bugs and it is a pretty effective method to do it.
At first we have to realize that “exploratory” is very embracing word and it is also applied on technical side; It means that you can use “exploratory” way on performing tests like ad hoc or other one, you are really “exploring” the software and you are able to call it as a exploratory testing, but you can’t say that exploratory testing are ad hoc tests because they are completely different. What I am saying is: the ad hoc tests are kind of exploratory tests but exploratory testing are not ad hoc tests. It will be clear on further.
There also a formal technique to perform exploratory testing created by James Bach, when you can document and get the steps traceability, it is Session Based Testing.
This work explains what is exploratory testing, the difference among ET and Ad hoc, some techniques to perform ET, positive & negative points as well as some case study where has been applied exploratory tests using the same techniques detailed here.

Exploratory tests definitionS

I prepared some phrases found on some books/links to define exploratory tests:

Ø  “What’s the big deal? Exploratory testing is random pounding on the keys.  Nothing to it. My toddler does it every day.”[1]

Ø  … “Oh, Exploratory Testing,” said one of the developers, “that’s where the tester does a bunch of wacky, random stuff, right?” … [2]

Ø  Exploratory, or ad hoc testing, can be an especially useful testing strategy… [3]

Ø  "Exploratory testing is an interactive process of concurrent product exploration, test design and test execution.” “To the extent that the next test we do is influenced by the result of the last test we did, we are doing exploratory testing.” James Bach, Satisfice (2001)


It is just to show up that different literature defines exploratory tests in a different concepts, people think different and defined their own perspective regarding topic. I am not trying to dictate the right concept, I am just telling you the most suitable concept to define exploratory test allowing for “exploratory” is pretty embracing word. 



... to be continued in next posts...

quinta-feira, 23 de outubro de 2014

How software testing fits the cloud computing aspects?

Cloud computing is still an IT industry hot topic and has been gaining space and importance; Meanwhile, cloud testing remains unknown even for testing professionals around the world. It happens due to not only the limited documentation and references, but also because many companies already have a testing structure very well defined and does not have any application or product in the cloud. As mentioned before, much has been said about cloud computing, at this moment a question arises: How software testing fits the cloud computing aspects?
Let’s check below.

Cloud computing
Firstly we need to understand what cloud computing is.
In a few words cloud computing is a service to everything you want from IT industry. It means, store personal files, photos, documents and so on, where all files may be accessed anywhere and anytime with no software installation required.
Cloud computing is more than this simple example, which reflects your life as a user of cloud computing. The same idea is suitable to IT industry in totality. In a B2B (business-to-business) approach, it is possible use cloud service for an infrastructure, platform, software etc... In other words, everything-as-a-service (XaaS): infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS) and others.
A cloud computing advice would be: do not waste money in things that are not your business. In IaaS for example, it is not necessary to take servers inside the companies, they might rent as a service to fit infrastructure according to the business. Think about an online company that sells shoes; their business is to sell shoes, not to control technology for selling online, this is a typically scenario that consumes in a great way a cloud computing services.
The main idea of cloud computing is cost reduction and new opportunity.

Cloud testing
There is a subtle difference between testing in the cloud and testing cloud services. Testing applications that are in the cloud is a challenge on finding a test strategy to perform tests in this new computing platform; this approach will not been attended in this article. The focus here is present a cloud testing as a services that fits to software’s which are located in the cloud or not.
Cloud testing is a new way to manage your tests where companies offer testing as a service:
-          For Outsourcing: when another company implements your tests. The difference from traditional outsourcing remains on the fact that you get complete access on your repository to execute, get results or metrics and so on. Your pool of tests will be available to been managed anytime and anywhere.
-          For consuming infrastructure: when you manage your tests, to implement tests and to execute them. In this case, your repository is also available to been managed anytime and anywhere without demanding internal infrastructure. In additional you get access to several resources such as browsers (versions/platform), mobile (models/versions). This might been used for both automation and manual tests.
Benefits of cloud testing
The focus is cost reduction by consuming resources on demand. It means only pay for what really use. Some cloud testing providers offer cloud testing by hours per month where you get access to all platforms, devices and whatever the provider offer according to the contracted hours beyond offer follow-up of test creation and execution in a real time.
Another point is the security, in this case your company chooses for a dedicated storage in providers and off course, pay for it. This is seasonable for under development applications that are not in market yet or in case of some business strategy.
The great advantage on consuming cloud testing service, beyond price, is ease of access and virtualization environments without taking dedicated servers in house, making it simple, for developers and testers.

Conclusion

Your application does not need to be in the cloud to been tested in the cloud. This is a new way to test software in different approaches, such as, automation, manual, unit, black box, grey box.  To achieve a quality of software it does not matter where your software is going to be tested; the most important is apply best practices in software testing area. Cloud testing just makes life easier for tester’s professionals since it offers a several platforms and environment to test your application for a low cost.  The strategy to optimize cloud-testing services remains in the hands of the best professionals.

References
www.informationweek.com/ blog/229206200