SOGETI UK BLOG

It’s no secret that from time to time, us testers feel that we get the blame for delays in projects. The impression is that we have not tested well enough and as a result defects and issues cause projects to need extra time.

Sometimes this is true – when we are not fully prepared before the test period.

Maybe we “guesstimated” and not estimated the amount of testing work needed to be done, and then copied old (not updated) test scripts with the approach of testing everything all over again. Quite often we even try to test poor quality systems until the quality improves, and that can be a rather expensive approach with a low success rate. 

 So here is a list of possible reasons why the test team are delayed or not able to deliver:

– Activities in earlier phases, for example in requirements, design, development, were not fully completed before the next test phase started up.

– The code is not delivered to test according to the agreed schedule.

– Deliveries from earlier phases were handed over to the test phase with poor quality.

– The test team receives a delivery with defect(s) that make it impossible to start the test execution, and therefore a lot needs to be tested again.

– The test environment is not delivered according to the agreed schedule.

– The test environment is not complete with sub-systems, configuration management, databases etc., or is unstable.

– The test environment does not contain complete and coherent test data.

– Test resources are not delivered according to agreement (often key resources from business).

– Other projects change the code that is integrated with the code under test.

– Other projects disturbing with their activities in the test environment.

Let’s stop playing the blame game, find the bottlenecks and improve the development and test processes instead. How other areas of the business approach and achieve these is up to them, but testers can start by doing a Test Process Improvement (TPI) assessment and documenting the maturity level of the current test process. If you use the TPI NEXT® model it will also provide a better understanding of the correlation between the test process and adjacent processes. It will help extensively in discussing and addressing issues for improving the overall software development lifecycle process.

Gro Rognstad AUTHOR:
Gro Rognstad has a passion for Testing and Quality. She is Chief Technology Officer in Sogeti Norway and responsible for bringing our testing services and innovations to customers and employees. She is a senior advisor with a broad background and 27 year in IT - the last 17 years focusing on Testing and Quality.

Posted in: A testers viewpoint, Opinion, Performance testing, Test Environment Management, Working in software testing      
Comments: 0
Tags: , , ,

 

TestCenterMany companies have started their journey for setting up a Test Center of Excellence. They also allocate more of the overall IT Budget on increasing the quality on the applications. A Test Centre should be responsible for some of the IT Departments goals for delivering measurable improvements in quality, faster time-to-market, cost reduction and more efficient IT operational processes.

To build a successful Test Centre of Excellence the following corner stones need to be in place:

Industrialized Test Process

The test process must be structured, with predefined phases. This process should be followed by all projects. The test process must be adapted into other processes in the company (software development process, deliveries from suppliers, etc.) The testers should feel familiar with the established process and training is important.

The goal of an industrial process is to facilitate efficient deliveries with higher test coverage.

Methods, tools, and accelerators

The Test Center’s method should describe how testing should be carried out with a strategy, procedures, templates, techniques and tools. These must be available and followed by all.

Catalogue Based Testing Services

There is a clear advantage if the Test Center’s tasks are offered as testing services where the (internal and/or external) customers of the Test Center order according to what they need, and not a named person – “I need Anna to do this testing since she is the only one with this special competence…”. This ensures that the focus is on competence management, and dependency on key resources will be lower. Examples of services: test management, creating and maintaining test scripts (manual and automated), functional testing, performance testing, security testing, test environment management, test data, etc.

Governance Model

It is recommended to establish a governance model for the Test Center. Since test and quality activities allocate a larger part of the IT budget it is important to show that the investment leads to faster time to market, cost reduction, improved quality, and better risk management.

As part of the governance model, it is advisable to establish the Test Center with an SLA structure, either for the unit as a whole and/or for each service from the service catalog. It is also smart to define a set of strategic KPIs and/or tactical KPIs.

(Global) Delivery Organization

Test activities for the projects can be organized in test lines per domain/product. The test line consists of one core test team, with special knowledge on the domain/product and can be extended with flexible test resources when needed – either when it is a peak or special testing competence is needed. This enables a standard way of testing, and the resources can be utilized in a more efficient way. Because all test lines use the same industrialized basis for test processes and test tools, new test lines will be operational at short notice, and will profit from advantages of economies of scale with immediate operational flexed capacity. By organizing the testing in this way it is also possible to have a global team and offshore (some of the) test lines.

Continuous improvement

A Test Center should regularly measure their test maturity and have plans for continuous improvements. The process should be formalized with assessments where the team agrees on pain points and improvements that should be delivered during the next coming period.

 

Gro Rognstad AUTHOR:
Gro Rognstad has a passion for Testing and Quality. She is Chief Technology Officer in Sogeti Norway and responsible for bringing our testing services and innovations to customers and employees. She is a senior advisor with a broad background and 27 year in IT - the last 17 years focusing on Testing and Quality.

Posted in: functional testing, Software testing, SogetiLabs, Test Automation, Test Centers of Excellence, Test Environment Management, test framework, Test Tools      
Comments: 0
Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

 

testingproblemsIn recent years, many companies focus on improving their test process. The aim is that the improvements will lead to measurable improvements in quality, faster time-to-market, cost reduction and more efficient IT operational processes. Still, they think it is best to first establish a good test process and then consider if they need to improve their test environment.

The World Quality Report 2013-2014 found that many companies have challenges with the test environment and test data. It is a new area of concern.

The problems are often put forward as “statements” from the test managers and testers. They struggle with downtime and interference with each other during the tests. Too much is often going on in the same test environments simultaneously since projects often have the same delivery deadlines. At the same time, they  do not get much sympathy from management. The project teams are seldom responsible for delivering the test environments – it is delivered by an infrastructure partner. Reports “document” that the test environment is ‘up’ and available according to SLA requirements or other agreements with the infrastructure partner. However, I have been a test manager in several projects where the test environment was not available half of the test period.

Examples of problems:

– “Someone” has added new code or changes in the test environment leading to serious errors.

– “Someone” has given the test team the wrong version of the code because of a lack of configuration management.

– “Someone” is running a job that leads to poor performance.

– The data is too old and outdated.

– There are not enough devices available for testing apps. This leads to manual tests with the most common devices and poor test coverage.

Assessments we have done for clients show us that these challenges are seldom documented by the test team, and the management does not take decisions based on emotions. They need facts. Problems during the test periods must be documented so that the management can find the right balance between risk, design, management and support, and identifying the optimal level of investment in test environments. 40% of the total test costs are related to the testing hardware and infrastructure. Professional Test Environment Management (TEM) services enable the organizations to speed up their software release schedules by up to 25%, cut infrastructure costs by 5-10% and increase item team productivity by up to 30%.

In other words,  it will be profitable to improve the test environments. Test environments are complex and the challenges will increase with the need of apps testing on devices in addition to traditional tests. It will require professional test environment management and test data management, with SLA agreements that secure stable and available test environments.

Test Environment Management services provides a holistic approach to the area of technical testing services, from Service Management, Data, Build, Architecture, Infrastructure, Applications and Virtualization.

1. Test Environment Management (TEM) Services.
2. Test Data Management (TDM) Services.
3.Development Operations (DevOps) Services.
4. Infrastructure, Virtualization & Cloud (IS) Services.
5. Wide-angle Application Rationalization Program (WARP) Services.
6. Service Virtualization (SV) Services.

To secure the expected effect of the improvement of the test process, it is necessary to also improve test environment and test data management. It must be seen as key areas in the improvement program. Some good advice to all testers struggling: document and report the challenges you have with the test environment.

References:

– World Quality Report 2013-14

Gro Rognstad AUTHOR:
Gro Rognstad has a passion for Testing and Quality. She is Chief Technology Officer in Sogeti Norway and responsible for bringing our testing services and innovations to customers and employees. She is a senior advisor with a broad background and 27 year in IT - the last 17 years focusing on Testing and Quality.

Posted in: Apps, Big data, Infrastructure, SogetiLabs, Test Environment Management, World Quality Report      
Comments: 1
Tags: , , , , , , , , , , , , ,

 

quietplzI am not sure about that! I think our clients – the business – will expect more of us in the near future. The World Quality Report 2013-14 found that 46% organizations don’t have a specific approach to testing in agile projects. They are adapting testing methodologies based on waterfall development models. And as many as 87 % of businesses in the Nordics report that they are using agile, but larger companies are typically seen to implement only certain elements of agile and experience greater difficulties adapting their processes, methods and people to the agile ways. We have to dig in to these challenges and find solutions for the areas not adapted by the companies.

Since Agile Manifesto was written in 2001 we have learned that an agile environment gives the project members more responsibility, more challenges and interesting days at work. More fun at work! According to Everett M. Rogers’ adoption curve, we have reason to believe that we only have the laggards left in waterfall methods in the Nordics, and they might wait until the next big wave comes?

Diffusion of innovations

Figur 1 Everett M. Rogers 1963 – Relationship between types of adopters classified by innovativeness and their location on the adaption curve

For us testers, agile environments are an advantage compared to the traditional waterfall projects. The developers have not used all the money and time when we get something to test. However, agile environment in its current form is not the final destination. We can – and must – be even better. The clients expect us to deliver measurable improvements in quality, faster time-to-market, cost reduction and more efficient IT operational processes. There are some challenges in the development process which will force us to improve our development and test processes.

No fault forward

The majority of the projects are planning with sprint tests with good test coverage. In theory this should lead to less testing in the system (integration) tests. If it is a large project, responsible for developing one or several complex systems, it turns out most often that it is difficult to divide up the product backlog between the sprint teams. The consequence is that the sprint teams have challenges to deliver something that is testable at the next test level. We have learned that it often ends up with large tests at the end of a project (with a lot of integration defects between components developed in the sprints). This leads to overrun, delay and not the expected product quality.

The product owner seldom thinks like an IT-engineer

The client starts up a project because they want to achieve business goals. All the important goals are divided into items in the product backlog – functionality they expect us to deliver. The product owners are often very competent in their field, and in the projects they end up being busy prioritizing requirements and providing the development team with a lot of clarifications. I believe they feel that they are more involved in an agile environment, but at the same time they have been given more responsibility for the success. How can they keep track of all the items in the product backlog and at the same time keep an eye on the business goals? Have we given them the right tools so that after the project they can report back to the stakeholders that the projects achieve the business goals, expected ROI, manageable risks, and right product quality?

What’s next?

We have made ​​significant progress in the way the development team works. We work in a much more solid way than before. Now we must look beyond design and development.

We should focus on the Product Owner and the organization around and give real support to the important role of the Product Owner. For example, we should implement better management reporting with relevant information to the stakeholders. A good start is to respect that the business team are experts in something else than software development team. Instead of waiting for the business to adopt our way of working we need to change ourselves and find new ways to collaborate with the business, especially in the requirement and design phase. Good examples are clear quality gates and more use of Model Driven Development (and model based testing), virtual prototypes and testing, and simulation testing.

References:

  • World Quality Report 2013-14
  • Rogers, E.M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

Gro Rognstad AUTHOR:
Gro Rognstad has a passion for Testing and Quality. She is Chief Technology Officer in Sogeti Norway and responsible for bringing our testing services and innovations to customers and employees. She is a senior advisor with a broad background and 27 year in IT - the last 17 years focusing on Testing and Quality.

Posted in: Agile, Collaboration, Software Development, Sprint tests, waterfall, World Quality Report      
Comments: 0
Tags: , , , , , , , , , , , , , , , , , ,