Success of a test automation project can easily be compared to winning the Formula 1 championship title! In F1, the talent of the driver is a fundamental must-have; as is selecting the best car. In software testing, choosing the right automation team is a critical success factor in addition to identifying and selecting the best test automation tool for the project.

When it comes to tool selection, many organisations are struggling to identify, select and implement appropriate automation tools due to lacking internal skills, knowledge, practical experience and confusing around tooling trends.  Hence, they face challenges in realising the benefits of automation – time, cost, quality, time to market and improved test coverage. In some extreme cases, organisations have even terminated the automation project as it has backfired and had a negative outcome, providing no benefits.

In order to select the best automation tool, testing personnel should consider a number of diverse drivers which can influence the success of the automation project in both the short and long term. These drivers are:

• awareness of the automation tooling market/trends
• tool vendor strategy
• client business strategy/agility
• technology landscape
• system integration/consolidation/amalgamation
• effective combination of tool licensing models
• selection of multiple tools
• tool features, benefits and limitations, reliability and governance
• availability and capability of technical support
• incorporation with automation team competencies and the user community
• integration with various operating system/browsers/technology/hardware/tools
• embracing various testing types, levels & stages
• extending the tool to non-automation users
• test data management
• privacy

…and the list goes on!

In addition to this expansive list, we have seen many new automation tools released into the market, in recent years, which makes the tool identification process even dicier as there are more options to choose from!

Let’s for now focus on the client challenges, industry/tooling trends and technical dynamics. At Sogeti, the Automation Center of Excellence (CoE) team have used insights from Sogeti’s Global Innovation/SME team to create a fool-proof Automation Proof of Concept (PoC) Tool Kit, which consists of global best practices, proven systematic approaches, an industry leading methodology plus insights for tool identification, selection and models for evaluation. For example, The ‘Funnelling Model’ provides a systematic approach to gather, isolate, evaluate, select and deploy the best automation tool for the specific challenge or situation.

The Funnelling Model follows the seven stages below to provide the best outcome through tool evaluation and selection processes:

•         Stage 1: Tool Evaluation Daily Plan & Schedule – This document provides the list of critical daily Proof of Concept activities to be performed in order to achieve success in the tool evaluation and selection project.
•         Stage 2: Automation Feasibility Assessment Checklist – This industry leading assessment questionnaire enables the organisation to conduct an in-depth assessment of the current and predicted future state of the project, in order to understand the considerations required for the tool selection as part of test automation strategy.
•         Stage 3: Proof of Concept Entry & Exit Criteria – This document lists key quality gates and checkpoints to be reached in order to conduct successful tool evaluation and selection.
•         Stage 4: Tool Features Comparison Matrix – This utility compares and analyses industry leading commercial and open-source automation tools available in the market against 11 criteria and 200+ sub-criteria – all of which are key and critical to tool comparison success. Based on the comparison, the utility provides graphs and metrics at criteria level for the user to quickly isolate and cherry pick the most suitable automation tools. Further evaluation is carried out at Stage 5.
•         Stage 5: Tool Evaluation & Selection Checklist – This stage provides a systematic and structured approach, allowing the user to define requirements and evaluate the tools selected at Stage 4 against an additional 16 criteria and more than 220+ sub-criteria. Post-evaluation, the model again provides graphs and metrics both at summary and individual criteria level for the user to narrow down the scope, allowing the best automation tool to be selected for both current and future project requirements.
•         Stage 6: Tool Evaluation Report & Presentation – These documents provide detailed evaluation summaries of the journey, tools considered for evaluation and the reasons why they were selected plus details of evaluation criteria used, scope, deliverables, risks, issues, observations, tool recommendations, pricing, automation benefits realisation plan and predicted Return On Investment (ROI).
•         Stage 7: ROI & Savings Calculator – These are designed to calculate the ‘how much’ and ‘when’; the financial savings and other automation benefits to be gained following the deployment and execution of automated test scripts. The model is designed to support decisions on tool selection, plus which areas and how much of manual testing should be automated, by calculating anticipated savings on an annual basis.

And so you can see that, although selecting the right Automation tools may at first seem daunting, there are ways to make the selection process a little simpler and more measured. Why not try the Funnelling Model next time your tools are up for renewal?

Sogeti can assist in the selection and implementation of automation tools. To find out more, please visit our website.

Ganesh Practoor AUTHOR:
Ganesh is a managing consultant, UK Test Automation CoE, for Sogeti UK.

Posted in: Automation Testing, Requirements, SDLC, Software testing, Test Tools, Working in software testing      
Comments: 0
Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,


Anyone in the testing profession will tell you that automated testing has garnered much attention over the past few years. The shift to automated testing is not only due to emerging platforms which require it, such as mobile applications, but also because of the efficiencies and financial benefits it provides.

As a Managing Consultant at Sogeti UK, my primary role is to head the UK Test Automation Centre of Excellence (CoE) & Subject Matter Expert (SME) community, support test automation projects and sales activities in the UK. In addition, I am leading the Global Test Automation Innovation initiative at a group level. I am exposed to the breadth of industry sectors globally looking to invest in test automation within their armoury of testing methods. These sectors include financial services, telecommunications, utilities, energy services, life insurance and retail. But, as you might expect, it’s not always as easy as simply implementing automation and then assuming ­– or even hoping – it will work.

Automated testing is more than a process or a technology, it’s an attitude. The best testing professionals and consultants look at testing activities as a whole and figure out how to implement automation effectively and consistently throughout the testing life cycle.

In order to successfully implement test automation, there needs to be a thorough understanding of each client’s ambitions and business goals. We then need to consider whether automation can effectively realize these ambitions and business goals.

At Sogeti, we are changing the way organisations look at and implement automated testing in the following ways:

– Structured Approach: We have developed a standard automation approach (consistent with our proven TMap NEXT® methodology). Our standard approach also includes Return on Investment (ROI) and Break Even Point (BEP) calculators, which are used to quantify both time and money viability in advance of significant investment in automation in order to ensure we realise client’s ambitions and business goal within the required timeframe.

Implement: Sogeti is dedicated to providing thought leadership, innovation and industry-leading best practices. We work with each of our clients to implement robust strategies, accelerators, ready-made automation frameworks, re-usable libraries, scripts and effective risk management techniques.

– Success: The best way of achieving a business goal is to define it! Defining automation goal is not as easy as it sounds. Sogeti takes the time to understand each client’s industry sector through a SMART (Specific, Measurable, Attainable, Realistic, and Time-bound) goal approach to define test automation metrics in support of our client’s unique business goals.

To read more about how Sogeti develops automated testing, check out our whitepaper or read some of our opinions here.


Ganesh Practoor AUTHOR:
Ganesh is a managing consultant, UK Test Automation CoE, for Sogeti UK.

Posted in: A testers viewpoint, Opinion, Performance testing, Software testing, Test Automation, Test Methodologies, Testing and innovation, TMap, Working in software testing      
Comments: 0
Tags: , , , , , , , , , , , , , ,