SOGETI UK BLOG

We’ve all heard the IoT statistics and it’s clear that 26 billion objects predicted to be connected and $1.9 trillion generated by 2020* and an average of 25 vulnerabilities per connected device**, adds up to an urgent need for an effective IoT-specific test strategy. Enter Tom van de Ven, Jaap Bloem and Jean Pascal Duniau, whose book “IoTMap – Testing in an IoT Environment” was launched at the end of April.

We interviewed lead author Tom van de Ven to discover his 5 step approach and the building blocks required to implement a successful test strategy for a multi-layered IoT solution.

What would be your elevator pitch for your new book?

The book outlines a future-proof, 5 step approach to IoT testing that demonstrates that there are 3 key areas of focus required for success:

1. Testing IoT solutions is all about testing the Customer Experience (CX) and the building blocks of a strong test strategy must be focused on the end user by bringing together business intelligence, high tech and mobile testing expertise.

2. Ensuring quality, security and an excellent CX requires a shift from relying on functional unit and regression testing, to focusing on non-functional quality attributes through, for example, interoperability, performance, recovery, reliability, security and usability testing.

3. Crowdtesting and storytelling are an excellent way to understand the needs and habits of the end user and keep pace with rapid releases.

The book uses real life examples and clear jargon-free descriptions to illustrate that if you understand the 4 layer IoT model and apply the 5 step approach with CX, non-functional testing and crowdtesting as your focus, you can create a powerful and comprehensive IoT test strategy.

How is the test strategy in the book future-proof?

The 4 layer IoT model I use as a basis for the test strategy can be dismantled, adapted and repurposed to embrace new research by technology leaders such as SogetiLabs, so that the 5 step test approach will remain relevant to solving IoT challenges even as they evolve and grow in complexity.

1. The “Thing” layer comprising the device sensors, hardware, electronics and mechatronics

2. The “Bridge” connecting the thing to a platform for data storage and wifi, bluetooth or wired connection.

3. The “App Data” platform layer where data gets stored such as Azure or Bluemix

4. The “Application” layer where the level of end user interaction is at its peak whether you are dealing with a touchscreen for a smartphone app or navigation device or a web interface.

These layers can all be combined when dealing with the types of IoT objects we are seeing today, but can also be viewed as separate elements and combined with new elements that arise as IoT solutions and technology develop in the future. The book is also future proof in the sense that there are chapters devoted to exciting, cutting edge ideas, such as using Artificial Intelligence in your test strategy.

Which industries are going to be impacted first?

Banking and Financial Services particularly Insurance, will be heavily impacted.  Currently insurance companies distribute risk across a huge number of people, but the IoT will enable companies to mine a vast range of highly accurate data, very fast, in real time, giving a much clearer individual picture. Insurers will need to rethink how they will calculate our insurance premiums!

What are the challenges testing in an IoT environment?

Determining how to test in an IoT environment means considering what will be tested, how to test it and crucially, as the potential levels of connectivity are almost infinite, when to stop. My mantra for decision-making is Connect, Talk, Think, Act.  Are we testing the entire device or the component parts? Do we need to simulate the whole IoT environment such as a connected home or just simulate the immediate application environment? Does the environment require extremes in temperature, terrain or speed? If the device is a smart fridge, is the end user likely to connect it only to their mobile phone or is it also going to be talking to a smart watch, fitness device and separate freezer unit? It’s important to know at what point in the possible chain of connections we can safely and sensibly stop testing, as after 3-4 connections there are so many variables that testing becomes extremely complex and increasingly expensive.

The fact that we are dealing with a physical object comprised of complex technology is the biggest challenge in IoT testing and of course security is one of the most important considerations. It should be built-in but IoT objects are composed of several component parts and different developers and vendors have hugely varying security policies. As testers we need to look beyond the usual security tests and assess the device from the perspective of whether we can break into it remotely and in person, exploring all the ways it can be misused and putting ourselves into the head of a new kind of IoT Cybercriminal.

Another challenge is the potential skills gap in IoT testing. As end-to-end IoT testing requires virtually every area of testing expertise, even companies with a dedicated test team will not have the resources to test effectively. The answer here is greater levels of collaboration both with dedicated test partners and also with businesses in other industries which have the required expertise and access to the necessary information. For example insurance companies are now working with mobile phone providers to access data about car drivers and their vehicles. These practical challenges are overcome by my 5 step strategy and tooling approach, which is described in detail in the book and which we have already used at Sogeti on several client projects, to great effect. However the IoT also demands a new way of thinking and a cultural shift. To ensure that connected Things are safe and secure and provide a brilliant CX, we all need to take inspiration from services like Amazon Dash and switch our thinking to be more like a collaborative and nimble start-up.

IoTMap – Testing in an IoT Environment is now available in print and downloadable versions here.

*Gartner: http://www.gartner.com/newsroom/id/2636073
**HPE: http://go.saas.hpe.com/fod/internet-of-things

Tom Van de Ven AUTHOR:
Tom van de Ven has been active in the field of High Tech testing for over 12 years. As a High Tech test expert he is a frequently asked sparring partner for Sogeti High Tech customers with regard to starting/professionalizing test projects. Besides a multitude of test assignments (eg. in the field of healthcare, semiconductors, agriculture and automotive) he is an active member of the Sogeti High Tech Test Competence Centre and a speaker for High Tech seminars. Tom uses his experience in a role as a coach for (beginner) High Tech test engineers and is constantly looking for improvements in High Tech test methodologies. He completed his new book that combines Internet of Things and TMap: IoTMap in April 2016. He also teaches and develops several testing courses in the embedded and high tech domain. If not teaching, testing a tunnel or promoting “Quick Tech Testing” you can find him setting up a high tech test automation framework for the odd customer.

Posted in: A testers viewpoint, Innovation, Internet of Things, IT strategy, Software testing, sogeti employees      
Comments: 0
Tags: , , , , ,

 

An Internet of Things (IoT) solution is built up from items that together create new functionality. The separate elements form an IoT ecosystem. Testing the entire system must be thought of at an early stage in IoT product development.

Forget functional testing
Define a product with a set of requirements and all functionality (that is written down) can be tested. As said an IoT solution consists of separate elements that can be functionally tested . The IoT ecosystem as a whole behaves according to different rules. The possible connections to 3rd party IoT products are endless. As a result it will be impossible to describe the behaviour in every situation. One of the elements of setting up an IoT test strategy is looking at the non-functional behaviour of the whole solution.

Choose the right quality attributes!
Functionality is tested enough. We must search for ways of testing all other behaviour of the IoT ecosystem. Using quality attributes we can make good effort in filling this gap. I will give you some examples of quality attributes that can be used in IoT environments:

  • Installability: upgrades can be sent to “Things” and remotely installed. Test situations where this can go wrong (think of power loss during download or installing) and check if recovery mechanisms work.
  • Interoperability: information exchange between (parts of) IoT systems must work between different software and/or hardware versions. All known combinations can be tested but what to do with software versions not yet released? What happens when new hardware is on the market that should work with your solution as well?
  • Usability: users will only launch an App once or twice to form their opinion on usability. Usability testing is key in making a full IoT solution a success.

Create your IoT test strategy
Choosing the right quality attributes is one of the steps in defining an IoT test strategy. Read more on the other steps and IoT quality attributes my book: “IoTMap: Testing in an IoT environment”.

Book launch event
The book is launched on the 21st of April in the Netherlands during a book launch event where you can get a free copy of the book. More information and the program (in Dutch) can be found here.

 

Tom Van de Ven AUTHOR:
Tom van de Ven has been active in the field of High Tech testing for over 12 years. As a High Tech test expert he is a frequently asked sparring partner for Sogeti High Tech customers with regard to starting/professionalizing test projects. Besides a multitude of test assignments (eg. in the field of healthcare, semiconductors, agriculture and automotive) he is an active member of the Sogeti High Tech Test Competence Centre and a speaker for High Tech seminars. Tom uses his experience in a role as a coach for (beginner) High Tech test engineers and is constantly looking for improvements in High Tech test methodologies. He completed his new book that combines Internet of Things and TMap: IoTMap in April 2016. He also teaches and develops several testing courses in the embedded and high tech domain. If not teaching, testing a tunnel or promoting “Quick Tech Testing” you can find him setting up a high tech test automation framework for the odd customer.

Posted in: functional testing, Internet of Things, IT strategy, Quality Assurance, User Experience      
Comments: 0
Tags: , , , , , , ,

 

The promise of the Internet of Things (IoT) is growing by the day. Thanks to the increasing connectivity of devices and sensors and the increasing flow of valuable information it brings to create new and imaginative vistas. IoT creates new products and adds functionality to existing products. We can couple endless series of products and let them communicate with each other. This means another approach to quality and testing.

An IoT solution contains a broad set of expertise. Think of data storage, business intelligence, (internet) connections, apps and web portals or the “Thing” itself. Testing in an IoT environment is new but on the other hand lets us reuse a lot from existing test environments.

From TMap to IoTMap

TMap is proven methodology to set up and execute tests. With building blocks the right techniques can be chosen to implement a test strategy. For IoT solutions we can define a set of IoT layers that make the IoT stack. Existing knowledge from TMap can be applied to each layer. This creates a set of test expertise areas for each part of the IoT stack. Here lies the basis that makes IoT testing different.

Less functional testing more IoT experience

Testing an IoT solution is all about combining test expertise. In each IoT layer functionality and specific quality attributes is tested. For example testing confidentiality in relation to data storage or interoperability when looking at connections between different “Things”.

At full IoT system level, it is more about looking at the IoT experience than functionality. Functionality should work and can be covered within each IoT layer. Testing in IoT environment shows a shift from functional testing to testing the IoT experience (with focus on quality attributes).

The difference with traditional testing

Testing in an IoT environment differs from testing as we know it. IoT distinguishes from regular testing with:

  • Combining a broad set of test expertise
  • Testing a physical “Thing” within the IoT chain
  • Shift focus from functional testing to IoT experience testing

Five steps to IoT testing

A detailed description to go from the IoT stack model to an IoT test strategy can be found in my book ‘IoT Map: Testing in an IoT environment’. In five clear steps the book guides you to set up testing with the correct quality attributes at the right moment and gives you new IoT building blocks with examples. IoT test environments and IoT test automation is covered in order to give you everything you need to test your IoT solution.

IoTMap book launch 21st of April

Also curious on testing in an IoT environment? After the 21st of April the book (an ePub) can be ordered. On the 21st of April a book launch event is organized in the Netherlands where you can learn more on IoT testing and receive the book for free. More information (in Dutch) and how to apply for the event, can be found here.

Tom Van de Ven AUTHOR:
Tom van de Ven has been active in the field of High Tech testing for over 12 years. As a High Tech test expert he is a frequently asked sparring partner for Sogeti High Tech customers with regard to starting/professionalizing test projects. Besides a multitude of test assignments (eg. in the field of healthcare, semiconductors, agriculture and automotive) he is an active member of the Sogeti High Tech Test Competence Centre and a speaker for High Tech seminars. Tom uses his experience in a role as a coach for (beginner) High Tech test engineers and is constantly looking for improvements in High Tech test methodologies. He completed his new book that combines Internet of Things and TMap: IoTMap in April 2016. He also teaches and develops several testing courses in the embedded and high tech domain. If not teaching, testing a tunnel or promoting “Quick Tech Testing” you can find him setting up a high tech test automation framework for the odd customer.

Posted in: Business Intelligence, Digital strategy, functional testing, Innovation, Internet of Things, IT strategy, Quality Assurance, Research, SogetiLabs, Test environment, test framework, Testing and innovation, Transformation, User Experience      
Comments: 0
Tags: , , , , , , , , , , , ,