A Beginners Guide to Interoperability Testing [2023 Update]

Introduction

Nowadays, you’d be hard pressed to find a complex multifunctional software project that completely consists of custom-made components. As a rule, projects are built from a number of pre-made synchronised solutions. This is done because:

• it’s cheaper;
• it’s more reliable (since each component has already been tested);
• it’s faster.

Such obvious advantages have greatly streamlined the development process. However, the trend for using pre-made software has also made interoperability testing a must. And that’s what this article is all about.

What is interoperability testing?

Interoperability testing is a kind of software testing that checks whether the software is able to interact with other programming components and systems. 

Side note: here, the word “components” doesn’t refer to elements of a single software; otherwise, we’d be using the term “integration testing”.

In this case, interoperability is the indicator of interaction between two or more programs. This indicator is expressed through the task acceptance criteria, as said tasks are being completed. The factual result – that is, the interaction between programs – is supposed to satisfy the requirements set beforehand.

We should also note that multisystem interoperability is extremely common nowadays. More and more projects of this sort appear in classic web and mobile development, as well as in IoT.

What are the goals of interoperability testing?

Before we list the aims of interoperability testing, let’s take a website that deals with medical tourism as an example. Its features include a database of free booking slots in clinics and sanatoriums, the ability to book airline tickets, and a number of payment systems. But even if a user can log into the site and book a bed in one of the clinics, that doesn’t mean that the site’s integrated third-party services function correctly. This poses a number of questions for a developer:

• Does the booking process work correctly on both your side and the side of the third-party service that the clinic uses?
• Is the information on the number of available booking slots up to date?
• Is there a quality way for medical workers to receive all the necessary documentation?
• Is the website secure?
• Is the interaction stable?
• etc.

With those questions in mind, the goals of interoperability testing can be described as follows:

• to provide a high-quality uninterrupted connection between two different software systems;
• to facilitate data security and safe data transfer between systems;
• to ensure a uniform data type and data format among all integrated software systems.

What are the pros and cons of interoperability testing?

We’ve asked Bamboo Agile’s QA Engineer Darya Tochilenko to share her thoughts about this type of testing testing:

So, such testing offers undeniable advantages. However, there are also a number of negatives you need to consider. Let’s see how the two sides stack up.

Advantages:

• It aids in the construction of a link between two systems. This connection helps you gain a better understanding of how the system will function in conjunction with other items. It also promotes improved communication.
• It improves efficiency. Decision-making can be simplified when data is presented consistently.
• Data types should be consistent and compatible between all systems to avoid connectivity issues. This testing ensures that the data types conveyed are uniform and that there are no data type mismatches. 
• It ensures consistent data formatting, so that there is no conflict due to incompatible file extensions.
• It ensures that all interconnected systems use the same semantics and methods.

Disadvantages:

• It necessitates precise measurements for all systems to function properly in an end-to-end environment. 
• This testing increases network complexity because all components must be tested.
• The lack of standardisation and documentation regarding data exchange, communication interfaces and protocols.

What are the types of interoperability testing?

This article discusses interoperability testing as a whole. But in practice, interoperability testing is split into five types. Let’s look at each of them separately to avoid any future confusion. 

1. Interoperability testing by data type

This means that all data has to remain the same when exiting one system and entering another. The main task is to avoid any contradictions: if the data sent out is a type A symbol, then the data received on the other end has to be a type A symbol as well.

2. Semantic interaction testing

This type of testing checks the reliability of the data transfer algorithm.

3. Physical interoperability testing

Physical interoperability testing checks connection quality between two systems. This type is particularly popular in IoT, where connection ports for various data transfers play a big role.

4. Protocol interaction testing

This one regards the protocol used for data transfer between two integrated software systems. What it does is ensure that the protocol provides effective data protection.

5. Data format interoperability testing

Do not confuse this kind with data type testing from point one. The data format has to remain the same on both ends of the data transfer. For example, all data sent in binary code has to remain binary on the receiving end.

What are the challenges of interoperability testing?

The medical tourism example from the beginning of the article points to the fact that generic applications, where users follow the same user flow despite the app employing multiple different systems, are pretty easy to test. Since the user flow remains more or less the same, it’s easier to find holes in the process.

But in some fields, the user flow is prone to constant changes. This directly affects the integrated systems, because the entry and exit points can vary from case to case. Since we’re already talking about medicine, let’s modify the example a little bit: now it regards not just the process of booking a slot, but the software that hospitals use – simply put, Healthcare IT Testing for Interoperability.  

In healthcare systems, every new client (regardless of their health condition) is considered a separate case. For example, one hospital can get all these at the same time:

• a burn victim in critical condition, treatment uncertain;
• an appendicitis patient in critical condition, surgery necessary;
• a patient with a strong headache and possible concussion, needs immediate examination to exclude the possibility of cerebral hemorrhage;
• a patient with a broken finger, needs to be queued for a cast.

State Transition Diagrams and Cause-and-Effect Diagrams are perfect for testing this system. They let you see the whole picture, as well as split the system into its functional elements and their relationships. This can be done regardless of where and how a separate system is activated.

The main thing that the tester needs to understand is that there’s a clear hierarchy here, one that places everything exactly where it belongs. Systems like that usually treat every hospital unit as a separate system of its own. Each unit (app) has its sub-units (modules), and each module has its sections.

Of course, testing these will cause a lot of problems:

• The main one is covering all possible variants. It’s almost unfeasible to cover all attainable combinations of system interactions.
• One has to account for the fact that the base software’s performance is affected by the performance of outer systems (that’s that IoT) – i. e. a patient’s condition is inferred from their MRI results, and said patient is transferred either a level lower or higher depending on the degree of supervision needed.
• Found defects analysis can require a lot of time and specialized knowledge. Note that consultations with industry professionals are absolutely necessary when testing highly specialized systems.

These hurdles may seem tough to overcome, but there are plenty of effective methods for optimizing the testing process. Let’s see what we can do to make our interoperability testing as accurate as possible.

How does one overcome these challenges?

Us, testers, have a sequence that streamlines testing medical systems and interoperability testing for applications of different complexity levels. 

1. If a QA engineer gets a ready-made working system, they have to go the way of Dr House and construct a differential diagnosis for it. What problems did it have? How were those problems replicated? How were they solved?
2. Plan your testing. If it employs the abovementioned diagrams, use the problem spots from point 1 and state transition points to weave the web of test cases.
3. You’ll need to organize a test environment. But there’s a nuance: while web and mobile applications are pretty easy to deal with, IoT can cause you some trouble (after all, a hospital is unlikely to let you take their MRI for a spin). To make up for the lack of the necessary devices, you’ll have to use your imagination and special emulators. Alternatively, you can ask the programmers to write a simple device replacement program suitable for testing. Remember that the results produced by these replacements must be valid for the devices they replicate.
4. Make sure that the systems integrated into the one you’re testing have been tested, too.
5. Start simple. As you progress towards testing the more complex aspects of the system, analyze the results of each testing stage, of every error, every atypical situation.

While this sequence of actions uses healthcare software systems as an example, it works wonders for any type of application. It’s less about the particular industry and more about a thorough and consistent approach to the process of interoperability testing.

How to perform interoperability testing?

This type of testing can be done by following the Deeming wheel, or the PDCA cycle. It stands for Plan, Do, Check, and Act. Let’s see what each of these stages is about.

Plan

You can’t launch testing without a strategy. To create a good testing plan, you must understand how each and every app and system works and connects to the other components. Another thing you should do before starting testing is full functional and non-functional testing. You can read more about the former here. All that naturally needs to be documented and accounted for in the general plan.

You can actually just tweak your regular Test Plan for interoperability. Create your test conditions – note that they shouldn’t be restricted to specific apps, but have to be based on the data flow between them. Set them in a way that covers as many applications in the network, optimally all of them. Developers typically use an RTM (Requirements Traceability Matrix) to map out the test cases and test conditions in a neat and organised way. 

Do

“Do” is the amount of time you spend carrying out your plan. Plan your time wisely to complete both functional and non-functional testing within reasonable deadlines. During this phase, you execute the test cases, document defects, follow up with the development team to have them rectified, re-test and regression test the system as a whole, submit the test findings, and move it to close.

Check

“Check” is the phase in which you examine your test results and attempt to map them with the RTM. This is when you validate whether all of the expected requirements have been met and all applications have been traversed. You ensure that data is correctly and smoothly navigated and shared across all applications and systems. Also make sure that the data is not altered during travel.

Once that is done, consider conducting a retrospective of the full testing process. Identify the areas that went well, those that did not, and any action items that need to be addressed.

Act

“Act” is aimed to act on retrospective items. The points recognised as “good practices” should be continued to be implemented, while the areas that could be improved should be highlighted. You need to devise which actions should be taken to correct them, and implement said actions. 

See which steps brought more harm than good and try to exclude them from future processes if possible – if not, revise the way you go about them. After all, it’s all about learning from one’s mistakes.

Interoperability testing on mobile

If your mobile app is using third-party integrations, you also need to conduct an interoperability test. When planning for this type of testing on mobile devices, you must consider several factors:

• There are numerous types of devices on the market. You’ll need to make a list of all the models and OS versions your application is meant to run on and test them accordingly.
• All mobile operating systems are written in a distinct programming language. As a result, you should be ready to deal with the peculiarities of each system variant when testing.
• Understand the legal obligations and regional contracts regarding certain types of apps (Healthcare, Finance, Automotive).
• Account for different screen sizes and resolutions.
• The impact on integrated mobile apps on the system must also be considered. For example, a social media app can interact with the built-in camera app, so this interaction needs to be tested. Some apps may use a lot of memory or CPU. The other issue is that built-in applications can receive updates that can lead to compatibility issues.

So, just like you do for computer-based application testing, you would need to plan and develop an RTM for interoperability testing on mobiles. Here, the objective, strategy, risks, and execution will be the same, but the tools and approaches used will differ.

Interoperability testing example

Now let’s make an example of such type of testing for a healthcare company.

A large healthcare organisation is implementing a new electronic health record (EHR) system. The new EHR system is designed to be interoperable with the organisation’s existing EHR system, but there are concerns that the two systems might not be able to exchange data successfully.

So the company conducts testing between the two EHR systems. The testing involves a variety of tests, including data exchange tests, functionality tests, performance tests, and security tests. The testing is conducted in a realistic environment that simulates the real-world conditions in which the systems will be used.

As a result, the testing identifies several issues that need to be addressed before the new EHR system can be implemented. These issues include data format errors, authentication problems, and performance bottlenecks. The issues are then addressed by the software vendor of the new EHR system, and the two systems are made to be successfully interoperable.

So, how has this healthcare company benefited from testing? It helped to ensure that the new EHR system could exchange data successfully with the existing EHR solution. This allowed the organisation to achieve its goal of having a single, integrated view of patient data. This type of testing also helped to identify and address any potential security vulnerabilities in the two systems.

Conclusion

Nowadays, projects with multiple integrated systems are at every corner. They are becoming more and more common in the classical web and mobile development, and are widely applied in the internet of things. 

Interoperability testing has earned its place of honour in the Test Execution stage of the Software Testing Life Cycle. And, of course, interoperability functionality is considered its own scope of tasks within the Software Development Life Cycle.

Do you have a solution in need of interoperability testing or general QA? Bamboo Agile’s experienced QA team will be happy to lend a hand.

Sign up for a free consultation with our specialists by filling out this simple form

FAQ