What is integration, how should it be done, and what can it achieve?
What is integration?
Software integration means the integration of separate applications and systems.
Integration may involve both on-premise and cloud-based apps. For example, integration allows a company’s ERP system to retrieve address information from the customer account management application or the enterprise product planning software to retrieve parts availability information from the inventory management system.
Integration aims to eliminate the need for manual data transfer. However, integration may not be necessary if the data input is infrequent and does not take long. On the other hand, if it is an error-prone and critical phase for the organization’s operations, it is worth automating the transfer of information.
Different types of integration
If only a few different systems exist, they can be connected directly as point-to-point integration between systems. As the number of systems increases, the number of direct connections increases exponentially. In addition to the actual transfer of data, integration must also take care of data aggregation, transformation, enrichment, and validation, as well as logging, monitoring, and error management. In point-to-point integration, the above steps must be done separately for each site whenever new integration needs arise.
When integration is centralized, everything is in one place and easy to manage. On the other hand, centralized integration is a server and resource-intensive solution that cannot be recommended as a small, single-integration solution. As the number of sites to be integrated increases, it is worth considering a centralized integration solution.
A more scalable integration approach is to use a centralized integration platform (Enterprise Service Bus, ESB). Each system communicates directly only with a common operator, which takes care of data transmission, the transformation of data formats, and the control of integrations. This model makes it easier to add new systems because the integrator only needs to build one connection between the new system and the integration platform instead of building a connection from the new system to each existing system. Data can also be easily retrieved from the different files or interfaces of multiple systems.
If your company already has an integration platform, it should be used if possible. Point-to-point integration and centralized integration can also be used in parallel if you can make this combination work. If there are problems or needs for improvement in the old point-to-point integrations, moving even the old integrations to a centralized integration platform is worth considering. In this way, the monitoring of the functioning of the disconnected integration can be transferred to centralized control.
Integration as microservices
Integrations can also be done using microservice architecture. Microservices are small, standalone applications that do not require a product to run under them. They can work separately as standalone units or together to form a larger functional whole. The use of microservices allows easier development and maintenance of the components of the whole and supports reusability.
A register of all available microservices must be kept, showing where the service can be found. The router, on the other hand, receives external calls, distributes jobs, and calls the microservice. Microservices report their status at regular intervals to support services, which see the status and load on servers, configure services and ensure the continuous integration of services.
Microservice architecture is best suited to situations where scalability is required, i.e., where the number of service requests varies widely. When using microservices, the load is automatically distributed among the available services. New service instances can be set up or disabled on the fly, either manually or automatically.
Limitations and possibilities of integrations
Synchronous and asynchronous integration
Integration can be synchronous or asynchronous. In synchronous integration, the source system sends information to the target system and waits for a response from the target system. In asynchronous integration, the source system does not wait for the target system to finish its work.
Synchronous integration is appropriate when it is assumed that the target system will perform its task quickly. But if the target operation takes longer, the source system cannot wait for the user. In that case, the integration must be asynchronous.
Integration Platform as a Service (iPaaS)
Integrations are increasingly moving to cloud-based iPaaS-services such as Oracle Integration Cloud Service, Boomi, MuleSoft, and Microsoft Azure Logic Apps. In the iPaaS model, the company does not have to worry about background servers or installing or updating the integration platform.
An alternative to integration: Robotic Process Automation – RPA
Integration becomes problematic when a system does not provide interfaces, the system vendor does not want to open them or it is expensive to create interfaces. In this case, the data transfer can be implemented using software robotics.
A software robot uses the same interfaces as a human would but does so in an automated and fast way. Software robotics can also be used for smaller, one-off, and ad hoc data transfer cases. Data transfer by integration is preferable to the use of software robotics because integration takes advantage of the services and interfaces provided by the systems and makes data processing faster.
Software robotics is in danger of becoming an isolated part of the development process, lacking proper version control. However, it is a matter of practice, so the development of a software robot must follow the same sound software development principles as integration.
The pitfalls of integration
Integration can stop working when systems are upgraded. People may also start to use systems differently from how they were intended or just differently from how they used to. Common stumbling blocks are dates, times, and time zones given in different formats. Special characters entered in fields can also cause problems.
A typical structural integration mistake is to make a copy of a service over and over again, even though one and the same service could be used in more than one place. Integration can also be made asynchronous when it would be better to make it synchronous (and vice versa). In asynchronous integration, the message order can also go wrong, so it is important for the integration platform to take care of the message order.
From integration to service
As a by-product of its business, a company may generate information that its customers could use. The customer can then be offered limited access to the company’s databases for a fee. It may not even require building an interface to provide the service, but the desired information can be integrated directly into the customer’s systems. All companies should therefore consider whether they could increase their business by offering some value-added information to their customers.
Factors for integration
Many system vendors integrate their products. They know their system well and can use that knowledge when implementing integration. However, integrations are not the core business of system vendors, and they do not inherently have superior knowledge of other systems into which their own product is integrated.
Companies specializing in integrations
Companies that do integrations alone are focused on doing integrations and have experience in various systems. However, integrations are usually only part of an organization’s business process development, which also involves interfering with many other digital processes.
Software companies with broader software development skills, in addition to integration, are good partners when integrations are aimed at improving old or creating new business processes. They are familiar with general good integration practices and know how to implement integrations in the best possible way, but they can also help with other software development needs.
Keys to the successful integration
The technical implementation of the integration is faster when the systems to be connected are familiar to the integrator. However, understanding the customer’s business and processes is the most significant factor in successful integration. The better the integrator understands what information the customer needs and wants to use it for, the better the outcome.
A versatile software company can also offer the organization much more to improve business efficiency.
Please, contact Juhani, who works in our sales.