Navigating the End-to-End Integration Journey
Table of Contents
When we use the term ‘end-to-end integration’ what exactly do we mean? There’s an answer, but it might not be the one you’re expecting.
Depending on your perspective or your day-to-day role, you probably have a different idea of what you want to see within an overall business or process flow.
One way to look at people within a company is in terms of personas. A persona is just a way of characterizing what someone in a given role should know about a given subject, or in this case, what/how much visibility do they need into your company’s integration systems.
And they’re helpful because some of the personas we will consider here are going to affect this idea of what we mean by end-to-end visibility and end-to-end integration.
Your different personas might range from general “line of business” personnel to the integration teams who are the ones building out and using the integration tools.
Perhaps it may include the operations team responsible for maintaining the business processes at the technical level, and the customer service folks who are responsible for talking to customers and figuring out what exactly needs to be done in order to support them.
In terms of visibility, you must look at each of those personas and determine what they might require from a visibility solution. Exactly what does each need to see in order to do their job?
The line of business persona will look at the business (or department or business unit) as a whole.
It’s about having visibility into the activity of the business -- not necessarily a particular integration flow, but more trending information, e.g., how are the orders varying today, compared with yesterday, or last week?
Compare that with the visibility needs of your customer service team.
They’re going to be looking at a set of customers they’re responding to, either trying to be proactive with some of those bigger customers, or also looking to be reactive as customers encounter different problems.
The customer service team is going to need end-to-end visibility over an individual customer’s transactions.
The integration team is going to want visibility into the individual integration flows underneath the business processes and how they’re connected.
They want to know what the right flow of information is, where it’s coming from and going to, whether it’s from a business partner’s system or from their own internal environment, and how the flow needs to be architected when it’s put together.
An operations persona is looking at a similar type of process, but also brings a different perspective to it.
Primarily, they want to make sure everything is running smoothly. But if it’s not, they want to be able to very quickly get to the source of where the problem is, and obviously then, determine how that problem should be resolved.
So, while the integration team is looking at determining how to design and build the integrations, and how they should be robust and deliver the business process they want, the operations team is more looking at the running of that business process, or how to operate and optimize it.
There’s one persona that we’ve left out so far, which is arguably more important than any we’ve discussed, and that’s the customers themselves.
If your platform is capable of it, you want to give your customers visibility into the integration flows as well.
So, if something is not going as they think it should, you can provide them with a window into their own transactions to see exactly what is going on with those interactions, because it might just be that you know something on their side is not working effectively, or it is something that they can resolve themselves by just looking at the information.
From all this, it’s clear that end-to-end visibility means different things for different people, justifying the need for different visibility requirements for different personas.
There’s the business process side, which cares about delivering a particular business result. There’s a physical process that potentially underlies that.
And then there’s an integration process that enables that business process to happen. But it’s not even as straightforward as all that.
Think about the end-to-end business processes your company relies on. For example, if you’re in manufacturing, maybe it’s an order-to-cash process that has traditionally relied on EDI, and in it today you’re experiencing a steady increase of eCommerce business flows. That suggests your process now incorporates non-EDI environments too, such as marketplaces like Amazon.
Let’s take a quick look at a plausible eCommerce integration process to see where exactly it fits in.
In this process, we have a variety of orders coming in.
Maybe it is a traditional EDI type of order with large quantities for dealing with retailers. Or it could be individual orders coming from an Amazon marketplace, or an eCommerce site, such as Shopify.
Depending on how you’re receiving the orders, it might be an API call to receive orders from a marketplace, or it could be an AS2 file-based interaction coming from a customer with that type of underlying format.
From there, we’ll do some type of data transformation, and then we’ll put that record into our back-end system.
And while there are surely more complex transactions than this going on, let’s just simplify this to acknowledge that the order comes from somewhere, the appropriate data transformation happens, and then it gets put into a back-end system.
That represents the integration process that forms part of the overall business process that’s supporting our end-to-end physical process.
Because when we talk about end-to-end integration visibility, we are really talking about the top-level process that is supporting those other two processes, and also, by implication, providing visibility into those processes, too.
When it comes to your end-to-end integration visibility, another key thing you need to consider is; What is the scope of the business process that you need to see?
Look at that business process view, so you know what the scope of the longer-running transaction is and how you can support that.
Think too about what your partners’ systems are capable of and how you can account for them. In other words, figure out how you can do as much as you can on your side to allow for what your partners’ limitations might end up being.
From there, look at what can you do to optimize visibility of any engine that you cannot control. Use the very best protocols that you can in order to see what is coming and going to and from your partners.
Then learn how you can maximize the information that is visible by deploying the right protocols and also making sure that you bubble up that information from those protocols.
When something goes wrong, you must know how easy it is to figure out what happened, so always think about that too when you’re designing the integrations.
Finally, back to those personas.
You must consider all the personas that need differing levels of visibility into your integration business flows and figure out what each one needs so you can design your integrations to enable that.