Managed File Transfer and File-based Integration – What’s the Difference and Why it Matters

Welcome to the Think Tank with Frank Kenney

With decades of analyst and integration industry experience, Frank Kenney is a fountain of knowledge on all things tech. Now, he aims to share that awareness with you. Come back every other Thursday for your biweekly dose of thought leadership in this blog from one of technology’s most insightful thinkers and gain perspective on a variety of topics ranging from what’s happening in integration today to what’s on the horizon, poised to disrupt the integration space going forward.

***

When you’re an analyst, one of the most important things that you have to remember is to be respectful of the work that’s come before you. Truth be told, most of the established concepts, including terms and category definitions are not your darlings to kill. This isn’t a complaint. Instead, as an analyst, you author new perspectives built upon a collective foundation of analyst acumen that proceeded your arrival, and that encompasses years of research notes, presentations, and inquiries. So, regardless of whether you agree or disagree with a given concept, the reality of being an analyst means it’s essential to maintain consistency throughout the ongoing narrative.

The story of how Managed File Transfer got a name

Fifteen years ago, in the summer of 2004, I faced an uphill battle. I was trying to change the narrative. I was younger then, and brash. And so inevitably, I was dutifully reminded that I was being too cavalier about a particular technology stack; one I was happily calling “secure file transfer.”  Some of the smartest security analysts in the world told me the term I had crafted was too close to the widely used and well established SFTP protocol. They pointed to the likelihood that my further talking about it within the context of application integration was confusing to our clients, and potentially to the market overall. Eventually, my boss at that time directed me to come up with another name, and I eventually settled on Managed File Transfer (MFT). Since then the topic of MFT integration has only became more relevant.

All too often, we make the mistake of forgetting that life is many-shaded, full of nuance and subtlety; that the space between two points – the journey as it were – matters as much as where you started or the destination striving to reach; that there is a world of grays between what’s black and that which is white. I believe I made that mistake as well. For in my ambition (did I mention I was young and brash?) to differentiate this particular technology platform that I so very much believed in, and in my efforts to bring into focus a multimillion-dollar market that I could see was so apparently critical to information technology, I purposely left off one important word off its name for the sake of avoiding confusion: Integration.

What is File-based Integration?

At its most uncomplicated form, integration is about enabling two or more applications that were designed independently to work together. And so, one of the purest forms of integration is file-based integration. Here’s a high-level snapshot:

File-based Integration Process

1. One application exports the data it wants to share and leaves it in a folder on a server somewhere.
2. The other application gets this data and imports it. Now, the two applications are in “sync.”

In reality, a number the nuances come into play between points A and B. For even the dangerously simplistic flow described above to work, a number of priority questions need answering before effective integration is possible.

What if the applications use two different data formats?

Who is responsible for the transformation of the file?

How does the second application know when the file is ready for pickup and import?

Who is monitoring all of this to ensure that it is happening in a timely fashion?

How often am I doing this synchronization?

The few questions laid out only begin to tie together the necessary business context with exigent digital components. And frankly, the list goes on, and on, and on from here.

File-based integration should, therefore, be thought of as integration using files. It’s that simple because, that’s really what’s going on under the covers, and that’s where the focus should be.

All of the required negotiation, mediation, security, management, and automation fall squarely under the auspices of integration. These activities are consistent whether we are talking message-based integration, like what we would do with a message queuing environment or API-based integration what many of the cloud integration platforms are offering today.

File-based integration as part of HIP

Consistent with the current views of technology firms such as Gartner and Forrester, companies are slowly recognizing the need to architect and deploy hybrid integration platforms which allow them to solve a variety of integration problems using a variety of integration patterns. One of those key integration patterns is, of course, file-based integration.

What’s in a name?

I think it’s important to understand that file-based integration isn’t a new concept. Instead, it’s a new set of words that are being used within a new context to finally allow this specific pattern of integration to stand tall amongst some of the other more merchandized styles of integration. For that I’m pleased.

If I were able to go back, setting aside youthful exuberance and new-job anxieties, maybe I would rechristen MFT to something like “Governed File Based Integration” of GFBI for short. But there’s no sense in lamenting in the past, what’s done is done.  I can safely move to beyond the aforementioned naming mistake that neglected the concept of integration because MFT has come full circle.

We should recognize “Managed File Transfer” in a name and as a technology that remains essential to enabling file-based integration processes between applications and in multi-enterprise B2B patterns as part of a larger hybrid integration platform strategy.

So, in the words of the late great radio host, Paul Harvey, “and now you know the rest of the story…”

PS It was always about integration!

***

Complete MFT Implementation Guide