Batch APIs

Welcome to this week's Think Tank with Frank Kenney

With decades of analyst and integration industry experience, Frank Kenney is a fountain of knowledge on all things tech. Now, he aims to share that awareness with you. Come back every other Thursday for your biweekly dose of thought leadership in this blog from one of technology's most insightful thinkers and gain perspective on a variety of topics ranging from what's happening in integration today to what's on the horizon, poised to disrupt the integration space going forward.

***

Batch isn't dead. It's just moved... to APIs.

A funny thing happened on the way...

Now, that's is a classic opener. A story-starter cliché right up there with, "It was a dark and stormy night," or "Once upon a time..." But, instead of the macabre or fantastical, it's usually a sign of something vaudevillian, tipping off a light-hearted tale that's comical, farcical, or coincidental.

So, without further ado... A funny thing happened on the way to the corner store this morning.

To get to the little bodega in my neighborhood, it takes exactly seven minutes door-to-door. That's each way, and yes, I'm counting the time it takes to find my keys and get on the road. The corner grocery is an oasis in the countryside suburbs of Tampa. And at least once every few weeks, I jump in the car and head there.

On this particular morning, I jump in the car, hit the keyless ignition, and give the volume knob a little clockwise twist. When the speakers kick in, over the airwaves comes "Old Town Road" by Lil Nas X. (The radio was tuned to whatever station was last playing, not that it's important, but I thought you should know.)

Anyway, I'm not exactly a fan of meme-as-music and Old Town Road is not really my cup of tea, so I quickly change the station. Lo and behold, the same song is playing on the next preset channel. Okay, so, time to try this again third time's a charm, as they say. And besides, I've almost hit the gas station already at the halfway point of my drive and I want to hear at least a couple minutes of a song I can get down to. So, I change the station again and again, it's "Old Town Road."

"Old Town Road"...The same song on three separate stations. How does that even happen? Resigned to my fate, I play it out until the fading banjitar and mournful whistle signify the end. Song's over and just before a commercial break, the DJ mentions one fact about the song, and for me, the penny dropped.

It makes perfect sense when you learn that this week marks 18 straight weeks at the top of the Billboard charts making it the longest number one run in the history of the charts. In fact, it beat the previous record last week. Kudos to that young man!

To give you a sense of how I'm wired, the first thing I thought was, "Ha! That's just like batch transfer! It's inescapable. You can't get away from it!" And now here we are. As you can imagine, I came home, immediately jumped behind my computer and started writing this post.

Batch. It's the gift that keeps on giving and giving and giving.

Last week, I spent time with Cleo's Dylan Lee and Arvind Venugopal learning about all the cool new features and functionality in the platform. They were telling me about differentiation in platform support, specifically about a way to elegantly handle batch APIs.  I corrected them with a reminder that we've been doing batch EDI for two decades. Why is this anything new? They responded how any great product marketing manager and product manager should respond mountains of research and evidence that companies increasingly moving towards API-based B2B integration are leveraging batch APIs to modernize their connections with their suppliers, providers, customers, and partners but also support their partner's requirements and best practices.

In many cases, companies are only connecting a few times a day to grab a thousand orders or a thousand invoices or a record of the thousand events. For all the real-time, transactional panaceas that we've been promised with APIs, companies are recognizing the need for API-based batch processing when it comes to working with their partners.

What I continue to say still holds, APIs don't replace the need for fully-governed data movement solutions that can work in real-time, just-in-time and batch. This is why we will continue to see so many companies experimenting with a new API-led environment that is sitting right alongside a modernized EDI environment. It's a coalescence of API and EDI that capable of multiple support requirements and (increasingly the modern, expectation-driven) styles of trading partner, supplier and customer interactions.

My question to my team was, "Does it necessitate a new type of functionality to work with these batch-style API-based processes?" Their answer was unequivocally, yes.

Moving to an architecture that facilitates grabbing 1000 transactions in a single request, correlating the data, and even transforming the transactional payload on the fly, as part of one unit of processing, or one step of a larger process is bound to keep the operations, networking and integration teams very happy.

More importantly it allows me to pull up another old anecdote that persistently follows us around:  Amazon is a very cool company that will enable its customers and partners to interface with it using real-time transactions for buying and selling, status lookups and track-and-trace. But at the end of the day, UPS only comes to my house once a day. Like Jell-O, there's always room for batch.

So in the realm of business integration, batch is a megahit. It's resided on the charts for decades, and will end up dominating a whole lot longer than "Old Town Road," "Despacito" and "One Sweet Day."  Maybe, it's time we celebrate its success.

***