Moving Big Data: Securely, Reliably, and Efficiently

Data is exploding around us. According to Harvard Business Review, we are creating 2.5 exabytes (billion gigabytes) of data every day. Every hour of that day, Walmart handles more than 1 million customer transactions and generates more than 2.5 petabytes (2560 terabytes) of data — the equivalent of 167 times the information contained in all the books in the US Library of Congress. Facebook stores 300 million uploaded photo images each day. And the volume of business data worldwide doubles every 1.2 years.

This data explosion has created a new term in the IT industry called Big Data.

Big Data presents a huge opportunity for businesses to use data more effectively to improve their market understanding, competitiveness, and operational efficiency. Due to the complexity in managing and moving larger quantities of data, this can be a disruptive change across many industries; however businesses that become experts at using Big Data will thrive in the new era.

Gartner, the world’s leading information technology research and advisory company, defines Big Data as follows, “Big data are high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.” The volume, velocity, and variety of data in the Big Data era fundamentally require new approaches and supporting technologies. In addition, privacy and usability of big data are two important requirements that present significant challenges.