Energy Corporation Pilots Object-based Storage for Archiving Unstructured Seismic Data

Challenge

A multi-national energy corporation wanted to evaluate and test an object-based storage solution against competing options. WWT and Dell EMC partnered to deploy EMC's Elastic Cloud Storage (ECS) Appliance, powered by ViPR, in a proof of concept (POC) in our Advanced Technology Center (ATC).

The solution needed to bring 100 petabytes of seismic data stored on tape online while meeting resiliency and availability requirements. The POC's scope involved the restoration and reuse of many years of seismic data that had been generated and archived to cold storage.The customer wanted to avoid the risks associated with traditional tape, such as formatting consistency and corruptions. They also wanted to gain the benefits of having data stored in an online repository, such as self-checking, real-time repairing and anytime access.

Because of the customer's storage environments, the POC had to simulate performance within geographically disparate data centers. Working against a compressed timeframe, the customer needed to finish evaluating and testing a solution within a month.

Solution

WWT and Dell EMC proposed the Elastic Cloud Storage (ECS) Appliance, powered by ViPR, as a complete hyperscale infrastructure that essentially functioned as a local private storage cloud. Benefits to the customer included the fact that ECS offers universal access to object and HDFS, as well as block protocols, on a single platform. The ECS Appliance uses ViPR, Dell EMC's software-defined platform that abstracts, pools and automates a data center's underlying physical storage infrastructure.

The ECS Appliance is built using commodity components, with commodity-based nodes managing a set of commodity disks, to ensure cost-effective storage. ViPR data fabric is installed on each node, which provides local element manager services and acts as a distributed cluster manager.

A notable aspect of the engagement was WWT's awareness of Dell EMC's ability to show an integrated NFS object service. With this knowledge, we partnered with Dell EMC to bring the customer on-site to a lab to demonstrate a native CIFS/NFS interface. We had expertise with the interface because we were one of the first Dell EMC partners to use native CIFS/NFS and object-based storage on the same platform to talk back to the object-based hardware engine (ECS Appliance).

Critical to the customer was the ability of ECS to deliver an integrated storage cloud that could support performing analysis against data in place, without having to move the data. Several other aspects appealed to the customer as well: primarily the ability to run analysis against archived data, plus the appliance's use of Scale IO to perform block storage along with object, NFS and CIFS at a compelling price point. Finally, the customer also considered ECS' ability to support Hadoop file systems, a factor that would prove critical in supporting downstream big data integration.

Implementation

WWT's strong understanding of the customer's business challenges allowed our high-performance team to execute the pilot in weeks instead of months. Our national technical architect specializing in object-based storage, Rich Harper, spent a week on-site with the customer as a subject matter expert supporting POC requirements. Rich helped develop documentation and participated in a 16-hour conference call focused on completing the buildout of the lab's management operations according to specific feature requirements.

Replicating the customer's environment for the pilot required a level of effort equal to setting up six data centers, with three sets of dual sites, to test geographically disparate data center capabilities. Our lab services team was brought in to assist with developing the networking and virtualization elements necessary to ensure required latency (15ms) between sites.

Outcome and benefits

The POC was successfully completed in just over three weeks. WWT and Dell EMC demonstrated a complete hyperscale infrastructure solution providing data storage for 3rd Platform applications, as well as universal access to object and HDFS and block protocols, on a single platform.

With only two weeks of ATC lead time, we successfully built the test environment and simulated a geographically disparate data center scenario using customer data that met their requirements for high availability and resiliency. We also demonstrated the solution's potential for employing a Hadoop file system, paving the way for use with a big data stack.

Highlights

ATC was used to accelerate project timeline for a complex, high-touch proof of concept.
WWT and Dell EMC innovated to solve a new storage-related use case generated by the customer's internal seismic data group.
Solution met requirements for resiliency and high availability of unstructured seismic data, as well as future integration within advanced architectural applications, including big data.
Pilot was completed in less than four weeks.

Conclusion

The market for object-based storage is quickly moving from bleeding edge to accepted among Fortune 1000 organizations. Organizations see the challenges of increasing data demands, especially of data that use traditional storage methods. Because of this, an investigation into new options such as object-based storage is underway.

The success of this ATC object-based storage pilot for a multi-national energy corporation was the result of a strong partnership between Dell EMC and WWT, a high-performance team effort during the POC, and the diverse capabilities of our ATC's lab environment.