Barclays had successfully deployed a data lake to transition away from an enterprise data warehouse. The challenge they faced with enterprise adoption was to get the data in the hands of the users as quickly as possible, while preventing data sprawl and maintaining the integrity and consistency of the system. Partnering with Dell EMC, a solution was developed called the Elastic Data Platform that was based on 5 key principles:
Using technologies provided by Dell EMC, Barclays has deployed this Elastic Data Platform to allow Data Scientists to provision their own environments and scale them up or down – adding/removing compute nodes – using BlueData EPIC to deploy Cloudera and Spark on Docker containers. Additionally, the compute layer of the environment was decoupled from the storage layer and deployed in a tiered architecture using Dell EMC Isilon.
This presentation will discuss the Barclays journey for adoption, the architecture for the solution, and describe the underlying technologies that enable it.