Data Management Layer

The LEXIS Data Management layer, built on top of the infrastructure layer, will connect the participating HPC and HPDA sites together. It provides an optimum basis for a consistent data-management process in the LEXIS framework, as required for Big Data analytics and HPDA.

As a core of the Distributed Data Infrastructure (DDI) to be set up, we deploy an iRODS- and EUDAT-based data-management system across LEXIS sites. On top of this system, where data can be accessed transparently from all sites, we build REST APIs in order for the LEXIS ecosystem (orchestrator, portal, billing engine) to smoothly interact with the data layer. Monitoring functionalities are implemented as well, and can be utilised for billing as well as for performance optimisation of the LEXIS data infrastructure.

The LEXIS data-management layer consists of the Data System Core (green rectangle – Distributed Data Infrastructure with APIs) and local storage systems.

A moderately-sized local storage system (LEXIS experimental storage, 150TB) installed at LRZ