LEXIS brings HPC technologies closer to SMEs, industry and society

A new platform can help industry, society and especially small and medium-sized enterprises (SMEs) to easily access and use efficient heterogeneous distributed infrastructures without the need for advanced computing knowledge. To lower the entry barriers to the worlds of supercomputing, cloud, and big data, an EU-funded project is testing its platform in Aeronautics, Earthquake & Tsunami, and Weather & Climate domains.

The increasing quantity of data poses an enormous challenge for organisations seeking to extract knowledge critical for business, operations and research. Combinations of high-performance computing (HPC), Cloud and Big Data technologies are key to meet the needs of large and small organisations alike. However, this has been difficult since the supercomputing centres served merely the academic research.

“The LEXIS (Large-scale EXecution for Industry & Society) project is building an advanced engineering platform at the confluence of HPC, Cloud and Big Data, which uses large-scale geographically-distributed resources from the existing HPC infrastructure, employs Big Data analytics solutions and augments them with Cloud services,” explains the project coordinator Dr. Jan Martinovic from IT4Innovations. The platform is driven by the requirements of three initial pilot use cases.

The aeronautics pilot mainly focuses on accelerating turbomachinery and rotating-parts simulation workflows by direct code acceleration. In addition, advanced hydrodynamics simulation capabilities are now used to predict fluid flow around complex geometries under turbulent motion in the Rotating Parts use-case.

Earthquake and Tsunami simulations and warnings are in the focus of the second pilot. The main achievement was a clear formalisation of the complex workflows, which runs under time-constrained conditions. The computational efficiency of the tsunami simulation code was improved and therefore delivering results of a potential tsunami wave inundation in less than a minute enables faster and more accurate emergency warnings.

Last but not least, the Weather and Climate large-scale pilot deals with advanced workflows to predict e.g. flash floods, agricultural yields, forest fires and air quality, all based on numerical weather forecasts. Prototypes of the workflows are being implemented and tested on the LEXIS infrastructure which paves the way for the subsequent assimilation of multiple observational weather data in order to make all our predictions more reliable.

Based on the LEXIS pilots, several needs shared by SMEs and industry were identified within a number of different fields:

• dynamic data-aware orchestration of complex workflows;
• data sharing between Cloud and HPC resources and distributed data management with appropriate data backend to the orchestration solutions;
• access to HPC/BigData/Cloud resources for SMEs and industry and
• easy control of workflows and data through a user-friendly web interface.

To deliver the best user experience, typical user scenarios were collected together with the pilot partners giving invaluable feedback. Based on these, a LEXIS Portal as a one-stop-shop for usage of the platform is being designed.

“LEXIS has opted for the security-by-design approach where security is one of the pillars of the architecture as crucial as compute and data management. A federated, fault-tolerant and modern authentication and authorization framework has been implemented. It provides LEXIS users single-sign-on and role-based authorizations across the entire data and service providers community,” points out Marc Levrier from Atos company.

The solutions provided by LEXIS will enable improved cooperation between industry and academia. The new platform will ensure that SMEs and industries are able to use the appropriate resources for their applications in a user-friendly manner.

“To improve the quality of the LEXIS platform, an Open Call is planned, in which partners from academia and business will be asked to test their applications on the LEXIS platform and provide feedback. This should increase the impact of the project and establish a presence in various communities of potential users,” adds Dr. Martinovic.