Modeling High-throughput Applications for in situ Analytics - Systèmes Répartis, Calcul Parallèle et Réseaux Accéder directement au contenu
Article Dans Une Revue International Journal of High Performance Computing Applications Année : 2019

Modeling High-throughput Applications for in situ Analytics

Résumé

With the goal of performing exascale computing, the importance of I/Omanagement becomes more and more critical to maintain system performance.While the computing capacities of machines are getting higher, the I/O capa-bilities of systems do not increase as fast. We are able to generate more databut unable to manage them eciently due to variability of I/O performance.Limiting the requests to the Parallel File System (PFS) becomes necessary. Toaddress this issue, new strategies are being developed such as online in situanalysis. The idea is to overcome the limitations of basic post-mortem dataanalysis where the data have to be stored on PFS rst and processed later.There are several software solutions that allow users to specically dedicatenodes for analysis of data and distribute the computation tasks over dier-ent sets of nodes. Thus far, they rely on a manual resource partitioning andallocation by the user of tasks (simulations, analysis).In this work, we propose a memory-constraint modelization for in situ anal-ysis. We use this model to provide dierent scheduling policies to determineboth the number of resources that should be dedicated to analysis functions,and that schedule eciently these functions. We evaluate them and show theimportance of considering memory constraints in the model. Finally, we discussthe dierent challenges that have to be addressed in order to build automatictools for in situ analytics.
Fichier principal
Vignette du fichier
paper.pdf (525.55 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02091340 , version 1 (05-04-2019)

Identifiants

Citer

Guillaume Aupy, Brice Goglin, Valentin Honoré, Bruno Raffin. Modeling High-throughput Applications for in situ Analytics. International Journal of High Performance Computing Applications, 2019, 33 (6), pp.1185-1200. ⟨10.1177/1094342019847263⟩. ⟨hal-02091340⟩
254 Consultations
393 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More