Simple, efficient allocation of modelling runs on heterogeneous clusters with MPI

Environmental Modelling and Software
By:

Links

Abstract

In scientific modelling and computation, the choice of an appropriate method for allocating tasks for parallel processing depends on the computational setting and on the nature of the computation. The allocation of independent but similar computational tasks, such as modelling runs or Monte Carlo trials, among the nodes of a heterogeneous computational cluster is a special case that has not been specifically evaluated previously. A simulation study shows that a method of on-demand (that is, worker-initiated) pulling from a bag of tasks in this case leads to reliably short makespans for computational jobs despite heterogeneity both within and between cluster nodes. A simple reference implementation in the C programming language with the Message Passing Interface (MPI) is provided.

Additional publication details

Publication type Article
Publication Subtype Journal Article
Title Simple, efficient allocation of modelling runs on heterogeneous clusters with MPI
Series title Environmental Modelling and Software
DOI 10.1016/j.envsoft.2016.11.003
Volume 88
Year Published 2017
Language English
Publisher Elsevier Science Ltd.
Publisher location Oxford
Contributing office(s) Eastern Geographic Science Center
Description 10 p.
First page 48
Last page 57