SWARM

SWARM (SWift Adaptive Runtime Machine)

  • Simplify: Makes physical parallelism and complexity invisible to the user
  • Future-Proof: Delivers through a single programming interface:
    - Portability
    across heterogeneous systems
    - Scalability
    of applications from desktop to supercomputer
    - Stability
    of code base across system generations
  • Accelerate: Boosts performance through increased parallel efficiency and scalability 
  • Burst: Enables on-demand cloud resource allocation 
  • Stabilize: Future versions will provide resiliency through micro-checkpointing and graceful degradation
  • Conserve: Pushing towards energy efficiency
SWARM Execution Overview

What is SWARM?

ETI SWARM (SWift Adaptive Runtime Machine) is a full function parallel computing framework to deliver superior performance through advances in both efficiency and scalability. ETI SWARM exploits innovative methods of dynamic adaptive resource management and task scheduling to make best use of runtime information for guided control and allocation of heterogeneous resources achieving superior utilization of all system components. Programmability is enhanced by offloading many user decisions to the runtime system software. Performance portability is realized through adaptive techniques to fit the problem demands to available machine resources including heterogeneity.

ETI SWARM operates on a multitude of platforms, including x86 multi-core desktops, servers, and clusters, GPGPUs, and Intel’s new Xeon Phi.  The goal of ETI SWARM is to enable developers to quickly build quality solutions to compute-intensive (HPC) and data-intensive (Big Data) problems on distributed, heterogeneous computing systems.

 

Runtime Comparison

The ETI SWARM parallel computing framework provides a transformative solution for peta-scale systems rooted in deep dataflow technology for total system efficiency and resiliency.

  • Lightweight, thread-virtual machine technology is rooted in dataflow and asynchronous, fine-grain multithreading and synchronization
  • Exposure of implicit parallelism hides latency and allows users to exploit full processing power of multi-core systems
  • Dynamic scheduling capabilities and resource management expose extreme concurrency by mapping tasks to resources in real time based on the satisfaction of specific dependencies and constraints
  • Execution model based on codelets enables migration of data structures, work,  and global control
  • Multi-chip, multi-core, shared memory multiprocessing eliminates OS message-passing penalties and hardware-coherent caching dependencies
  • Application awareness allows monitoring of hardware availability and dynamic application requirements

 

Unification of Programming Models

Scalecc and SCALE

Today, users of ETI SWARM can leverage heterogeneous systems by coding to the SWARM API in C, or by using SCALE (SWARM Codelet Association LanguagE), an object-oriented extension to C that simplifies use of basic runtime constructs.  ETI SWARM supports multi-core distributed heterogeneous systems with x86, GPUs (via OpenCL and CUDA), and Xeon Phi.

Who can benefit from ETI SWARM now?

  • ISVs can leverage ETI SWARM to parallelize and scale their software for desktops, clusters, and clouds.
  • Scripting language ISVs, such as Matlab or Mathematica, can develop high-performance routines to enable their users to exploit parallelism on today’s workstations and clusters.
  • Industry developers can accelerate and scale up cluster-/cloud-based in-house programs, even on commodity clusters.
  • ISVs with new task-based algorithms that require dependency analysis, extremely low-latency scheduling, and dynamic resource allocation can leverage ETI SWARM as the only commercial parallel computing framework that can support these needs.

ETI is currently adding many features to ETI SWARM such as asymmetric schedulers, global address space support, garbage collection, and fault tolerance.  GPU support is being extended to be more user-friendly. Networking is being extended to support Infiniband directly, without using network sockets. ETI SWARM is the enabling technology for ETI's high performance Big Data analytics processing platform, SWARM-MR, to be released in Q2 2013.

In addition, ETI is creating a high-level language to take full advantage of ETI SWARM by simplifying the expression of common parallel design patterns and distributed data layouts.  This approach will enable the rapid development of highly parallel programs and libraries that can take advantage not only of today’s multi-core, homogeneous platforms, but also the many-core, heterogeneous platforms of tomorrow.

Click here to see SWARM Case Studies

SWARM Documents

To learn more about SWARM, check out the SWARM Documentation, which includes Programmer's Guide to the SWARM API, SWARM API Reference, and Example Programs. Also check out the SCALE Documentation, SWARM Release Notes, and SCALE Release Notes.

Please note that the documents posted here are from the latest release of SWARM. To access documentation from earlier releases, please Register or Sign-In to download SWARM Beta.

For more information on SWARM, see:

pdficon.png SWARM Whitepaper

pdficon.png SWARM for DOE

pdficon.png SWARM for Application Customers


Download SWARM Beta

Please REGISTER or Sign-in to download the SWARM beta and scalecc binaries.

pdficon.png White Paper

SWARM:Scalable Performance Optimization for Multi-core/Multi-node 

downloadbutton.gif

 

clcikicon.png SWARM v0.14

Register or Sign-in to download software

registerbutton.png

 

clcikicon.png Contact ETI

Want to learn more?  We can help.

contactbutton.gif

 

clcikicon.png Site Search

Find what you need on our site, quickly & easily

searchbutton.png