Elsevier Science Home
Computer Physics Communications Program Library
Full text online from Science Direct
Programs in Physics & Physical Chemistry
CPC Home

[Licence| Download | New Version Template] afam_v1_0.tar.gz(3702 Kbytes)
Manuscript Title: HASEonGPU - An adaptive, load-balanced MPI/GPU-Code for calculating the amplified spontaneous emission in high power laser media
Authors: C. Eckert, E. Zenker, M. Bussmann, D. Albach
Program title: HASEonGPU
Catalogue identifier: AFAM_v1_0
Distribution format: tar.gz
Journal reference: Comput. Phys. Commun. 207(2016)362
Programming language: C++, Matlab.
Computer: GPU cluster or workstation with CUDA-capable GPUs (compute capability ≥ 2.0).
Operating system: Linux.
Has the code been vectorised or parallelized?: Yes, can utilise 1 CPU core per compatible GPU.
RAM: Several Gb, depending on input size and number of GPUs. 4000000000 bytes (4 GB) per GPU is recommended.
Keywords: Amplified Spontaneous Emission, CUDA, GPU Cluster, Massively Parallel, Monte Carlo Integration, High Power Laser.
Classification: 4.13, 6.5, 15.

External routines: CUDA, Boost Program Options, OpenMPI

Nature of problem:
The algorithm described by D. Albach in [1,2] uses ray-tracing techniques and Monte Carlo integration to calculate Amplified Spontaneous Emission (ASE) with high precision. It requires a high number of sampling points as well as a high number of rays to reach the desired results. Additionally, reflections on the upper and lower surface of the medium increase the workload by an order of magnitude. On traditional CPU-based systems the computation is time-consuming, which limits the number of simulations that can be performed.

Solution method:
HASEonGPU uses a non-uniform distribution of sampling points within the gain medium to focus computation on areas of interest. This is further improved by combining the Monte Carlo integration with importance sampling [3]. To improve execution time further, the algorithm is highly parallelized to run on a GPU and supports adaptive sampling resolutions and random restarts. It can also be executed in a GPU cluster, where linear scaling is achieved by a coarse-granular load balancing that distributes the workload among all GPUs in a master-worker-scheme over MPI.

Presently, the number of rays used for the Monte Carlo integration of a single sampling point within the gain medium is limited by the available memory on the GPU (about 108 rays per GB of GPU memory). Furthermore, when using MPI as a workload distribution mechanism, one of the MPI processes will act as a scheduling master and its GPU can not participate in the computation.

Unusual features:
The software can run on a workstation (threaded) as well as on a large-scale GPU cluster (MPI) that provides the required GPU hardware. The simulation parameters include polychromatic laser pulses as well as surface coatings, cladding, and refractive indices of the gain medium. This also allows the simulation of reflections on the upper and lower surface of the medium. If a desired mean square error metric is not met with a set number of rays, the algorithm can automatically increase the number of rays to improve the results.

Additional comments:
The source code also includes a MATLAB script that can be used to call HASEonGPU directly from MATLAB code to integrate it into existing simulation setups. There are also examples included on how to execute HASEonGPU from the command line as well as an example experiment that uses MATLAB and the provided script. More detailed information can be found in the README file.

Running time:
Depending on the number of sampling points, desired sampling resolution for each point, and number of GPUs, the execution time can vary strongly. A typical cylindrical gain medium of 6cm diameter simulated with 4210 non-uniformly distributed sampling points can be simulated with a sufficient precision in 1min on a single NVIDIA Tesla K20m GPU. Running time as well as precision can be further optimized through various parameters.

[1] D. Albach, J.-C. Chanteloup, G. l. Touz e, Influence of ASE on the gain distribution in large size, high gain Yb3+ : Y AG slabs, Opt. express 17 (5) (2009) 37923801.
[2] D. Albach, Amplified spontaneous emission and thermal management on a high average-power diode-pumped solid-state laser-the Lucia laser system, Ph.D. thesis, Palaiseau, Ecole polytechnique (2010).
[3] E. C. Anderson, Monte Carlo methods and importance sampling, 1999.