UMBC High Performance Computing Facility

How to run MATLAB programs on tara

Introduction

Running MATLAB on HPC's cluster nodes is similar to running any other serial job. Make sure you've read the tutorial for C programs first, to understand the basics. We will not demonstrate any parallel code here, so reading just the serial section is okay for now. A basic introduction to running MATLAB in the computer labs at UMBC is available on the CIRC webpage here.

For more information about the software, see the MATLAB website.

Performing Calculations on the Cluster Nodes

Let's try to run this sample MATLAB program, given below

% Generate two 100x100 matrices with random contents:
A=rand(100);
B=rand(100);

% Multiply the two matrices:
AB=A*B;

% Calculate the sum of the contents:
sumAB=sum(AB(:));

% Save the AB and sumAB variables to the Matlab save file out.mat:
save out.mat AB sumAB;

Download: ../code/matrixmultiply-matlab/matrixmultiply.m

As always, we will need a batch script too

#!/bin/bash
#SBATCH --job-name=matrixmultiply
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop

matlab -nodisplay -r "matrixmultiply, exit"

Download: ../code/matrixmultiply-matlab/run.slurm

Note that by using the serial queue, we've requested a single core of one node for our job. This will help to yield the best throughput of MATLAB jobs on the cluster. See the technical report HPCF-2009-1 (Sharma & Gobbert) on the publications page for more details. We can run our batch script in the usual way

[araim1@tara-fe1 matrixmultiply-matlab]$ sbatch run.slurm
sbatch: Submitted batch job 2621
[araim1@tara-fe1 matrixmultiply-matlab]$

After your job completes, you should see an out.mat MATLAB save file in your directory. Later on, if you want to get the data out of that file, you can use the the load command in MATLAB:

>> load out.mat

which will load in the AB and sumAB variables that you saved using your save command. Also in your directory, there should also be slurm.out and slurm.err files. The slurm.err file should be empty and the slurm.out file should contain something like this

[araim1@tara-fe1 matrixmultiply-matlab]$ cat slurm.out
                            < M A T L A B (R) >
                  Copyright 1984-2008 The MathWorks, Inc.
                         Version 7.6.0.324 (R2008a)
                             February 10, 2008


  To get started, type one of these: helpwin, helpdesk, or demo.
  For product information, visit www.mathworks.com.

[araim1@tara-fe1 matrixmultiply-matlab]$

You should be able to use any of the usual non-graphical MATLAB functionality if you follow the directions in this section. If you want to generate graphics in your MATLAB jobs, continue to the next section.

Generating Plots on the Cluster Nodes

As with all cluster jobs, you will need a batch script in order to run MATLAB

#!/bin/bash
#SBATCH --job-name=plotsine
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop

matlab -nodisplay -r "plotsine, exit"

Download: ../code/plotsine-matlab/run.slurm

Now you'll need the plotsine.m file that the script tries to run

zero_to_2pi=linspace(0,2*pi,1000);
them_sine=sin(zero_to_2pi);

plot(zero_to_2pi,them_sine);
print -dpng sine.png
print -deps sine.eps
print -djpeg sine.jpeg

Download: ../code/plotsine-matlab/plotsine.m

Now submit the batch script and wait for it to finish. After it finishes, you should see the following files

[araim1@tara-fe1 plotsine-matlab]$ ls
run.slurm      plotsine.m  sine.eps        sine.jpeg     
sine.png       slurm.err   slurm.out
[araim1@tara-fe1 plotsine-matlab]$

The sine.eps, sine.jpeg and sine.png files contain a plot of sin(x) from x=0..2*pi. The files are encapsulated postscript (.eps), joint photographic experts group (.jpeg) and portable network graphics files (.png), respectively. The slurm.err file should be empty and the slurm.out file should contain the same text as in the previous section. The three images you made should look something like this

PNG sine plot

The encapsulated postscript file (sine.eps) will be in greyscale since I used -deps instead of -depsc. Here are links to the three output files if you want to download them

Checking memory in Matlab programs

On the How to check memory usage page, we discuss various ways of monitoring memory usage, including logging it directly from your C code. In Matlab, there doesn't seem to be a built-in way to do this (not in the Linux version at least). But with a small amount of work, we can add the capability ourselves.

First grab the following C files, which are also used in How to check memory usage

The following C code is written in a specific form which Matlab can interface to. It will call the get_memory_usage_kb function defined in the C files above. The code retrieves the VmRSS and VmSize quantities for the current process (see How to check memory usage for more information), and returns them as a pair to Matlab.

#include <sys/types.h>
#include <unistd.h>
#include "mex.h"
#include "memory.h"

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    int* data;
    long vmrss;
    long vmsize;

    if (nrhs > 0)
    {
        mexErrMsgTxt("Too many input arguments.");
    }

    get_memory_usage_kb(&vmrss, &vmsize);

    plhs[0] = mxCreateNumericMatrix(1, 1, mxUINT32_CLASS, mxREAL);
    data = mxGetData(plhs[0]);
    data[0] = vmrss;

    plhs[1] = mxCreateNumericMatrix(1, 1, mxUINT32_CLASS, mxREAL);
    data = mxGetData(plhs[1]);
    data[0] = vmsize;
}

Download: ../code/check_memory-matlab/getmemusage.c

To compile this code, we need to use the Matlab MEX compiler. This is already installed on the cluster. We can use it as follows to compile our code. If the compilation succeeds, the file getmemusage.mexa64 is created.

[araim1@tara-fe1 check_memory-matlab]$ mex getmemusage.c memory.c
[araim1@tara-fe1 check_memory-matlab]$ ls
getmemusage.c  getmemusage.mexa64  memory.c  memory.h
[araim1@tara-fe1 check_memory-matlab]$

Now we can start up Matlab and call our new getmemusage function just like any usual function

[araim1@tara-fe1 check_memory-matlab]$ matlab -nodisplay

                                                   < M A T L A B (R) >
                                         Copyright 1984-2009 The MathWorks, Inc.
                                       Version 7.9.0.529 (R2009b) 64-bit (glnxa64)
                                                     August 12, 2009

 
  To get started, type one of these: helpwin, helpdesk, or demo.
  For product information, visit www.mathworks.com.
 
>> [vmrss, vmsize] = getmemusage

vmrss =

      101140


vmsize =

      933132
>> A = rand(5000, 5000);
>> [vmrss, vmsize] = getmemusage 

vmrss =

      298572


vmsize =

     1128448

>>

Note that this approach has a few limitations. It can only keep track of memory used in the current process. Matlab may invoke external processes for some tasks, whose memory usage will not be counted by this method.

Parallel Programming

Access to the Parallel Computing Toolbox is now available on tara. This allows simple multicore programming, however it is limited to single node jobs. There are several programming constructs available in the Parallel Computing Toolbox:

Parallel for provides a simple way to parallelize "for" loops.
spmd provides a "single process multiple data" programming model, which can be thought of as a simplified MPI
Support for distributed data structures

We will provide a simple example below. Detailed documentation on the use of the Parallel Computing Toolbox is available from MathWorks. Consider the following program, which is a multicore Hello World.

poolobj = parpool(8);
spmd
    msg = sprintf('Hello world from process %d of %d', labindex, numlabs);
end

for i=1:poolobj.NumWorkers
    disp(msg{i});
end
delete(poolobj);

Download: ../code/matlab-pct-hello/driver.m

The code starts up a matlabpool with 8 workers on the local machine. Within the "spmd" block, the string "msg" is built on each process. Notice the special variables "labindex" (ID for parallel worker) and numlabs (number of parallel workers).

After the "spmd" block, the "msg" data is available as a data structure that can be manipulated in serial. Here it consists of eight strings; we loop through and print each one. Notice here that the number of workers can be accessed by "matlabpool('size')".

#!/bin/bash
#SBATCH --job-name=matlab-pct
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop
#SBATCH --ntasks-per-node=8

matlab -nodisplay -r "driver, exit"

Download: ../code/matlab-pct-hello/run.slurm

A run of the code is shown below.

[araim1@tara-fe1 matlab-pct-hello]$ sbatch run.slurm
[araim1@tara-fe1 matlab-pct-hello]$ cat slurm.err
[araim1@tara-fe1 matlab-pct-hello]$ cat slurm.out

                            < M A T L A B (R) >
                  Copyright 1984-2012 The MathWorks, Inc.
                    R2012b (8.0.0.783) 64-bit (glnxa64)
                              August 22, 2012
 
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
 
Starting matlabpool using the 'local' profile ... connected to 8 workers.
Hello world from process 1 of 8
Hello world from process 2 of 8
Hello world from process 3 of 8
Hello world from process 4 of 8
Hello world from process 5 of 8
Hello world from process 6 of 8
Hello world from process 7 of 8
Hello world from process 8 of 8
Sending a stop signal to all the workers ... stopped.

[araim1@tara-fe1 matlab-pct-hello]$

Next is a simple example of "parallel for". We can replace simple loops with parallel loops with minimal programming effort.

poolobj = parpool(8);
x = zeros(1, 40);
parfor i = 1:40
  x(i) = i;
end
delete(poolobj);

x

Download: ../code/matlab-pct-parfor/driver.m

#!/bin/bash
#SBATCH --job-name=matlab-parallel-toolkit
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop
#SBATCH --ntasks-per-node=8

matlab -nodisplay -r "driver, exit"

Download: ../code/matlab-pct-parfor/run.slurm

[araim1@tara-fe1 matlab-pct-parfor]$ sbatch run.slurm
[araim1@tara-fe1 matlab-pct-parfor]$ cat slurm.err
[araim1@tara-fe1 matlab-pct-parfor]$ cat slurm.out

                            < M A T L A B (R) >
                  Copyright 1984-2012 The MathWorks, Inc.
                    R2012b (8.0.0.783) 64-bit (glnxa64)
                              August 22, 2012
 
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
 
Starting matlabpool using the 'local' profile ... connected to 4 workers.
Sending a stop signal to all the workers ... stopped.

x =

  Columns 1 through 13

     1     2     3     4     5     6     7     8     9    10    11    12    13

  Columns 14 through 26

    14    15    16    17    18    19    20    21    22    23    24    25    26

  Columns 27 through 39

    27    28    29    30    31    32    33    34    35    36    37    38    39

  Column 40

    40

[araim1@tara-fe1 matlab-pct-parfor]$