High Performance Computing with Maple, Part 1

Many people who use Maple on Sheffield's High Performance Computing (HPC) cluster do so interactively. They connect to the system, start a graphical X-Windows session and use Maple in exactly the same way as they would use it on their laptop. Such usage does have some benefits: giving access to more CPU cores and memory than you'd get on even the most highly specified of laptops, for example.

Interactive usage on the HPC system also has problems. Thanks to network latency, using a Graphical User Interface over an X-Windows connection can be a painful experience. Additionally, long calculations can tie up your computer for hours or days and if anything happens to the network connection during that time, you risk losing it all!

If you spend a long time waiting for your Maple calculations to run, it's probably time to think about moving to batch processing.

Batch processing

The idea behind batch processing is that you log in to the system, send your computation to a queue and then log out and get on with your life. The HPC system will process your computation when resources become available and email you when it's done. You can then log back in, transfer the results to your laptop and continue your analysis.

So, batch processing frees up your personal computer but it can also significantly increase your throughput. With batch processing, you can submit hundreds of computations to the queue simultaneously. The system will automatically process as many of them as it can in parallel -- allowing you to make use of dozens of large computers at once.

Batch processing is powerful but it comes at a price and that price is complexity.

Converting interactive worksheets to Maple language files

You are probably used to interacting with Maple via richly formatted worksheets and documents. These have the file extension .mw or .maple. Unfortunately, it is not possible to run Maple worksheets in batch mode so it is necessary for us to convert them to maple language files instead.

A Maple Language File has the extension .mpl and is a pure text file. To convert a worksheet to a Maple Language File, open the worksheet and click on File->Export As->Maple Input

Convert to mpl

An example

Here is an example .maple worksheet and the corresponding .mpl Maple Language File, created using the conversion process detailed above. We also have a Job submission script that will be explained later.

If you look at the .mpl file in a text editor, you will see that it contains plain text versions of all the Maple input commands that were present in the original worksheet.

myseries := series(sin(x), x = 0, 10);
poly := convert(myseries, polynom);
plot(poly, x = -2*Pi .. 2*Pi, y = -3 .. 3);

This is the file that we can run on the HPC system in batch mode.

The job submission script is a set of instructions to the HPC system's scheduler. It tells the system how much memory you want to use, what program you want to run and so on. Here is its contents

!/bin/bash
# Request 4 gigabytes of real memory (mem)
# and 4 gigabytes of virtual memory (mem)
#$ -l mem=4G -l rmem=4G

#Make Maple 2015 available
module load apps/binapps/maple/2015

#Run Maple with the input file, **series_example.mpl**
maple < series_example.mpl

To run the example on the system:

  • Transfer series_example.mpl and run_maple_job.sh to a directory on the HPC system. They both need to be in the same directory.
  • Log in to the system using a command line terminal and cd to the directory containing the files.
  • Use the ls command to confirm you really are in the right directory. You should see something like this:
[ab1test@testnode02 maple_example]$ ls

run_maple_job.sh  series_example.maple  series_example.mpl
  • Submit the job to the queue with the qsub command
qsub run_maple_job.sh
  • You should see something like
[ab1test@testnode02 maple_example]$ qsub run_maple_job.sh

Your job 1734126 ("run_maple_job.sh") has been submitted

The job number will differ from the one above. It is automatically allocated by the system and uniquely identifies the job.

  • At this point, you could log off the system and do something else if you wished but, since this is such a short job, it won't be long before the results appear. A few seconds to a minute under normal conditions.

  • Run the ls command again to see the results files.

[te1st@testnode02 maple_example]$ ls

run_maple_job.sh  run_maple_job.sh.e1734126  run_maple_job.sh.o1734126  series_example.maple  series_example.mpl

There are two new files:

  • run_maple_job.sh.e1734126 - contains any error messages. Hopefully empty here
  • run_maple_job.sh.o1734126 - Contains the results of your job

The numbers at the end refer to the job number.

This completes your first batch submission using Maple.

Issues with graphs in Maple batch mode

The Maple worksheet we used in this example includes a plot command. This looks great in a Maple worksheet but looks very retro when performed in batch mode!

> plot(poly, x = -2*Pi .. 2*Pi, y = -3 .. 3);

                                      3+                                 H     
                                       +                                 H     
                                       +                                HH     
                                      2+                                H      
                                       +                                H      
                                       +                               HH      
                                       +                               H       
                                      1+    HHHHHHHHHH                 H       
           HHHHHHHH                    +  HHH         HHH             H        
          HH      HHH                  +HHH             HH           HH        
  --+-+-+-*+-+--+-+-**-+-+--+-+-+--+-+-**-+-+--+-+-+--+-+-**-+-+--+-+*-+-+-+--
   -6    H     -4     HH   -2        H0*           2       HHH 4    HH     6   
         H             HHH         HHH +                     HHHHHHHH          
        H                 HHHHHHHHHH -1+                                       
        H                              +                                       
       HH                              +                                       
       H                               +                                       
       H                             -2+                                       
       H                               +                                       
      H                                +                                       
      H                              -3+                                       

You probably want to have something that looks a little nicer. The way to do this is to modify the Maple plot command so that it specifies an output file. For example, if we want to create a .gif file, our Maple Language File becomes

myseries := series(sin(x), x = 0, 10);
poly := convert(myseries, polynom);

plotsetup(gif,plotoutput="plot.gif"):
plot(poly, x = -2*Pi .. 2*Pi, y = -3 .. 3);

The plotsetup command can output a number of file types. See Maple's plotsetup documentation for details.

A full batch example for you to try is available below

The results of transferring these files to the system and submitting with qsub run_maple_job_2.sh should include a file called plot.gif that looks like this

Maple plot

Future articles

In future articles, we'll be looking at how to make use of Maple's parallel computing constructs long with more advanced scheduling tricks that allow us to run 100s of jobs simultaneously.

Further reading