Many people who use Maple on Sheffield’s High Performance Computing (HPC) cluster do so interactively. They connect to the system, start a graphical X-Windows session and use Maple in exactly the same way as they would use it on their laptop. Such usage does have some benefits: giving access to more CPU cores and memory than you’d get on even the most highly specified of laptops, for example.
Interactive usage on the HPC system also has problems. Thanks to network latency, using a Graphical User Interface over an X-Windows connection can be a painful experience. Additionally, long calculations can tie up your computer for hours or days and if anything happens to the network connection during that time, you risk losing it all!
If you spend a long time waiting for your Maple calculations to run, it’s probably time to think about moving to batch processing.
The idea behind batch processing is that you log in to the system, send your computation to a queue and then log out and get on with your life. The HPC system will process your computation when resources become available and email you when it’s done. You can then log back in, transfer the results to your laptop and continue your analysis.
So, batch processing frees up your personal computer but it can also significantly increase your throughput. With batch processing, you can submit hundreds of computations to the queue simultaneously. The system will automatically process as many of them as it can in parallel – allowing you to make use of dozens of large computers at once.
Batch processing is powerful but it comes at a price and that price is complexity.
## Converting interactive worksheets to Maple language files
You are probably used to interacting with Maple via richly formatted worksheets and documents. These have the file extension .mw or .maple. Unfortunately, it is not possible to run Maple worksheets in batch mode so it is necessary for us to convert them to maple language files instead.
A Maple Language File has the extension .mpl and is a pure text file. To convert a worksheet to a Maple Language File, open the worksheet and click on File->Export As->Maple Input
Here is an example .maple worksheet and the corresponding .mpl Maple Language File, created using the conversion process detailed above. We also have a Job submission script that will be explained later.
If you look at the .mpl file in a text editor, you will see that it contains plain text versions of all the Maple input commands that were present in the original worksheet.
myseries := series(sin(x), x = 0, 10);
poly := convert(myseries, polynom);
plot(poly, x = -2*Pi .. 2*Pi, y = -3 .. 3);
This is the file that we can run on the HPC system in batch mode.
The job submission script is a set of instructions to the HPC system’s scheduler. It tells the system how much memory you want to use, what program you want to run and so on. Here is its contents
!/bin/bash
# Request 4 gigabytes of real memory (mem)
# and 4 gigabytes of virtual memory (mem)
#$ -l mem=4G -l rmem=4G
#Make Maple 2015 available
module load apps/binapps/maple/2015
#Run Maple with the input file, **series_example.mpl**
maple < series_example.mpl
To run the example on the system:
run_maple_job.sh series_example.maple series_example.mpl
* Submit the job to the queue with the **qsub** command
qsub run_maple_job.sh
* You should see something like
[ab1test@testnode02 maple_example]$ qsub run_maple_job.sh
Your job 1734126 (“run_maple_job.sh”) has been submitted
The job number will differ from the one above. It is automatically allocated by the system and uniquely identifies the job.
* At this point, you could log off the system and do something else if you wished but, since this is such a short job, it won't be long before the results appear. A few seconds to a minute under normal conditions.
* Run the **ls** command again to see the results files.
[te1st@testnode02 maple_example]$ ls
run_maple_job.sh run_maple_job.sh.e1734126 run_maple_job.sh.o1734126 series_example.maple series_example.mpl
There are two new files:
* run_maple_job.sh.e1734126 - contains any error messages. Hopefully empty here
* run_maple_job.sh.o1734126 - Contains the results of your job
The numbers at the end refer to the job number.
This completes your first batch submission using Maple.
## Issues with graphs in Maple batch mode
The Maple worksheet we used in this example includes a plot command. This looks great in a Maple worksheet but looks very retro when performed in batch mode!
plot(poly, x = -2Pi .. 2Pi, y = -3 .. 3);
3+ H
+ H
+ HH
2+ H
+ H
+ HH
+ H
1+ HHHHHHHHHH H
HHHHHHHH + HHH HHH H
HH HHH +HHH HH HH --+-+-+-*+-+--+-+-**-+-+--+-+-+--+-+-**-+-+--+-+-+--+-+-**-+-+--+-+*-+-+-+-- -6 H -4 HH -2 H0* 2 HHH 4 HH 6
H HHH HHH + HHHHHHHH
H HHHHHHHHHH -1+
H +
HH +
H +
H -2+
H +
H +
H -3+
You probably want to have something that looks a little nicer. The way to do this is to modify the Maple plot command so that it specifies an output file. For example, if we want to create a .gif file, our Maple Language File becomes
myseries := series(sin(x), x = 0, 10); poly := convert(myseries, polynom);
plotsetup(gif,plotoutput=”plot.gif”): plot(poly, x = -2Pi .. 2Pi, y = -3 .. 3); ```
The plotsetup command can output a number of file types. See Maple’s plotsetup documentation for details.
A full batch example for you to try is available below
The results of transferring these files to the system and submitting with qsub run_maple_job_2.sh should include a file called plot.gif that looks like this
In future articles, we’ll be looking at how to make use of Maple’s parallel computing constructs long with more advanced scheduling tricks that allow us to run 100s of jobs simultaneously.
For queries relating to collaborating with the RSE team on projects: rse@sheffield.ac.uk
Information and access to JADE II and Bede.
Join our mailing list so as to be notified when we advertise talks and workshops by subscribing to this Google Group.
Queries regarding free research computing support/guidance should be raised via our Code clinic or directed to the University IT helpdesk.