The fatanode cluster consists of:
fatanode-01 through fatanode-18. Each node
has dual Intel Xeon E5-2697A v4 CPUs with thermal scaling disabled, 512GBytes RAM,
and (currently) runs Ubuntu 20.04. MPI, OpenMP, C++ native threads and pthreads are available.fatanode-head, which can be used for job coordination etc.fatanode-data, whose disks can be accessed through any of the other nodes.The fataepyc cluster consists of:
fataepyc-01 through fataepyc-18. Each node
has dual AMD EPYC 7643 CPUs with thermal scaling disabled, 2TBytes RAM,
and (currently) runs Ubuntu 22.04. MPI, OpenMP, C++ native threads and pthreads are available.fataepyc-head, which can be used for job coordination etc.fataepyc-data, whose disks can be accessed through any of the other nodes.Access to the cluster is managed cooperatively through a
shared Google calendar. To gain access to this calendar, talk to Ciaran McCreesh, Blair Archibald,
or William Pettersson. You will need to provide an email address that has an associated Google account. Alternatively,
for occasional one-off bookings, we can make them on your behalf. You can book nodes by creating a calendar entry
for the time you want, stating your name and which nodes you are using in the title of the entry. Please use an entry
like node01-10 for the fatanodes, and epyc01-02 for the fataepycs.
Please book only the time you need, rather than speculative block bookings in case you have a last minute panic. If you do have a last minute panic and there are people using the cluster, emailing and asking nicely will usually persuade them to interrupt what they're doing for you. With that in mind, you are welcome to make long bookings for lots or even all of the nodes, if that is what you need.
If you already have a department Unix account, it should work on the cluster nodes too. If you don't, or it doesn't work, email support-socs requesting access. To access the machines, you will need to either VPN in to the School, or connect via the ssh gateway. You cannot access the machines directly from outside. It is probably not currently possible to give access to non-Glasgow collaborators.
Your usual Unix home directory is available. If you need more space, you can make yourself a directory in
either /scratch or /cluster. The /cluster directory is shared between
all the nodes using NFS (and is different between the two clusters). If you're doing anything I/O intensive, such
as doing a big git checkout or taring up your results directory, you can ssh onto fatanode-data
or fataepyc-data and things
will run faster on there. We don't book the data nodes or the head nodes in the calendar. Be aware that
neither /scratch nor /cluster is backed up, and you should only
use them for temporary files.
If you need software installed that you can't run from your home directory, you can ask support-socs to install it. Please CC in Ciaran so we can keep track of what we have and why.
These machines are optimised for parallel compute loads, and so single core runtimes are probably slower than a recent desktop machine. Using the fatanodes is probably only a good idea if you're using software that parallelises well, or if you can run many jobs simultaneously. Training on how to do this is available from Ciaran or Blair. If you can't scale past four cores, you may be better using a spare postgrad PC for experiments.
For Java users: we have heard reports that you should probably turn off parallel GC, and limit each VM's RAM to something like 32GBytes, or the JVM will behave very badly. (If you investigate this systematically, please email Ciaran with details so this document can be updated.)
It is helpful if we know what you're running on the cluster, so we can tell everyone how widely used and useful it is for supporting research and justify buying more hardware. If you can, please email Ciaran with a one paragraph description of what you've used the cluster for and any related outputs.