What is High Throughput Computing?
High throughput computing (HTC) is where many independent computations are run over many machines. IT Services for Research support HTC using Condor, by using Condor to turn unproductive CPU cycles on idle machines into productive CPU cycles that further the research aims of researchers on campus.
Is High Throughput Computing right for my research?
HTC is different to HPC in its approach. Due to the nature of HPC, jobs run simultaneously over many processors, usually requiring communication between processors.
Typically in HTC there is no communication between processes, and jobs run as available CPU cores become free. HTC is particularly suitable for 'parameter' sweep and Monte-Carlo style studies where you might want to run many simulations, all with slightly different conditions, and for 'pleasingly' parallel problems. รง have experience of significantly reducing the run time of calculations by converting code for use with Condor.
What High Throughput Computing facilities are available?
Currently ITS has two Condor pools. Condor is used to provide dynamic backfill to the HPC services running on the NW-GRID cluster and on the cluster in the School of MACE. There are also significant faculty-based resources available to EPS researchers.
We can work with you to make good use of these resources and assist in deploying your own.
Example use cases
Computational primatology: You have a code that analyses the gait of various creatures. To do this requires many attempts at different patterns of walking until a successful step is made. Farming this work out to an HTC system allows the use of many more machines than would be possible using non-HTC techniques.
Computational Chemistry: You are working to develop new types of sensor arrays for analysing air quality. To interpret the responses of your chosen material to contaminants requires the use of a genetic algorithm. For a code based around this to find the optimal solution it needs to do many runs to learn. HTC is perfect in this environment.
How do I apply?
Please contact us to discuss what help IT Services for Research can give (e.g. in converting your code for HTC use) and to obtain access to our resources. We can also help you deploy HTC systems like Condor on you own hardware, to make use of resources on currently unproductive machines (e.g. teaching clusters and spare lab machines).