December 16, 2014 Technical note Open Access

JOB PACKING: OPTIMIZED CONFIGURATION FOR JOB SCHEDULING

Dal Pra, Stefano

The default behaviour of a batch system is to dispatch jobs to nodes having the lower
value of some load index. Whilst this causes jobs to be equally distributed among all the
nodes in the farm, there are cases when different types of behaviour may be desirable, such as
having a completely full node before dispatching jobs to another one, or having similar jobs
dispatched to nodes already running jobs of the same kind. This work defines the packing
concept, different packing policies and useful metrics to evaluate how good the policy is. A
simple farm simulator has been written to evaluate the expected impact on a farm of different
packing policy. The simulator is run against a sequence of real jobs, whose parameters have
been taken from the accounting database of INFN-Tier1. The effectiveness of two packing
policies of interest, namely relaxed and exclusive, are compared. The exclusive policy proves
to be better, at the cost of unused cores in the farm, whose number is estimated. The
possibility of implementing the exclusive policy on a specific batch system, LSF 7.06, is
exploited. Relevant configurations are shown and an overall description of the mechanism is
presented.