News & Announcements
Important Announcement for NIH Biowulf Users (Biowulf)Date: 15 June 2010 12:06:16
From: steven fellini (sfellini@NIH.GOV)
This message is for Biowulf users who specify memory sizes (for instance, "m2048" or "m4096") in their 'qsub' or 'swarm' commands. Effective immediately, additional PBS batch system properties have been defined to allow the specification of memory sizes for nodes allocated to a job. The rationale for this change is described in the last part of this e-mail. For the next 30 days, both old- and new-style properties will be accepted by the batch system, but after July 15 specifying the old properties will result in an error. Two types of properties will be available to specify the memory sizes of nodes: m: the amount of memory per processor core g: the total amount of memory in a node [ Note: "mem=" should continue to be used to specify memory sizes on the shared large-memory node(s) ] Specifying memory per core -------------------------- Use the "m
" property where is the size of memory in gigabytes. Valid properties are "m1", "m2", and "m4". Most nodes in the cluster are "m2" (ie, 2 GB/core). You should use this property for most jobs requiring a specific memory size. It is particularly useful for submitting swarms, since 'swarm' may place your jobs on nodes with differing numbers of cores. This property ensures that each process will have access to the same amount of memory regardless of the number of cores on the node. Specifying memory per node -------------------------- Use the "g " property where is the total memory in the node in gigabytes. Valid properties are "g4", "g8", "g24" and "g72". This property should be used when your job requires the entire memory of the node for a single process. To check availability of nodes with a particular memory size, use the "freen" command. Beginning JUNE 23, 2010 jobs submitted with the old memory properties will result in a warning that the property has been deprecated and the job will be automatically modified by the batch system to use the new property (for instance "m4096" will be translated to "m2"). Starting JULY 16, 2010, using the old properties (m1024, m2048, m4096, m8192) will result in an error and the job will not be accepted by the batch system. -------- Rationale: when memory properties were first created for the Biowulf cluster in 1999 and for several years afterwards, there was always a 1:1 relationship between "processor sockets" and "processing cores". Nodes with dual-socket motherboards always had 2 cores, and the memory property always referred to the total node memory. However, beginning in 2006 changes in processor technology resulted in the installation of nodes with dual-core chips, and for consistency we maintained the memory properties to refer to the memory per 2-cores (e.g., dual-core nodes with 8 GB had an m4096 property). Since then quad-core nodes have been installed, and the future will bring 8, 12 and 16 core chips. It is no longer intuitive to have the 'm' property refer to the memory per 2-cores, and therefore it is time to redefine the memory properties to reflect multi-core technology.