Important Announcement for NIH Biowulf Users (Biowulf)

This message is for Biowulf users who specify memory sizes
(for instance, "m2048" or "m4096") in their 'qsub' or 'swarm' 
commands.

Effective immediately, additional PBS batch system properties 
have been defined to allow the specification of memory sizes
for nodes allocated to a job.  The rationale for this change 
is described in the last part of this e-mail.  

For the next 30 days, both old- and new-style properties will 
be accepted by the batch system, but after July 15 specifying 
the old properties will result in an error.


Two types of properties will be available to specify the memory 
sizes of nodes:

m: the amount of memory per processor core
g: the total amount of memory in a node

[ Note: "mem=" should continue to be used to specify memory
sizes on the shared large-memory node(s) ]


Specifying memory per core
--------------------------

Use the "m" property where  is the size of memory in 
gigabytes.  Valid properties are "m1", "m2", and "m4".  Most
nodes in the cluster are "m2" (ie, 2 GB/core). 

You should use this property for most jobs requiring a specific
memory size.  It is particularly useful for submitting 
swarms, since 'swarm' may place your jobs on nodes with 
differing numbers of cores.  This property ensures that each 
process will have access to the same amount of memory regardless
of the number of cores on the node.

Specifying memory per node
--------------------------

Use the "g" property where  is the total memory in 
the node in gigabytes.  Valid properties are "g4", "g8", "g24" 
and "g72".

This property should be used when your job requires the entire
memory of the node for a single process. 


To check availability of nodes with a particular memory size,
use the "freen" command. 

Beginning JUNE 23, 2010 jobs submitted with the old memory
properties will result in a warning that the property has been
deprecated and the job will be automatically modified by 
the batch system to use the new property (for instance "m4096"
will be translated to "m2").  Starting JULY 16, 2010, using
the old properties (m1024, m2048, m4096, m8192) will result in 
an error and the job will not be accepted by the batch system.

--------
Rationale: when memory properties were first created for the
Biowulf cluster in 1999 and for several years afterwards, there 
was always a 1:1 relationship between "processor sockets" and
"processing cores".  Nodes with dual-socket motherboards always
had 2 cores, and the memory property always referred to the 
total node memory.  However, beginning in 2006 changes in 
processor technology resulted in the installation of nodes with
dual-core chips, and for consistency we maintained the memory 
properties to refer to the memory per 2-cores (e.g., dual-core 
nodes with 8 GB had an m4096 property).  Since then quad-core 
nodes have been installed, and the future will bring 8, 12 and 
16 core chips.  It is no longer intuitive to have the 'm' 
property refer to the memory per 2-cores, and therefore it is 
time to redefine the memory properties to reflect multi-core 
technology.