Resetting Drained Slurm Node

After barking up the wrong tree for a while I discovered via StackExchange how to reset a Slurm Node reporting as drained.

Run the scontrol command then update the node as shown in the following example commands (amend the word jacob here to the name of your node):

[root@machine]# scontrol
scontrol: update NodeName=jacob State=DOWN Reason="undraining"
scontrol: update NodeName=jacob State=RESUME
scontrol: show node jacob
NodeName=jacob Arch=x86_64 CoresPerSocket=12
   CPUAlloc=0 CPUErr=0 CPUTot=96 CPULoad=0.01 Features=Xeon,IVB
   NodeAddr=jacob NodeHostName=jacob Version=15.08
   OS=Linux RealMemory=257668 AllocMem=0 FreeMem=254573 Sockets=4 Boards=1
   State=IDLE ThreadsPerCore=2 TmpDisk=49971 Weight=1 Owner=N/A
   BootTime=2015-12-19T10:13:10 SlurmdStartTime=2015-12-26T21:52:49
   CurrentWatts=0 LowestJoules=0 ConsumedJoules=0
   ExtSensorsJoules=n/s ExtSensorsWatts=0 ExtSensorsTemp=n/s

scontrol: quit


Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>