Deleting many (more than 10k) jobs from an OpenPBS queue

Today I notices that the showq command of PBS may not list all jobs, you can configure the number of jobs it shows (it was set to 2000 on our system).

That’s fine, but it’s a bit funny that running qdel all as a user to delete all your jobs in the queue does NOT delete all your jobs: it only deletes the ones that showq shows.

I had more than 55,000 jobs in the queue, and deleting them 2000 at a time is too annoying.

Here is what I did, in the bash shell:
for JOBID in $(qstat | grep '^[0-9]' | awk -F . '{print $1}' | tr "\n" " "); do qdel $JOBID; done

This extracts the job id from each line of the qstat output, and runs qdel <jobid>> for each id.

The format of the qstat output is the following:

Job id Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
1486.odysseus somejobname you 351:02: R long
1487.odysseus somejobname you 355:05: R long
...

The above shell command ignores all lines which do not start with a number (the first 2 header lines) using grep, than extracts the process ID (the part before the first dot) using awk, then replaces the newlines by spaces using tr.

Versions used:
showq 3.2.6p20
qstat 4.2.7

Advertisements

About dfspspirit

PhD student in bioinformatics, interested in photography, level design, digital image manipulation, architecture and, of course, bioinformatics.
This entry was posted in IT and computers, supercomputing, troubleshooting and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s