GPUs Basel 2018 - Revision history

Stefano Scaramuzza at 13:22, 27 August 2018

2018-08-27T13:22:30Z

Stefano Scaramuzza at 12:50, 27 August 2018

2018-08-27T12:50:54Z

Stefano Scaramuzza at 10:00, 27 August 2018

2018-08-27T10:00:27Z

Stefano Scaramuzza at 09:58, 27 August 2018

2018-08-27T09:58:41Z

Stefano Scaramuzza at 08:22, 27 August 2018

2018-08-27T08:22:02Z

Stefano Scaramuzza at 15:18, 20 August 2018

2018-08-20T15:18:20Z

Stefano Scaramuzza at 14:59, 20 August 2018

2018-08-20T14:59:56Z

Stefano Scaramuzza at 14:11, 20 August 2018

2018-08-20T14:11:22Z

Stefano Scaramuzza at 13:57, 20 August 2018

2018-08-20T13:57:18Z

Stefano Scaramuzza at 12:55, 20 August 2018

2018-08-20T12:55:29Z

@@ Line 38: / Line 38: @@
 * While logged in to your sciCORE account, activate dynamo:
   <tt>source /scicore/home/s-gpu-course/GROUP/dynamo_activate_linux_shipped_MCR.sh</tt>
 * Untar the Dynamo project:
-  <tt>dynamo dvuntar myProject.tar </tt>
+  <tt>dynamo </tt>
 * Create a blank SLURM submission script (text file) named ''submit_job.sh'':
@@ Line 65: / Line 70: @@
 echo "dvunfold myProject" >> dcommands.sh
 dynamo dcommands.sh
-chmod u=rxw ./myProject.m
+chmod u=rxw ./myProject.exe
-./myProject.m</nowiki>
+./myProject.exe</nowiki>

@@ Line 1: / Line 1: @@
 Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the same steps described here on your own project.
-The GPUs we use are located on the high performance computing cluster of the University of Basel called sciCORE (https://scicore.unibas.ch) which uses the SLURM queuing system. A queuing system coordinates the access to the GPUs and is needed when there are many users using a limited amount of GPUs.
+The GPUs we use are located on the high performance computing cluster of the University of Basel called sciCORE (https://scicore.unibas.ch) which uses the SLURM queuing system. A queuing system coordinates the access to the GPUs and is needed when there are many users using a limited amount of GPUs. You have been given the credentials needed to log in on the cluster at the beginning of the workshop.
 The main idea is that we create an alignment project locally, move it to the cluster on sciCORE and then run it using a pre-installed Dynamo standalone version on sciCORE. How you can do that is described in the following steps.
@@ Line 31: / Line 31: @@
 * Login to your sciCORE account:
   <tt>ssh -Y USERNAME@login.scicore.unibas.ch</tt>
-When asked to continue type "yes". Use the provided password.
+If asked to continue type "yes". Use the provided password.

@@ Line 77: / Line 77: @@
   <tt>squeue -u USERNAME</tt>
-See all users in queue for the K80 GPU:
+See all users in the queue:
   <tt>squeue -q 30min</tt>
 To cancel the job type ''scancel'' followed by the job ID that was shown by the squeue command:

@@ Line 24: / Line 24: @@
 * Open a local Linux terminal and navigate to the directory where you just created the tutorial dataset and project. Copy the project data (particles) to sciCORE:
-  <tt>rsync -avuP myParticles USERNAME@login.bc2.unibas.ch:/scicore/home/PATH/dynamo_projects</tt>
+  <tt>rsync -avuP myParticles USERNAME@login.scicore.unibas.ch:/scicore/home/s-gpu-course/USERNAME/dynamo_projects</tt>
 * Copy the previously created tar file of the project to sciCORE:
-  <tt>rsync -avuP myProject.tar USERNAME@login.bc2.unibas.ch:/scicore/home/PATH/dynamo_projects</tt>
+  <tt>rsync -avuP myProject.tar USERNAME@login.scicore.unibas.ch:/scicore/home/s-gpu-course/USERNAME/dynamo_projects</tt>
 * Login to your sciCORE account:
@@ Line 37: / Line 37: @@
 * While logged in to your sciCORE account, activate dynamo:
-  <tt>source PATH/dynamo_activate_linux_shipped_MCR.sh</tt>
+  <tt>source /scicore/home/s-gpu-course/GROUP/dynamo_activate_linux_shipped_MCR.sh</tt>
 * Untar the Dynamo project:
@@ Line 45: / Line 45: @@
   <tt>nano submit_job.sh</tt>
-* Copy (and adapt) the following lines into the newly created script. In any case you have to change the path following the command ''cd'' to the path of your project. Depending on your project you might also have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. Note the that the K80 and TitanX GPUs use different scripts. Also, if you are using the K80 and think that your job will run longer than 30 minutes, set the ''qos'' to ''6hours'' and adapt the time to anything between 30 minutes and 6 hours:
+* Copy (and adapt) the following lines into the newly created script. Depending on your project you might have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. If your job will run longer than 30 minutes, set the ''qos'' to ''6hours'' and adapt the time to anything between 30 minutes and 6 hours:
   <nowiki>#!/bin/bash -l
@@ Line 58: / Line 56: @@
 #SBATCH --ntasks-per-node=1
 #SBATCH --cpus-per-task=1
-#SBATCH --partition=k80
+#SBATCH --partition=pascal
 #SBATCH --gres=gpu:1
 module load CUDA/7.5.18
-source PATH/dynamo_activate_linux_shipped_MCR.sh
+source /scicore/home/s-gpu-course/GROUP/dynamo_activate_linux_shipped_MCR.sh
-cd PATH/dynamo_projects
+cd $HOME/dynamo_projects
 echo "dvput myProject -gpu_identifier_set $CUDA_VISIBLE_DEVICES" > dcommands.sh
 echo "dvunfold myProject" >> dcommands.sh
@@ Line 69: / Line 68: @@
 ./myProject.m</nowiki>
 * You can now run your alignment project by submitting the previously created script to SLURM with:

← Older revision		Revision as of 08:22, 27 August 2018
Line 31:		Line 31:
	* Login to your sciCORE account:		* Login to your sciCORE account:
	<tt>ssh -Y USERNAME@login.scicore.unibas.ch</tt>		<tt>ssh -Y USERNAME@login.scicore.unibas.ch</tt>
		+	When asked to continue type "yes". Use the provided password.

@@ Line 1: / Line 1: @@
-Here we describe on how to use the GPUs provided for the Basel Workshop 2018. We go through each step by using a simple tutorial dataset and project as an example. You can use the same steps described on your dataset/project.
+Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the same steps described here on your own project.
-The GPUs we use are located on the high performance computing cluster of the University of Basel called sciCORE (https://scicore.unibas.ch) which uses the SLURM queuing system. A queuing system coordinates the access to the GPUs and is needed when there are many users using just a few GPUs.
+The GPUs we use are located on the high performance computing cluster of the University of Basel called sciCORE (https://scicore.unibas.ch) which uses the SLURM queuing system. A queuing system coordinates the access to the GPUs and is needed when there are many users using a limited amount of GPUs.
-The main idea is that we create an alignment project locally, move it to the cluster on sciCORE and then run it using a pre-installed Dynamo standalone version on sciCORE. To do that, we follow the following steps:
+The main idea is that we create an alignment project locally, move it to the cluster on sciCORE and then run it using a pre-installed Dynamo standalone version on sciCORE. How you can do that is described in the following steps.
 '''On your local Matlab session with Dynamo loaded:'''
-* Create the tutorial project:
+* Create a tutorial project with Dynamo:
   <tt>dtutorial myParticles -p myProject -M 128</tt>
 We now have a tutorial dataset with 128 particles in the directory <code>myParticles</code> and a tutorial alignment project <code>myProject</code>.
@@ Line 16: / Line 16: @@
 and under ''computing environment'' select ''GPU (standalone)'' as an environment.
-* Check and Unfold the project.
+* Check and unfold the project.
-* Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then "create a tarball".
+* Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then ''create a tarball''.
 '''On your local Linux terminal:'''
-* Copy the project data (particles) to sciCORE:
+* Open a local Linux terminal and navigate to the directory where you just created the tutorial dataset and project. Copy the project data (particles) to sciCORE:
   <tt>rsync -avuP myParticles USERNAME@login.bc2.unibas.ch:/scicore/home/PATH/dynamo_projects</tt>
 * Copy the previously created tar file of the project to sciCORE:
-  <tt>rsync -avuP dTutorial.tar USERNAME@login.bc2.unibas.ch:/scicore/home/PATH/dynamo_projects</tt>
+  <tt>rsync -avuP myProject.tar USERNAME@login.bc2.unibas.ch:/scicore/home/PATH/dynamo_projects</tt>
 * Login to your sciCORE account:
@@ Line 35: / Line 35: @@
 '''While logged in to your sciCORE account:'''
-* Activate dynamo:
+* While logged in to your sciCORE account, activate dynamo:
   <tt>source PATH/dynamo_activate_linux_shipped_MCR.sh</tt>
@@ Line 42: / Line 42: @@
 * Create a blank SLURM submission script (text file) named ''submit_job.sh'':
-  <tt>nano  submit_job.sh</tt>
+  <tt>nano submit_job.sh</tt>
-* Copy and adapt the following lines into the newly created script (note the difference between K80 and TitanX GPUs):
+* Copy (and adapt) the following lines into the newly created script. In any case you have to change the path following the command ''cd'' to the path of your project. Depending on your project you might also have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. Note the that the K80 and TitanX GPUs use different scripts. Also, if you are using the K80 and think that your job will run longer than 30 minutes, set the ''qos'' to ''6hours'' and adapt the time to anything between 30 minutes and 6 hours:
 For using the K80 GPUs:
@@ Line 89: / Line 89: @@
 chmod u=rxw ./myProject.m
 ./myProject.m</nowiki>
 * You can now run your alignment project by submitting the previously created script to SLURM with:
@@ Line 115: / Line 113: @@
 To check the last average load the standalone Dynamo environment by typing <code>dynamo</code> into the terminal and use the usual commands, e.g.:
-  <tt>ddb dTutorial:a -v</tt>
+  <tt>ddb myProject:a -v</tt>

@@ Line 52: / Line 52: @@
 #SBATCH --job-name=dTest
 #SBATCH --qos=30min
-#SBATCH --time=00:60:00
+#SBATCH --time=00:30:00
 #SBATCH --mem=16G
 #SBATCH --nodes=1
@@ Line 74: / Line 74: @@
 #SBATCH --job-name=dTest
 #SBATCH --qos=emgpu
-#SBATCH --time=00:60:00
+#SBATCH --time=00:30:00
 #SBATCH --mem=16G
 #SBATCH --nodes=1
@@ Line 90: / Line 90: @@
 ./myProject.m</nowiki>
-* Note that depending on your project you might have to adapt the project name and the time requested (time=) in the script.
+* Note that depending on your project you might have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. If you are using the K80 and think that your job will run longer than 30 minutes, set the ''qos'' to ''6hours'' and adapt the time to anything between 30 minutes and 6 hours.
 * You can now run your alignment project by submitting the previously created script to SLURM with:

@@ Line 10: / Line 10: @@
 * Create the tutorial project:
   <tt>dtutorial myParticles -p myProject -M 128</tt>
-This creates a tutorial dataset with 128 particles in the directory <code>myParticles</code> and a tutorial alignment project <code>myProject</code>.
+We now have a tutorial dataset with 128 particles in the directory <code>myParticles</code> and a tutorial alignment project <code>myProject</code>.
 * Open the alignment project window:
@@ Line 18: / Line 18: @@
 * Check and Unfold the project.
-* Before moving the data to sciCORE we have to compress the project. In the dcp gui go to ''Tools'' and then "create a tarball"
+* Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then "create a tarball".
@@ Line 44: / Line 44: @@
   <tt>nano  submit_job.sh</tt>
-* Copy and adapt the following lines into the newly created script:
+* Copy and adapt the following lines into the newly created script (note the difference between K80 and TitanX GPUs):
 For using the K80 GPUs:
@@ Line 92: / Line 92: @@
 * Note that depending on your project you might have to adapt the project name and the time requested (time=) in the script.
-* You can now run your alignment project by submitting the previously created script to SLURM with:
+* With the following commands you can check the overall status of the submitted jobs:
-* To check your status in the queue type:
+Check your status in the queue:
-squeue -u USERNAME
+ <tt>squeue -u USERNAME</tt>
-To see all users in queue for the K80 GPU:
+See all users in queue for the K80 GPU:
-squeue -q 30min
+ <tt>squeue -q 30min</tt>
-To see all users in queue for the TitanX GPU:
+See all users in queue for the TitanX GPU:
-squeue -q empgu
+ <tt>squeue -q empgu</tt>
-To cancel the job type ''scancel'' and the job ID that was shown by the squeue command:
+To cancel the job type ''scancel'' followed by the job ID that was shown by the squeue command:
-scancel my_job_id
+ <tt>scancel my_job_id</tt>
 Some ways to check the last output:
-ls -rtl
+ <tt>ls -rtl</tt>
-tail -f slurm-45994509.out
+ <tt>tail -f slurm-45994509.out</tt>
-less slurm-45994509.out
+ <tt>less slurm-45994509.out</tt>
-To check the last average:
+To check the last average load the standalone Dynamo environment by typing <code>dynamo</code> into the terminal and use the usual commands, e.g.:
-dynamo
+ <tt>ddb dTutorial:a -v</tt>