GPUs Basel 2019 - Revision history

Raffaele Coray at 13:44, 2 December 2024

2024-12-02T13:44:30Z

Stefano Scaramuzza at 10:51, 27 August 2019

2019-08-27T10:51:04Z

Stefano Scaramuzza at 10:40, 27 August 2019

2019-08-27T10:40:58Z

Stefano Scaramuzza at 10:38, 27 August 2019

2019-08-27T10:38:06Z

Stefano Scaramuzza at 07:37, 27 August 2019

2019-08-27T07:37:22Z

Stefano Scaramuzza at 17:24, 26 August 2019

2019-08-26T17:24:12Z

Stefano Scaramuzza at 17:22, 26 August 2019

2019-08-26T17:22:29Z

Stefano Scaramuzza at 16:14, 26 August 2019

2019-08-26T16:14:48Z

Stefano Scaramuzza at 15:57, 26 August 2019

2019-08-26T15:57:02Z

Stefano Scaramuzza: Created page with "Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the..."

2019-08-26T09:52:04Z

Created page with "Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the..."

New page

Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the same steps described here on your own project.

The GPUs we use are located on the high performance computing cluster of the University of Basel called sciCORE (https://scicore.unibas.ch) which uses the SLURM queuing system. A queuing system coordinates the access to the GPUs and is needed when there are many users using a limited amount of GPUs. You should have received the credentials needed to log in on the cluster at the beginning of the workshop.

The main idea is to create an alignment project on your local machine, move it to the sciCORE cluster and then run it using a pre-installed Dynamo standalone version on sciCORE. How this is done is described in the following steps.

'''On your local Matlab session with Dynamo loaded:'''

* Create a tutorial project with Dynamo:
<tt>dtutorial myParticles -p myProject -M 128</tt>
We now have a tutorial dataset with 128 particles in the directory <code>myParticles</code> and a tutorial alignment project <code>myProject</code>.

* Open the alignment project window:
<tt>dcp myProject</tt>
and under ''computing environment'' select ''GPU (standalone)'' as an environment.

* Check and unfold the project.

* Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then ''create a tarball''.

'''On your local Linux terminal:'''

* Open a local Linux terminal and navigate to the directory where you just created the tutorial dataset and project. Copy the project data (particles) to sciCORE:
<tt>rsync -avuP myParticles USERNAME@login.scicore.unibas.ch:/scicore/home/s-gpu-course/USERNAME/dynamo_projects</tt>

* Copy the previously created tar file of the project to sciCORE:
<tt>rsync -avuP myProject.tar USERNAME@login.scicore.unibas.ch:/scicore/home/s-gpu-course/USERNAME/dynamo_projects</tt>

* Login to your sciCORE account:
<tt>ssh -Y USERNAME@login.scicore.unibas.ch</tt>
If asked to continue type "yes". Use the provided password.

'''While logged in to your sciCORE account:'''

* While logged in to your sciCORE account, activate dynamo:
<tt>source /scicore/home/s-gpu-course/GROUP/dynamo_activate_linux_shipped_MCR.sh</tt>

* Go to the location where you copied the data:
<tt>cd dynamo_projects </tt>

* Untar the Dynamo project:
<tt>dynamo </tt>
<tt>dvuntar myProject.tar </tt>
<tt>exit </tt>

* Create a blank SLURM submission script (text file) named ''submit_job.sh'':
<tt>nano submit_job.sh</tt>

* Copy (and adapt) the following lines into the newly created script. Depending on your project you might have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. If your job will run longer than 30 minutes, set the ''qos'' to ''6hours'' and adapt the time to anything between 30 minutes and 6 hours:

<nowiki>#!/bin/bash -l
#
#SBATCH --job-name=dTest
#SBATCH --qos=30min
#SBATCH --time=00:30:00
#SBATCH --mem=16G
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --partition=pascal
#SBATCH --gres=gpu:1
#SBATCH --reservation=dynamo
module load CUDA/7.5.18
source /scicore/home/s-gpu-course/GROUP/dynamo_activate_linux_shipped_MCR.sh
cd $HOME/dynamo_projects
echo "dvput myProject -gpu_identifier_set $CUDA_VISIBLE_DEVICES" > dcommands.sh
echo "dvunfold myProject" >> dcommands.sh
dynamo dcommands.sh
chmod u=rxw ./myProject.exe
./myProject.exe</nowiki>

* You can now run your alignment project by submitting the previously created script to SLURM with:
<tt>sbatch submit_job.sh</tt>

* With the following commands you can check the overall status of the submitted jobs:

Check your status in the queue:
<tt>squeue -u USERNAME</tt>

See all users in the queue:
<tt>squeue -q 30min</tt>

To cancel the job type ''scancel'' followed by the job ID that was shown by the squeue command:
<tt>scancel my_job_id</tt>

Some ways to check the last output:
<tt>ls -rtl</tt>
<tt>tail -f slurm-45994509.out</tt>
<tt>less slurm-45994509.out</tt>

To check the last average load the standalone Dynamo environment by typing <code>dynamo</code> into the terminal and use the usual commands, e.g.:
<tt>ddb myProject:a -v</tt>

← Older revision		Revision as of 13:44, 2 December 2024
Line 1:		Line 1:
		+	[[Category:WalkthroughsSpecific]]
		+
	Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the same steps described here on your own project.		Here we describe how to use the GPUs provided for the Basel Workshop 2018. We go through each step and use a simple tutorial dataset and project as an example. You can use the same steps described here on your own project.

← Older revision		Revision as of 10:40, 27 August 2019
Line 87:		Line 87:

	See all users in the queue:		See all users in the queue:
−	<tt>squeue -q ~~30min~~</tt>	+	<tt>squeue -q emgpu</tt>

← Older revision		Revision as of 17:24, 26 August 2019
Line 90:		Line 90:


−	To cancel the job type ''scancel'' followed by the job ID that was shown by the squeue command:	+	To cancel the job type ''scancel'' followed by the job ID that was shown by the previous ''squeue'' command:
	<tt>scancel my_job_id</tt>		<tt>scancel my_job_id</tt>

← Older revision		Revision as of 17:22, 26 August 2019
Line 75:		Line 75:
	./myProject.exe</nowiki>		./myProject.exe</nowiki>

		+
		+	* Save and close the ''nano'' text editor by pressing first ''ctrl''+''shift''+''x'' and then ''Y'' to save changes and finally ''enter''.

	* You can now run your alignment project by submitting the previously created script to SLURM with:		* You can now run your alignment project by submitting the previously created script to SLURM with:

← Older revision		Revision as of 15:57, 26 August 2019
Line 18:		Line 18:
	* Check and unfold the project.		* Check and unfold the project.

−	* Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then ''create a tarball''.	+	* Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then ''create a tarball''. In this example it does not matter whether you choose ''skipping'' or ''including'' results.

@@ Line 93: / Line 93: @@
   <tt>scancel my_job_id</tt>
-Some ways to check the last output:
+To see the progress type:
   <tt>ls -rtl</tt>
   <tt>tail -f slurm-45994509.out</tt>
- <tt>less slurm-45994509.out</tt>
+Exit the live view by typing ''ctrl''+''c''.
 To check the last average load the standalone Dynamo environment by typing <code>dynamo</code> into the terminal and use the usual commands, e.g.:
   <tt>ddb myProject:a -v</tt>

@@ Line 52: / Line 52: @@
   <tt>nano submit_job.sh</tt>
-* Copy (and adapt) the following lines into the newly created script. Depending on your project you might have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. If your job will run longer than 30 minutes, set the ''qos'' to ''6hours'' and adapt ''time'' to anything between 30 minutes and 6 hours:
+* Copy (and adapt) the following lines into the newly created script. Depending on your project you might have to adapt the project name and the time requested (''time=hh:mm:ss'') in the script. The limit for the parameter ''time'' is 6 hours:
   <nowiki>#!/bin/bash -l
+#
 #SBATCH --job-name=dTest
-#SBATCH --qos=30min
+#SBATCH --qos=emgpu
 #SBATCH --time=00:30:00
 #SBATCH --mem=16G
@@ Line 63: / Line 63: @@
 #SBATCH --ntasks-per-node=1
 #SBATCH --cpus-per-task=1
-#SBATCH --partition=pascal
+#SBATCH --partition=titanx
 #SBATCH --gres=gpu:1
 #SBATCH --reservation=dynamo

@@ Line 19: / Line 19: @@
 * Before moving the data to sciCORE we have to compress the project. In the project window go to ''Tools'' and then ''create a tarball''. In this example it does not matter whether you choose ''skipping'' or ''including'' results.
 '''On your local Linux terminal:'''
-* Open a local Linux terminal and navigate to the directory where you just created the tutorial dataset and project. Copy the project data (particles) to sciCORE:
+* Open a new local Linux terminal and navigate to the directory where you just created the tutorial dataset and project. Copy the project data (particles) to sciCORE:
   <tt>rsync -avuP myParticles USERNAME@login.scicore.unibas.ch:/scicore/home/s-gpu-course/USERNAME/dynamo_projects</tt>