MPI Cluster - Revision history

Daniel Castaño: /* Using a cluster under Matlab */

2017-09-18T11:09:01Z

Using a cluster under Matlab

Daniel Castaño: /* Compilation */

2016-05-18T09:17:11Z

Compilation

Daniel Castaño: /* Compilation */

2016-05-18T09:12:02Z

Compilation

Daniel Castaño: /* Parallelization system */

2016-05-18T09:07:49Z

Parallelization system

Daniel Castaño at 09:01, 18 May 2016

2016-05-18T09:01:06Z

Daniel Castaño: /* GUI */

2016-05-18T08:54:50Z

GUI

Daniel Castaño: /* GUI */

2016-05-18T08:51:53Z

GUI

Daniel Castaño at 08:51, 18 May 2016

2016-05-18T08:51:07Z

Daniel Castaño at 08:23, 18 May 2016

2016-05-18T08:23:45Z

Daniel Castaño at 08:22, 18 May 2016

2016-05-18T08:22:55Z

← Older revision		Revision as of 11:09, 18 September 2017
Line 99:		Line 99:
	Additionally, in this setting the averaging step is parallelized exactly the same as the aligning step.		Additionally, in this setting the averaging step is parallelized exactly the same as the aligning step.

−	Note that the ''engine'' should be deployed and maintained on your cluster. Having a single licence on the login node (even if the Parallel Computing Toolbox is active there) will NOT be sufficient to run ''Dynamo''-Matlab jobs on your cluster.	+	Note that the ''engine'' should be deployed and maintained on your cluster. Having a single licence on the login node (even if the Parallel Computing Toolbox is active there) will NOT be sufficient to run ''Dynamo''-Matlab jobs on your cluster. Unfortunately, not many institutions offer Distributed Computing Engine licenses.

@@ Line 7: / Line 7: @@
 == Compilation ==
-The executables delivered in the ''Dynamo'' distribution should work in a Linux workstation for parallel jobs run on the same machine (through OMP threads). They are NOT enough to run parallel jobs in a cluster of different machines, which requires the MPI libraries.  Thus, compiling ''Dynamo'' on your cluster requires a {{t|cc}} compiler that links the MPI libraries. In most systems, you can run the command:
+The executables delivered in the ''Dynamo'' distribution should work in a Linux workstation for parallel jobs run on the same machine (through OMP threads). They are NOT enough to run parallel jobs in a cluster of different machines, which requires the MPI libraries.  Thus, compiling ''Dynamo'' on your cluster requires an additional step involving a  {{t|cc}} compiler available on your side that links the MPI libraries. In most systems, you can run the command:
 <tt>module avail</tt>
-on the shell of your login node to check the available modules. Modules for parallel computation typically will include an mpi-enabled compiler. You need to load one of them, for instance:
+on the shell of the login node of your cluster to check the available ''modules'' that you can switch on. Modules for parallel computation typically will include an mpi-enabled compiler. You need to load one of them, for instance:
 <tt> module load mpiCC</tt>

@@ Line 19: / Line 19: @@
 <tt>which mpicc</tt>
-should give you the complete path to a compiler called <tt>mpicc</tt> on your {{t|$PATH}}. If this is not the case, try with alternative syntax.
+should give you the complete path to a compiler called <tt>mpicc</tt> on your {{t|$PATH}}. If this is not the case, try with alternative syntax ({{t|miCC}}), etc.
 If you are lucky  enough, your cluster environment should have some information system (like a webpage) that tells you the modules that you are expected to use for MPI-compatible compilation, and the attached compilers.
@@ Line 28: / Line 28: @@
 <tt>source dynamo_compile_mpi.sh mpicc</tt>
-If you get an error during compilation, try with a different module of a different compiler inside the same module.
+Note that you pass the name of the compiling command as second argument of {{t|source}}. If you get an error during compilation, try with a different module of a different compiler inside the same module.
 ==Cluster header file==

← Older revision		Revision as of 09:07, 18 May 2016
Line 90:		Line 90:

	Additionally: currently the alignment step is distributed along all the available tasks, but the averaging itself is processed by a single core. In most cases averaging is not a bottleneck, and this is not a problem: averaging is typically a small fraction of time compared to alignment. This can however be a problem when you have several thousands particles and the alignment step only covers a few Euler triplets.		Additionally: currently the alignment step is distributed along all the available tasks, but the averaging itself is processed by a single core. In most cases averaging is not a bottleneck, and this is not a problem: averaging is typically a small fraction of time compared to alignment. This can however be a problem when you have several thousands particles and the alignment step only covers a few Euler triplets.
		+
		+	=== Algorithms ===
		+	The algorithms in the MPI version are the same used in the Matlab or Standalone versions.

	==Using a cluster under Matlab==		==Using a cluster under Matlab==

@@ Line 1: / Line 1: @@
-''Dynamo'' can be run as {{standalone}} on a cluster of CPUs. This works for alignment and classification projects. Using the ''Dynamo'' standalone in  a CPU cluster requires some additional steps compared to the execution on a single server during an interactive session.
+''Dynamo'' can be run as [[standalone]] on a cluster of CPUs. This works for alignment and classification projects. Using the ''Dynamo'' standalone in  a CPU cluster requires some additional steps compared to the execution on a single server during an interactive session.
 # Compile specifically for your cluster
@@ Line 83: / Line 83: @@
 ===Several MCR extraction folders===
 ''Dynamo'' includes the experimental option of selecting a different {{t|MCR_CACHE_ROOT}} for each one of the spawned MPI tasks.  You should try this option if you notice a too slow performance.
 ==Using a cluster under Matlab==
 This is a totally different scenario. If your cluster supports running [[Matlab]] jobs through the Distributed Computing Engine... that's perfect. You don't need to use the MPI version of ''Dynamo'': no need to compile the MPI executables, no need to design a cluster header file. You just use the {{t|destination}} parameter {{t|matlab_parfor}} and Matlab will take care of everything.
 Note that the ''engine'' should be deployed and maintained on your cluster. Having a single licence on the login node (even if the Parallel Computing Toolbox is active there) will NOT be sufficient to run ''Dynamo''-Matlab jobs on your cluster.

@@ Line 45: / Line 45: @@
 * Make certain that the {{t|Parallelized averaging step}} in the bottom panel is set to zero. This option only applies to Matlab based computations.
 * Pass the path to the cluster header file.
 ===Command line===

@@ Line 40: / Line 40: @@
 === GUI===
- After opening a project in the {{t|dcp GUI}} you need to set fo {{t|Computing Environment}} GUI
+After opening a project in the {{t|dcp GUI}}, switch to the {{t|Computing Environment}} GUI and:
 * Make certain that you are dialing the {{t|cluster MPI}}  option on the.
 * Select the number of cores on the field {{t|CPU cores}}. Each one will be handled by a separate MPI task.

@@ Line 5: / Line 5: @@
 # Each time you create a project, tell it to use the cluster header to produce a project execution script (extension <tt>.sh</tt>)
 # Submit the execution script representing the project to your cluster.
 == Compilation ==
+The executables delivered in the ''Dynamo'' distribution should work in a Linux workstation for parallel jobs run on the same machine (through OMP threads). They are NOT enough to run parallel jobs in a cluster of different machines, which requires the MPI libraries.  Thus, compiling ''Dynamo'' on your cluster requires a {{t|cc}} compiler that links the MPI libraries. In most systems, you can run the command:
 <tt>module avail</tt>
@@ Line 23: / Line 19: @@
 <tt>which mpicc</tt>
-should give you a complete path to a compiler called <tt>mpicc</tt> on your path. If this is not the case, try with alternative syntax.
+should give you the complete path to a compiler called <tt>mpicc</tt> on your {{t|$PATH}}. If this is not the case, try with alternative syntax.
-If you are fortunate enough, your cluster environment should have some information system (like a webpage) that tells you the modules that you are expected to use for compilation, and the attached compilers.
+If you are lucky  enough, your cluster environment should have some information system (like a webpage) that tells you the modules that you are expected to use for MPI-compatible compilation, and the attached compilers.
 Once you know the name of the compiler that you are going to use (say, {{t|mpicc}}),  you can proceed compile the MPI executables:
@@ Line 34: / Line 30: @@
 If you get an error during compilation, try with a different module of a different compiler inside the same module.
+==Cluster header file==
 A ''cluster header'' file allows ''Dynamo'' to produce an [[execution script]] for a project that will be understood by the specific syntax of your cluster.
@@ Line 41: / Line 36: @@
 == Preparing a project ==
-You need first to tune several parameters in your project, through the GUI or the command line
+You need first to tune several parameters in your project, through the GUI or the command line.
 === GUI===
@@ Line 50: / Line 46: @@
 * Make certain that the {{t|Parallelized averaging step}} in the bottom panel is set to zero. This option only applies to Matlab based computations.
 * Pass the path to the cluster header file.
 ===Command line===
 All the steps above can be performed through the command line, using the names of the project parameters. You can follow the examples below, where a project called {{t|myProject}} gets its parameters tuned with the command {{t|dvput}}
@@ Line 58: / Line 55: @@
 * <tt>dvput myProject cluster myClusterHeader.sh </tt>
 Remember that the ''Dynamo'' command {{t|dvhelp}} will list the different project parameters that can be edited by the user through {{t|dvput}}
 ==Performance==
 In some clusters, following the above procedure without further tuning can lead ''Dynamo'' to show a very slow performance. This is normally related to the fact ''Dynamo'' as standalone works on the [[MCR libraries]]. These libraries might need some tuning for your system.
 ==Using a cluster under Matlab==
-If your cluster supports running Matlab jobs through the Distributed Computing Engine... that's perfect. You don't need to use the MPI version of ''Dynamo'': no need to compile the MPI executables, no need to design a cluster header file. You just use the {{t|destination}} parameter {{t|matlab_parfor}} and Matlab will take care of everything.
+This is a totally different scenario. If your cluster supports running [[Matlab]] jobs through the Distributed Computing Engine... that's perfect. You don't need to use the MPI version of ''Dynamo'': no need to compile the MPI executables, no need to design a cluster header file. You just use the {{t|destination}} parameter {{t|matlab_parfor}} and Matlab will take care of everything.