Pipelines for Workshop Heidelberg 2018 - Revision history

Daniel Castaño at 08:27, 22 August 2018

2018-08-22T08:27:15Z

Daniel Castaño at 08:16, 22 August 2018

2018-08-22T08:16:28Z

Daniel Castaño at 07:40, 22 August 2018

2018-08-22T07:40:28Z

Daniel Castaño at 06:45, 22 August 2018

2018-08-22T06:45:31Z

Daniel Castaño: /* Preprocessing */

2018-08-22T06:32:32Z

Preprocessing

Daniel Castaño at 06:21, 22 August 2018

2018-08-22T06:21:22Z

Daniel Castaño at 23:06, 21 August 2018

2018-08-21T23:06:03Z

Daniel Castaño at 23:05, 21 August 2018

2018-08-21T23:05:09Z

Daniel Castaño: Created page with "These materials describe the procedural steps required for high resolution tomography. = Pipeline = == Input data == We will start with two tilt series. A drift correcti..."

2018-08-21T19:40:51Z

Created page with "These materials describe the procedural steps required for high resolution tomography. = Pipeline = == Input data == We will start with two tilt series. A drift correcti..."

New page

These materials describe the procedural steps required for high resolution tomography.

= Pipeline =

== Input data ==
We will start with two tilt series. A drift correction has already been applied on each tilt using motioncorr2, so that each micrograph in a tilt series is an averaged movie. The pixel size is 2.7A (the tilt series have been binned by fourier cropping to ease the computation ), and the structures of interest form spherical lattices.

== Procedural steps ==

=== Preprocessing steps ===
As we will initially work with already drift-corrected tilt series,
* Defocus estimation with <tt>ctffind4</tt>
* Alignment of tilt series with ''Dynamo''
* Exposure filter on the aligned tilt series.
* Stripe-based CTF correction (with <tt>ctfphaseflip</tt> from Imod)
* Creation of tomograms with ''Dynamo''

These preprocessing operations that go from the tilt series to the tomograms can be used for any sample.

=== Tomogram management ===

This part is specific to the particular geometry of the proteins of interest.
* Cataloguing the tomograms (archiving)
* Annotation of vesicle positions.
* Extraction of putative particles based on the vesicle positions.
* Ellimination of repeated particles.

=== Subtomogram averaging ===

* Extraction of true particles.
* Splitting in odd and even data sets.
* Independent alignment and resolution determination.

= Material organization =

← Older revision		Revision as of 08:27, 22 August 2018
Line 120:		Line 120:

	== Management of tomogram data ==		== Management of tomogram data ==
		+
		+	Now we need to organize the different tilt series computed independently into an object that indexes all of them: the catalogue.

	=== Archiving of tomograms ===		=== Archiving of tomograms ===

@@ Line 122: / Line 122: @@
 === Archiving of tomograms ===
 === Modelling of vesicles ===
@@ Line 129: / Line 141: @@
   <pre>
-</pre>
+Here, 40 is the number of pixels of separation between particles on the surface of each vesicle.
 ==== Extraction of particles ====
 With the table and the file that indexes the tomograms, we extract the located positions into a data folder.
 ==== First average ====
@@ Line 138: / Line 154: @@
 We compute a first average to feed an alignment project. Although the average is relatively featureless, it has enough signal to attract reinforcement of coherent signal, and a few iteration rounds will be enough to show the basic features of the protein.
-  <tt>o = daverage('');</tt>
+  <tt>o = daverage('oversampling_5.2.Data','t','oversampling_5.2.Data/crop.tbl','fc',1);</tt>
 ==== Creation of a project ====
 We create an alignment project.
@@ Line 150: / Line 167: @@
 == Subtomogram averaging ==
-Once the particles have been located, we need to
+Once the particles have been located, subtomogram averaging can start. We divide the data set in two halves by creating two different tables.

@@ Line 76: / Line 76: @@
 * Output: <tt>work/i/imodDefocus.txt</tt>
-<pre>for i=1:2
+ <pre>for i=1:2
     emboDefocusEstimation('work',i,pixelSize);
 end </pre>
 ===  tilt series alignment ===
 * Input:    <tt>work/i/rawStack</tt>, <tt>work/i/nominalAngles.tlt</tt>
 * Output:  <tt>work/X/aligner.AWF/alignedFullStack.mrc </tt>
-<pre>for i=1:2
+ <pre>for i=1:2
     emboAlignment('work',i,pixelSize);
 end </pre>
@@ Line 90: / Line 92: @@
 ===  dose filtering ===
 * Input:
 * Output: <tt>work/X/doseStack.mrc</tt>
 ===  CTF correction of tilt series ===
@@ Line 98: / Line 103: @@
 * Input:    <tt>work/X/</tt>,<tt>work/X/doseStack.mrc</tt>
 * Output: <tt>work/X/finalStacl</tt>
 ===  Reconstruction of tomograms ===
@@ Line 112: / Line 121: @@
 == Management of tomogram data ==
 == Subtomogram averaging ==

@@ Line 69: / Line 69: @@
 ===  defocus estimation ===
 The function <tt>emboDefocusEstimation</tt> will run <tt>ctffind4</tt> on the input data, creating the following items:
-* Input:
+* Input:    <tt>work/i/rawStack</tt>
-* Output:
+* Output: <tt>work/i/imodDefocus.txt</tt>
 <pre>for i=1:2
-emboDefocusEstimation('work',i,pixelSize);
+   emboDefocusEstimation('work',i,pixelSize);
 end </pre>
 ===  tilt series alignment ===
-* Input:
+* Input:    <tt>work/i/rawStack</tt>, <tt>work/i/nominalAngles.tlt</tt>
-* Output:
+* Output:  <tt>work/X/aligner.AWF/alignedFullStack.mrc </tt>
+<pre>for i=1:2
 ===  dose filtering ===
 * Input:
-* Output:
+* Output: <tt>work/X/doseStack.mrc</tt>
 ===  CTF correction of tilt series ===
-===  Reconstruction of tomogram ===
+The results of the dose filtering are sent to Imod's<tt>ctfphaseflip</tt>
-* Input:
+* Input:    <tt>work/X/</tt>,<tt>work/X/doseStack.mrc</tt>
-* Output:
+* Output: <tt>work/X/finalStacl</tt>
 == Management of tomogram data ==

@@ Line 60: / Line 60: @@
 They are intended to be callable on a loop like:
-  <pre> for i=1:2
+  <pre>for i=1:2
-emboFunction('work',i);
+   emboFunction('work',i);
 end </pre>
-If new tilt series are added to increase resolution, you just need to apply the function on additional indices.
+If new tilt series are added in order to increase the attainable resolution, you just need to apply the function on additional indices.
 ===  defocus estimation ===
-The function <tt>emboDefocusEstimation</tt> will run <tt>ctffind4</tt> on the input data, creating the following items
+The function <tt>emboDefocusEstimation</tt> will run <tt>ctffind4</tt> on the input data, creating the following items:
 * Input:
 * Output:
 ===  tilt series alignment ===

← Older revision		Revision as of 06:21, 22 August 2018
Line 1:		Line 1:
−	These materials describe the procedural steps required for high resolution tomography.	+	These materials describe the procedural steps required for high resolution tomography. Materials on more basic aspects of ''Dynamo'' can be consulted [[Materials Basel 2018\| here]]:

@@ Line 39: / Line 39: @@
 The initial structure of the folder contains two items
-  <pre> ls work/1
+  <pre>ls work/1
 rawStack.mrc
 nominalTilts.tlt
@@ Line 91: / Line 91: @@
 == Management of tomogram data ==
-== Subtomogram averaging ===

← Older revision		Revision as of 23:05, 21 August 2018
Line 34:		Line 34:

	= Material organization =		= Material organization =
		+
		+	The data corresponding to each tilt series is under the folders <tt>work/1</tt> and <tt>work/2</tt>. Intermediate results corresponding to each tilt series will be stored in the corresponding folders.
		+
		+	The initial structure of the folder contains two items
		+
		+	<pre> ls work/1
		+	rawStack.mrc
		+	nominalTilts.tlt
		+	</pre>
		+
		+	The additional processing steps will gradually generate more elements pert tilt.
		+
		+	== Copying data to your account ==
		+
		+	You will need to copy the provided data into you account:
		+
		+	<tt>cp -r /g/embl/workshop/teach11/data/work . </tt>
		+
		+	== Matlab environment ==
		+
		+	Matlab provides a convenient syntax for manipulation of sets of paths.
		+
		+	== Preprocessing ==
		+	Each of the preprocessing steps can be computed separately for each tilt series. We have thus provided a set of functions, each one a wrapper that accepts as input one index that identifies a tilt series and creates an output with the right naming convention.
		+	They are intended to be callable on a loop like:
		+
		+	<pre> for i=1:2
		+	emboFunction('work',i);
		+	end </pre>
		+
		+	If new tilt series are added to increase resolution, you just need to apply the function on additional indices.
		+
		+	=== defocus estimation ===
		+
		+	The function <tt>emboDefocusEstimation</tt> will run <tt>ctffind4</tt> on the input data, creating the following items
		+
		+	* Input:
		+	* Output:
		+
		+	=== tilt series alignment ===
		+	* Input:
		+	* Output:
		+
		+	=== dose filtering ===
		+	* Input:
		+	* Output:
		+
		+	=== CTF correction of tilt series ===
		+	* Input:
		+	* Output:
		+
		+	=== Reconstruction of tomogram ===
		+	* Input:
		+	* Output:
		+
		+	== Management of tomogram data ==
		+
		+	== Subtomogram averaging ===