Tilt series alignment workflow folder - Revision history

Daniel Castaño: /* Markers */

2019-02-06T16:06:02Z

Markers

← Older revision		Revision as of 16:06, 6 February 2019
Line 67:		Line 67:
	Files representing markers contain (among further information) the <tt>x,y</tt> coordinates of the markers on each tilt micrograph where the marker is present.		Files representing markers contain (among further information) the <tt>x,y</tt> coordinates of the markers on each tilt micrograph where the marker is present.

	Makers files are stored in <tt>.dms</tt> files, which are binary files that contain ''Dynamo''-native <tt>markerSet</tt> objects. They can be converted into an ''indexed cloud'' (numerical matrix) by reading the <tt>.dms<tt> file into memory and then using <tt>getIndexedCloud();</tt>		Makers files are stored in <tt>.dms</tt> files, which are binary files that contain ''Dynamo''-native <tt>markerSet</tt> objects. They can be converted into an ''indexed cloud'' (numerical matrix) by reading the <tt>.dms</tt> file into memory and then using <tt>getIndexedCloud();</tt>

	<tt>m = dread('markers.dms'); </tt>		<tt>m = dread('markers.dms'); </tt>
Line 98:		Line 98:
	The positions on the full stack lay in the file <tt>markers.dms<tt>.		The positions on the full stack lay in the file <tt>markers.dms<tt>.

	They are computed by fitting the positions in the <tt>finalWorkingMarkers</tt> (originally computed by locating the centers of binned gold beads) to full resolution gold beads.		They are computed by fitting the positions in the <tt>finalWorkingMarkers</tt> (originally computed by locating the centers of binned gold beads) to full resolution gold beads.

	==Reconstructions ==		==Reconstructions ==
	==Metadata ==		==Metadata ==

Daniel Castaño at 14:34, 6 February 2019

2019-02-06T14:34:52Z

@@ Line 65: / Line 65: @@
 == Markers ==
-Makers files are stored in <tt>.dms</tt> files, which are binary files.
+Files representing markers contain (among further information) the <tt>x,y</tt> coordinates of the markers on each tilt micrograph where the marker is present.
 === Markers on binned stack ===
 ==== Rough alignment ====
-==== Initial track  ====
+Those are tracks of markers found by comparing shifted versions of micrographs. They do not include any 3D model of the projection geometry, and are used internally by ''Dynamo'' to provide  an initial track identity to the gold markers.
 ==== Final tracks ====
 === Marker on full stack ===

Daniel Castaño at 14:04, 6 February 2019

2019-02-06T14:04:13Z

← Older revision		Revision as of 14:04, 6 February 2019
Line 65:		Line 65:

	== Markers ==		== Markers ==
			Makers files are stored in <tt>.dms</tt> files, which are binary files.

	=== Markers on binned stack ===		=== Markers on binned stack ===
			==== Rough alignment ====
			==== Initial track ====
			==== Final tracks ====

	=== Marker on full stack ===		=== Marker on full stack ===
	The positions on the full stack lay in the file <tt>markers.dms<tt>		The positions on the full stack lay in the file <tt>markers.dms<tt>.

			They are computed by fitting the positions in the <tt>finalWorkingMarkers</tt> (originally computed by locating the centers of binned gold beads) to full resolution gold beads.
	==Reconstructions ==		==Reconstructions ==
	==Metadata ==		==Metadata ==

Daniel Castaño at 13:38, 6 February 2019

2019-02-06T13:38:04Z

← Older revision		Revision as of 13:38, 6 February 2019
Line 5:		Line 5:
	== Workflow object ==		== Workflow object ==

	The workflow object itself appears as a file called <tt>object.mat</tt>. This is a binary file, which is used internally by ''Dynamo'' when performing alignments and reconstructions. ~~It is~~ rarely ~~needed by users~~.		The workflow object itself appears as a file called <tt>object.mat</tt>. This is a binary file, which is used internally by ''Dynamo'' when performing alignments and reconstructions. Although it contains information about every aspect of the workflow (used parameters, location of input and output data, etc), regular users rarely need to explicitly manipulate it for standard operations.

	When it is needed (see examples below), It can be called into memory with the generic <tt>dread</tt> command:		When it is needed (see examples below), It can be called into memory with the generic <tt>dread</tt> command:
Line 33:		Line 33:

	Some micrographs might be '''manually''' rejected because of visually obvious errors, like defocus or shift jumps. An explicit decision by the user on eliminating a given (set of) micrograph(s) can be taken a priori, or during the alignment procedure with the GUI.		Some micrographs might be '''manually''' rejected because of visually obvious errors, like defocus or shift jumps. An explicit decision by the user on eliminating a given (set of) micrograph(s) can be taken a priori, or during the alignment procedure with the GUI.
	In both cases, the indices are referred to the nominal angles seen by the workflow.		In both cases, the indices are referred to the nominal angles seen by the workflow, and are stored in a text file called '''<tt>discardedTiltIndicesManual.txt</tt>''' .

	~~===~~ Tilts ~~rejected a priori ===~~		Tilts included in this file are not counted by ''Dynamo'' when computing chains of markers.

	This option is normally used for batch processing using unsupervised alignment. Before launching the automatic alignment process, the user quickly supervises the different tilt series in the batch and marks the tilts that are .		==== Tilts rejected a priori ====

	=== Tilts rejected during manual alignment ===		The user has the option of manually creating the file <tt>manuallyDiscardedTiltIndices.txt</tt> before starting the workflow (as automatic process or through the GUI). This option is normally used for batch processing using unsupervised alignment. Before launching the automatic alignment process, the user quickly supervises the different tilt series in the batch and marks the tilts that are not suitable for processing (typically because of defocus or shift jumps ).

			==== Tilts rejected during manual alignment ====
			When operating the GUI, some micrographs can be identified as invalid. They can be marked on the fly, and the file <tt>discardedTiltIndicesManual.txt</tt> will be updated. Inversely, one can restore tilts that had been deselected a-priori.

			=== Tilts rejected during automatic alignment procedure ===
			Tilts can be rejected by the alignment algorithm if they don't contain enough markers, or if the error in the markers detected on the tilts is too big. The tilt indices that fall in this category are stored in the file <tt>discardedTiltIndicesComputational.txt</tt>

			=== Reconstruction angles ===
			The indices of the angles from <tt>nominal.tlt</tt> that survive both manual and computational rejection are kept in the file <tt>reconstructionTiltIndices.txt</tt>

	== Stacks ==		== Stacks ==

			=== Raw binned stack ===
			''Dynamo'' uses a binned, unaligned matrix in order to speed up the alignment procedure. It is stored under the name <tt>workingMatrix</tt>.

			=== Aligned binned stack ===
			The aligned binned stack is stored in the file <tt>alignedBinnedStack.mrc</tt>.
			It is generated by applying the fitter generated by the <tt>finalWorkingMarkers</tt> to the raw binned stack.

			This stack does '''not'''contain the micrographs corresponding to tilts that have been rejected, both manually or computationally. Thus, the order an identity of micrographs in the stack corresponds to the order and identity of angles indexed by <tt>reconstructionTiltIndices</tt>

			=== Aligned full stack ===
			The aligned full stack is stored in <tt>alignedFullStack.mrc</tt>

			== Markers ==
			=== Markers on binned stack ===
			=== Marker on full stack ===
			The positions on the full stack lay in the file <tt>markers.dms<tt>
	==Reconstructions ==		==Reconstructions ==
	==Metadata ==		==Metadata ==

Daniel Castaño at 11:53, 6 February 2019

2019-02-06T11:53:51Z

← Older revision		Revision as of 11:53, 6 February 2019
Line 26:		Line 26:
	<tt>nominal.tlt</tt>		<tt>nominal.tlt</tt>

	This file is normally generated by ''Dynamo'' when the alignment workflow is created . Note that these angles ~~can~~ also be stored in a different location and linked to the workflow, but this procedure is ~~exceptional~~.		This file is normally generated by ''Dynamo'' when the alignment workflow is created. Note that these angles could also be stored in a different location and linked to the workflow, but this procedure is non-standard.

	~~This~~ text file should contain an angle (in degrees) for each micrograph in the data stack, and appear in the same order.		In any case, the text file should contain an angle (in degrees) for each micrograph in the data stack, and appear in the same order.

	=== Explicitly rejected tilt angle indices ===		=== Explicitly rejected tilt angle indices ===
Line 36:		Line 36:

	=== Tilts rejected a priori ===		=== Tilts rejected a priori ===

			This option is normally used for batch processing using unsupervised alignment. Before launching the automatic alignment process, the user quickly supervises the different tilt series in the batch and marks the tilts that are .

	=== Tilts rejected during manual alignment ===		=== Tilts rejected during manual alignment ===

Daniel Castaño at 11:30, 6 February 2019

2019-02-06T11:30:46Z

← Older revision		Revision as of 11:30, 6 February 2019
Line 22:		Line 22:
	== Angles ==		== Angles ==

	=== ~~Raw~~ angles ===		=== Nominal tilt angles ===
	The '''nominal'''tilt angles of the raw stack are given as a text file with the name:		The '''nominal'''tilt angles of the raw stack are given as a text file with the ''default'' name:
	<tt>nominal.tlt</tt>		<tt>nominal.tlt</tt>

			This file is normally generated by ''Dynamo'' when the alignment workflow is created . Note that these angles can also be stored in a different location and linked to the workflow, but this procedure is exceptional.

			This text file should contain an angle (in degrees) for each micrograph in the data stack, and appear in the same order.

			=== Explicitly rejected tilt angle indices ===

			Some micrographs might be '''manually''' rejected because of visually obvious errors, like defocus or shift jumps. An explicit decision by the user on eliminating a given (set of) micrograph(s) can be taken a priori, or during the alignment procedure with the GUI.
			In both cases, the indices are referred to the nominal angles seen by the workflow.

			=== Tilts rejected a priori ===

			=== Tilts rejected during manual alignment ===

	== Stacks ==		== Stacks ==
	==Reconstructions ==		==Reconstructions ==
	==Metadata ==		==Metadata ==

Daniel Castaño at 10:43, 6 February 2019

2019-02-06T10:43:11Z

@@ Line 5: / Line 5: @@
 == Workflow object ==
-The workflow object itself appears as a file called <tt>object.mat</tt>. This is a binary file.
+The workflow object itself appears as a file called <tt>object.mat</tt>. This is a binary file, which is used internally by ''Dynamo'' when performing alignments and reconstructions. It is rarely needed by users.
 == Angles ==
 == Stacks ==
 ==Reconstructions ==
 ==Metadata ==

Daniel Castaño at 17:08, 5 February 2019

2019-02-05T17:08:07Z

← Older revision		Revision as of 17:08, 5 February 2019
Line 3:		Line 3:
	The folder .AWF contains different categories of files:		The folder .AWF contains different categories of files:

	*Workflow object		== Workflow object ==
	*Raw data
	*Stacks		The workflow object itself appears as a file called <tt>object.mat</tt>. This is a binary file.
	*Angles
	*Reconstructions		== Raw data ==
	*Metadata		== Angles ==
			== Stacks ==
			==Reconstructions ==
			==Metadata ==

Daniel Castaño at 17:06, 5 February 2019

2019-02-05T17:06:14Z

← Older revision		Revision as of 17:06, 5 February 2019
Line 1:		Line 1:
	'''Article under construction, sorry'''		'''Article under construction, sorry'''

			The folder .AWF contains different categories of files:

			*Workflow object
			*Raw data
	*Stacks		*Stacks
	*Angles		*Angles
	*Reconstructions		*Reconstructions
			*Metadata

Daniel Castaño at 17:04, 5 February 2019

2019-02-05T17:04:03Z

← Older revision		Revision as of 17:04, 5 February 2019
Line 2:		Line 2:


			*Stacks
	Angles		*Angles
			*Reconstructions
	Reconstruction