Difference between revisions of "Walkthrough on GUI based tilt series alignment"

From Dynamo
Jump to navigation Jump to search
Line 4: Line 4:
  
 
We have prepared version a tilt series from the EMPIAR entry [XXXX]
 
We have prepared version a tilt series from the EMPIAR entry [XXXX]
−
  <tt>/html/w/doc/data/hiv/align/b001ts001.mrc</tt>
+
  <tt> wget  https://wiki.dynamo.biozentrum.unibas.ch/html/w/doc/data/hiv/align/b001ts001.mrc</tt>
  
 
In this binned version, the pixel size is 2'7 Angstrom (original was 1.35, this is bin level 1 in <tt>Dynamo</tt>)
 
In this binned version, the pixel size is 2'7 Angstrom (original was 1.35, this is bin level 1 in <tt>Dynamo</tt>)
Line 12: Line 12:
 
The basic function for invoking the GUI is <tt>dtsa</tt> (short form of <tt>dynamo_tilt_series_alignment</tt>;  
 
The basic function for invoking the GUI is <tt>dtsa</tt> (short form of <tt>dynamo_tilt_series_alignment</tt>;  
  
−
   <tt>dtsa('folder','firstAlign'');</tt>
+
   <tt>u = dtsa('folder','firstAlign'');</tt>
 +
 
 +
[[ File:alignGUI_creatingAWorkflow.png |thumb|center| 600px|creating a workflow]]
  
 
The data for the workflow (tilt series, angles) could have been introduced directly when creating the workflow, we however will introduce each item explicitly in this walkthrough.  
 
The data for the workflow (tilt series, angles) could have been introduced directly when creating the workflow, we however will introduce each item explicitly in this walkthrough.  
  
 +
 +
[[ File:alignGUI_inputWorkflowParameters.png |thumb|center| 600px|input workflow parameters]]
 
= GUI description =
 
= GUI description =
  
 
The GUI guides of the process of creating and editing a  set of "working markers" (in the areas of '''detection''', '''reindexing''' and '''refinement'''), and then using them to create aligned stacks ('''alignment''' tab), analyse and correct their CTF (through wrappers to ctfffind4 and imod) and then create reconstructions.  
 
The GUI guides of the process of creating and editing a  set of "working markers" (in the areas of '''detection''', '''reindexing''' and '''refinement'''), and then using them to create aligned stacks ('''alignment''' tab), analyse and correct their CTF (through wrappers to ctfffind4 and imod) and then create reconstructions.  
  
 +
[[ File:alignGUI_alignmentGUIBeforeCompletionOfAnyStep.png |thumb|center| 600px|alignment GUI before completion of any step]]
 
== Settings==
 
== Settings==
  
Line 40: Line 45:
 
== Execution Areas==
 
== Execution Areas==
 
The GUI is divided in five areas, which are sets of steps that can be executed sequentially.  Sequential execution is the natural way of proceeding; but steps can be visited in any order, i.e., a step can be reexecuted in a later moment after updating its information.
 
The GUI is divided in five areas, which are sets of steps that can be executed sequentially.  Sequential execution is the natural way of proceeding; but steps can be visited in any order, i.e., a step can be reexecuted in a later moment after updating its information.
 +
 +
[[ File:alignGUI_passingTheCursorOnTheGUIElementsShowsAHelpElement.png |thumb|center| 600px|passing the cursor on the GUI elements shows a help element  ]]
  
 
== Steps ==
 
== Steps ==
Line 70: Line 77:
 
=== Detection of gold beads ===
 
=== Detection of gold beads ===
  
−
== Indexing ==
+
[[ File:alignGUI_completedStepsBecomeBoldFontSize.png |thumb|center| 600px|completed steps become bold font size]]
  
−
== Alignment ==
 
−
=== Fixed the markers ===
 
−
 
−
== CTF  ==
 
−
In this GUI CTF estimation and correction are delegated to CTFFIND4 and Imod's ctfphaseflip programs respectively.
 
−
 
−
== Reconstruction ==
 
−
 
−
=== Binned reconstruction ===
 
  
−
=== Full sized reconstruction ===
 
−
[[ File:alignGUI_creatingAWorkflow.png |thumb|center| 600px|creating a workflow]]
 
−
[[ File:alignGUI_inputWorkflowParameters.png |thumb|center| 600px|input workflow parameters]]
 
−
[[ File:alignGUI_creationOfWorkflowGUITakesSomeSeconds.png |thumb|center| 600px|creation of workflow GUI takes some seconds]]
 
−
[[ File:alignGUI_alignmentGUIBeforeCompletionOfAnyStep.png |thumb|center| 600px|alignment GUI before completion of any step]]
 
−
[[ File:alignGUI_passingTheCursorOnTheGUIElementsShowsAHelpElement.png |thumb|center| 600px|passing the cursor on the GUI elements shows a help element  ]]
 
−
[[ File:alignGUI_completedStepsBecomeBoldFontSize.png |thumb|center| 600px|completed steps become bold font size]]
 
 
[[ File:alignGUI_thePlusIconAtCornerShowsTheValuesOfTheParametersOfAGivenStep.png |thumb|center| 600px|the plus icon at corner shows the values of the parameters of a given step]]
 
[[ File:alignGUI_thePlusIconAtCornerShowsTheValuesOfTheParametersOfAGivenStep.png |thumb|center| 600px|the plus icon at corner shows the values of the parameters of a given step]]
 
[[ File:alignGUI_openTomoshowOnTheFullSizedMatrixToCheckTheSizeOfAGoldBead.png |thumb|center| 600px|open tomoshow on the full sized matrix to check the size of a gold bead]]
 
[[ File:alignGUI_openTomoshowOnTheFullSizedMatrixToCheckTheSizeOfAGoldBead.png |thumb|center| 600px|open tomoshow on the full sized matrix to check the size of a gold bead]]
Line 102: Line 93:
 
[[ File:alignGUI_toolboxOptionForTestingTheParametersOnACentralImage.png |thumb|center| 600px|toolbox option for testing the parameters on a central image]]
 
[[ File:alignGUI_toolboxOptionForTestingTheParametersOnACentralImage.png |thumb|center| 600px|toolbox option for testing the parameters on a central image]]
 
[[ File:alignGUI_resultsOfTestingOnACentralImage.png |thumb|center| 600px|results of testing on a central image]]
 
[[ File:alignGUI_resultsOfTestingOnACentralImage.png |thumb|center| 600px|results of testing on a central image]]
 +
 +
=== Computation of observation features s===
 
[[ File:alignGUI_patchGalleryForFeatures.png |thumb|center| 600px|patch gallery for features]]
 
[[ File:alignGUI_patchGalleryForFeatures.png |thumb|center| 600px|patch gallery for features]]
 
[[ File:alignGUI_theGalleryAllowsRelatingAppearanceOfGoldBeadAnValueOfTheFeatures.png |thumb|center| 600px|the gallery allows relating appearance of gold bead an value of the features]]
 
[[ File:alignGUI_theGalleryAllowsRelatingAppearanceOfGoldBeadAnValueOfTheFeatures.png |thumb|center| 600px|the gallery allows relating appearance of gold bead an value of the features]]
Line 109: Line 102:
 
[[ File:alignGUI_coloringOptionForScatteredPlots.png |thumb|center| 600px|coloring option for scattered plots]]
 
[[ File:alignGUI_coloringOptionForScatteredPlots.png |thumb|center| 600px|coloring option for scattered plots]]
 
[[ File:alignGUI_eachPointRepresentsACcPeak.png |thumb|center| 600px|each point represents a cc peak]]
 
[[ File:alignGUI_eachPointRepresentsACcPeak.png |thumb|center| 600px|each point represents a cc peak]]
 +
 +
=== Selection of best gold beads===
 
[[ File:alignGUI_checkObservationCloudAfterSelectionOrClustering.png |thumb|center| 600px|check observation cloud after selection or clustering ]]
 
[[ File:alignGUI_checkObservationCloudAfterSelectionOrClustering.png |thumb|center| 600px|check observation cloud after selection or clustering ]]
 
[[ File:alignGUI_observationCloudHasBeenThinned.png |thumb|center| 600px|observation cloud has been thinned]]
 
[[ File:alignGUI_observationCloudHasBeenThinned.png |thumb|center| 600px|observation cloud has been thinned]]
 
[[ File:alignGUI_checkMarkerPositionsAfterClustering.png |thumb|center| 600px|check marker positions after clustering]]
 
[[ File:alignGUI_checkMarkerPositionsAfterClustering.png |thumb|center| 600px|check marker positions after clustering]]
 
[[ File:alignGUI_markerPositionsAfterClustering.png |thumb|center| 600px|marker positions after clustering]]
 
[[ File:alignGUI_markerPositionsAfterClustering.png |thumb|center| 600px|marker positions after clustering]]
 +
 +
 +
== Indexing ==
 +
 +
=== Rough Alignment ===
 
[[ File:alignGUI_occupancySchemeOfTheResultsOfAStep.png |thumb|center| 600px|occupancy scheme of the results of a step]]
 
[[ File:alignGUI_occupancySchemeOfTheResultsOfAStep.png |thumb|center| 600px|occupancy scheme of the results of a step]]
 
[[ File:alignGUI_roughAlignmentSelectsManyPotentialTrails.png |thumb|center| 600px|rough alignment selects many potential trails ]]
 
[[ File:alignGUI_roughAlignmentSelectsManyPotentialTrails.png |thumb|center| 600px|rough alignment selects many potential trails ]]
 +
 +
=== Selection of stable trails===
 
[[ File:alignGUI_trailsWithFewMarkersAreThenRejected.png |thumb|center| 600px|trails with few markers are then rejected]]
 
[[ File:alignGUI_trailsWithFewMarkersAreThenRejected.png |thumb|center| 600px|trails with few markers are then rejected]]
 
[[ File:alignGUI_noteWorkingMarkersCanBeRestoredToTheResultOfAPreviousStep.png |thumb|center| 600px|note working markers can be restored to the result of a previous step]]
 
[[ File:alignGUI_noteWorkingMarkersCanBeRestoredToTheResultOfAPreviousStep.png |thumb|center| 600px|note working markers can be restored to the result of a previous step]]
 +
 +
 +
=== Iterative reindexing ===
 
[[ File:alignGUI_parametersForIterativeReindexing.png |thumb|center| 600px|parameters for iterative reindexing]]
 
[[ File:alignGUI_parametersForIterativeReindexing.png |thumb|center| 600px|parameters for iterative reindexing]]
 
[[ File:alignGUI_resultOfIterativeReindexing.png |thumb|center| 600px|result of iterative reindexing]]
 
[[ File:alignGUI_resultOfIterativeReindexing.png |thumb|center| 600px|result of iterative reindexing]]
 +
 +
===Tilt gap filling ===
 
[[ File:alignGUI_parametersForTiltGapFiller.png |thumb|center| 600px|parameters for tilt gap filler]]
 
[[ File:alignGUI_parametersForTiltGapFiller.png |thumb|center| 600px|parameters for tilt gap filler]]
 
[[ File:alignGUI_occupancyAfterTiltGapFiller.png |thumb|center| 600px|occupancy after tilt gap filler]]
 
[[ File:alignGUI_occupancyAfterTiltGapFiller.png |thumb|center| 600px|occupancy after tilt gap filler]]
 
[[ File:alignGUI_weHave66TracesAfterGapFiller.png |thumb|center| 600px|we have 66 traces after gap filler ]]
 
[[ File:alignGUI_weHave66TracesAfterGapFiller.png |thumb|center| 600px|we have 66 traces after gap filler ]]
 
[[ File:alignGUI_resultsOfTiltGapFiller.png |thumb|center| 600px|results of tilt gap filler]]
 
[[ File:alignGUI_resultsOfTiltGapFiller.png |thumb|center| 600px|results of tilt gap filler]]
 +
 +
===Trail extension===
 
[[ File:alignGUI_parametersOfTiltExtensor.png |thumb|center| 600px|parameters of tilt extensor]]
 
[[ File:alignGUI_parametersOfTiltExtensor.png |thumb|center| 600px|parameters of tilt extensor]]
 
[[ File:alignGUI_trailExtensorAddsAFewNewTrails.png |thumb|center| 600px|trail extensor adds a few new trails]]
 
[[ File:alignGUI_trailExtensorAddsAFewNewTrails.png |thumb|center| 600px|trail extensor adds a few new trails]]
 
[[ File:alignGUI_someOfTheNewTrailsHaveEmptySpaces.png |thumb|center| 600px|some of the new trails have empty spaces]]
 
[[ File:alignGUI_someOfTheNewTrailsHaveEmptySpaces.png |thumb|center| 600px|some of the new trails have empty spaces]]
 
[[ File:alignGUI_resultOfTrailExtensor.png |thumb|center| 600px|result of trail extensor]]
 
[[ File:alignGUI_resultOfTrailExtensor.png |thumb|center| 600px|result of trail extensor]]
 +
 +
 +
===Refinement ===
 
[[ File:alignGUI_tabForRefinement.png |thumb|center| 600px|tab for refinement]]
 
[[ File:alignGUI_tabForRefinement.png |thumb|center| 600px|tab for refinement]]
 
[[ File:alignGUI_allStepsOfRefinementAreaAreBeingComputed.png |thumb|center| 600px|all steps of refinement area are being computed]]
 
[[ File:alignGUI_allStepsOfRefinementAreaAreBeingComputed.png |thumb|center| 600px|all steps of refinement area are being computed]]
Line 132: Line 144:
 
[[ File:alignGUI_trimmingOfMarkersCanBeAccessedThroughTheScissorIcon.png |thumb|center| 600px|trimming of markers can be accessed through the scissor icon]]
 
[[ File:alignGUI_trimmingOfMarkersCanBeAccessedThroughTheScissorIcon.png |thumb|center| 600px|trimming of markers can be accessed through the scissor icon]]
 
[[ File:alignGUI_residualsAtEachObservation.png |thumb|center| 600px|residuals at each observation]]
 
[[ File:alignGUI_residualsAtEachObservation.png |thumb|center| 600px|residuals at each observation]]
 +
 +
== Alignment ==
 +
 +
=== Fix the markers ===
 +
 
[[ File:alignGUI_areaForCreationOfAnAlignedTiltSeries.png |thumb|center| 600px|area for creation of an aligned tilt series]]
 
[[ File:alignGUI_areaForCreationOfAnAlignedTiltSeries.png |thumb|center| 600px|area for creation of an aligned tilt series]]
 +
 +
=== Align the tilt series ===
 
[[ File:alignGUI_visualizationOfAlignedTiltStack.png |thumb|center| 600px|visualization of aligned tilt stack]]
 
[[ File:alignGUI_visualizationOfAlignedTiltStack.png |thumb|center| 600px|visualization of aligned tilt stack]]
 
[[ File:alignGUI_binnedAlignedTiltStack.png |thumb|center| 600px|binned aligned tilt stack]]
 
[[ File:alignGUI_binnedAlignedTiltStack.png |thumb|center| 600px|binned aligned tilt stack]]
 +
 +
== CTF  ==
 +
In this GUI CTF estimation and correction are delegated to CTFFIND4 and Imod's ctfphaseflip programs respectively.
 +
''This part will not be used in the workshop.'''
 +
 +
== Reconstruction ==
 +
 +
=== Binned reconstruction ===
 
[[ File:alignGUI_reconstructionArea.png |thumb|center| 600px|reconstruction area]]
 
[[ File:alignGUI_reconstructionArea.png |thumb|center| 600px|reconstruction area]]
 
[[ File:alignGUI_chooseAVisualization.png |thumb|center| 600px|choose a visualization]]
 
[[ File:alignGUI_chooseAVisualization.png |thumb|center| 600px|choose a visualization]]
 
[[ File:alignGUI_binnedWeightedBackProjectionReconstruction.png |thumb|center| 600px|binned  weighted back projection reconstruction]]
 
[[ File:alignGUI_binnedWeightedBackProjectionReconstruction.png |thumb|center| 600px|binned  weighted back projection reconstruction]]
 +
 +
=== Full sized reconstruction ===
 
[[ File:alignGUI_thePinToolIsUsefulForLocatingIsolatedPoints.png |thumb|center| 600px|the pin tool is useful for locating isolated points]]
 
[[ File:alignGUI_thePinToolIsUsefulForLocatingIsolatedPoints.png |thumb|center| 600px|the pin tool is useful for locating isolated points]]
 
[[ File:alignGUI_wePutAPinInTheCenterOfAnObjectOfInterest.png |thumb|center| 600px|we put a pin in the center of an object of interest]]
 
[[ File:alignGUI_wePutAPinInTheCenterOfAnObjectOfInterest.png |thumb|center| 600px|we put a pin in the center of an object of interest]]
Line 145: Line 174:
 
[[ File:alignGUI_theFullSizeReconstructionIncludesOnlyTheObjectOfInterest.png |thumb|center| 600px|the full size reconstruction includes only the object of interest]]
 
[[ File:alignGUI_theFullSizeReconstructionIncludesOnlyTheObjectOfInterest.png |thumb|center| 600px|the full size reconstruction includes only the object of interest]]
 
[[ File:alignGUI_YouMightNeedToActivateTheBandpassToSeeTheVLP.png |thumb|center| 600px|You might need to activate the bandpass to see the VLP ]]
 
[[ File:alignGUI_YouMightNeedToActivateTheBandpassToSeeTheVLP.png |thumb|center| 600px|You might need to activate the bandpass to see the VLP ]]
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
]

Revision as of 08:33, 27 August 2019

Dynamo includes a package for automated aligned and reconstruction of tilt series. This walkthrough guides you through the steps on how to use it with a GUI. A different page in this wiki

Data

We have prepared version a tilt series from the EMPIAR entry [XXXX] 

  wget  https://wiki.dynamo.biozentrum.unibas.ch/html/w/doc/data/hiv/align/b001ts001.mrc

In this binned version, the pixel size is 2'7 Angstrom (original was 1.35, this is bin level 1 in Dynamo)

Creating an alignment workflow

The basic function for invoking the GUI is dtsa (short form of dynamo_tilt_series_alignment; 

 u = dtsa('folder','firstAlign);
creating a workflow

The data for the workflow (tilt series, angles) could have been introduced directly when creating the workflow, we however will introduce each item explicitly in this walkthrough.


input workflow parameters

GUI description

The GUI guides of the process of creating and editing a set of "working markers" (in the areas of detection, reindexing and refinement), and then using them to create aligned stacks (alignment tab), analyse and correct their CTF (through wrappers to ctfffind4 and imod) and then create reconstructions.

alignment GUI before completion of any step

Settings

Acqusition settings

Here you enter pixel size (2.7 Angstroms), Cs (2.2), nominal defocus (-4.0 microns, used if the CTF area is connected).

Controls

markers GUI

The icon of the markers GUI allows you to check the position of the working markers in the micrograph. If you want to make editions of the GUI permanent, you need to save the current markers.

Occupancy plot

Shows a scheme of which markers are present in which tilt numbers.

Gold bead gallery

Crops the gold beads currently contained in the working markers and shows them as a gallery.

Execution Areas

The GUI is divided in five areas, which are sets of steps that can be executed sequentially. Sequential execution is the natural way of proceeding; but steps can be visited in any order, i.e., a step can be reexecuted in a later moment after updating its information.

passing the cursor on the GUI elements shows a help element

Steps

Steps needs to be open (letters in grey) in order to be executed. They become open when the input they require gets computed and stored during execution of the workflow. When a step is under execution, its pushbutton becomes green.

Step Toolbox

The tool icon can be secondary clicked and will provide a popup menu with specific tools for each steps and some common one. The common ones are handles to inspect and list the items that are needed (input items) or generated (output items) by the step. The View provides tools specific for visualisation of results of the step.

Entering the data

The minimal data to initiate. w work

  • tilt series matrix.
  • tilt angles.
  • discarded tilt angles

Discarded tilt angles

Tilt angles to be discarded can be selected directly with the dmarkers GUI using the x and shift + x buttons.


The areas in the alignment task

Detection

Visualization matrix

This step just creates a binned version of the tilt series stack to accelerate visualizations that will be invoked in a later point.

Detection of gold beads

completed steps become bold font size


the plus icon at corner shows the values of the parameters of a given step
open tomoshow on the full sized matrix to check the size of a gold bead
the ruler tool allows to measure in pixels the diameter of a gold bead
secondary click in the view panel opens a menu to access the different results of a step
marker sets can be viewed as cloud of points
cloud of unindexed cc peaks
shows the detected spots on the video GUI
static video annotations cannot be edited
Each patch will be a putative gold bead
the set of cc peaks is shown as a montage
example of failed search
toolbox option for testing the parameters on a central image
results of testing on a central image

Computation of observation features s

patch gallery for features
the gallery allows relating appearance of gold bead an value of the features
feature plotter allows checking of the feature values on the whole set
different plotting options
scatter 2d option
coloring option for scattered plots
each point represents a cc peak

Selection of best gold beads

check observation cloud after selection or clustering
observation cloud has been thinned
check marker positions after clustering
marker positions after clustering


Indexing

Rough Alignment

occupancy scheme of the results of a step
rough alignment selects many potential trails

Selection of stable trails

trails with few markers are then rejected
note working markers can be restored to the result of a previous step


Iterative reindexing

parameters for iterative reindexing
result of iterative reindexing

Tilt gap filling

parameters for tilt gap filler
occupancy after tilt gap filler
we have 66 traces after gap filler
results of tilt gap filler

Trail extension

parameters of tilt extensor
trail extensor adds a few new trails
some of the new trails have empty spaces
result of trail extensor


Refinement

tab for refinement
all steps of refinement area are being computed
better fiiting is attained
trimming of markers can be accessed through the scissor icon
residuals at each observation

Alignment

Fix the markers

area for creation of an aligned tilt series

Align the tilt series

visualization of aligned tilt stack
binned aligned tilt stack

CTF

In this GUI CTF estimation and correction are delegated to CTFFIND4 and Imod's ctfphaseflip programs respectively. This part will not be used in the workshop.'

Reconstruction

Binned reconstruction

reconstruction area
choose a visualization
binned weighted back projection reconstruction

Full sized reconstruction

the pin tool is useful for locating isolated points
we put a pin in the center of an object of interest
we check the distance of the object of interest
we use the option of expressing the center of the full sized tomogram in terms of a position in the binned tomogram
as an specific viewer is not available for this step we can use the toolbox of the steps items
the full size reconstruction includes only the object of interest
You might need to activate the bandpass to see the VLP









]

Retrieved from "https://www.dynamo-em.org//w/index.php?title=Walkthrough_on_GUI_based_tilt_series_alignment&oldid=3264"