From a859d935c6fc0d9ba2db9be1233120b5ea196ee8 Mon Sep 17 00:00:00 2001 From: Peter Petersen Date: Tue, 7 Apr 2020 23:06:50 -0400 Subject: [PATCH 1/5] docu update --- docs/datastructure/data-structure-and-format.md | 6 +++--- docs/pipeline/custom-calculations.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/datastructure/data-structure-and-format.md b/docs/datastructure/data-structure-and-format.md index d3117d03..ea518cd6 100644 --- a/docs/datastructure/data-structure-and-format.md +++ b/docs/datastructure/data-structure-and-format.md @@ -115,7 +115,7 @@ A Matlab struct `session` stored in a .mat file: `sessionName.session.mat`. The * `equipment` : hardware used to acquire the data ### Spikes -A Matlab struct `spikes` stored in a .mat file: `sessionName.spikes.cellinfo.mat`. It can be generated with [loadSpikes.m](https://github.com/petersenpeter/Cell-Explorer/blob/master/calc_CellMetrics/loadSpikes.m). The Cell Inspector's pipeline `calc_CellMetrics.m` used the script `loadSpikes.m`, to automatically load spike-data from either KiloSort, Phy or Neurosuite and saves it to a spikes struct. `sessionName.spikes.cellinfo.mat` should be located in the clustering path. The struct has the following fields: +A Matlab struct `spikes` stored in a .mat file: `sessionName.spikes.cellinfo.mat`. It can be generated with [loadSpikes.m](https://github.com/petersenpeter/Cell-Explorer/blob/master/calc_CellMetrics/loadSpikes.m). The Cell Inspector's pipeline `ProcessCellMetrics.m` used the script `loadSpikes.m`, to automatically load spike-data from either KiloSort, Phy or Neurosuite and saves it to a spikes struct. `sessionName.spikes.cellinfo.mat` should be located in the clustering path. The struct has the following fields: * `ts`: a 1xN cell-struct for N units each containing a 1xM vector with M spike events in samples. * `times`: a 1xN cell-struct for N units each containing a 1xM vector with M spike events in seconds. * `cluID`: a 1xN vector with inherited IDs from the applied clustering algorithm. @@ -158,7 +158,7 @@ A Matlab struct `eventName` stored in a .mat file: `sessionName.eventName.events * `duration`: duration of event (in seconds; calculated from timestamps; Px1). * `detectorinfo`: info about how the events were detected. -The `*.events.mat` files should be stored in the basepath. Any `events` files located in the basepath will be detected in the pipeline (calc_CellMetrics) and an average PSTHs will be generated. +The `*.events.mat` files should be stored in the basepath. Any `events` files located in the basepath will be detected in the pipeline (ProcessCellMetrics.m) and an average PSTHs will be generated. ### Manipulations A Matlab struct `manipulationName` stored in a .mat file: `sessionName.eventName.manipulation.mat` with the following fields: @@ -173,7 +173,7 @@ A Matlab struct `manipulationName` stored in a .mat file: `sessionName.eventName * `duration`: duration of event (in seconds; calculated from timestamps; Px1). * `detectorinfo`: info about how the events were detected. -The `*.manipulation.mat` files should be stored in the basepath. `events` and `manipulation` files are similar in content, but only manipulation intervals are excluded in the pipeline. Any `manipulation` files located in the basepath will be detected in the pipeline (calc_CellMetrics) and an average PSTH will be generated. Events and manipulation files are similar in content, but only manipulation intervals are excluded in the pipeline. +The `*.manipulation.mat` files should be stored in the basepath. `events` and `manipulation` files are similar in content, but only manipulation intervals are excluded in the pipeline. Any `manipulation` files located in the basepath will be detected in the pipeline (ProcessCellMetrics.m) and an average PSTH will be generated. Events and manipulation files are similar in content, but only manipulation intervals are excluded in the pipeline. ### Channels A matlab struct `ChannelName` stored in a .mat file: `sessionName.ChannelName.channelinfo.mat` with the following fields: diff --git a/docs/pipeline/custom-calculations.md b/docs/pipeline/custom-calculations.md index c156e165..403999e4 100644 --- a/docs/pipeline/custom-calculations.md +++ b/docs/pipeline/custom-calculations.md @@ -6,7 +6,7 @@ nav_order: 5 --- # Custom calculations {: .no_toc} -The Cell Explorer pipeline has a subfolder for calculations to exist outside the main pipeline, such that updates can be applied without affecting your own additions to the pipeline. Please save your scripts to the folder `calc_CellMetrics/+customCalculations/` and follow the template already in that folder to integrate your own calculations into the regular pipeline. +The Cell Explorer pipeline has a subfolder for calculations to exist outside the main pipeline, such that updates can be applied without affecting your own additions to the pipeline. Please save your scripts to the folder `+customCalculations/` and follow the template already in that folder to integrate your own calculations into the regular pipeline. Your metrics has to follow the Cell Explorer [cell_metrics standard]({{"/datastructure/your-own-metrics/"|absolute_url}}). Any `events` or `manipulation` files located in the basepath will be detected in the pipeline and PSTHs will be generated automatically. Events and manipulation files are similar in content, but only manipulation intervals are excluded in the pipeline. From 9b9e88ff3d27d997b4b892c728aa8b0b05d2d316 Mon Sep 17 00:00:00 2001 From: Peter Petersen Date: Wed, 8 Apr 2020 08:02:23 -0400 Subject: [PATCH 2/5] docu update --- docs/datastructure/standard-cell-metrics.md | 7 ++++--- docs/interface/capabilities.md | 11 ++++++----- docs/pipeline/running-pipeline.md | 4 ++-- docs/tutorials/ground-truth-tutorial.md | 4 ++-- docs/tutorials/optotagging-tutorial.md | 16 +++++++++------- 5 files changed, 23 insertions(+), 19 deletions(-) diff --git a/docs/datastructure/standard-cell-metrics.md b/docs/datastructure/standard-cell-metrics.md index 41dfa037..11cfc07d 100644 --- a/docs/datastructure/standard-cell-metrics.md +++ b/docs/datastructure/standard-cell-metrics.md @@ -135,6 +135,7 @@ The spatial metrics are all based on average firing rate map. ## Response curve metrics * `responseCurves`: response curves. -## Ground truth metrics -* `groundTruthClassification`: Opto-tagged/ground truth cell tags. More than one tag can be assigned to each cell. - +## Group data +* `groups`: Cell groups. Each cell can be assigned to one or more groups. +* `tags`: Each cell can be assigned to one or more tags. +* `groundTruthClassification`: Opto-tagged/ground truth cell groups. Each cell can be assigned to one or more groups. diff --git a/docs/interface/capabilities.md b/docs/interface/capabilities.md index 0ac2f797..c27394a7 100644 --- a/docs/interface/capabilities.md +++ b/docs/interface/capabilities.md @@ -20,7 +20,8 @@ You can do direct classification in the GUI. The following types of classificati * **Brain region**: Allen institute atlas. * **Deep-superficial**: Deep superficial assignment can be done in the Cell Explorer cell-wise and in a separate gui channel-wise. * **Labels**: You can assign your own labels to any cell. -* **Tags**: A selection of predetermined tags can also be assigned. +* **Tags**: Tags can be assigned. +* **Groups**: Groups can be created. * **Ground truth cell types**: Ground truth data can be analysed directly in the GUI. ### Interface for deep-superfial classification curation @@ -63,7 +64,7 @@ You can save your combined cell metrics from a study into a single mat file that ### Export figures There are two ways to export figures. -1. You can export the whole interface from the top menu from the top menu `File` -> `Export figure`. This will open the Matlab Figure Export Setup dialog box (`exportsetupdlg`). +1. You can export the whole interface from the top menu from the top menu `File` -> `Export figure`. This will open the Matlab Figure Export Setup dialog box `exportsetupdlg`. 2. Single cell figures 1. Select a number of cells, using the mouse and press `space`, this opens the action dialog. If no selection is done before pressing `space` a selection dialog will be shown.

@@ -72,7 +73,7 @@ There are two ways to export figures.

### Work in batch-mode while handling metrics on a single session level -Using the Cell Explorer on a batch of sessions, will load metrics into one struct allowing you to visualize and classify your data across recordings and classify cells across sessions, while still maintaining the data handling on a single session level, writing your changes back to the original files. You can save metrics from a batch of sessions, and still load the data back into the Cell Explorer. +The Cell Explorer can handle batches of sessions. It will load metrics into one struct allowing you to visualize and classify your data across recordings and classify cells across sessions, while still maintaining the data handling on a single session level, writing your changes back to the original files. You can save metrics from a batch of sessions, and still load the data back into the Cell Explorer. -### Autosave -The Cell Explorer automatically saves your manual curation every 6 classification action (actions include changes to cell-type, deep-superficial assignment and brain region). You can turn this feature off or adjust the autosave-interval in preferences. The autosave only saves your progress to the workspace and you have to save your changes to the original cell_metrics file through the Cell Explorer interface. +### Track changes and autosave +The Cell Explorer tracks your actions, which includes cell-type classifications, deep-superficial assignment, brain regions, labels, tags, groups and ground truth classifications. Reverse an action by pressing `ctrl+Z`. Further it autosaves your actions to your workspace every 6th action (You can turn the autosave feature off or adjust the autosave-interval in preferences). You still have to save your changes to the original cell_metrics file through the Cell Explorer interface. diff --git a/docs/pipeline/running-pipeline.md b/docs/pipeline/running-pipeline.md index 44454e54..da4a9f4b 100644 --- a/docs/pipeline/running-pipeline.md +++ b/docs/pipeline/running-pipeline.md @@ -22,8 +22,8 @@ First step is creating the session struct. This struct contains all metadata nec ![](https://buzsakilab.com/wp/wp-content/uploads/2020/03/FlowChart_sessionStruct.png) -### Running pipeline -Following the definition of metadata, the cell metrics calculation process can be performed. A single script processes all default cell_metrics (which can be customized and expanded). The process is fully automatic, except for the detection of monosynaptic connections, which displays a graphical interface for further manual curation (manual curation can be turned off). See the [full list of default cell_metrics here]({{"/datastructure/standard-cell-metrics/"|absolute_url}}). Below follows two flowcharts: a simple with the minimal inputs and an advanced flowchart. The advanced chart shows all relevant files that is loaded by the cell_metrics calculation process. +### Processing cell_metrics +Following the definition of metadata, the cell metrics calculation process can be performed. A single script processes all default cell_metrics (which can be customized and expanded). The process is fully automatic, except for the detection of monosynaptic connections, in which a graphical interface is shown for further manual curation (the manual step can be turned off). See the [full list of default cell_metrics here]({{"/datastructure/standard-cell-metrics/"|absolute_url}}). Below follows two flowcharts: a simple with the minimal inputs and an advanced flowchart. The advanced chart shows all relevant files that is loaded by the cell_metrics calculation process. ![](https://buzsakilab.com/wp/wp-content/uploads/2020/03/FlowChart_pipeline.png) diff --git a/docs/tutorials/ground-truth-tutorial.md b/docs/tutorials/ground-truth-tutorial.md index bfb41301..2301dd52 100644 --- a/docs/tutorials/ground-truth-tutorial.md +++ b/docs/tutorials/ground-truth-tutorial.md @@ -7,10 +7,10 @@ nav_order: 7 # Ground truth data tutorial {: .no_toc} -The Cell Explorer contains a select set of ground truth data located in `groundTruthData/`. This tutorial will guide you through using the ground truth data included with the Cell Explorer. +The Cell Explorer contains a select set of ground truth data located in `+groundTruthData/`. This tutorial will guide you through using the ground truth data included with the Cell Explorer. 1. Launch the Cell Explorer -2. From the top menu `Ground truth`, select `Define ground truth data`. This will display the dialog below with a list of ground truth cells from the `groundTruthData/` folder. The data is orgazied by sessions, where each session contains at least one tagged cell but can contain more. +2. From the top menu `Ground truth`, select `Define ground truth data`. This will display the dialog below with a list of ground truth cells from the `+groundTruthData/` folder. The data is orgazied by sessions, where each session contains at least one tagged cell but can contain more. ![](https://buzsakilab.com/wp/wp-content/uploads/2020/03/GroundTruthCellsDialog.png) 3. Select the cells you would like to load as ground truth data, press OK and the data will be loaded. 4. From the `Ground truth` menu, you can select how to display the ground truth data: as scatter points, as a density map (image), or double histograms. diff --git a/docs/tutorials/optotagging-tutorial.md b/docs/tutorials/optotagging-tutorial.md index 21d2c89f..5e96a0bc 100644 --- a/docs/tutorials/optotagging-tutorial.md +++ b/docs/tutorials/optotagging-tutorial.md @@ -6,7 +6,7 @@ nav_order: 6 --- # Opto-tagging tutorial {: .no_toc} -This tutorial will guide you through the process of tagging your cells by assigning groundTruthClassification-tags to your data in the Cell Explorer. If you have data that you are interested in sharing please contact us. You can push ground truth cells back the the Cell Explorer GitHub repository, or you can send your data by email to us, see instructions below. Cells assigned as ground truth will have a classification label in `cell_metrics.groundTruthClassification`. Opto-tagged/ground truth cells can have one or more labels. +This tutorial will guide you through the process of tagging your cells by assigning groundTruthClassification-tags to your data in the Cell Explorer. If you have data that you are interested in sharing please contact us. You can push ground truth cells back the the Cell Explorer GitHub repository, or you can send your data by email to us, see instructions below. Cells assigned as ground truth will have a classification label in `cell_metrics.groundTruthClassification`. Opto-tagged/ground truth cells can be assigned to one or more groups. ## Table of contents {: .no_toc .text-delta } @@ -15,17 +15,19 @@ This tutorial will guide you through the process of tagging your cells by assign {:toc} ### Add your opto-tagged cells to the `cell_metrics` struct -Opto-tagged/ground truth cells have one or more labels in `cell_metrics.groundTruthClassification`. If you already have determined the identity of your cells you can simply add them to the `cell_metrics.groundTruthClassification`: +Opto-tagged/ground truth cells can be assigned to one or more groups in `cell_metrics.groundTruthClassification`. If you already have determined the identity of your cells you can simply add them to `cell_metrics.groundTruthClassification`: ```m cellIDs_optoTagged = [1,5,10]; % IDs (UIDs) of the opto-tagged cells -name_optoTagged = 'PV+'; % Cell line name -cell_metrics.groundTruthClassification(cellIDs_optoTagged) = repmat({ { name_optoTagged } },length(cellIDs_optoTagged),1); +name_optoTagged = 'PV_pos'; % Cell line name +cell_metrics.groundTruthClassification.(name_optoTagged) = cellIDs_optoTagged; +% or: +cell_metrics.groundTruthClassification.PV_pos = [1,5,10]; ``` ### Tagging your cells in the Cell Explorer 1. Launch the Cell Explorer. 2. Activate the manual curation of ground truth classification from the top menu `Ground truth` -> `Perform ground truth cell type classification in current session(s)`. This opens a tab group in the Cell Assignment tab menu titled `G/T` in the right side-panel. -3. Adjust the highlighted cells using the menu option `Ground truth` -> `Show ground truth cell types in current session(s)`. This plot option is separate from the plotting option for centralized ground truth data. +3. Adjust the highlighted cells using the menu option `Group data` -> `Open group data dialog`. This dialog allows you to define how to visualize your tagged cells. 4. Assign the ground truth tag label to your cells. You can add more tags in the Cell Explorer and in the preference file `CellExplorer_Preferences.m`. Each cell can have one or more ground truth classification tags assigned. 5. Once complete, save the session using the top menu `File` -> `Save classification`. @@ -35,9 +37,9 @@ cell_metrics.groundTruthClassification(cellIDs_optoTagged) = repmat({ { name_opt Following the process described in the previous sections on tagging your cells, you can submit your opto-tagged cells to the `groundTruth` folder through the Cell Explorer UI, allowing you to share your cells and use them across sessions. 1. Launch the Cell Explorer. -1. Go to the `Ground truth` top menu and select `Save manual classification to groundTruth folder` to submit your ground truth cells to the `groundTruthData/` data folder (centralized ground truth data). The `groundTruthData/` folder is organized by sessions. +1. From the `Ground truth` top menu select `Save tagging to groundTruthData folder` to submit your ground truth cells to the `+groundTruthData/` data folder (centralized ground truth data). The files in the `+groundTruthData/` folder is organized by sessions. ### Submit your cells to the Cell Explorer repository You can submit your cells to the Cell Explorer repository such that other people can take advantage of your ground truth cells. This allows the community to share their tagged cells such that others can researchers can benefit. 1. __Push metrics back to main branch of the Cell Explorer Github repository__: If you cloned or forked the Cell Explorer Github repository, you can submit a pull-request to the main Cell Explorer branch. We will verify your data and submit then to the main branch of the Cell Explorer. -2. __Email metrics to us__: You can email your ground truth cells to us at petersen.peter@gmail.com. Provide the `cell_metrics` files saved to `groundTruthData/`. The ground truth cells are organized by sessions, where the files in `groundTruthData/` only contains cells with groundTruthClassification from the original session. We will verify your data and submit them to the main branch of the Cell Explorer. +2. __Email metrics to us__: You can email your ground truth cells to us at petersen.peter@gmail.com. Provide the `cell_metrics` files saved to `+groundTruthData/`. The ground truth cells are organized by sessions, where the files in `+groundTruthData/` only contains cells with groundTruthClassification from the original session. We will verify your data and submit them to the main branch of the Cell Explorer. From aa1e8ff918e1ce6929ad48686586d9ba5e5ad03a Mon Sep 17 00:00:00 2001 From: Peter Petersen Date: Wed, 8 Apr 2020 08:38:51 -0400 Subject: [PATCH 3/5] Update db_example.m --- db/db_example.m | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/db/db_example.m b/db/db_example.m index 397607b8..38702e5f 100644 --- a/db/db_example.m +++ b/db/db_example.m @@ -44,7 +44,7 @@ %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % Running the Cell Explorer pipeline via the db -cell_metrics = calc_CellMetrics('sessionName',sessionName); +cell_metrics = ProcessCellMetrics('sessionName',sessionName); cell_metrics = CellExplorer('metrics',cell_metrics); %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % From 30b2ab993bef49cd5028c50b4ab9522a2d7b1529 Mon Sep 17 00:00:00 2001 From: Peter Petersen Date: Wed, 8 Apr 2020 10:29:48 -0400 Subject: [PATCH 4/5] doc update --- docs/interface/capabilities.md | 21 +++++++-------------- docs/pipeline/pipeline.md | 2 +- docs/pipeline/running-pipeline.md | 17 ++++++++++------- docs/tutorials/export-figure.md | 26 ++++++++++++++++++++------ 4 files changed, 38 insertions(+), 28 deletions(-) diff --git a/docs/interface/capabilities.md b/docs/interface/capabilities.md index c27394a7..94f18345 100644 --- a/docs/interface/capabilities.md +++ b/docs/interface/capabilities.md @@ -14,6 +14,7 @@ The Cell Explorer is a graphical user interface that allow you to explore your d 1. TOC {:toc} + ## Classification You can do direct classification in the GUI. The following types of classification can be performed. * **Putative cell type**: You can create new cell types directly in the GUI. @@ -24,15 +25,15 @@ You can do direct classification in the GUI. The following types of classificati * **Groups**: Groups can be created. * **Ground truth cell types**: Ground truth data can be analysed directly in the GUI. -### Interface for deep-superfial classification curation -![](https://buzsakilab.com/wp/wp-content/uploads/2020/02/gui_deepSuperficial.png){: .mt-4} - ## Monosynaptic connections Monosynaptic connections are determined in the pipeline, and you can visualize the connections in the GUI and redo the manual curation directly from the GUI. You can adjust connections from the Cell Explorer by launching the monosyn interface. [Please see the tutorial on manual curation of monosynaptic connections]({{"/tutorials/monosynaptic-connections-tutorial/"|absolute_url}}). ### Interface for monosynaptic connections curation ![](https://buzsakilab.com/wp/wp-content/uploads/2020/02/monosyn.png) +### Interface for deep-superfial classification curation +![](https://buzsakilab.com/wp/wp-content/uploads/2020/02/gui_deepSuperficial.png){: .mt-4} + ## Database capabilities The Cell Explorer is capable of loading datasets from and writing to the Buzsaki lab database. Please setup your credentials and local paths as [described here]({{"/database/preparation/"|absolute_url}}). @@ -45,16 +46,15 @@ To help you characterize your own data, you can load reference data provided by There are a subset of ground truth cell types provided. ### Raincloud plot -To estimate single dimensional variations in your data you can generate a [raincloud plot](https://github.com/RainCloudPlots/RainCloudPlots). You can generate the plot from the top menu `View` -> `Generate rain cloud metrics plot`. +To quantify single dimensional variations in your data you can generate a [raincloud plot](https://github.com/RainCloudPlots/RainCloudPlots). You can generate the plot from the top menu `View` -> `Generate rain cloud metrics plot`. The comparison line widths signify significance levels, `linewidth=1` signifies p>0.05, `linewidth=2` signifies p<0.05 and `linewidth=3` signifies p<0.001. Significance levels is determined using [Two-sample Kolmogorov-Smirnov test](https://www.mathworks.com/help/stats/kstest2.html) (a nonparametric hypothesis test). You can generate a raincloud plot from any color grouping, e.g. cell types, deep-superficial or animal. Below plot shows a raincloud a comparison across putative cell types: ![raincloud cell types](https://buzsakilab.com/wp/wp-content/uploads/2020/02/raincloud-cell-types.png) - ### Significance matrix -The significance matrix can help find metrics that your data into groups, e.g. deep-superfical labels. You can generate the plot from the top menu `View` -> `Generate significance matrix` or by pressing `K`. Please select a group of size 2 beforehand. This will show a dialog for selecting which metrics to use. +The significance matrix can help quantify the modality of your data, e.g. using the deep-superficial labels or the cell types. You can generate the plot from the top menu `View` -> `Generate significance matrix` or by pressing `K`. Please select a group of size 2 beforehand. This will show a dialog for selecting which metrics to process.

The colors in the matrix signify significance level (right color bar in log10), `*` signifies p<0.05 and `**` signifies p<0.001. Selected metrics are shown on the left side of the matrix. Significance levels is determined using [Two-sample Kolmogorov-Smirnov test](https://www.mathworks.com/help/stats/kstest2.html) (a nonparametric hypothesis test). @@ -63,14 +63,7 @@ The colors in the matrix signify significance level (right color bar in log10), You can save your combined cell metrics from a study into a single mat file that can be shared together with a publication. This allows peers to verify your classification or use your cell metrics directly. You can save the mat file from the Cell Explorer from the menu `File` -> `Save classification`. ### Export figures -There are two ways to export figures. -1. You can export the whole interface from the top menu from the top menu `File` -> `Export figure`. This will open the Matlab Figure Export Setup dialog box `exportsetupdlg`. -2. Single cell figures - 1. Select a number of cells, using the mouse and press `space`, this opens the action dialog. If no selection is done before pressing `space` a selection dialog will be shown. -

- 2. Select either of the three `MULTI PLOT OPTIONS` - 3. In the new dialog, toogle `Save figures`, and define format and file path. -

+Figures can be exported using the GUI, either the main Cell Explorer window or through cell selection actions dialog. For more information please see the [figure export tutorial]({{"/tutorials/export-figure/"|absolute_url}}). ### Work in batch-mode while handling metrics on a single session level The Cell Explorer can handle batches of sessions. It will load metrics into one struct allowing you to visualize and classify your data across recordings and classify cells across sessions, while still maintaining the data handling on a single session level, writing your changes back to the original files. You can save metrics from a batch of sessions, and still load the data back into the Cell Explorer. diff --git a/docs/pipeline/pipeline.md b/docs/pipeline/pipeline.md index 272c5cd4..0c4da833 100644 --- a/docs/pipeline/pipeline.md +++ b/docs/pipeline/pipeline.md @@ -6,4 +6,4 @@ has_children: true --- # Processing pipeline {: .no_toc} -Every metric is calculated in the same processing pipeline from a single call. To run the pipeline one have to prepare the compatible data structure and metadata. You can add your own metrics and custom calculations to the pipeline. +The pipeline has three main processing steps: 1. Gathering metadata, 2. Processing cell_metrics and 3. Running Cell Explorer. All metrics are calculated using a single processing script. To run the pipeline one have to prepare the compatible data structure and metadata. You can add your own metrics and custom calculations to the pipeline. diff --git a/docs/pipeline/running-pipeline.md b/docs/pipeline/running-pipeline.md index da4a9f4b..eb56c14b 100644 --- a/docs/pipeline/running-pipeline.md +++ b/docs/pipeline/running-pipeline.md @@ -5,7 +5,10 @@ parent: Processing pipeline nav_order: 1 --- # Running pipeline -The pipeline has three main processing steps: 1. Gathering metadata, 2. Processing cell_metrics and 3. Running Cell Explorer. +The pipeline has three main processing steps: +1. Gathering metadata +2. Processing cell_metrics +3. Running Cell Explorer {: .no_toc} ## Table of contents @@ -15,10 +18,10 @@ The pipeline has three main processing steps: 1. Gathering metadata, 2. Processi {:toc} ## Flowcharts -The flowcharts below shows the processes in details. The boxes are color coded according to external files (blue), database (purple), script (green), Cell Explorer structs and .mat files (yellow). +The flowcharts below show the processes in details. The boxes are color coded according to external files (blue), database (purple), script (green), Cell Explorer structs and .mat files (yellow). ### Gathering metadata -First step is creating the session struct. This struct contains all metadata necessary for calculating the cell metrics. You can use the sessionTemplate to extract and define the parameters and visualize it with the graphical interface gui_session. The templates will scan the basepath for specific files to minimize the manual entry. You can customize the template script to fit and extract information relevant to your data. [The session struct is defined here]({{"/datastructure/data-structure-and-format/#session-metadata"|absolute_url}}). The session struct follows the database structure of the Buzsaki lab and all metadata can be loaded directly from the database for database sessions. See the example code below on how perform the actions in Matlab. +First step is creating the session struct. This struct contains all metadata necessary for calculating the cell metrics. You can use the `sessionTemplate` to extract and define the parameters and visualize them with the graphical interface `gui_session`. The templates will scan the basepath for specific files to minimize the manual entry. You can customize the template script to fit and extract information relevant to your data. [The session struct is defined here]({{"/datastructure/data-structure-and-format/#session-metadata"|absolute_url}}). The session struct follows the database structure of the Buzsaki Lab and all metadata can be loaded directly from the database for database sessions. See the example code below on how perform the actions in Matlab. ![](https://buzsakilab.com/wp/wp-content/uploads/2020/03/FlowChart_sessionStruct.png) @@ -28,7 +31,7 @@ Following the definition of metadata, the cell metrics calculation process can b ![](https://buzsakilab.com/wp/wp-content/uploads/2020/03/FlowChart_pipeline.png) ### Running Cell Explorer -The Cell Explorer can be used to display single cell_metrics files as well as batches. Batch loading is performed with the script LoadCellMetricsBatch. The advanced flowchart below further details the capabilities of loading various GUIs from the Cell Explorer (gui_session, gui_MonoSyn and gui_DeelSuperficial) as well as do spike raster plots, that requires access to the local spikes struct and potentially also manipulation and events files if plotting PSTHs. +The Cell Explorer can be used to display single cell_metrics files as well as batches. Batch loading is performed with the script LoadCellMetricsBatch. The advanced flowchart below further details the capabilities of loading various GUIs from the Cell Explorer (`gui_session`, `gui_MonoSyn` and `gui_DeelSuperficial`) as well as do spike raster plots, that requires access to the local spikes struct and potentially also manipulation and events files when plotting PSTHs. ![](https://buzsakilab.com/wp/wp-content/uploads/2020/03/FlowChart_CellExplorer.png) ## Running pipeline from a data path @@ -44,7 +47,7 @@ You can also view the session struct in a GUI: session = gui_session(session); ``` -To run the pipeline from the Matlab Command Window from the session struct type: +To run the processing script from the Matlab Command Window from the session struct type: ```m cell_metrics = ProcessCellMetrics('session', session); ``` @@ -52,7 +55,7 @@ You can also run it directly from a basepath and generate the session struct dir ```m cell_metrics = ProcessCellMetrics; ``` -When calling the pipeline with the sessionTemplate, a GUI will be shown allowing you to edit the metadata both for the input parameters and the session struct. +When calling the processing script with the sessionTemplate, a GUI will be shown allowing you to edit metadata, both input parameters and the session struct. Once complete, view the result in the Cell Explorer by typing: ```m @@ -85,4 +88,4 @@ sessionNames = {'sessionName1','sessionName2','sessionName3'}; cell_metrics = LoadCellMetricsBatch('sessions',sessionNames); cell_metrics = CellExplorer('metrics',cell_metrics); ``` -As you perform classifications in the Cell Explorer, you may save back to the original cell metrics stored with the sessions defined above. You can perform the batch mode from a list of paths as well. +As you perform classifications in the Cell Explorer in batch mode, you can save your progress to the original sessions. You can work in batch mode from a list of paths as well. diff --git a/docs/tutorials/export-figure.md b/docs/tutorials/export-figure.md index 775b63bf..e98e5c15 100644 --- a/docs/tutorials/export-figure.md +++ b/docs/tutorials/export-figure.md @@ -6,7 +6,10 @@ nav_order: 11 --- # Tutorial on exporting Cell Explorer figures {: .no_toc} -Exporting figures in Matlab can be a headache, so here is a small tutorial to help with this. The steps below shows how to save a PDF file of the main interface of the Cell Explorer. Saving a PNG (image file) is more straight forward. +Exporting figures in Matlab can be a headache, so here are two small tutorials to help with this: exporting the main interface and individual actions plots. + +## Exporting the Cell Explorer interface +The steps below shows how to save a PDF file of the main interface of the Cell Explorer. Saving a PNG (image file) is more straight forward. 1. Launch the Cell Explorer 2. Select `File`-> `Export figure` from the top menu. This will open a [Export Setup dialog](https://www.mathworks.com/help/matlab/ref/exportsetupdlg.html). Before the dialog is shown the paper size is set to the current figure size and the renderer is set to painter. @@ -16,12 +19,23 @@ Exporting figures in Matlab can be a headache, so here is a small tutorial to he 4. If you altered any settings, click the button `Apply to Figure`. 5. Click the button `Export...` to bring up the Save As dialog to specify location and file name. -If the export figure dialog is not sufficient for your need, you can bring up the main figure menu by pressing `m`. +When applying the settings to the figure (`Apply to Figure`), the figure sometimes resizes to a smaller initial size. Just resize the figure back to the full size before clicking `Export`. -When applying the settings to the figure (`Apply to Figure`), the figure sometimes resizes to a smaller initial size. Just resize the figure back to the full size before clicking `Export`. +If the export figure dialog is not sufficient for your need, you can bring up the main figure menu by pressing `m`. -Any other figures produced by the Cell Explorer can be saved in similar fashion using the File menu options `Save As` or `Export Setup...`. +Following the tutorial should provide you with a .pdf figure, looking like the figure below: +![Cell Explorer](https://buzsakilab.com/wp/wp-content/uploads/2019/11/Cell-Explorer-example.png) -Following the tutorial should provide you with a .pdf figure as shown below: +## Exporting figures using the action dialog +1. Select a set of cells, using the mouse and open the actions dialog (press `space` or the `Actions` button in the right panel). If no cell selection is done beforehand, a cell selection dialog will be shown first: +

+2. Select one of the three `MULTI PLOT OPTIONS` in the actions dialog. +3. In the multi plot dialog shown below, select the plots to generate, check the `Save figures` toggle, and define file format (.png or .pdf) and file path (Save to the clustering paths, to a Cell Explorer or a user defined path). : +

-![Cell Explorer](https://buzsakilab.com/wp/wp-content/uploads/2019/11/Cell-Explorer-example.png) +When you select to save your figures to the Cell Explorer path or Clustering path, they will be saved to a subfolder named `summaryFigures`. + +Be aware that saving pdf files, is substantial slower than saving png files, but the figures will be saved with vector graphics. + +## Exporting remaining figures +Any other figures produced by the Cell Explorer can be saved in similar fashion using the File menu options `Save As` or `Export Setup...`. From ef1a40fdc428855d70d28df747f5020affd8cb7c Mon Sep 17 00:00:00 2001 From: Peter Petersen Date: Wed, 8 Apr 2020 10:34:12 -0400 Subject: [PATCH 5/5] doc update --- docs/interface/capabilities.md | 8 +++----- docs/tutorials/export-figure.md | 8 +++++++- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/interface/capabilities.md b/docs/interface/capabilities.md index 94f18345..4c17fb67 100644 --- a/docs/interface/capabilities.md +++ b/docs/interface/capabilities.md @@ -25,16 +25,14 @@ You can do direct classification in the GUI. The following types of classificati * **Groups**: Groups can be created. * **Ground truth cell types**: Ground truth data can be analysed directly in the GUI. -## Monosynaptic connections +### Monosynaptic connections Monosynaptic connections are determined in the pipeline, and you can visualize the connections in the GUI and redo the manual curation directly from the GUI. You can adjust connections from the Cell Explorer by launching the monosyn interface. [Please see the tutorial on manual curation of monosynaptic connections]({{"/tutorials/monosynaptic-connections-tutorial/"|absolute_url}}). - -### Interface for monosynaptic connections curation -![](https://buzsakilab.com/wp/wp-content/uploads/2020/02/monosyn.png) +![](https://buzsakilab.com/wp/wp-content/uploads/2020/02/monosyn.png){: .mt-4} ### Interface for deep-superfial classification curation ![](https://buzsakilab.com/wp/wp-content/uploads/2020/02/gui_deepSuperficial.png){: .mt-4} -## Database capabilities +### Database capabilities The Cell Explorer is capable of loading datasets from and writing to the Buzsaki lab database. Please setup your credentials and local paths as [described here]({{"/database/preparation/"|absolute_url}}). ### Reference data diff --git a/docs/tutorials/export-figure.md b/docs/tutorials/export-figure.md index e98e5c15..7069cbca 100644 --- a/docs/tutorials/export-figure.md +++ b/docs/tutorials/export-figure.md @@ -8,6 +8,12 @@ nav_order: 11 {: .no_toc} Exporting figures in Matlab can be a headache, so here are two small tutorials to help with this: exporting the main interface and individual actions plots. +## Table of contents +{: .no_toc .text-delta } + +1. TOC +{:toc} + ## Exporting the Cell Explorer interface The steps below shows how to save a PDF file of the main interface of the Cell Explorer. Saving a PNG (image file) is more straight forward. @@ -38,4 +44,4 @@ When you select to save your figures to the Cell Explorer path or Clustering pat Be aware that saving pdf files, is substantial slower than saving png files, but the figures will be saved with vector graphics. ## Exporting remaining figures -Any other figures produced by the Cell Explorer can be saved in similar fashion using the File menu options `Save As` or `Export Setup...`. +Any other figures produced by the Cell Explorer can be saved using the File menu options `Save As` or `Export Setup...`.