Workflow
The Workflow module of the platform allows to connect all modules and allow users to create projects on single click without creating individual component. The workflow function facilitates multiple views of ensemble on a single window eliminating the need to visit different modules for the purpose. The steps involved in creating a workflow includes:
- 1.Click on "Workflow" in the left panel to get a snapshot view of all workflows created by the user.
- 2.Select "CREATE WORKFLOW" from the top right corner of the "Workflow" page.
- 3.Provide Project Name, Project Description (if any) and select a workflow template. An already saved workflow template can also be selected for a project.
- 4.Click on the "CREATE WORKFLOW" button
“VIEW WORKFLOW" available on each workflow will provide a view of the selected workflow. Workflow summary view will provide the following information.
- Workflow Name
- Status
- Workflow Description
- Last updated on
- Workflow Category
The workflow screen shows the project pipeline with various tabs on the right side of the window. Drag and drop the tabs onto the worksheet to create a workflow as per the project scope.
Use Case: Real Time Deployment Project
To create a deployment project workflow - select and link dataset, algorithm and deployment elements from the project pipeline.
Dataset Element – The dataset tab can be used to select an existing dataset by clicking the search icon and selecting the uploaded dataset. In absence of a dataset, new dataset will be automatically created which will reflect on the ‘My Dataset’ dashboard once the workflow is saved.
Algorithm Element – An Algorithm can be selected from the list of available algorithms by clicking on ‘search’ icon.
Deployment Element – A deployment project with the selected dataset and algorithm will be created on which the AI outputs will be visible.
To create annotation workflow - select and link the dataset, annotation template and annotation elements from the project pipeline.
Dataset Element – Select or create a dataset.
Annotation Template Element – The ‘annotation template’ element can be used to select existing annotation template by clicking the search icon. In absence of an annotation template, new template will be automatically created which will reflect on the ‘Annotation Template’ module once the workflow is saved along.
Annotation Element – A new ‘Annotation Project’ will be created once the workflow is saved. User can assign Assigner and Reviewer from Annotation element.
Ensemble models is a Machine Learning approach to combine multiple models in the prediction process. At CARPL, we provide features to ensemble models by passing the outcome of each model through an operation (Average, Maximum, Minimum) to get the final output of the model. It is a solution to overcome challenges like high variance or low accuracy in some models. It also helps in eliminating the false negatives which provides a better patient care in Medical Imaging.
Since there is a huge variation between the AI findings for various algorithms, CARPL provides user with an ability to map AI findings of different Algorithm into a single finding which a user can name as per the intervention and finally get the output through a desired operation. Currently CARPL supports three operations:
- 1.Average: Ensemble averaging is the technique of engineering multiple different models and allow them to form an opinion towards the final prediction. Equal weight is assigned to each models’ prediction and final output is the arithmetic mean of all the individual predictions for each finding. This is widely used to solve the problem of overfitting for any model.
- 2.Maximum: This technique allows user to get the Maximum of all the predicted probabilities for each finding. This helps in clinical purposes for eliminating the false negatives as the final predication is the maximum which enables the highest probability of occurrence of any abnormality.
- 3.Minimum: This technique allows user to get the Minimum of all predicted probabilities for each finding. This helps in eliminating the false positives and can be much useful for many interventions. CARPL also allows user to view the combined ROIs and Secondary Capture of each algorithm into a single window through CARPL Viewer. User can differentiate the ROI of each algorithm with a different colour and markings.
To create an ensemble workflow, the user can select and link a dataset with Testing and Monitoring or Deployment elements and multiple algorithms can be selected for the use case.
Dataset Element – Select or create a dataset.
Algorithm Element – An Algorithm can be selected from the list of available algorithms by clicking on ‘search’ icon. User can select more than one algorithm for the use case for each algorithm element.
Deployment Element – Select individual deployment element for each algorithm.
Connector Element – Create an ensemble project by linking individual deployment projects into a single deployment project through a connector. Select connector type as ensemble filter.
Ensemble Mapping Filter – Once the algorithms are selected, select the findings and type of operation (maximum/ minimum/ average) for predicted probabilities for each finding.
Note: Select ‘Null’ if there are no findings available for an algorithm
To create multiple deployment project workflow in a single frame as per different use cases (modality specific) - select and link dataset with multiple deployment elements and algorithms from the project pipeline.
Dataset Element – Select or create a dataset.
Algorithm Element – An Algorithm can be selected from the list of available algorithms by clicking on ‘search’ icon. User can select more than one algorithm for the use case for each algorithm element.
Deployment Element – Deployment project with the selected dataset and algorithm will be created on which the AI outputs will be visible. Select individual deployment element for each algorithm.
To create multiple Testing and Monitoring project workflow in a single frame as per different use cases (modality specific) - select and link dataset with multiple Testing and Monitoring elements and algorithms from the project pipeline.
Dataset Element – Select or create a dataset.
Algorithm Element – An Algorithm can be selected from the list of available algorithms by clicking on ‘search’ icon. User can select more than one algorithm for the use case for each algorithm element.
Testing and Monitoring Element – Testing and Monitoring project with the selected dataset and algorithm will be created on which the AI outputs will be visible. Select individual Testing and Monitoring element for each algorithm.
Click on ‘Draft’ to make changes in the template later. Click on ‘Publish’ to create a project. Once a template is published, no further changes can be made.
Once the workflow is developed, it could be saved as a template for future use and reference using the ‘Save Template’ option.
Click on ‘View Logs’ to view the workflow activity consisting of name of the element, ID, success status of that particular workflow and type of error (if any).
To edit the workflow’s name and description.
Use “Delete Workflow” to delete the workflow.
Last modified 1mo ago