Project with set of CUDA accelerated medical segmentation metrics.Mathemathical basis for metrics calculations are based on the work of the Taha et al. [1].
]add MedEval3D
Example for calculating Mahalinobis distance
using MedEval3D
using MedEval3D.BasicStructs
using MedEval3D.MainAbstractions
arrGold = CUDA.ones(3,3,3)
arrAlgoCPU = ones(3,3,3)
arrAlgoCPU[1,1,1]=0
arrAlgoCPU[3,3,3]=0
arrAlgoCPU[3,2,3]=0
arrAlgoCPU[3,2,2]=0
arrAlgo =CuArray(arrAlgoCPU)
conf= ConfigurtationStruct(md=true)
numberToLookFor = UInt8(1)
preparedDict=MainAbstractions.prepareMetrics(conf)
res= calcMetricGlobal(preparedDict,conf,arrGold,arrAlgo,numberToLookFor)
res.md # will give 0.127
Programming model is based on the two phase metric evaluation.
First phase is invoked as a preparation step in order to calculate variables that are constant across kernel given image array dimensions. Those constants include thread block dimensions and number of required thread blocks to optimize occupancy using Occupancy API. Other constants are mainly related to precalculation of loop sizes and appropriate GPU memory allocations. Preparation step is designed to be invoked once for each dataset and can be cached and reused given image array size and GPU hardware will not change.
Second phase is invoked with an image array and gold standard segmentation together with variables calculated in the preparation step. Additionally to enable reliable calculation in multiple function invocations all data stractures are set to initial values (ussually 0).
For more theory You may look into paper that is currently in development https://www.overleaf.com/project/60f54dd02d12a4796b60026d. In case of any questions to this package or to the rest of medical segmentation framework (currently also visualization and segmentation tool at https://github.com/jakubMitura14/MedEye3d.jl) ask me on github or linkedIn https://www.linkedin.com/in/jakub-mitura-7b2013151/
For executing othe metrics we need to specify appropriate ConfigurtationStruct, we can do multiple metrics at once and it will run in similar time as single metric (most of the computations are reused)
First we need to specify input data
arrGold = CuArray(...) # 3 dimansional CUDA array representing gold standard mask
arrAlgo = CuArray(...) # 3 dimansional CUDA array representing output of our algorithm - we want to compare it against gold standard ...
Next proper configuration
numberToLookFor = ... # number in arrGold and arrAlgo that is marking the structure of intrest for example 1
conf = ConfigurtationStruct(...) # we set in the Configuration struct to true field representing metric of intrest - we can mark multiple to true or just one - reference in the end of read me file.
We invoke the preparation step - this needs to be invoked only once unless there is change in ConfigurtationStruct or in GPU hardware.
preparedDict=MainAbstractions.prepareMetrics(conf)
Last step is invoking metrics
res = calcMetricGlobal(preparedDict,conf,arrGold,arrAlgo,numberToLookFor))
Now we access the Result using the field name of the ResultMetrics struct (reference in the end)
Reference to ConfigurtationStruct and ResultMetrics
struct ConfigurtationStruct
dice::Bool = false #dice coefficient
jaccard::Bool = false #jaccard coefficient
gce::Bool = false #global consistency error
vol::Bool = false# Volume metric
randInd::Bool= false # Rand Index
ic::Bool= false # interclass correlation
kc::Bool= false # Kohen Cappa
mi::Bool= false # mutual information
vi::Bool= false # variation Of Information
mutable struct ResultMetrics
dice::Float64 = -1.0 #dice coefficient
jaccard::Float64 = -1.0 #jaccard coefficient
gce::Float64 = -1.0 #global consistency error
vol::Float64 = -1.0 # Volume metric
randInd::Float64 = -1.0 # Rand Index
ic::Float64 = -1.0 # interclass correlation
kc::Float64 = -1.0 # Kohen Cappa
mi::Float64 = -1.0 # mutual information
vi::Float64 = -1.0 # variation Of Information
md::Float64 = -1.0 # mahalanobis distance
end
All experiments were conducted on Windows PC (Intel Core I9 10th gen., GeforceRTX 3080), and Data from CT-ORG [9] dataset, on image of size (512×512×826). Time needed to calculate metrics was estimated in case of Julia code using BenchamrkTools.jl [10]. For testing python libraries internal python module timeit was utilized. Results of experiments mentioned above are summarized in Figure 1, and in Table 1. In all cases data was already in RAM memory for CPU computation or in GPU memory for CUDA computations - hence memory transfer times were not included. As visible in the implementation of CUDA acceleration of described package in most cases led to from 40 up to 214 times shorter execution times. The only exception is in case of CUDA accelerated Monai Dice metric algorithm MedEval3D is slower (24.1 ms vs 52.8 ms) . However from all of the metrics described Monai implements only Dice metric, although admittedly most of others could be potentially calculated from Monai confusion matrix. What is also worth pointing out is a field in a table named ”all Conf metr based” this tested calculating all of the metrics jointly apart from interclass correlation and mahalanobis distance - time of execution in such case both in case of MedEval3d and Pymia algorithms was similar to calculation of just one of those metrics. This can be explained by the fact that in all of those cases the most computation intensive work is related to calculation of confusion matrix, and this calculation is reused in both packages between diffrent metrics. 11
Figure 1: Comparison of median times needed to calculate given metrics in log scale,
for Monai only CUDA accelerated algorithm was taken into account. Dimensionality
of data was (512x512x826)
If You will find usefull my work please cite it
@Article{Mitura2021,
author = {Mitura, Jakub and Chrapko, Beata E.},
journal = {Zeszyty Naukowe WWSI},
title = {{3D Medical Segmentation Visualization in Julia with MedEye3d}},
year = {2021},
number = {25},
pages = {57--67},
volume = {15},
doi = {10.26348/znwwsi.25.57},
keywords = {OpenGl, Computer Tomagraphy, PET/CT, medical image annotation, medical image visualization},
}
[1] Taha, A.A., Hanbury, A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15, 29 (2015). https://doi.org/10.1186/s12880-015-0068-x