MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning

1 Introduction

• Manual review of growing volumes of high-resolution medical images motivates automatic pre-segmentation of abnormal features.The paper notes that physicians must inspect larger numbers of slices as imaging use and resolution increase.Source: §1 · p. 1
• Standalone segmentation pipelines are often dataset-specific, while general frameworks lack medical-image I/O, preprocessing, imbalance handling, and modular customization.These limitations make objective comparisons between architectures difficult because each comparison can require a custom pipeline.Source: §1 · p. 1
• MIScnn targets faster pipeline setup with preprocessing, augmentation, patch slicing, batch creation, and configurable model choices and evaluation.It supports switching among modern models, adding custom architectures, and adjusting training and preprocessing options.Source: §1 · pp. 1–2

2 Methods

• MIScnn combines medical-image I/O, preprocessing, augmentation, patch-wise or full-image analysis, models, metrics, training, prediction, and automatic evaluation.Its listed interfaces also support custom models, data I/O, preprocessing, postprocessing, and metrics.Source: §2 · p. 2
• The pipeline flowchart runs from data I/O to a deep-learning model, framing the framework's end-to-end workflow.The core feature list includes the intervening preprocessing, augmentation, model-use, and evaluation functions.Source: Figure 1 · p. 2, §2 · p. 2

2.1 Data Input

• MIScnn provides a NIfTI interface for loading MRI and CT data, alongside support for other custom data I/O interfaces.NIfTI files carry 3D image matrices and metadata such as MRI slice thickness.Source: §2.1 · p. 2

Custom Data I/O Interface

• Custom data I/O interfaces let MIScnn handle other formats and preserve format-specific information such as MRI slice thickness without format conversion.Format-specific preprocessing can be performed in the interface before returning the image matrix to the workflow.Source: Custom Data I/O Interface · p. 2

Pixel Intensity Normalization

• MIScnn can include or exclude intensity scaling and standardization to address varying signal ranges across biomedical images.Scaling maps values to a predefined range, while Z-score standardization centers values close to a normal distribution.Source: Pixel Intensity Normalization · p. 3
• Organ-specific clipping can constrain CT intensities to a chosen range because the same organs or tissue types are expected to have consistent values across scanners.Values outside the range are clipped to its minimum or maximum.Source: Pixel Intensity Normalization · p. 3
• Resampling adjusts image dimensions and can harmonize slice thickness or voxel spacing required for model training.Downsampling also reduces GPU memory requirements for training and prediction.Source: Pixel Intensity Normalization · p. 3

One Hot Encoding

• For segmentation labels with more than two classes, MIScnn automatically creates a binary variable for each class through One Hot encoding.The paper distinguishes this multi-class representation from binary labels represented with two states, zero and one.Source: One Hot Encoding · p. 3

2.2 Patch-wise and Full Image Analysis

• MIScnn supports slice-by-slice 2D analysis, configurable 3D patches, and full 3D images, with patch-wise analysis helping fit data to available GPU memory.The paper gives 128x128x128 as an example patch size and notes that high-resolution volumes can exceed available GPU memory.Source: §2.2 · p. 3

2.3 Data Augmentation for Training

• Augmentation creates plausible variations of training images to expand small datasets and help avoid overfitting.Medical imaging often has few samples for a studied condition [21] [22].Source: §2.3 · p. 3
• MIScnn integrates batchgenerators for spatial transforms, elastic deformations, brightness, contrast, gamma, and noise augmentations.Supported transforms include translations, rotations, scaling, and Gaussian noise.Source: §2.3 · p. 3

2.4 Sampling and Batch generation

• MIScnn can exclude patches labeled entirely as background during patch-wise training to avoid time spent fitting uninformative patches.The passage notes that augmentation can multiply these blank patches without adding learning information.Source: §2.4 · p. 4

Batch Management

• Prepared batches can be saved for faster repeated access without reprocessing, or generated on the fly when datasets or disk-space constraints make storage unsuitable.Batch size depends on available GPU memory and must be configured accordingly.Source: Batch Management · p. 4
• MIScnn shuffles the processing sequence of batches at each epoch’s end, reducing fitting variance and overfitting risk without rearranging data into new batches.Source: Batch Management · p. 4
• Stored prepared batches support parallel CPU loading alongside GPU computing and can avoid preprocessing bottlenecks on CPU/GPU clusters.Source: Batch Management · p. 4

2.5 Deep Learning Model Creation

• MIScnn’s open model interface supports configurable 2D and 3D architectures, including U-Net, and integration of custom models.Configuration options include neuron counts, input sizes, dropout, batch normalization, and enhanced architecture variants.Source: §2.5 · p. 4
• MIScnn supports Dice and Jaccard metrics and uses Tversky loss, which addresses data imbalance with a precision–recall trade-off.Class-wise Dice accounts for performance by segmentation class, recommended for commonly class-unbalanced images; custom metrics can also be integrated.Source: §2.5 · pp. 4–5
• Figure 2 presents a flowchart of the deep-learning model creation and architecture-selection process.Source: Figure 2 · p. 5
• Keras standard metrics, including accuracy and cross-entropy, can also be used for training or evaluation in MIScnn.Source: §2.5 · p. 5

2.6 Model Utilization

• Training requires configuring the learning rate and epoch count after data augmentation and batch-management settings, and fitted models can be saved for later reuse.A model can be trained, used for prediction, evaluated through repeated training and prediction, or saved and loaded.Source: §2.6 · p. 5
• MIScnn can record fitting scores and metrics in a tab-separated file or plot them, while Keras reports model performance and remaining fitting time.Source: §2.6 · p. 5
• For each pixel, the model predicts a sigmoid value per class; multi-class output assigns the class with the highest value.The sigmoid values represent probability estimates for their associated labels.Source: §2.6 · p. 5
• For overlapping prediction patches, MIScnn’s default merge averages each class’s sigmoid predictions at pixels covered by multiple patches.The overlap approach addresses reduced prediction power at patch edges.Source: §2.6 · p. 5
• Predicted segmentation matrices retain the original image shape and, with the default NIfTI interface, are saved in NIfTI format without additional metadata.Source: §2.6 · p. 6
• MIScnn automates k-fold, leave-one-out, percentage-split, and detailed validation, with plots and files for metrics and visual comparisons to ground truth.Except for detailed validation, the methods use random sampling to create training and testing sets.Source: §2.6 · p. 6

3.1 Kidney Tumor Segmentation Challenge 2019 (KiTS19)

• KiTS19 provides arterial-phase abdominal CT scans from 300 patients, with expert pixel labels for background, kidney, and tumor.The challenge released 210 scans with ground truth during training and 90 scans without published ground truth during submission.Source: §3.1 · p. 6

3.2 Validation on the KiTS19 data set with MIScnn

• MIScnn ran an automatic 3-fold cross-validation on a 120-scan KiTS19 subset with slice-thickness normalization.The pipeline used multi-class 80x160x160 patches, augmentation, and a standard 3D U-Net trained with Tversky loss for 1000 epochs.Source: §3.2 · p. 7
• The cross-validation evaluated kidney and tumor segmentation using Tversky loss, soft Dice, class-wise Dice, and categorical cross-entropy plus soft Dice.Scores were computed during fitting and prediction and summarized across folds.Source: §3.2 · p. 7, Figure 4 · p. 8
• Kidney Dice coefficient had a median around 0.9544, while tumor segmentation had a median around 0.7912.The authors describe the tumor result as weaker, and report that training used 80 scans while 40 were selected for testing.Source: Table 1 · p. 7, §3.2 · p. 7
• MIScnn also generated visualizations for manual comparison of predicted kidney and tumor segmentations against ground truth.The authors report that the predicted semantic segmentations were highly accurate.Source: §3.2 · p. 8, Figure 5 · p. 8

4.1 MIScnn framework

• MIScnn combines configurable preprocessing, augmentation, model selection, and utilization with interfaces for custom models, image formats, and fitting metrics.Its switchable models and automatic evaluation are intended to support robust, unbiased model comparisons and setup by novices in a few lines of code.Source: §4.1 · p. 8

4.2 Use Case: Kidney Tumor Segmentation Challenge

• A cross-validation pipeline using the standard U-Net produced strong kidney-tumor segmentation predictions on a subset of the KiTS19 CT data.The authors note that a more precise architecture might improve performance but would require more fitting time and GPU memory than their hardware schedule allowed.Source: §4.2 · p. 9

4.3 Road Map and Future Direction

• Planned work includes broader image-format support, expanded preprocessing and augmentation, more efficient patch skipping, custom preprocessing interfaces, and additional models.The roadmap also calls for an objective comparison of U-Net variants using the same pipeline.Source: §4.3 · p. 9

4.4 MIScnn Availability

• MIScnn is installable with pip, released under GPLv3, and accompanied by the cross-validation experiment notebook in its Git repository.The repository also provides the source code.Source: §4.4 · p. 9

• MIScnn brings data I/O, preprocessing, augmentation, patch-wise analysis, metrics, deep-learning models, training, prediction, and automatic evaluation into a Python library.The listed evaluation functionality includes cross-validation.Source: §5 · p. 9
• High configurability and multiple open interfaces let users customize pipelines and rapidly assemble them with only a few lines of code.Source: §5 · p. 9
• A cross-validation on the Kidney Tumor Segmentation Challenge 2019 CT data set yielded a powerful predictor, demonstrating MIScnn’s functionality.Source: §5 · p. 9