MATLAB Deep Learning Toolbox
Expert skill for deep learning in MATLAB, focused on medical image analysis workflows.
Cross-Toolbox Note: For image preprocessing (filtering, morphology), see matlab-image-processing-toolbox. For medical I/O (DICOM, NIfTI), see matlab-medical-imaging-toolbox. For wavelet features, see matlab-wavelet-toolbox.
When to Use This Skill
- •Building CNN classifiers for medical diagnosis (X-ray, dermatology, histopathology)
- •Semantic segmentation (U-Net, DeepLabv3+ for organ/lesion segmentation)
- •Object detection (YOLO, Faster R-CNN for nodule/cell detection)
- •Transfer learning with pretrained networks (ResNet, VGG, EfficientNet)
- •Custom training loops with
dlarrayand automatic differentiation - •3D volumetric deep learning for CT/MRI
- •GPU-accelerated training and multi-GPU workflows
- •Model deployment (ONNX export, GPU Coder)
Read Before Coding
| Task | Knowledge Card | Key Functions |
|---|---|---|
| Classify images | cards/classification.md | trainNetwork, classify, pretrained nets |
| Transfer learning | cards/classification.md | resnet50, layerGraph, replace layers |
| Semantic segmentation | cards/segmentation-semantic.md | unetLayers, deeplabv3plusLayers, semanticseg |
| Instance segmentation | cards/segmentation-instance.md | maskrcnn, Mask R-CNN |
| Object detection | cards/object-detection.md | yolov4ObjectDetector, fasterRCNNObjectDetector |
| Custom training | cards/custom-training.md | dlarray, dlfeval, dlgradient, adamupdate |
| Data pipelines | cards/data-pipeline.md | imageDatastore, augmentedImageDatastore, minibatchqueue |
| Network layers | cards/network-architecture.md | dlnetwork, layerGraph, custom layers |
| GPU/parallel | cards/gpu-parallel.md | gpuArray, 'ExecutionEnvironment', multi-GPU |
| Deploy models | cards/deployment.md | exportONNXNetwork, GPU Coder |
| Medical workflows | cards/medical-imaging.md | 3D networks, DICOM pipelines |
⚠️ IMPORTANT: API Changes in R2024b+
Several functions shown in this skill are deprecated in R2024b and later. Use the modern equivalents for new projects:
| Deprecated Function | Modern Replacement | Notes |
|---|---|---|
trainNetwork | trainnet | Returns dlnetwork instead of DAGNetwork |
unetLayers | unet | Returns dlnetwork directly |
unet3dLayers | unet3d | Returns dlnetwork directly |
deeplabv3plusLayers | deeplabv3plus | Returns dlnetwork directly |
classificationLayer | Use trainnet with "crossentropy" loss | Loss specified at training time |
pixelClassificationLayer | Use trainnet with "crossentropy" loss | For segmentation |
Modern pattern:
net = dlnetwork(lgraph); % Create dlnetwork net = trainnet(ds, net, "crossentropy", opts); % Train with loss function
The examples below use the legacy API for compatibility with pre-R2024b code.
Critical Rules
Rule 1: Data Type and Normalization
% WRONG: Using uint8 directly net = trainNetwork(uint8Images, labels, layers, options); % Poor convergence % CORRECT: Normalize to [0,1] or [-1,1] images = im2single(uint8Images); % Now [0,1] % Or for pretrained networks expecting specific preprocessing: images = (im2single(uint8Images) - 0.485) / 0.229; % ImageNet normalization
Rule 2: Match Input Sizes
% Check network input size inputSize = net.Layers(1).InputSize; % e.g., [224 224 3] % Resize images to match augDs = augmentedImageDatastore(inputSize(1:2), imds); % Or images = imresize(images, inputSize(1:2));
Rule 3: dlarray Format Strings
% Format: S=Spatial, C=Channel, B=Batch, T=Time, U=Unspecified x = dlarray(data, 'SSCB'); % H×W×C×B (standard image batch) x = dlarray(data, 'SSSCB'); % H×W×D×C×B (3D volume batch) x = dlarray(data, 'CBT'); % C×B×T (sequence) % Common error: wrong format causes dimension mismatch % WRONG: x = dlarray(data, 'BCSS'); % Batch and Channel swapped
Rule 4: Gradient Computation
% Gradients only flow through dlarray operations % WRONG: Using non-differentiable ops mask = Y > 0.5; % Thresholding breaks gradient flow % CORRECT: Use soft operations mask = sigmoid((Y - 0.5) * 10); % Differentiable approximation
Rule 5: GPU Memory Management
% Check available GPU memory before training
gpu = gpuDevice;
fprintf('Available: %.2f GB\n', gpu.AvailableMemory/1e9);
% Reduce batch size if OOM
options = trainingOptions('adam', 'MiniBatchSize', 8); % Start small
% Clear GPU memory between experiments
reset(gpuDevice);
Transform Selection
Task?
├── Image Classification
│ ├── Binary/Multiclass → trainNetwork + softmax + crossentropy
│ └── Multi-label → trainNetwork + sigmoid + binary crossentropy
├── Segmentation
│ ├── Semantic (pixel labels) → unetLayers/deeplabv3plusLayers + semanticseg
│ ├── Instance (object masks) → Mask R-CNN
│ └── 3D volumetric → unet3dLayers
├── Object Detection
│ ├── Real-time → YOLO v4 (yolov4ObjectDetector)
│ ├── High accuracy → Faster R-CNN (fasterRCNNObjectDetector)
│ └── Small objects → RetinaNet
├── Custom Architecture
│ └── dlnetwork + custom training loop + dlfeval/dlgradient
└── Generative
├── Image synthesis → GAN (generator + discriminator)
└── Variational → VAE with reparameterization
Quick Patterns
Transfer Learning (Classification)
% Load pretrained network
net = resnet50;
lgraph = layerGraph(net);
% Replace final layers for your classes
numClasses = 4; % e.g., Normal, Pneumonia, COVID, TB
lgraph = removeLayers(lgraph, {'fc1000', 'fc1000_softmax', 'ClassificationLayer_fc1000'});
newLayers = [
fullyConnectedLayer(numClasses, 'Name', 'fc_new')
softmaxLayer('Name', 'softmax_new')
classificationLayer('Name', 'output')];
lgraph = addLayers(lgraph, newLayers);
lgraph = connectLayers(lgraph, 'avg_pool', 'fc_new');
% Train
options = trainingOptions('adam', ...
'MaxEpochs', 10, ...
'MiniBatchSize', 32, ...
'InitialLearnRate', 1e-4, ...
'ValidationData', valDs, ...
'Plots', 'training-progress');
trainedNet = trainNetwork(trainDs, lgraph, options); % Legacy API
% Modern: net = trainnet(trainDs, dlnetwork(lgraph), "crossentropy", options);
U-Net Segmentation
% Create U-Net for medical image segmentation
imageSize = [256 256 1];
numClasses = 2; % Background + Lesion
% Legacy API (deprecated in R2024b+, use unet() instead):
lgraph = unetLayers(imageSize, numClasses, ...
'EncoderDepth', 4, ...
'NumFirstEncoderFilters', 64); % Default is 64, not 32
% Modern API (R2024b+):
% net = unet(imageSize, numClasses, EncoderDepth=4);
% Prepare pixel label datastore
classNames = ["Background", "Lesion"];
pixelLabelIDs = [0, 1];
pxds = pixelLabelDatastore('masks/', classNames, pixelLabelIDs);
ds = combine(imageDatastore('images/'), pxds);
% Train with dice loss
options = trainingOptions('adam', ...
'MaxEpochs', 50, ...
'MiniBatchSize', 8, ...
'InitialLearnRate', 1e-3);
net = trainNetwork(ds, lgraph, options); % Legacy API
% Modern: net = trainnet(ds, dlnetwork(lgraph), "crossentropy", options);
% Inference
mask = semanticseg(testImage, net);
Custom Training Loop
% For advanced control: custom loss, metrics, logging
net = dlnetwork(lgraph);
numEpochs = 50;
learnRate = 1e-3;
[avgGrad, avgSqGrad] = deal([]);
for epoch = 1:numEpochs
shuffle(mbq);
while hasdata(mbq)
[X, Y] = next(mbq);
% Forward + backward
[loss, gradients, state] = dlfeval(@modelLoss, net, X, Y);
net.State = state;
% Update with Adam
[net, avgGrad, avgSqGrad] = adamupdate(net, gradients, ...
avgGrad, avgSqGrad, iteration, learnRate);
end
end
function [loss, gradients, state] = modelLoss(net, X, Y)
[Ypred, state] = forward(net, X);
loss = crossentropy(Ypred, Y);
gradients = dlgradient(loss, net.Learnables);
end
Object Detection (YOLO)
% Create YOLO v4 detector
detector = yolov4ObjectDetector('csp-darknet53-coco');
% For custom classes, create from scratch
name = "nodule_detector";
classes = ["nodule"];
anchorBoxes = {[32 32; 64 64]; [128 128; 256 256]}; % Multi-scale
detector = yolov4ObjectDetector(name, classes, anchorBoxes, ...
'InputSize', [416 416 3]);
% Train
options = trainingOptions('adam', ...
'MaxEpochs', 80, ...
'MiniBatchSize', 8, ...
'InitialLearnRate', 1e-3);
[detector, info] = trainYOLOv4ObjectDetector(trainData, detector, options);
% Detect
[bboxes, scores, labels] = detect(detector, testImage);
Function Quick Reference
Network Training
| Function | Purpose | Example |
|---|---|---|
trainNetwork | Train from layers/lgraph | net = trainNetwork(ds, layers, opts) |
trainnet | Train dlnetwork with loss | net = trainnet(ds, net, lossFcn, opts) |
trainingOptions | Configure training | opts = trainingOptions('adam', ...) |
Network Architecture
| Function | Purpose | Example |
|---|---|---|
dlnetwork | Create trainable network | net = dlnetwork(lgraph) |
layerGraph | Network with branches | lgraph = layerGraph(net) |
addLayers | Add layers to graph | lgraph = addLayers(lgraph, newLayers) |
connectLayers | Connect layer outputs | lgraph = connectLayers(lgraph, src, dst) |
removeLayers | Remove layers | lgraph = removeLayers(lgraph, names) |
Segmentation
| Function | Purpose | Example |
|---|---|---|
unetLayers | Create U-Net | lgraph = unetLayers([256 256 1], 2) |
unet3dLayers | Create 3D U-Net | lgraph = unet3dLayers([128 128 128 1], 2) |
deeplabv3plusLayers | Create DeepLabv3+ | lgraph = deeplabv3plusLayers(...) |
semanticseg | Pixel classification | mask = semanticseg(img, net) |
pixelLabelDatastore | Label datastore | pxds = pixelLabelDatastore(...) |
Object Detection
| Function | Purpose | Example |
|---|---|---|
yolov4ObjectDetector | YOLO v4 detector | det = yolov4ObjectDetector(...) |
fasterRCNNObjectDetector | Faster R-CNN | det = fasterRCNNObjectDetector(...) |
ssdObjectDetector | SSD detector | det = ssdObjectDetector(...) |
detect | Run detection | [bboxes, scores] = detect(det, img) |
trainYOLOv4ObjectDetector | Train YOLO | det = trainYOLOv4ObjectDetector(...) |
Custom Training
| Function | Purpose | Example |
|---|---|---|
dlarray | Differentiable array | x = dlarray(data, 'SSCB') |
dlfeval | Evaluate with gradients | [loss, grad] = dlfeval(@fn, net, x) |
dlgradient | Compute gradients | grad = dlgradient(loss, params) |
adamupdate | Adam optimizer step | [net, ag, asg] = adamupdate(...) |
sgdmupdate | SGD with momentum | [net, vel] = sgdmupdate(...) |
forward | Forward pass | [y, state] = forward(net, x) |
predict | Inference (no gradients) | y = predict(net, x) |
Data Handling
| Function | Purpose | Example |
|---|---|---|
imageDatastore | Image folder datastore | imds = imageDatastore(folder) |
augmentedImageDatastore | With augmentation | augDs = augmentedImageDatastore(...) |
imageDataAugmenter | Define augmentations | aug = imageDataAugmenter(...) |
minibatchqueue | Custom batching | mbq = minibatchqueue(ds, ...) |
transform | Apply function | tds = transform(ds, @myFcn) |
combine | Combine datastores | cds = combine(imds, labelds) |
Pretrained Networks
| Network | Function | Input Size | Use Case |
|---|---|---|---|
| ResNet-50 | resnet50 | 224×224 | General classification |
| ResNet-101 | resnet101 | 224×224 | Higher capacity |
| VGG-16 | vgg16 | 224×224 | Feature extraction |
| EfficientNet-B0 | efficientnetb0 | 224×224 | Efficient inference |
| Inception-v3 | inceptionv3 | 299×299 | Fine details |
| DenseNet-201 | densenet201 | 224×224 | Feature reuse |
Knowledge Cards Summary
~2,400 lines of curated content:
| Card | Lines | Focus |
|---|---|---|
classification.md | ~300 | Transfer learning, pretrained networks |
segmentation-semantic.md | ~350 | U-Net, DeepLabv3+, loss functions |
segmentation-instance.md | ~200 | Mask R-CNN |
object-detection.md | ~300 | YOLO, Faster R-CNN, anchor boxes |
custom-training.md | ~350 | dlarray, gradients, optimizers |
data-pipeline.md | ~250 | Datastores, augmentation |
network-architecture.md | ~200 | Layers, custom layers |
gpu-parallel.md | ~150 | GPU, multi-GPU training |
deployment.md | ~150 | ONNX, code generation |
medical-imaging.md | ~300 | Medical-specific workflows |
See knowledge/INDEX.md for full navigation.
Cross-Toolbox Integration
For preprocessing → matlab-image-processing-toolbox
% Before DL: enhance, denoise, normalize img = adapthisteq(img); % CLAHE img = imgaussfilt(img, 1.5); % Denoise img = im2single(img); % Normalize
For wavelet features → matlab-wavelet-toolbox
% Multi-scale features as network input
[C, S] = wavedec2(img, 3, 'db4');
features = cat(3, appcoef2(C,S,'db4'), ...
detcoef2('h',C,S,1), detcoef2('v',C,S,1), detcoef2('d',C,S,1));
For medical I/O → matlab-medical-imaging-toolbox
% Load with spatial referencing
V = medicalVolume('brain.nii');
% Process slices for DL
for k = 1:V.NumTransverseSlices
slice = extractSlice(V, k, 'transverse');
prediction = semanticseg(slice, net);
end
Source: MathWorks Deep Learning Toolbox Documentation (R2025b) Note: Legacy API examples (trainNetwork, unetLayers) shown for compatibility. Use trainnet and unet for new projects.