In this tutorial, we demonstrate how to load a few pre-trained CNNs in MatConvNet and test a small data set through them. Specifically, we load GoogleNet, AlexNet and VGGNet-s.

First, clear your path and figures and set up MatConvNet

		  

% Clean up experimental workspace
clc, clear all, close all;
% Point to MatConvNet Setup File 
run ..\matconvnet-1.0-beta23\matlab\vl_setupnn                      

Next, download the pre-trained CNNs from the MatConvNet repository.

		  

%----------------------------------------------------------
% Download Pre-Trained CNNs
%   - AlexNet
%   - GoogleNet
%   - VGGNet
%----------------------------------------------------------

urlwrite(...-
  'http://www.vlfeat.org/matconvnet/models/imagenet-googlenet-dag.mat', ...                 % Download GoogleNet, DagNN version
  'GoogleNet-dag.mat');

urlwrite(...
  'http://www.vlfeat.org/matconvnet/models/imagenet-caffe-alex.mat', ...                    % Download MatConvNet Implementation of AlexNet, SimpleNN version
  'AlexNet.mat');

urlwrite(...
  'http://www.vlfeat.org/matconvnet/models/imagenet-vgg-s.mat', ...                         % Download MatConvNet Implementation of AlexNet, SimpleNN version
  'VGG_s.mat');                  

Now we can load the pre-trained CNN's. GoogleNet is a DagNN network while VGG-s and AlexNet are SimpleNN networks. We also acquire the normalization image size so that we know the spatial size of the data that these networks have each been trained over.

		  

%----------------------------------------------------------
% Load and setup Pre-Trained CNNs
%----------------------------------------------------------

netStruct1 = load('GoogleNet-dag.mat');                                                     % Load Google's GoogleNet
net1 = dagnn.DagNN.loadobj(netStruct1);                                                     % GoogleNet is DagNN format, build it as such                                         
d1_size = net1.meta.normalization.imageSize(:,[1 2]);

net2= load('VGG_s.mat');                                                                    % Load Microsoft's vggNet-101
net2 = vl_simplenn_tidy(net2) ;                                                             % AlexNet is SimpleNN format, build it as such
d2_size = net2.meta.normalization.imageSize(:,[1 2]);

net3 = load('AlexNet.mat');                                                                 % Load in AlexNet 
net3 = vl_simplenn_tidy(net3) ;                                                             % AlexNet is SimpleNN format, build it as such
d3_size = net3.meta.normalization.imageSize(:,[1 2]);                 

Now we can begin preparing the testing data. To do this we provide a relative path to where we are storing the testing data, and then read each sub-folder for the corresponding class names. Next we make a string array of the class names. It is to note that the strings() command started in matlab version 2016, so a equivalent version or higher will be needed for that command to run.

		  

%----------------------------------------------------------
% Load in Testing Data
%----------------------------------------------------------
    
results1 = {};                                                                              % GoogleNet Results, will contain all test data info for confusion matrix
results2 = {};                                                                              % vggNet-101 Results, will contain all test data info for confusion matrix
results3 = {};                                                                              % AlexNet Results, will contain all test data info for confusion matrix

class_locations = 'data/test_data/';                                                        % Test data classes are stored here
class_folders = dir([class_locations 'Class_*']);                                           % Gather all test data folders
types = strings(1, length(class_folders));                                                  % Initalize string array for folder names

for i = 1:length(class_folders)
   types(i) = class_folders(i).name;                                                        % Assign folder names
end
              

After all the class names have been stored, we start with the first class, and iterate through all N classes. We do this with a function called ChooseData().

		  

[data_folder, test_data, keywords] = ChooseData(types(k));                              % Specify Class type based on test data 
       

ChooseData is a function that will take in the current class name, and return the test_data from that class, the relative path to that class, and keywords that will help our CNN understand when it guesses correctly during the testing phase. This function will be called for each class name present in the string array constructed above.

		  

function [folder, test_data, keywords] = ChooseData(class)
    
    switch class                                                                            % Switch Statement to manage test data sets
        
        case 'Class_Bear'                                                                   % Test Case: Bear class
            folder = 'data/test_data/Class_Bear/';                                          % Bear Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for bear detection
            keywords = ["bear", "bruin", "ursus"];
        
        case 'Class_Bike'                                                                   % Test Case: Bike Class
            folder = 'data/test_data/Class_Bike/';                                          % Bike Class Folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for bike detection
            keywords = ["off-roader", "bike", "cycle", "moped"];

        case 'Class_Burger'                                                                 % Test Case: Burger Class
            folder = 'data/test_data/Class_Burger/';                                        % Burger Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for burger detection
            keywords = ["burger"];

        case 'Class_Bus'                                                                    % Test Case: Bus Class
            folder = 'data/test_data/Class_Bus/';                                           % Bus Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for bus detection
            keywords = ["bus", "trolley"];
 
        case 'Class_Cat'                                                                    % Test Case: Cat Class
            folder = 'data/test_data/Class_Cat/';                                           % Cat Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for cat detection
            keywords = ["cat"];
            
        case 'Class_Golfcart'                                                               % Test Case: Golfcart Class
            folder = 'data/test_data/Class_Golfcart/';                                      % Golfcart Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for golfcart detection
            keywords = ["golfcart"];
                        
        case 'Class_Microwave'                                                              % Test Case: Microwave Class
            folder = 'data/test_data/Class_Microwave/';                                     % Microwave Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for microwave detection
            keywords = ["microwave"];
                
        case 'Class_Plane'                                                                  % Test Case: Plane Class
            folder = 'data/test_data/Class_Plane/';                                         % Plane Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for plane detection
            keywords = ["plane", "air", "jet"];
        
        case 'Class_Saxophone'                                                              % Test Case: Saxaphone Class
            folder = 'data/test_data/Class_Saxophone/';                                     % Saxaphone Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for saxaphone detection
            keywords = ["saxophone"];
            
        case 'Class_Sheep'                                                                  % Test Case: Sheep Class
            folder = 'data/test_data/Class_Sheep/';                                         % Sheep Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for sheep detection
            keywords = ["sheep"];
            
        case 'Class_Soccer'                                                                 % Test Case: Soccer Class
            folder = 'data/test_data/Class_Soccer/';                                        % Soccer Class folder
            test_data = dir([folder '*.JPEG']);                                             % Keywords for soccer detection
            keywords = ["soccer"];
    end

end              

Next, we can look at the full testing phase section. The testing phase will loop over the string array mentioned above. Each iteration, it will select a new class_name that will be passed into ChooseData() which will return the test_data images and corresponding keywords. After that, the test data needs to be formatted for each CNN (GoogleNet, VGGNet, Alexnet). The formatting consists of converting each sample to single precision, resizing each sample to the correct input sizes, and performing zero-mean normalization. Afterwards, we will pass each sample through all the CNNs and then record their results. The testing phase will record the results of each class (10 classes total) for each CNN. So each CNN will have 10 sets of results associated with it.

		  

for k = 1:length(types)                                                                     % Loop over each test data class

    [data_folder, test_data, keywords] = ChooseData(types(k));                              % Specify Class type based on test data 
    
    googleNet_pos = 0;                                                                      % Keep track of GoogleNet classifications
    vggNet_pos = 0;                                                                         % Keep track of vggNet classifications
    alexNet_pos = 0;                                                                        % Keep track of AlexNet classifications
    
    for i = 1:length(test_data)                                                             % Loop over all test samples from class N
      
        c_sample = test_data(i);                                                            % Select current sample
        c_sample = imread(strcat(data_folder, c_sample.name));                              % Load in data
        sample_ = single(c_sample);                                                         % Convert image to single precision, 0-255 range
   
        sample.GN = imresize(sample_, d1_size);                                             % Resize to GoogleNets input
        sample.RN = imresize(sample_, d2_size);                                             % Resize to vggNets input
        sample.AN = imresize(sample_, d3_size);                                             % Resize to AlexNets input
   
        sample.GN = bsxfun(@minus, sample.GN, net1.meta.normalization.averageImage);        % Zero-mean GoogleNet
        sample.RN = bsxfun(@minus, sample.RN, net2.meta.normalization.averageImage);        % Zero-mean vggNet-101
        sample.AN = bsxfun(@minus, sample.AN, net3.meta.normalization.averageImage);        % Zero-mean AlexNet

        output.googleNet = Compute_output(c_sample, sample.GN, net1, 'DagNN');              % Run sample through GoogleNet, DagNN
        output.vggNet = Compute_output(c_sample, sample.RN, net2, 'SimpleNN');              % Run sample through vggNet-101, DagNN
        output.alexNet = Compute_output(c_sample, sample.AN, net3, 'SimpleNN');             % Run sample through AlexNet, SimpleNN
   
        fprintf('Testing sample %g for each CNN:\n', i);
        
        varyMe = Match_keyword(output, keywords);                                           % Evaluate ouput of each CNN
        
        if(varyMe.googleNet)
            googleNet_pos = googleNet_pos + 1;                                              % GoogleNet guesses correctly
        end
        
        if(varyMe.vggNet)
            vggNet_pos = vggNet_pos + 1;                                                    % vggNet101 guesses correctly
        end
        
        if(varyMe.alexNet)
            alexNet_pos = alexNet_pos + 1;                                                  % AlexNet guesses correctly
        end
               
    end

    class_r1 = Confusion_Matrix(googleNet_pos, types(k), i);                                % Gather results from GoogleNet
    class_r2 = Confusion_Matrix(vggNet_pos, types(k), i);                                   % Gather results from vggNet101
    class_r3 = Confusion_Matrix(alexNet_pos, types(k), i);                                  % Gather results from AlexNet
    
    results1{end + 1} = class_r1;                                                           % Record each class results GoogleNet
    results2{end + 1} = class_r2;                                                           % Record each class results vggNet-101
    results3{end + 1} = class_r3;                                                           % Record each class results AlexNet
    
end      

Compute_output is a function that takes in the formatted sample and runs it through the argument provided CNN. This function will accomodate both SimpleNN and DagNN networks. This function outs the CNN's classification from the given sample.

		

function [output] = Compute_output(im, im_, net, type)
    
    if(strcmp(type, 'DagNN'))                                           % If the CNN is DagNN Type 
        net.eval({'data', im_});                                        % Run input through CNN
        scores = net.vars(net.getVarIndex('prob')).value;               % obtain output
        scores = squeeze(gather(scores));                               % Gather all scores
    else                                                                % If CNN isn't DagNN, its SimpleNN
        res = vl_simplenn(net, im_);                                    % Run test sample through CNN
        scores = squeeze(gather(res(end).x));                           % Gather all scores
    end
    
    [bestScore, best] = max(scores);                                    % Take the max score from scores batch
    figure(1) ; clf ; imagesc(im) ;                                     % Show test sample
    title(sprintf('%s (%d), score %.3f',...
    net.meta.classes.description{best}, best, bestScore)) ;             % Print label on test sample with accuracy and best match
    output = net.meta.classes.description{best};                        % Return best match label
       
end   

Match_keyword is a function that takes in the classification output of the CNN and the provided keywords to determine if the CNN made a correct guess.

		

function [varyMe] = Match_keyword(output, keywords)
    
    varyMe.googleNet = 0;                                                       % Initalize GoogleNet value 
    varyMe.vggNet = 0;                                                          % Intialize ResNet value
    varyMe.alexNet = 0;                                                         % Initalize AlexNet value
    
    for j = 1:length(keywords)                                                  % Loop over each keyword
        if(contains(output.googleNet, keywords(j)))                             % If a keyword is found in the output
            varyMe.googleNet = 1;                                               % GoogleNet guesses correctly
            break;                                                              
        end
    end
    
    for j = 1:length(keywords)                                                  % Loop over each keyword
        if(contains(output.vggNet, keywords(j)))                                % If a keyword is found in the output
            varyMe.vggNet = 1;                                                  % ResNet guesses correctly
            break;                                                              
        end
    end 
    
    for j = 1:length(keywords)                                                  % Loop over each keyword
        if(contains(output.alexNet, keywords(j)))                               % If a keyword is found in the output
            varyMe.alexNet = 1;                                                 % AlexNet guesses correctly
            break;                                                              
        end
    end
      
end  

Confusion Matrix is a function that provides statistical results of each test class for each pre-trained CNN.

		

function [class_test] = Confusion_Matrix(true_pos, type, total_samples)

    false_pos = total_samples - true_pos;                                       % Any wrong guesses are false positives
    current_accuracy = true_pos/total_samples;                                  % Class Accuracy is true_positives/iterations

    class_test.class = type;                                                    % Test data catagory
    class_test.accuracy = current_accuracy;                                     % Test data class accuracy
    class_test.samples = total_samples;                                          % Test data samples 
    class_test.true_pos = true_pos;                                             % Test data true positives
    class_test.false_pos = false_pos;                                           % Test data false positives

end

Lastly, we can store the finalized results into files

		

save('GoogleNet_Results.mat', 'results1', '-v7.3');                                         % GoogleNet Results for each class
save('VggNet-s_Results.mat', 'results2', '-v7.3');                                          % VggNet-s Results for each class
save('AlexNet_Results.mat', 'results3', '-v7.3');                                           % AlexNet Results for each class