How To Programmatically Create A Deep Neural Network In Python Caffe

When you are working with Caffe, you need to define your deep neural network architecture in a ‘.prototxt’ file. These prototxt files usually consist of hundreds of lines, defining layers and corresponding parameters. Before you start training your neural network, you need to create these files and define your architecture. One way to do this is manually write all these lines into a file. But sometimes, it’s beneficial to dynamically create this architecture depending on our needs. In such cases, creating a deep neural network programmatically can be very useful. Let’s go ahead and see how to do it in Python Caffe, shall we?  

Prerequisites

Before you proceed, make sure you have Caffe installed on your machine. Since we will be using Python here, you need to have PyCaffe installed too. We also need training data to define the architecture. Create a file called “train.h5list” and add the following line:

/path/to/train/data.h5

The file “data.h5” is the HDF5 dataset file that you need to create from your training data. We don’t really need this file for our code to work, so it’s okay if this file doesn’t exist for now. Just make sure you have it before you start training because Caffe will look for this file. Once we go through the steps, you will see that this can be extended to other types of datasets too. We are just taking this as a representative example to create a neural network programmatically.

Defining the network

Let’s look at the code. Import the necessary packages:

import caffe
from caffe import layers as cl

Define a function to create a neural network

def create_neural_net(input_file, batch_size=50):
    net = caffe.NetSpec()

The above line create an empty network. Let’s add the data and a couple of other parameters to this network:

    net.data, net.label = cl.HDF5Data(batch_size=batch_size, source=input_file, ntop=2)

Depending on the type of database you have, you can change HDF5Data() to ImageData() or Data(). Now let’s define a network that has 4 fully connected layers, 4 ReLU layers, along with a Softmax layer:

    net.fc1 = cl.InnerProduct(net.data, num_output=100, weight_filler=dict(type='xavier'))
    net.relu1 = cl.ReLU(net.fc1, in_place=True)
    net.fc2 = cl.InnerProduct(net.relu1, num_output=50, weight_filler=dict(type='xavier'))
    net.relu2 = cl.ReLU(net.fc2, in_place=True)
    net.fc3 = cl.InnerProduct(net.relu1, num_output=20, weight_filler=dict(type='xavier'))
    net.relu3 = cl.ReLU(net.fc3, in_place=True)
    net.fc4 = cl.InnerProduct(net.relu3, num_output=1, weight_filler=dict(type='xavier'))
    net.loss = cl.SoftmaxWithLoss(net.fc4, net.label)

Convert the structure into proto and return it:

    return net.to_proto()

How to create the output ‘.prototxt’ file?

We will be using the file ‘train.h5list’ as input here. The following code will create an output file called ‘train.prototxt’. The first argument to the code is the input file (train.h5list) and the second argument is the output file (train.prototxt):

if __name__=='__main__':
    train_h5list_file = sys.argv[1]
    output_file = sys.argv[2]

    batch_size = 50
    with open(output_file, 'w') as f:
        f.write(str(create_neural_net(train_h5list_file, batch_size)))

Go ahead and open the file ‘train.prototxt’ and you’ll see all the layers defined in the Caffe syntax.

—————————————————————————————-