When you are working with Caffe, you need to define your deep neural network architecture in a ‘.prototxt’ file. These prototxt files usually consist of hundreds of lines, defining layers and corresponding parameters. Before you start training your neural network, you need to create these files and define your architecture. One way to do this is manually write all these lines into a file. But sometimes, it’s beneficial to dynamically create this architecture depending on our needs. In such cases, creating a deep neural network programmatically can be very useful. Let’s go ahead and see how to do it in Python Caffe, shall we?
Before you proceed, make sure you have Caffe installed on your machine. Since we will be using Python here, you need to have PyCaffe installed too. We also need training data to define the architecture. Create a file called “train.h5list” and add the following line:
The file “data.h5” is the HDF5 dataset file that you need to create from your training data. We don’t really need this file for our code to work, so it’s okay if this file doesn’t exist for now. Just make sure you have it before you start training because Caffe will look for this file. Once we go through the steps, you will see that this can be extended to other types of datasets too. We are just taking this as a representative example to create a neural network programmatically.
Defining the network
Let’s look at the code. Import the necessary packages:
import caffe from caffe import layers as cl
Define a function to create a neural network
def create_neural_net(input_file, batch_size=50): net = caffe.NetSpec()
The above line create an empty network. Let’s add the data and a couple of other parameters to this network:
net.data, net.label = cl.HDF5Data(batch_size=batch_size, source=input_file, ntop=2)
Depending on the type of database you have, you can change HDF5Data() to ImageData() or Data(). Now let’s define a network that has 4 fully connected layers, 4 ReLU layers, along with a Softmax layer:
net.fc1 = cl.InnerProduct(net.data, num_output=100, weight_filler=dict(type='xavier')) net.relu1 = cl.ReLU(net.fc1, in_place=True) net.fc2 = cl.InnerProduct(net.relu1, num_output=50, weight_filler=dict(type='xavier')) net.relu2 = cl.ReLU(net.fc2, in_place=True) net.fc3 = cl.InnerProduct(net.relu1, num_output=20, weight_filler=dict(type='xavier')) net.relu3 = cl.ReLU(net.fc3, in_place=True) net.fc4 = cl.InnerProduct(net.relu3, num_output=1, weight_filler=dict(type='xavier')) net.loss = cl.SoftmaxWithLoss(net.fc4, net.label)
Convert the structure into proto and return it:
How to create the output ‘.prototxt’ file?
We will be using the file ‘train.h5list’ as input here. The following code will create an output file called ‘train.prototxt’. The first argument to the code is the input file (train.h5list) and the second argument is the output file (train.prototxt):
if __name__=='__main__': train_h5list_file = sys.argv output_file = sys.argv
batch_size = 50 with open(output_file, 'w') as f: f.write(str(create_neural_net(train_h5list_file, batch_size)))
Go ahead and open the file ‘train.prototxt’ and you’ll see all the layers defined in the Caffe syntax.