How To Extract Feature Vectors From Deep Neural Networks In Python Caffe

1 mainConvolutional Neural Networks are great at identifying all the information that makes an image distinct. When we train a deep neural network in Caffe to classify images, we specify a multilayered neural network with different types of layers like convolution, rectified linear unit, softmax loss, and so on. The last layer is the output layer that gives us the output tag with the corresponding confidence value. But sometimes it’s useful for us to extract the feature vectors from various layers and use it for other purposes. Let’s see how to do it in Python Caffe, shall we?  

Let’s import the necessary packages:

import sys
import cv2
import numpy as np
sys.path.insert(0, '/path/to/caffe/python'))
import caffe

We will extract the feature vector from the following input image file:

input_image_file = sys.argv[1]

This is the output text file where the line-separated feature vector will be stored:

output_file = sys.argv[2]

We will be using a pretrained model file. You can download it from here (http://dl.caffe.berkeleyvision.org/bvlc_reference_caffenet.caffemodel):

model_file = '/path/to/bvlc_reference_caffenet.caffemodel'

Specify the corresponding deploy prototxt file:

deploy_prototxt = '/path/to/deploy.prototxt'

We are now ready to initialize the convolutional neural network:

net = caffe.Net(deploy_prototxt, model_file, caffe.TEST)

Let’s say we want to extract the feature vector from the layer ‘fc7’ in the network. Define it:

layer = 'fc7'
if layer not in net.blobs:
    raise TypeError("Invalid layer name: " + layer)

We need to specify the image mean file for the image transformer:

imagemean_file = '/path/to/ilsvrc_2012_mean.npy'

We need to define the transformer in order to preprocess the input image before feeding it into the network:

transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_mean('data', np.load(imagemean_file).mean(1).mean(1))
transformer.set_transpose('data', (2,0,1))
transformer.set_raw_scale('data', 255.0)

Reshape the network blob (if needed) to the shape needed for the current CNN architecture being used:

net.blobs['data'].reshape(1,3,227,227)

Load the input image:

img = caffe.io.load_image(input_image_file)

Run the image through the preprocessor:

net.blobs['data'].data[...] = transformer.preprocess('data', img)

Run the image through the network:

output = net.forward()

Extract the feature vector from the layer of interest:

with open(output_file, 'w') as f:
    np.savetxt(f, net.blobs[layer].data[0], fmt='%.4f', delimiter='\n')

Go ahead and open the output text file. You will see a text containing 4096 lines, where each line contains a floating point value. This is the 4096-dimensional feature vector!

———————————————————————————————-

4 thoughts on “How To Extract Feature Vectors From Deep Neural Networks In Python Caffe

  1. Hello! Where do you use the `layer = ‘fc7’`. I tried the above method, and I get a 1000 size array. Clearly something is amiss.

  2. I got this error:
    I0509 13:10:44.140213 5230 upgrade_proto.cpp:44] Attempting to upgrade input file specified using deprecated transformation parameters: bvlc_reference_caffenet.caffemodel
    I0509 13:10:44.140246 5230 upgrade_proto.cpp:47] Successfully upgraded file specified using deprecated data transformation parameters.
    W0509 13:10:44.140251 5230 upgrade_proto.cpp:49] Note that future Caffe releases will only support transform_param messages for transformation fields.
    I0509 13:10:44.140254 5230 upgrade_proto.cpp:53] Attempting to upgrade input file specified using deprecated V1LayerParameter: bvlc_reference_caffenet.caffemodel
    I0509 13:10:44.481170 5230 upgrade_proto.cpp:61] Successfully upgraded file specified using deprecated V1LayerParameter
    I0509 13:10:44.543210 5230 net.cpp:746] Ignoring source layer loss

  3. I got this error:
    I0509 13:10:44.140213 5230 upgrade_proto.cpp:44] Attempting to upgrade input file specified using deprecated transformation parameters: bvlc_reference_caffenet.caffemodel
    I0509 13:10:44.140246 5230 upgrade_proto.cpp:47] Successfully upgraded file specified using deprecated data transformation parameters.
    W0509 13:10:44.140251 5230 upgrade_proto.cpp:49] Note that future Caffe releases will only support transform_param messages for transformation fields.
    I0509 13:10:44.140254 5230 upgrade_proto.cpp:53] Attempting to upgrade input file specified using deprecated V1LayerParameter: bvlc_reference_caffenet.caffemodel
    I0509 13:10:44.481170 5230 upgrade_proto.cpp:61] Successfully upgraded file specified using deprecated V1LayerParameter
    I0509 13:10:44.543210 5230 net.cpp:746] Ignoring source layer loss

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s