Forums

Help - Caching OpenCV

Hi there,

I'm currently running a Django webapp that uses OpenCV (cv2) and Numpy for image processing/face detection. One of my Caffe files is 500mb and everytime I hit my API it has to load these files making the request several seconds to execute. I was wondering if there's a way to load these files up into memory / cache the files. When I try with the normal caching methods I get a pickling error.

My code to load up my Caffe files is:

def load_face_net():
    THIS_FOLDER = os.path.dirname(os.path.abspath(__file__))
    FACE_PROTO_PATH = os.path.join(THIS_FOLDER, FACE_PROTO)
    FACE_MODEL_PATH = os.path.join(THIS_FOLDER, FACE_MODEL)
    # face_net = cache.get('face_net')
    # if face_net is None:
    face_net = cv2.dnn.readNetFromCaffe(FACE_PROTO_PATH, FACE_MODEL_PATH)
    # cache.set('face_net', face_net, 3600)
    return face_net

def load_gender_net():
    THIS_FOLDER = os.path.dirname(os.path.abspath(__file__))
    GENDER_MODEL_PATH = os.path.join(THIS_FOLDER, GENDER_MODEL)
    GENDER_PROTO_PATH = os.path.join(THIS_FOLDER, GENDER_PROTO)
    # gender_net = cache.get('gender_net')
    # if gender_net is None:
    gender_net = cv2.dnn.readNetFromCaffe(GENDER_MODEL_PATH, GENDER_PROTO_PATH)
    # cache.set('gender_net', gender_net, 3600)
    return gender_net

If it is the file that never changes you can load it while web app starts not during request handling.

Yes it's a file that never changes, how would I load it while the web app starts?

You can just use a global variable that starts out None then, in each of your load_* functions, you can either return the global variable if it's not None or load the file and set the global variable with the result.