Forums

Hi, I'm trying to load a simple model to the memory before making any calls/requests. My goal with that is to avoid re-loading the model everytime after recieving a request. The problem is that the script is not working if I define and load the model outside of the request method (hello_world by default). However I was able to run the code locally without any errors.

import cv2
import base64
import numpy as np
from keras.models import load_model
from flask import Flask, request, jsonify

app = Flask(__name__)

model = load_model('full_model.h5')  ----> not working

@app.route('/upload', methods=['POST'])

def hello_world():
    model = load_model('full_model.h5')  ----> working

I suspect the problem might be because of the way websites are started up; firstly a single Python process starts up, which imports all of your code. It then forks off "worker" processes to handle requests -- although there would only be one such worker process in a free account like yours, the fork still happens. So if you load the model at import time, it may be that the fork isn't handling copying it over to the new processes correctly.

A workaround would be to use a helper function to access it, something like this:

import cv2
import base64
import numpy as np
from keras.models import load_model
from flask import Flask, request, jsonify

app = Flask(__name__)

model = None
def get_model():
    global model
    if model is None:
        model = load_model('full_model.h5')
    return model


@app.route('/upload', methods=['POST'])
def hello_world():
    model = get_model()

This would load the model inside the request function the first time the endpoint is accessed, but then cache it for later use.