Forums

Help accessing SQL database inside Flask App

I am running into a strange error: I am able to successfully access my wordpress SQL database using SSH tunnel inside my Flask app like this:

import json
from turtle import position
from flask import Flask, request, jsonify
import pandas as pd
import numpy as np
import xgboost as xgb
from xgboost.sklearn import XGBRegressor
from itertools import combinations
import pickle
from flaskext.mysql import MySQL
from sshtunnel import SSHTunnelForwarder
from phpserialize import *
from os.path import expanduser
import paramiko

app = Flask(__name__)

mypkey = paramiko.RSAKey.from_private_key_file(r"/home/joeyengelberg/mysite/id_rsa")
sql_hostname = 'XXX.X.X.X'
sql_username = 'XXXXXXXX'
sql_password = 'XXXXXXXXXXXXXXX'
sql_main_database = 'XXXXXXX'
sql_port = 3306
ssh_host = 'XXXX.XXX.XXXXXX.XXX'
ssh_user = 'XXXXXXX'
ssh_port = 22
sql_ip = '1.1.1.1.1'

tunnel =  SSHTunnelForwarder(
    (ssh_host, ssh_port),
    ssh_username=ssh_user,
    ssh_pkey=mypkey,
    remote_bind_address=(sql_hostname, sql_port))

#Start Tunnel
tunnel.start()

mysql = MySQL()
app.config['MYSQL_DATABASE_USER'] = sql_username
app.config['MYSQL_DATABASE_PASSWORD'] = sql_password
app.config['MYSQL_DATABASE_DB'] = 'XXXXXXX'
app.config['MYSQL_DATABASE_HOST'] = '127.0.0.1'
app.config['MYSQL_DATABASE_PORT'] = tunnel.local_bind_port

mysql.init_app(app)

conn = mysql.connect()
cursor = conn.cursor()

post_key = "1912"

cursor.execute("SELECT meta_value FROM wp_postmeta WHERE post_id = %s AND meta_key = 'sp_statistics', post_key)
data = cursor.fetchone()
output = unserialize(bytes(data[0], 'utf-8'))

conn.close()

def Merge(dict1, dict2):
  res = {**dict1, **dict2}
  return res

@app.route('/')
def hello():
  resp = jsonify({"message":"honey"})
  resp.status_code = 200
  return resp


@app.route('/predict', methods = ['POST','GET'])
def predict():
  if request.method == 'POST':
     resp = jsonify({"message":"post"})
     resp.status_code = 200
     return resp

if request.method == 'GET':
    try:

        post_key = request.args.get("id")

        resp = jsonify(post_key)
        resp.status_code = 200
        return resp

    except Exception as e:
        print(e)

else:
    resp = jsonify({"message":"Something went wrong"})
    resp.status_code = 405
    return resp

I can confirm that the above code works because when I print(output) I get exactly what I want from the SQL database. HOWEVER, when I move the cursor.execute commands inside the if request.method == 'GET': block (which is what I want to do) it no longer works. Specifically, when I go to my web app site and set the id value to 1912 like this:

http://joeyengelberg.pythonanywhere.com/predict?id=1912

the web app just hangs and eventually gives a 504-backend error. Specifically, this code will not work:

import json
from turtle import position
from flask import Flask, request, jsonify
import pandas as pd
import numpy as np
import xgboost as xgb
from xgboost.sklearn import XGBRegressor
from itertools import combinations
import pickle
from flaskext.mysql import MySQL
from sshtunnel import SSHTunnelForwarder
from phpserialize import *
from os.path import expanduser
import paramiko

app = Flask(__name__)

mypkey = paramiko.RSAKey.from_private_key_file(r"/home/joeyengelberg/mysite/id_rsa")
sql_hostname = 'XXX.X.X.X'
sql_username = 'XXXXXXXX'
sql_password = 'XXXXXXXXXXXXXXX'
sql_main_database = 'XXXXXXX'
sql_port = 3306
ssh_host = 'XXXX.XXX.XXXXXX.XXX'
ssh_user = 'XXXXXXX'
ssh_port = 22
sql_ip = '1.1.1.1.1'

tunnel =  SSHTunnelForwarder(
    (ssh_host, ssh_port),
    ssh_username=ssh_user,
    ssh_pkey=mypkey,
    remote_bind_address=(sql_hostname, sql_port))

#Start Tunnel
tunnel.start()

mysql = MySQL()
app.config['MYSQL_DATABASE_USER'] = sql_username
app.config['MYSQL_DATABASE_PASSWORD'] = sql_password
app.config['MYSQL_DATABASE_DB'] = 'XXXXXXX'
app.config['MYSQL_DATABASE_HOST'] = '127.0.0.1'
app.config['MYSQL_DATABASE_PORT'] = tunnel.local_bind_port

mysql.init_app(app)

conn = mysql.connect()
cursor = conn.cursor()

def Merge(dict1, dict2):
  res = {**dict1, **dict2}
  return res

@app.route('/')
def hello():
  resp = jsonify({"message":"honey"})
  resp.status_code = 200
  return resp


@app.route('/predict', methods = ['POST','GET'])
def predict():
  if request.method == 'POST':
     resp = jsonify({"message":"post"})
     resp.status_code = 200
     return resp

if request.method == 'GET':
    try:

        post_key = request.args.get("id")

        cursor.execute("SELECT meta_value FROM wp_postmeta WHERE post_id = %s AND meta_key='sp_statistics', post_key)
        data = cursor.fetchone()
        output = unserialize(bytes(data[0], 'utf-8'))
        conn.close()

        resp = jsonify(output)
        resp.status_code = 200
        return resp

    except Exception as e:
        print(e)

else:
    resp = jsonify({"message":"Something went wrong"})
    resp.status_code = 405
    return resp

Any idea why accessing the SQL database outside of the if block works just fine but inside the if block fails?

I'm a little bit confused by the indentation in your code above, but I'm guessing that if/else with try/Except belongs to predict, right? Have you checked your web app's error log yet? Could you try putting the whole code that creates the connection into the function that is using it?

"but I'm guessing that if/else with try/Except belongs to predict, right?"

Correct.

"Have you checked your web app's error log yet?"

Yes. Here is the error which I believe just reflects the fact that the connection to the database was not performed so that noting was returned:

2022-07-16 06:00:25,952: Exception on /predict [GET] Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2051, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1502, in full_dispatch_request return self.finalize_request(rv) File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1521, in finalize_request response = self.make_response(rv) File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1684, in make_response raise TypeError( TypeError: The view function for 'predict' did not return a valid response. The function either returned None or ended without a return statement.

"Could you try putting the whole code that creates the connection into the function that is using it?"

I have tried that and, unfortunately, it does not work. I have also checked two other things (1) id is passing into the post_key variable correctly because if I ask for it to return post_key I get the correct 1912 and (2) when I run the code on my local machine it runs fine. So it must be something unique with Python Anywhere that I am not appreciating.

Can you show the version of code you used to create the connection inside the view? Did you create the tunnel as well?

You were correct that putting everything, including the tunneling, into the block makes it work. Thanks for the suggestion and the help!

Glad to hear that!