Forums

requests.get not working during API calls

Account: Free user

I am trying to send a message to my telegram chat using URL: https://api.telegram.org.

** This url is whitelisted for free users**

Below is the code which i have for the sending Telegram Message when running code with run command from pythonanywhere file editor everything works properly and iam able to get the response and message in telegram

import requests
from env import TELE_BOT_TOKEN as api_token

chat_id=158775078

def send_message(userName, emailId, message):
     text = f"""*Name*: {userName}
 *MailID*: {emailId}
 *Message*:
 {message}"""

    url = f'https://api.telegram.org/bot{api_token}/sendMessage'
    params = { 'chat_id':chat_id, 'text':text, 'parse_mode':'Markdown' }

    try:
        res = requests.get(url, params=params )
        return res.__dict__
    except requests.exceptions.RequestException as e:
        print(e)
        return 404

if __name__ == "__main__":
    print(send_message("vimal", "vimal@gmail.com", "test"))

My Api endpoint When calling via API am not able to get the response and the error message in console is not helping me much

@api.post('/sendmsg')
def send_tele_msg():
    username, emailid, message = request.json.get('userName'), request.json.get('emailId'), request.json.get('message')
    print(request.__dict__)
    print(username, emailid, message)
    msg = telebot.send_message(**request.json)
    return {'status':  True}

when trying from postman got the error

Error code: 504-loadbalancer

Kindly help me to resolve the issue. Same code is working properly in my local Machine

This is a duplicate post of https://www.pythonanywhere.com/forums/topic/27934/#id_post_82612

A 504-loadbalancer means that your view did not return a response to send back to the browser within a few minutes, so it timed out. That suggests to me that your call to telebot.send_message is not returning. What code are you using to create the telebot object?

Hi giles, Thanks for the response, yes i think res = requests.get(url, params=params ) is not returning response, my concern here is, when i am running it in pythonanywhere file editor run button it is working issue comes only when a request from postman or from UI. and i also saw this message in server logs.

2020-07-12 11:53:24 Sun Jul 12 11:53:24 2020 - *** HARAKIRI ON WORKER 1 (pid: 6, try: 1) ***
2020-07-12 11:53:24 Sun Jul 12 11:53:24 2020 - HARAKIRI !!! worker 1 status !!!
2020-07-12 11:53:24 Sun Jul 12 11:53:24 2020 - HARAKIRI [core 0] 10.0.0.52 - POST /api/v1/sendmsg since 1594554203
2020-07-12 11:53:24 Sun Jul 12 11:53:24 2020 - HARAKIRI !!! end of worker 1 status !!!
2020-07-12 11:53:24 DAMN ! worker 1 (pid: 6) died, killed by signal 9 :( trying respawn ...
2020-07-12 11:53:24 Respawned uWSGI worker 1 (new pid: 11)

i am using same logic to fetch gist from github and those requests are working properly

telebot object is the import of the code snippet which is there in op

The message in the logs is just what happens when the request times out -- it's essentially the log-side equivalent of a 504 error.

To debug this, it might help if you add some prints to your telebot.py module to find out which line it is blocking on. If you do this:

print(f"{datetime.now()} about to do X", flush=True)

...above each line, saying what the thing the code is about to do, then in the server log you'll get timestamped messages. My guess is that it will show that it is blocking on the requests.get, but it would be good to confirm that so that we don't head off down the wrong path when debugging.

Hi giles, thanks. yes the program was not processing after request.get. when i tried with print statement, code hang when executing requests.get

2020-07-12 17:28:30.863959 POST request, sendmsg
2020-07-12 17:28:30.864391 Inputs from request name: test, email: test@test.com, message: test
2020-07-12 17:28:30.864438 About to send telegram message

As it was working when i executed the file in console. i tried executing my script with subprocess and all the methods started working.

print(f"{datetime.datetime.now()} send tele message, trying subprocess", flush=True)
import subprocess
proc = subprocess.Popen(f'python3 telebot.py vimal vimal@g.com thisismsg', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
m,n = proc.communicate()
print(f"{datetime.datetime.now()} output: {m}", flush=True)
print(f"{datetime.datetime.now()} message: {n}", flush=True)

Ouptut:

2020-07-12 17:59:55.566586 POST request, sendmsg
2020-07-12 17:59:55.567100 Inputs from request name: test, email: test@test.com, message: test
2020-07-12 17:59:55.567209 send tele message, trying subprocess
2020-07-12 17:59:56.464343 output: b'200\n'
2020-07-12 17:59:56.464416 message: b'DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.telegram.org:443\nDEBUG:urllib3.connectionpool:https://api.telegram.org:443 "GET /bot<token>/sendMessage?chat_id=158775078&text=%2AName%2A%3A+vimal%0A%2AMailID%2A%3A+vimal%40g.com++%0A%2AMessage%2A%3A++%0Athisismsg&parse_mode=Markdown HTTP/1.1" 200 491\n'
2020-07-12 17:59:56.464432 Going to send telegram message, trying requests.get
2020-07-12 17:59:57.000888 Completed
2020-07-12 17:59:57.000958 About to send telegram message
2020-07-12 17:59:57.524078 Telegram Message sent
2020-07-12 17:59:57.524125 Going to send mail
2020-07-12 17:59:58.489538 Mail Sent

To experiment more, i removed subprocess code again and tried api call, now again execution hang at request.get

2020-07-12 18:12:09.150283 POST request, sendmsg
2020-07-12 18:12:09.150822 Inputs from request name: test, email: test@test.com, message: test
2020-07-12 18:12:09.150935 Going to send telegram message, trying requests.get

Any thoughts on this ?

So you are no longer doing it using your web app? How does the current code look like?

Hi fjl, I am currently using 3 methods to send message to telegram

1. using subprocess
2. using requests.get
3. using python-telegram-bot module

with subprocess enabled all 3 methods are working properly. without it none works as mentioned in above message.

Please find the code below

API ENDPOINT

@api.post('/sendmsg')
def send_tele_msg():

    username, emailid, message = request.json.get('userName'), request.json.get('emailId'), request.json.get('message')
    text = f"""*Name*: {username}\n*MailID*: {emailid}\n*Message*:\n{message}"""
    chat_id=158775078

    from env import TELE_BOT_TOKEN as api_token
    url = f'https://api.telegram.org/bot{api_token}/sendMessage'
    params = { 'chat_id':chat_id, 'text':text, 'parse_mode':'Markdown' }

    print(f"{datetime.datetime.now()} send tele message, trying subprocess", flush=True)
    import subprocess
    proc = subprocess.Popen(f'python3 telebot.py {username} {emailid} {message}', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    m,n = proc.communicate()

    print(f"{datetime.datetime.now()} Going to send telegram message, trying requests.get", flush=True)
    res = requests.get(url, params=params )
    print(f"{datetime.datetime.now()} Completed", flush=True)

    print(f"{datetime.datetime.now()} About to send telegram message - python-telegram-bot module", flush=True)
    res = tele_bot.send_message(chat_id=chat_id,
                text=text,
                parse_mode=telegram.ParseMode.MARKDOWN)
    print(f"{datetime.datetime.now()} Telegram Message sent", flush=True)

    return {'status':  True }

** telebot.py**

import requests
from os import getenv
from env import TELE_BOT_TOKEN as api_token

chat_id=158775078

def send_message(userName, emailId, message):
    text = f"""*Name*: {userName}
*MailID*: {emailId}  
*Message*:  
{message}"""

    url = f'https://api.telegram.org/bot{api_token}/sendMessage'
    params = { 'chat_id':chat_id, 'text':text, 'parse_mode':'Markdown' }

    res = requests.get(url, params=params )

    return res.status_code


if __name__ == "__main__":
    from sys import argv

    print(send_message(argv[1], argv[2], argv[3]))

That's strange. Are the responses that you're getting from the attempts in the web app code not giving you some sort of error message that you can use to work out why they failed?

No am not getting any error message. code execution hungs at

res = requests.get(url, params=params )

and then nothing :(

If a simple requests request like that is hanging in your web app, that would suggest to me that you're using (possibly by mistake) some sort of library that is modifying requests so that it needs threads or something like that or perhaps it's a fork of requests that you're using instead of the original requests library.

Thanks alot glenn, found the issue. I am was using gevent server in my local machine and monkey patched modules. this was the problem. after removing monkeypatch, it was working properly.

Ah yes! That would do it. Glad we could find it.

Yes, Thanks again :)

HI Team,

I am able to access an Open API from Covid through my local machine but getting 404 when accessing through Pythonanywhere server,

Below is the OPEN API link https://apisetu.gov.in/public/marketplace/api/cowin

However same API is working fine without any security or API key in local machine

<div class="codehilite"><pre><code> headers={'User-Agent':"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like ## Gecko) Chrome/90.0.4430.93 Safari/537.36"}

try: response_state = rq.get("https://cdn-api.co-vin.in/api/v2/admin/location/states",headers=headers,verify=False) json_data_state = json.loads(response_state.text) print(response_state.text) print(json_data_state)#['states'] except JSONDecodeError as e: print(e)

</code></pre></div> Can you please help on this?

Thanks MuxTrap

cdn-api.co-vin.in is already on the whitelist. Check whether the endpoint that you're accessing is redirecting to a different site and, if it is, send us a link to the API documentation for the new site and we'll consider it for the whitelist.

Hi, I am running the same issue while trying to access the api "https://api.dineoncampus.com". Can this be whitelisted? File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 696, in urlopen self._prepare_proxy(conn) File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 964, in _prepare_proxy conn.connect() File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 366, in connect self._tunnel() File "/usr/local/lib/python3.9/http/client.py", line 903, in _tunnel raise OSError(f"Tunnel connection failed: {code} {message.strip()}") OSError: Tunnel connection failed: 403 Forbidden

@ysulkin -- provide us with the official documentation for the API and we'll see what we can do.

I am writing to seek assistance with a technical issue I am facing while trying to connect APIs to the whitelist on my PythonAnywhere account. I have a PAID subscription account and am encountering a 403 error when attempting to access the APIs, despite them working seamlessly on my local server.

To provide a comprehensive overview of the situation, here are the details of the APIs I am trying to connect:

DVLA VEHICLE MOT HISTORY:
    Documentation: https://dvsa.github.io/mot-history-api-documentation/
    URL: https://beta.check-mot.service.gov.uk/
    Endpoints:
        /trade/vehicles/mot-tests?registration={registration}
        /trade/vehicles/mot-tests?page={page}
        /trade/vehicles/mot-tests?date={date}&page={page}
        /trade/vehicles/mot-tests?vehicleId={vehicleId}

DVLA VEHICLE ENQUIRY:
    Documentation: https://developer-portal.driver-vehicle-licensing.api.gov.uk/apis/vehicle-enquiry-service/vehicle-enquiry-service-description.html
    URL: https://driver-vehicle-licensing.api.gov.uk/vehicle-enquiry/v1/vehicles
    CURL Code:
    curl -L -X POST 'https://driver-vehicle-licensing.api.gov.uk/vehicle-enquiry/v1/vehicles' \
    -H 'x-api-key: REPLACE WITH YOUR API KEY' \
    -H 'Content-Type: application/json' \
    -d '{"registrationNumber": "TE57VRN"}'

VEHICLE IMAGE:
    Documentation: https://ukvehicledata.co.uk/Vehicle-Images
    URL: uk1.ukvehicledata.co.uk
    Endpoint:
    /api/datapackage/VehicleImageData?v=2&api_nullitems=1&auth_apikey={APIKEY}&key_VRM={keyword}

Despite my experience as a senior Full Stack Web Developer and my use of PAID subscription services, I have been unable to resolve the issue using the steps I usually employ for debugging. I have ensured that the correct API keys and permissions are in place, that whitelist configurations are correctly set up, and that firewall settings are not causing the issue.

I suspect there might be some nuances specific to PythonAnywhere that I might not be aware of. I kindly request your guidance in identifying and resolving the root cause of this 403 error. Your insights will be greatly appreciated.

If there are any specific settings, configurations, or checks I need to perform within the PythonAnywhere environment to ensure seamless API connectivity, please let me know. Additionally, if there are any logs or error messages that can shed light on the cause of the issue, I would be grateful to receive them.

Thank you for your prompt attention to this matter. I understand that you receive a high volume of inquiries, and I genuinely appreciate your support in helping me overcome this technical hurdle. ALLOW POST AND GET ALL REQUESTS

Could you show the exact response you get? I tried one of those urls and got 403, too, with content {'message': 'Missing Authentication Token'}. If that's similar in your case, I'd check if the tokens/keys you provide are interpolated correctly on PythonAnywhere.