problem with resolving swarm check-ins ID in the servers : Forums : PythonAnywhere

problem with resolving swarm check-ins ID in the servers

hi there, from yesterday i have a problem with my scripts that running on your servers. these script listen to twitter streaming API to detect swarm check-ins. after that, check-ins are resolved with the foursquare tutorial instruction. i have errors as below:

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Access to this page has been denied.</title> <link href="https://fonts.googleapis.com/css?family=Open+Sans:300" rel="stylesheet"> <style> html, body { margin: 0; padding: 0; font-family: 'Open Sans', sans-serif; color: #000; } a { color: #c5c5c5; text-decoration: none; } .container { align-items: center; display: flex; flex: 1; justify-content: space-between; flex-direction: column; height: 100%; } .container > div { width: 100%; display: flex; justify-content: center; } .container > div > div { display: flex; width: 80%; } .customer-logo-wrapper { padding-top: 2rem; flex-grow: 0; background-color: #fff; visibility: hidden; } .customer-logo { border-bottom: 1px solid #000; } .customer-logo > img { padding-bottom: 1rem; max-height: 50px; max-width: 100%; } .page-title-wrapper { flex-grow: 2; } .page-title { flex-direction: column-reverse; } .content-wrapper { flex-grow: 5; } .content { flex-direction: column; } .page-footer-wrapper { align-items: center; flex-grow: 0.2; background-color: #000; color: #c5c5c5; font-size: 70%; } @media (min-width: 768px) { html, body { height: 100%; } } </style>  </head> <body> <section class="container"> <div class="customer-logo-wrapper"> <div class="customer-logo"> <img src="" alt="Logo"/> </div> </div> <div class="page-title-wrapper"> <div class="page-title"> <h1>Please verify you are a human</h1> </div> </div> <div class="content-wrapper"> <div class="content"> <div id="px-captcha"> </div> <p> Access to this page has been denied because we believe you are using automation tools to browse the website. </p> <p> This may happen as a result of the following: </p> <ul> <li> Javascript is disabled or blocked by an extension (ad blockers for example) </li> <li> Your browser does not support cookies </li> </ul> <p> Please make sure that Javascript and cookies are enabled on your browser and that you are not blocking them from loading. </p> <p> Reference ID: #a897ecd0-d975-11e8-a76b-85eafc238d60 </p> </div> </div> <div class="page-footer-wrapper"> <div class="page-footer"> <p> Powered by <a href="https://www.perimeterx.com/whywasiblocked">PerimeterX</a> , Inc. </p> </div> </div> </section>  <script> window._pxAppId = 'PX0UqK4c76'; window._pxJsClientSrc = '/0UqK4c76/init.js'; window._pxFirstPartyEnabled = true; window._pxVid = ''; window._pxUuid = 'a897ecd0-d975-11e8-a76b-85eafc238d60'; window._pxHostUrl = '/0UqK4c76/xhr'; </script> <script src="/0UqK4c76/captcha/captcha.js?a=c&u=a897ecd0-d975-11e8-a76b-85eafc238d60&v=&m=0"></script>  </body> </html>

is there any change on servers that made this problem? or is it the foursquare servers problem? I think there is not any problem with foursquare. I make a test on the browser in my pc. please help to solve the problem as soon as possible.

i am looking forward to hearing from you. thanks.

sabzali | 31 posts | Oct. 29, 2018, 5:45 a.m. | permalink

The page says:

Access to this page has been denied because we believe you are using automation tools to browse the website. This may happen as a result of the following:

Javascript is disabled or blocked by an extension (ad blockers for example)
Your browser does not support cookies

Please make sure that Javascript and cookies are enabled on your browser and that you are not blocking them from loading.

If you are scraping their website, then it is up to them to decide if they want to block you or not.

conrad | 4232 posts | PythonAnywhere staff | Oct. 29, 2018, 3:49 p.m. | permalink

I use the foursqaure api for nearly two years. i thinks there is another problem. i make the same request on my PC browser without the mentioned problem. Do you have make any change on your servers nearly??

sabzali | 31 posts | Oct. 29, 2018, 4:19 p.m. | permalink

I run a python program on my own server to automate checkins using the Foursquare API. It suddenly stopped working two days ago with the same output. What's weird is it works fine in PHP.

Here's my code.

PHP: https://pastebin.com/JQfuCjsd

Python: https://pastebin.com/mv0Sh6X7

Output from checkin.text: https://pastebin.com/N9hB5Cz9

deleted-user-4742438 | 4 posts | Oct. 30, 2018, 3:34 a.m. | permalink

The reason for the error is in the response: "Access to this page has been denied because we believe you are using automation tools to browse the website." Foursquare is blocking you because they do not want you to access their site using automated tools. The PHP may be working just because they are not recognising it as an automated request for some reason.

glenn | 9498 posts | PythonAnywhere staff | Oct. 30, 2018, 12:01 p.m. | permalink

hi, what should i do?? any solution ??

sabzali | 31 posts | Oct. 30, 2018, 2:16 p.m. | permalink

Use their API in a way that is approved by them.

glenn | 9498 posts | PythonAnywhere staff | Oct. 30, 2018, 4:12 p.m. | permalink

This fixed it for me:

headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36'}

Then...

checkin = requests.post(url=checkinURL, params=checkinParams, headers=headers) print (checkin)

Now I get response 200.

As for why it suddenly stopped working when it worked without that before, I dont know.

deleted-user-4742438 | 4 posts | Oct. 30, 2018, 10:10 p.m. | permalink

This fixed it for me:

headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36'}

Then...

checkin = requests.post(url=checkinURL, params=checkinParams, headers=headers) print (checkin)

Now I get response 200.

As for why it suddenly stopped working when it worked without that before, I dont know.

deleted-user-4742438 | 4 posts | Oct. 30, 2018, 10:10 p.m. | permalink

hi, thanks. but how can i put user authentication (client_id and client_secret) in post request. in header or params? thanks.

sabzali | 31 posts | Oct. 31, 2018, 3:58 p.m. | permalink

{"meta":{"code":405,"errorType":"other","errorDetail":"This endpoint only supports GET.","requestId":"5bd9ed6ddd57971481dd6853"},"response":{}} I can't make post request. I received above error.

sabzali | 31 posts | Oct. 31, 2018, 6:01 p.m. | permalink

Are you trying to add a check-in? If so, that method is post. I also use my Oauth token, not client ID and secret to make one.

If this is something other people will be using, using an oauth generator will make it so much easier for them to use.

deleted-user-4742438 | 4 posts | Oct. 31, 2018, 6:09 p.m. | permalink

https://developer.foursquare.com/docs/api/checkins/resolve I am using this part of documentation. it said that this part receive GET requests.

sabzali | 31 posts | Oct. 31, 2018, 6:51 p.m. | permalink

These seem like problems you are having with the foursquare api. If you continue to struggle the best may be to hire some help.

conrad | 4232 posts | PythonAnywhere staff | Nov. 1, 2018, 4:36 p.m. | permalink

i fix the problem. thanks.

sabzali | 31 posts | Nov. 1, 2018, 5:39 p.m. | permalink