Python requests are being used and are currently preferable to Selenium if there is a way to make it work.
To confirm, yes, the same session/request workflow is run in PyCharm on a mac (10.13.6) with the intended results.
Testing further, I retried this particular example (posted below) as its own scheduled task multiple times, and it unexpectedly worked each time. However, again, when transferring the code to the main function/thread launched from a scheduled task, I get the same results (javascript validation page).
A minimum working example is posted below followed by the 'javascript validation' response text returned when the request doesn't work. Let me know if other details are needed.
Note: when attempting the main request by itself (i.e. even without the preliminary request for IDs), it appears to work the first time only. So, if it works when testing, it would need to be run again to be sure the IDs are being validated.
If running the example, you'll know that it works if "-- listings" is printed after JSON results. "-- HTML following error" with the error HTML would be printed otherwise.
MWE:
*As this may have been needlessly cluttering this post, it was removed (but can be posted or sent privately upon request)
Response when not working (side note: validation IDs still show in session.cookies when this occurs):
<!DOCTYPE html>
<html>
<head>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<meta http-equiv="cache-control" content="max-age=0" />
<meta http-equiv="cache-control" content="no-cache" />
<meta http-equiv="expires" content="0" />
<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT" />
<meta http-equiv="pragma" content="no-cache" />
<meta http-equiv="refresh" content="10; url=https://www.hubzu.com/distil_r_captcha.html?requestId=f06abe3b-f8ea-481b-969b-0ecb42465ecb&httpReferrer=%2FsearchResult%2Fcounty%2F1072431%2Ffl%2Fbroward-county%3FsrchBtnClk%3D1%26searchBy%3DBroward%2BCounty%2BFL%26pageNumber%3D3%26ajaxhtml%3Dtrue" />
<script type="text/javascript">
(function(window){
try {
if (typeof sessionStorage !== 'undefined'){
sessionStorage.setItem('distil_referrer', document.referrer);
}
} catch (e){}
})(window);
</script>
<script type="text/javascript" src="/gxgmeitaklswiexy.js" defer></script><style type="text/css">#d__fFH{position:absolute;top:-5000px;left:-5000px}#d__fF{font-family:serif;font-size:200px;visibility:hidden}#sfyaazqtacfrrw{display:none!important}</style><script type="text/javascript" src="https://n-cdn.areyouahuman.com/play/ZJFYkJE5SICN8qC78YCEaQsMw8PeMdzQFeIwtPBN?AYAH_P1=f06abe3b-f8ea-481b-969b-0ecb42465ecb&AYAH_P2=F3F9BF55-80F2-3663-B46E-FFBD19395FE5&AYAH_P3=18886A2C-D2F1-3CA0-B17D-4A9340440B2A&AYAH_F1=2968&AYAH_F2=12495"></script>
<noscript><img src="https://n-cdn.areyouahuman.com/noscript/ZJFYkJE5SICN8qC78YCEaQsMw8PeMdzQFeIwtPBN?AYAH_P1=f06abe3b-f8ea-481b-969b-0ecb42465ecb&AYAH_P2=F3F9BF55-80F2-3663-B46E-FFBD19395FE5&AYAH_P3=18886A2C-D2F1-3CA0-B17D-4A9340440B2A&AYAH_F1=2968&AYAH_F2=12495"></noscript></head>
<body>
<div id="distilIdentificationBlock"> </div>
</body>
</html>
conrad
|
4232
posts
|
PythonAnywhere staff
|
