Forums

I just want it to scrape a data page from 9am to 6pm. After that time, just take data from cache. Is that possible? Reason is the reduce the resources.

@app.route('/top', methods=['GET'])
@app.cache.cached(timeout=60)
def top():
    headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}
    page = requests.get('http://www.abc.com/ata?qrRAll',headers=headers)
return "is working"

Sure. Just check the time in the request and if it's between the times you want to serve the cached result, send the cached result. Otherwise get the new value.

Hi Glenn,

Yup, I know. But is there a function that I can call the data I last cache if it is out of the time range I set?

you would probably have to write out your own logic for that.

.

from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.cache import Cache
app.config['CACHE_TYPE'] = 'simple'
app.cache = Cache(app)

    @app.route('/thtop', methods=['GET'])
    @app.cache.cached(timeout=60)
    def thtop():
        now = datetime.now()
        now_time = now.time()
        if now_time >= time(3,30) and now_time <= time(16,30):
            rv = app.cache.get('last_response')
        else:
           rv = 'abcc'
            app.cache.set('last_response', rv, timeout=3600)
        return rv

however, when it is in the time range, it will return Exception on /thtop [GET]#012Traceback (most recent call last):#012 File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1687, in wsgi_app#012 response = self.full_dispatch_request()#012 File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1361, in full_dispatch_request#012 response = self.make_response(rv)#012 File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1439, in make_response#012 raise ValueError('View function did not return a response')#012ValueError: View function did not return a response

[edit by admin: formatting]

I don't know the Flask-Cache extension, but that error suggests that there's nothing in the cache during that interval -- which I think makes sense: when you put stuff into the cache it has a timeout of 3600 seconds, which is one hour. That would mean that one hour into your period when you always use the cache, there would be nothing left in there.