Forums

I'm trying to run a script for geocoding using a library called geocoder. The code is below:

@RateLimited(5)
def Geocode(addresses):
    latlong = {}
    for i in addresses:
        if i in latlong:
           break
        else:
           x = geocoder.google(i)
           latlong[i]=x.latlng
    return latlong

I'm also using a simple rate limiting function that I snitched from here to ensure that there aren't over 5 requests being sent out per second as well. I've tried Google, OSM, Bing Maps APIs and the function is returning only 3 results! The entire geocoding dict that I'm passing is just around a thousand addresses, so its not being rate-limited by the APIs I'm sure.

Any suggestions on how to resolve this? Thanks!

I guess the intention of:

 if i in latlong:
            break

is to avoid processing duplicates. If there aren't any duplicate addresses, this will never happen.

But if there -are- any duplicate addresses, it will always return on the first duplicate (because of the break) and never get past that point - maybe you mean pass instead of break?

Do you want to only allow one address lookup per call, for the rate limiting to work? In that case you need a break after the line latlong[i]=x.latlng, and of course the return dictionary would only ever have at most one item.

I suspect you may want to pass in the latlong dictionary and update it in each call, in which case latlong should be an argument, and don't initialise it inside this script.

HTH Jim

Uggh! Yes, the 'Break' was the problem. Thanks, got that sorted out finally!

Would love to get your suggestion on one thing - how would I go about pausing the program execution for 24 hours say, since Google/other map companies have restricted geocoding lookups to 2500 requests/day, and then restart execution from the last point? Would time.sleep be enough?

Sounds like a job for a scheduled task? See the Schedule tab on your dashboard...

Will check it out harry, thanks!