Forums

I have tried searching the forums for my previous post on this subject, but with no luck. As such I apologise that this is not a reply, but a new forum topic. Perhaps a link to individual users posts would be a useful addition?

My previous issue related to scraping Auction type websites using PythonAnywhere and Beautiful Soup. The response times were quite slow, and it was suggested this could be changed using requests rather than urllib. This has indeed improved the speed of my site responses.

I also found the following stack overflow link useful describing how to soupstrain BS4 . Speeding up beautifulsoup

The other issue related to strange responses from eBay that did not match those returned from my test environment. Harry Percival replied suggesting it could be a geolocation issue. This was indeed the case and I resolved this using a UK based proxy. (PythonAnywhere is hosted in US, despite being a UK company!) This works great most of the time, but is unreliable as I use free proxies. Does PythonAnywhere have any plans to expand into the Proxy market!? ;-) Failing that, can anyone suggest any other approaches I might explore to fool eBay into thinking I am a UK based user?

I also found the following website useful with regards eBay query string parameters.eBay Search URLs. Though a quick look at the advanced search eBay HTML shows that the specific form entries class names match the URL codes. (not surprising really, when you think about it?)

It doesn't really matter that it's a new topic.

We have no plans to provide any proxying features.