Robuta

Sponsor of the Day: Jerkmate

https://github.com/scrapy/scrapy/issues/6292 Handle robots.txt files not utf-8 encoded · Issue #6292 · scrapy/scrapy · GitHub Summary robots.txt files which are not utf-8 encoded make scrapy raise an UnicodeDecodeError atm. .venv/lib/python3.11/site-packages/scrapy/robotstxt.py robots txt files utf 8 scrapy github handle encoded https://github.com/scrapy/scrapy/issues/3903 Can I get remote server's ip address via response? · Issue #3903 · scrapy/scrapy · GitHub Can I get remote server's ip address via response? For some reason. I'll need get remote site's ip address when parsing response. I looked the document but... get remote ip address scrapy github server via https://github.com/scrapy/scrapy/issues/4135 Switch pickle to protocol 4 · Issue #4135 · scrapy/scrapy · GitHub protocol 4 scrapy github switch pickle issue