https://github.com/NikolaiT/Crawling-Infrastructure
GitHub - NikolaiT/Crawling-Infrastructure: Distributed crawling infrastructure running on top of...
Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues. -...
on topgithubcrawlinginfrastructuredistributed