Hydrosphere Mist is a serverless proxy for Spark cluster. Mist provides a new functional programming framework and deployment model for Spark applications.
This project aims to evaluate the feasibility of an architecture based on serverless technology for a web crawler using scrapy as a framework of choice.