Local Spark Dev Env with Docker
Development environment for k8s.
Using the spark-operator image to ensure it will be the same environment.
docker-compose up -d
Copy dependencies jars
docker cp jars/. spark:/opt/spark/jars
Create a local alias for spark-submit
alias spark-s="docker exec -it spark /opt/spark/bin/spark-submit"
Clean after work