Data Engineer Challenge

This goal of this repository is based on solving a technical challenge for the data engineering position.

Check the article here: Design, Development and Deployment of a simple Data Pipeline


  • Install Docker Desktop on Windows, it will install Docker Compose as well, Docker Compose will allow you to run multiple container applications.
  • Install git-bash for windows, once installed, open git bash and download this repository, this will download the docker-compose.yaml file, and other files needed.

ramse@DESKTOP-K6K6E5A MINGW64 /c
$ git clone
  • Once all the files needed were downloaded from the repository, let’s run everything. We will use the git bash tool again, go to the folder data-engineer-challenge and we will run the Docker Compose command:

ramse@DESKTOP-K6K6E5A MINGW64 /c
$ cd data-engineer-challenge

ramse@DESKTOP-K6K6E5A MINGW64 /c/data-engineer-challenge
$ cd code

ramse@DESKTOP-K6K6E5A MINGW64 /c/data-engineer-challenge/code
$ cd apps

@DESKTOP-K6K6E5A MINGW64 /c/data-engineer-challenge/code/apps
$ docker-compose up
  • After wait for a couple of minutes the final result of executing the above command should look like this:



  • let’s try the request to the API REST


  • let’s check the response from the API REST



View Github