All the components necessary to run a game of the OOO DC CTF finals.
This repo contains all the game components necessary to run an Attack-Defense CTF that OOO used from 2018–2021.
The design is based on adamd’s experience building the ictf-framework.
There are fundamental tenenats that we try to follow in the design of the system:
Spoke component model
The communication design of the components in the system (which you can kind of think of as micro-services) is a “spoke” model, where every component talks to the database (through a RESTish API), and no component directly talks to any other.
In this way, each component can be updated separately and can also be scaled independently using our k8s hosting.
This also made testing of each component easier, as the only dependence on a component is on the state of the database.
The only exception to this is the
patchbot (the component that needs to test the patches submitted by the teams).
The database API puts the
patchbot testing jobs into an RQ (Redis Queue), which all the
patchbot workers pull jobs from.
Append-only database design
Fundamentally, a CTF database needs to calculate scores (that’s essentially what the teams care about).
Prior design approaches that we’ve used would have a
score column in the
team table, and when they acquired or lost points, the app code would change this value.
However, many crazy things can happen during a CTF: recalculating scores or missed flags, even changing the scoring functions itself.
These can be difficult to handle depending on how the system is developed.
Therefore, we created a completely append-only database model, where no data in the DB is ever deleted or changed.
Even things like
service status (the GOOD, OK, LOW, BAD that we used) is not a column in the
Every change of status would created a new
StatusIndicator row, and the
services would pull the latest version from this table.
Related to the append-only database design, everything in the database was represented by events.
The database would store all game events (in our game over the years was
Then, the state of the game is based on these events.
An additional benefit is that these events could be shipped to the teams as part of the
Separate k8s clusters
How we ran this is with two k8s clusters: an admin cluster and a game cluster.
admin cluster ran all of these components.
game cluster ran all of the CTF challenges.
We used this design to do things like drop flags on the services.
kubectl to drop a flag onto a service running in the other cluster.
This also allowed us to lock down the
game cluster so that the vulnerable services couldn’t make external requests, could be scaled separately, etc.
This package is pip installable, and installs all dependencies. Do the following in a virtualenv:
$ pip install -e .
NOTE: If you want to connect to a mysql server (such as in prod or when deving against a mysql server), install the
mysqlclient dependency like so:
$ pip install -e .[mysql]
Make sure the tests pass before you commit, and add new test cases in test for new features.
Note the database API now checks that the timezone is in UTC, so you’ll need to specify that to run the tests:
$ TZ=UTC nosetests -v
If you’re using tmux, I created a script local_dev.sh
that will run a database-api, database-api frontend, team-interface
backend, team-interface frontend, gamebot, and an ipython session with
a database client created.
Just run the following
Deploy to prod
-p push the image to production registry.
$ ./deploy.sh -p
-r restart the running services, need to do:
$ ./deploy.sh -p -r
This has the tables for the database, a REST API to access it, and a python client to access the REST API.
See ooogame/database for details.
Responsible for putting new flags into all the services for every game tick.
See ooogame/flagbot for details.
Responsible for putting a new flags into a pod when it first comes up (from a team patching the service).
See ooogame/fresh_flagbot for details.
Responsible for incrementing the game’s ticks.
See ooogame/gamebot for details.
Responsible for extracting the King of the Hill (koh) scores from all
the koh pods every tick, and submitting them to the database.
See ooogame/koh_scorebot for details.
Responsible for providing an interface to the teams so that they can
submit flags, get pcaps, upload patches, and get their patch status.
Split into a backend flask REST API, which essentially wraps the
database-api, and a React frontend.
See ooogame/team_interface for details.
Responsible for picking up all the newly generated pcaps, anonymize
them, and if the service is releasing pcaps then release them.
See ooogame/pcapbot for details.
Responsible for creating the game state at every new tick and storing them in the nfs, and release them publicly.
See ooogame/gamestatebot for details.
This is also the component that pushes data to the public scoreboard