DIY Async I/O

Creating low-level foundations and abstractions for asynchronous programming in Python (i.e., implementing concurrency without using threads).

Step 0 concurrency with threads

Note on threads in Python, like C they are hardware – posix threads the GIL (Global Interpreter Lock) prevents threads in Python from running in parallel they can only run on a single CPU

Step 1 concurrency using callback functions

What some people call “callback hell” the demonstrates replacing loops and threads with small callback functions that rely on the Scheduler to switch between tasks.

Step 2 concurrency with coroutines driven by generator

Use the yield stmt to create async/await and replace counting functions with producers and consumers. See and

Step 3 concurrency combining callbacks and coroutines

Async IO in Python combines the models demonstrated in Step1 and Step 2. This is done using coroutines on top of a callback based Scheduler.


Each file is an executable.

In order of programming model progressing to final example which combines a callback based Scheduler and coroutines which in turn enables the lib to be used with either callback or coroutine I/O solutions.

Threads:, add producer/consumer to threads

Scheduler: recursive callbacks in func -

Yield and wrapped yield:

Queue and Result: and


Combine callback Scheduler and Task wrapped coroutines:

Add I/O tcp_server: