DIY Async I/O
Creating low-level foundations and abstractions for asynchronous programming in Python (i.e., implementing concurrency without using threads).
Step 0 concurrency with threads
Note on threads in Python, like C they are hardware – posix threads the GIL (Global Interpreter Lock) prevents threads in Python from running in parallel they can only run on a single CPU
Step 1 concurrency using callback functions
What some people call “callback hell” the async_producer.py demonstrates replacing loops and threads with small callback functions that rely on the Scheduler to switch between tasks.
Step 2 concurrency with coroutines driven by generator
Step 3 concurrency combining callbacks and coroutines
Async IO in Python combines the models demonstrated in Step1 and Step 2. This is done using coroutines on top of a callback based Scheduler.
Each file is an executable.
In order of programming model progressing to final example which combines a callback based Scheduler and coroutines which in turn enables the lib to be used with either callback or coroutine I/O solutions.
Threads: threads_expl.py, add producer/consumer to threads producer.py Scheduler: recursive callbacks in func - main.py Yield and wrapped yield: yield_it.py Queue and Result: async_producer.py and a_pro_clean.py __await__: async_await_producer.py Combine callback Scheduler and Task wrapped coroutines: async_cb_coro.py Add I/O tcp_server: async_io.py