Python

Demistify Python AsyncIO

Asynchronous programming is not new but still it creates lot of confusion and leads to programming bugs which become really tricky to solve.

With my experience in successfully using asyncio and websockets in production grade software, I am going to introduce you to async programming, esp in python, in an easy step by step manner from foundational principles to advanced usage.

Note: There are some terminology differences among different programming stacks. I will stick to python lingo

Traditional Async programming:

First let’s try to understand the traditional meaning of async programming.

1. A caller (method/function) submits a task to an API
2. The API returns immediately
3. The API runs the task in the background (typically in another thread)
4. The caller either explicitly waits for result

OR

the caller might have registered a call back in the step1

Let’s related it to the more familiar with Java way of doing / Javascript way of doing things

1. Java (ex:Main) thread create multiple threads,submits tasks and uses join OR Java thread creates an executor,submits tasks OR Java creates Future and awaits for results
2. Javascript submits an HTTP Request ,registers success callback and failure callback

The difference in Javascript is that the caller and the callbacks happen in the same single thread of execution. I would call it Application/User Single Thread .

Note: You can create threads in Javascript too. But that’s an advanced usage.

Single Thread of Execution :

There is an event loop(like a Queue) where user/application functions are added to the Queue. All the functions are picked from the Queue and executed in the same single thread

Controlled by OS or Controlled by programmer ?

Race Condition:

When shared memory/variables are accessed by different threads in an unpredictable way. The keyword is unpredictable . This is because Threads are scheduled by OS/Runtime as per thread scheduling and CPU availability . We can’t be very sure which thread is going to get CPU time at a particular point in time

Programmer is not in control. Runtime sequence is not predictable since runtime/OS uses scheduling algorithm.

We typically use locks/CAS to overcome the unpredictable sequence

Threads are controlled by runtime. They give up CPU as per runtime scheduler. Example : IO calls, Synchronous blocks/methods for locks etc.

No Race conditions :

In single thread frameworks why are there are no race conditions ? Because all the functions are executed in the same thread.

But does that mean shared memory/variables can be modified by different functions?Yes shared memory is still accessible to multiple functions but in a controlled way.

Programmer is in control however. You could still write a bad program of modifying shared variables impacting the application

Async functions/coroutines

Now the crucial part.

An Async function in python aka coroutine is one which gives up CPU and allows the event loop to execute the next function in the queue . The event loop engine resumes the function which gave up CPU whenever conditions are met. It is completely in programmer’s control

Async function is explicitly defined using async keyword. You are allowed to use await keyword inside async functions to give up CPU .

The thread is not suspended. The function is suspended

async def async_func(....):
   ...
   await ...
   ...
   await asyncio.sleep(15) # func gives up CPU
   ..

Regular function (also called Blocking function in Python) .

The reason it is called blocking is because

The thread is suspended unlike the above.

We will not be call regular function inside async function or vice versa. There are other ways

#Blocking/Regular function:
  def block_func():
   ...
    time.sleep(1)  # Thread gives up CPU
   ..

Managing/Creating Loop

An important topic about creating/managing loops

1. Let runtime manage loop lifecycle. Runs till async_function finishes. Could be the start of your program

asyncio.run( async_function() )

2. Explicit loop

loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_forever()

3. Stop loop explicitly

asyncio.get_running_loop().stop()

4. Get Current Loop explicitly

asyncio.get_event_loop()

5. Submit using asyncio module

asyncio.create_task
asyncio.run_coroutine_threadsafe

How to submit a task / coroutine to event Queuein Python

1. Add a task (invoke async function with args) and wait for its completion
2. submit a task and return immediately
3. submit a task and await explicitly
4. submit a task and register a callback handler in the same event loop/Queue . Callback handler is a regular function at a later time.You must pass in a callback reference followed by arguments.
5. submit a task to be executed in a different event loop/thread? Why is it threadsafe? because you are submitting to a different thread

#1
await async_func(...)  # Adds a task to queue and waits(function waits) for the task to complete
#2
await asyncio.create_task(async_func(...args)) # Adds a task to queue. Function waits for task to complete
#3
task1 = asyncio.create_task(async_func(...args)) # Adds a task to queue. Does not wait for task to complete. Returns immediately
await task1 #explicit wait
#4
# Add a callback handler
task2 = asyncio.create_task(async_func(...args))

# Note callback is a normal function not an async function
# Note callback is a just a function reference, we are not invoking it yet. The last arg is passing on which event loop/thread to execute this callback this
task2.add_done_callback(
functools.partial(callback,arguments) 

# 5
# Add an async function from one thread to another other thread (ex: another_loop )
asyncio.run_coroutine_threadsafe(async_func(...args), another_loop)

How to submit a blocking function

Pass a function reference followed by args

await asyncio.get_event_loop().run_in_executor(
                    pool, block_func, args )