Preparing for Migration

Before​ starting your migration it is important to take inventory of your use cases with Eventlet and so of your needs. This chapter help you to make this inventory.

Locate your Eventlet Usages 🔗

The first thing to do to prepare your migration is to locate all the Eventlet usages in your code base. That's pretty easy to realize by using a shell command. The snippet below should help you to collect 99 percents of your occurences.

#!/bin/bash
echo "Searching Eventlet Instances..."
grep -rnw 'import eventlet' . > eventlet_instances.txt
grep -rnw 'from eventlet' . >> eventlet_instances.txt
grep -rnw 'eventlet.spawn' . >> eventlet_instances.txt
grep -rnw 'eventlet.monkey_patch' . >> eventlet_instances.txt
echo "Results saved at eventlet_instances.txt"

Audit your Code 🔗

In accordance with the common scenario of Eventlet previously identified, the elements below will help you to triagge the occurences previously found. The scenario below will guide you to audit your code and will help you to identify which kind of usage is present in your code base.

from eventlet import wsgi, listen

def simple_app(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return [b'Hello, World!\n']

wsgi.server(listen(('', 8080)), simple_app)

import eventlet
from eventlet.green import urllib2

def fetch_url(url):
    return urllib2.urlopen(url).read()

urls = ['http://example.com', 'http://example.org']
pool = eventlet.GreenPool()

for body in pool.imap(fetch_url, urls):
    print(body)

import eventlet

def long_running_task():
    # Simulate a long-running task
    eventlet.sleep(10)
    print("Task completed")

eventlet.spawn(long_running_task)
print("Main thread continues to run")

from eventlet.event import Event

def deferred_task(evt):
    print("Waiting for event")
    evt.wait()
    print("Event occurred, executing task")

evt = Event()
eventlet.spawn_after(5, deferred_task, evt)
eventlet.sleep(5)
evt.send()

import eventlet

def task1():
    while True:
        print("Task 1 running")
        eventlet.sleep(1)

def task2():
    while True:
        print("Task 2 running")
        eventlet.sleep(1)

eventlet.spawn(task1)
eventlet.spawn(task2)

import eventlet
from eventlet.green import socket

def echo_server(port):
    server = socket.socket()
    server.bind(('0.0.0.0', port))
    server.listen(5)
    while True:
        client_socket, addr = server.accept()
        eventlet.spawn(handle_client, client_socket)

def handle_client(client_socket):
    while True:
        data = client_socket.recv(1024)
        if not data:
            break
        client_socket.sendall(data)
    client_socket.close()

echo_server(6000) 

import eventlet
from eventlet.websocket import WebSocket, websocket

@websocket('/echo')
def echo(ws):
    while True:
        msg = ws.wait()
        if msg is None:
            break
        ws.send(msg)

listener = eventlet.listen(('0.0.0.0', 7000))
ws = WebSocket(listener)
eventlet.spawn(ws.accept)

import eventlet
from eventlet.green import threading
import queue

def worker():
    while True:
        item = q.get()
        if item is None:
            break
        print(f"Processing {item}")
        q.task_done()

q = queue.Queue()
threads = []
for i in range(3):
    t = threading.Thread(target=worker)
    t.start()
    threads.append(t)

for item in range(10):
    q.put(item)

q.join()

for i in range(3):
    q.put(None)
for t in threads:
    t.join()

Is Eventlet Disablable? 🔗

Depending on your project kind, time to time, your application may support running with or without Eventlet. As said in the Eventlet section of this guide, Eventlet is often used to make existing code asynchronous, meaning, that your application could potentially run without Eventlet but in a blocking way. That mean that some Eventlet usages can be disabled.

This type of scenario can ease your migration. You just, if possible, of toggle monkey patching off to switch in synchronous mode. Starting from that point, you would just to transform your calls with the proposed alternatives to make them async again, without having to carre about Eventlet and its side effects in the middle.

Choosing an Alternative 🔗

This guide made the choice of AsyncIO and Threading as alternatives. We won't cover the other solutions like Curio, Gevent, Tornado, Twisted, however, if you volunteer to document them, then, feel free to propose a pull request to update that guide.

This section aim to help you to determine when using AsyncIO or Threading.

AsyncIO is great but is explicit nature can force you to rewrite entirely your application. Indeed, when you use AsyncIO, all your stack have to be scattered with the await and async keywords.

If your application is just a few lines of code and a couple of sub-modules that cannot be an issue. But, if your implicit async call made with Eventlet are deeply rooted in your call stack then refactoring your application with AsyncIO could become really painful.

In parallel, AsyncIO is tailored for asynchronous network I/O, making your application more efficient and less resource consuming.

Threads, on their side, can consume resources even if your application do nothing and is waiting for a network I/O. But threads won't force you to rewrite all your application due to a deeply rooted Eventlet async call.

So what is the solution? If your application is not composed of tons of sub-modules with deeply rooted Eventlet async calls, and, if these calls are mostly used to realize non-blocking network calls, then go straight to AsyncIO, else, prefer Threads.

Ideally it shouldn't be a all or nothing decision. If you the time and the resources to rewrite your use cases with the right solution, then, that would not be a problem to mix the solutions together. By example, if you are using the WSGI feature from Eventlet to implement an asynchronous server, then, you could consider migrate to the AsyncIO ecosystem and by example choosing to use aiohttp and uvicorn to refactor your WSGI scenario, and, in parallel you could consider to use threading to manage your background tasks that are CPU bound.

There is one last option that we need to speak about in this section. This option is named awaitlet. Awaitlet can allow you to mitigate the depth of your AsyncIO refactor. Awaitlet can be used as a glue between default synchronous code and part of your code that need async mechanisms and AsyncIO. Awaitlet would limit the impact of an AsyncIO refactor, and would allow you a more incremental migration scenario. We will describe how to manage the depth of your refactor later in this guide.

The graph below summarize your decision tree based on the solutions retained in this guide:

Split your Works 🔗

Services First

Time to time your application can be composed of one or many services and of one or many libraries. All these deliverables represent your application.

The majority of the time that's your services that monkey patch your environment, meaning that the occurences of Eventlet in your libraries is often just the result of a monkey patching at an higher level in your stack. If you migrate your libraries first you will surely broke something in your service or you will surely face unexpected side effects.

The right way to migrate this kind of scenario is to first completelly migrate your services, and once done, to drop the Eventlet occurences in your libraries.As we explained ealier in this guide the magic of Eventlet do not keep its promises and surely had side effects on all your stack. It is not rare to see libraries fixed with patches just to run with Eventlet (to avoid race conditions or side effects of monkey patching).

Migrate your services first, and, then, tackle your libraries.

Isolate your Application into Functional Parts

Sometimes a single codebase hosts different services or different parts of a service, endpoints, and entrypoints, which are not necessarly executed together in the same process. Identifying these functional parts can help you plan your migration in isolated, functionally autonomous sub-parts that are therefore easier to migrate.

Identifying these parts will allow you to incrementally migrating your deliverables without impacting everything in your app at the same time. It will give you more control and will allow you a more fine grained migration.

For more detailed guidance on handling applications with interconnected components, be sure to review the Migrating Applications with Internal Private APIs section later in this guide.

Divide and conquer.

Deprecate First

If your deliverables rely on Eventlet, there is chances that it also own parameters or config options related to Eventlet. We have to care about our end users and we have to avoid abrupt removal of these things. Removing them without advising firt could abruply broke their configurations or their environments, for this reason, we have to respect a deprecation period first.

This deprecation period will make your end users less nervous and frustrated. This deprecation period will show that you care about about your end users.

Explicit is better than implicit.

Secondly Executors

In general Greenthread are easy to identify and so easy to flush out. Adressing them first will drastically reduce the scope of Eventlet in your application, leaving you more brain resources to handle the complexe scenarios introduced by monkey patching and implicit async.

Migrate greenthread first, and, then, tackle wild monkey patching.

To conclude this section, this incremental approach is particularly suited for applications using internal private APIs, as it allows you to progressively migrate these APIs without disrupting other parts of the system.

Migrating Applications with Internal Private APIs 🔗

You may encounter situations where isolating parts of your application into separate processes is challenging because these parts rely on a shared internal private API using Eventlet. The goal here is to break this dependency without rewriting your entire codebase at once.

Here's a clear and incremental strategy you can follow:

1. Identify your API boundaries clearly.

   Start by precisely defining the boundary between your isolated sub-parts and the shared internal Eventlet-based API. Understand exactly which parts of your code depend on Eventlet functionalities.

   # Current Eventlet-based API
   def async_fetch_data():
       eventlet.spawn(...)
   
   def async_send_request():
       eventlet.spawn(...)

2. Create a neutral interface.

   Design an intermediate interface that doesn't expose Eventlet directly. This new interface can be synchronous (blocking) or use standard communication methods such as REST or RPC, abstracting away the specifics of Eventlet from the rest of your application.

   # New neutral interface without direct Eventlet references
   def fetch_data():
       pass
   
   def send_request():
       pass

3. Initially run your neutral interface alongside Eventlet.

   Initially, your new neutral interface might internally call the existing Eventlet-based implementation, ensuring compatibility and stability while allowing incremental migration.

   # Neutral API initially calling Eventlet-based implementation
   def fetch_data():
       return async_fetch_data()

4. Migrate each isolated part incrementally.

   Move your application parts one at a time to the neutral interface. Once a part uses this interface, you can safely run it in an isolated process since it no longer directly depends on Eventlet.

5. Remove Eventlet completely at the end.

   After migrating all the isolated parts to your neutral interface, replace the underlying Eventlet implementations incrementally with alternatives such as AsyncIO or Threading without impacting the isolated parts.

   # Final neutral API implemented with AsyncIO
   def fetch_data():
       asyncio.run(async_fetch())

Sequence Overview

Decouple, abstract, and incrementally migrate to minimize risk and maximize control.

Implementation Example of Migrating Applications with Internal Private APIs 🔗

1. Context and Objectives

Your application is built on a single codebase with several entry points that run dedicated business processes (for example, API Server, Compute Server, and XYZ Server). Up to now, these processes rely on Eventlet for concurrency management (e.g., green threads, monkey-patching, etc.). However, as your requirements evolve and for reasons such as performance, maintainability, or integration with other Python libraries, you have decided to migrate away from Eventlet to an alternative concurrency solution—whether that be asyncio, threading, or a combination of both.

Objectives:

• Decouple from Eventlet: Replace the Eventlet-dependent concurrency model with a more modern or suitable alternative.
• Ensure a smooth transition: Migrate the business processes incrementally to minimize risks and maintain service continuity.
• Maintain functionality: Guarantee that during and after the migration, the application’s business processes continue to interact with core services (database connections, RPC communications, etc.) correctly.

2. Step 1: Initial State (Before Migration)

Description

At this stage, all business processes (API Server, Compute Server, XYZ Server) are tightly coupled with Eventlet for concurrency. The Eventlet layer underpins how these processes manage asynchronous tasks, network I/O, and cooperative multitasking.

Diagram

Explanation

Vertical Silos: Represent the different business processes (API Server, Compute Server, XYZ Server) that execute concurrently.

Horizontal Silo: The Eventlet-based concurrency layer (Legacy) handles core functionalities like database connections, RPC, and security.

Tight Coupling: Every process directly depends on Eventlet, which makes it difficult to evolve or integrate with alternative concurrency frameworks.

3. Step 2: Progressive Migration (During Migration with One Process Migrated)

Description

In this phase, you begin the migration by refactoring one of the business processes (e.g., the API Server) to use a newly developed concurrency adapter. This adapter abstracts the underlying concurrency mechanism and redirects calls to the chosen alternative (asyncio, threading, or a hybrid approach). Meanwhile, the remaining processes continue to use Eventlet.

Diagram

Explanation

Partial Migration: Only the API Server has been refactored to use the concurrency adapter. This adapter bridges the existing functionality with the new concurrency mechanism.

Adapter Role: The adapter abstracts away the differences between Eventlet and the new concurrency approach, allowing the migrated process to operate with the new model without affecting the others.

Coexistence: The legacy (Eventlet) and the migrated concurrency models run in parallel, ensuring uninterrupted service for non-migrated processes.

4. Step 3: Complete Migration (After Migration, All Processes Migrated)

Description

After thorough testing and validation, all business processes are refactored to use the concurrency adapter, which now directs them to the new concurrency model (whether that's asyncio, threading, or a combination thereof). This marks the end of dependency on Eventlet.

Diagram

Explanation

Unified Abstraction: All business processes now leverage the concurrency adapter, ensuring consistent access to the new concurrency model.

Centralized Control: The adapter centralizes concurrency management, simplifying maintenance and future improvements.

Smooth Transition: This stage confirms that all components interact consistently with the new model, paving the way for complete deprecation of Eventlet.

5. Step 4: Adapter Removal (Simplified Final Architecture)

Description

Once the migration is fully validated and the new concurrency model proves stable across all business processes, the adapter can be removed. The business processes are then directly integrated with the new concurrency framework, further simplifying the architecture.

Diagram

Explanation

Removal of the Adapter: With the new concurrency model fully in place, the intermediary adapter becomes redundant and is removed.

Simplified Architecture: Direct integration with the new concurrency model reduces complexity and may improve performance.

Final State: This diagram represents the final state of your application, where all business processes are fully independent of Eventlet and operate under the chosen concurrency paradigm.

6. Conclusion

This report outlines a structured approach for migrating your application from an Eventlet-based concurrency model to an alternative solution (asyncio, threading, or a hybrid). The migration strategy comprises:

• Initial State: All business processes depend on Eventlet.
• Partial Migration: Begin by refactoring one process (API Server) to use a concurrency adapter while the others continue using Eventlet.
• Complete Migration: Transition all business processes to use the adapter, ensuring a unified migration to the new concurrency model.
• Final Simplification: Remove the adapter once the new concurrency model is fully stable, resulting in a simplified and modernized architecture.

By following this plan, you can minimize risks and ensure a smooth, controlled transition from Eventlet to your chosen alternative, ultimately enhancing the scalability and maintainability of your application.

Feel free to adjust this approach according to your specific requirements and add further details such as testing strategies, performance benchmarks, and error handling mechanisms to ensure a successful migration.