Building Efficient Crypto Trading Software With Python

Hello and welcome to this article where I will talk about the modern approach to building production-grade trading software with python programming language.

Python is widely used for its simplicity and clean, readable code. More than one million software developers can understand and support the code written in python.

But python is known for some disadvantages, such as speed. You can ignore this disadvantage most of the time as most libraries used for trading purposes use code snippets written in C and are fast enough for most common tasks.

We will discuss three main concepts: asynchronous programming, event-driven architecture, and microservices architecture.

As far as our software will use different data sources, API services, and more, we will have to make it asynchronous, so we will not wait for a data lag from any remote services. Asynchronous execution is a must.

The event-driven approach helps us better understand and control what is happening in the system, which works fast.

Microservices architecture makes it easier to develop and support projects with thousands of lines of code by giving us a chance to focus on a single problem which can be way harder with a monolithic architecture.

Asynchronous programming

It would be best if you had at least a basic understanding of asynchronous programming to build fast-reacting trading software.

The problem with the synchronous approach used most of the time is that our program must wait for data input from dedicated sources before proceeding to further actions. Connecting to WebSockets and receiving tick data updates or open order updates from exchanges using synchronous software is nearly impossible.

Asynchronous programming allows us to skip waiting for data input and proceed to the control stream or data processing stream. This approach radically improves our software speed and performance.

We will use asyncio python library for building an asynchronous application. We need to learn a few concepts to start building asynchronous software.

First, our functions are marked as asynchronous with async word, like in JavaScript. Example:

async def my_function()

Second, we use await word to tell the program where it can switch control from one stream to another. Example:

await asyncio.sleep(0)

This line of code does nothing but tell our program not to wait for data input or some long calculations, so our program can switch to another task.

Third, in building asynchronous software, we operate not only functions but tasks. A task can contain one or more functions inside.

Example:

async def input_func():
    pass


async def input_task():
    await asyncio.sleep(0)
    await self.input_func()

And, finally, loops. Our tasks are run inside loops, which can be called ‘control streams’.

Example:

ioloop = asyncio.get_event_loop()
tasks = [ioloop.create_task(input_task()), ioloop.create_task(another_task())]
wait_tasks = asyncio.wait(tasks)
ioloop.run_until_complete(wait_tasks)

This is an example of a simple asynchronous application. You can find it in this GitHub repo.

Event-driven approach

The event-driven approach is a way to think about software architecture. Developing and maintaining a considerable trading application while thinking in procedures and functions paradigm could be inefficient and even impossible.

We can think about trading software as a system that reacts to some random events in the real world. Something happens in the market; prices change, which is an event. We can process it inside our system and produce another event, order, for example, it conditions are met. We can also create a logging or notification event based on our needs.

The event-driven approach helps us better understand what is happening inside the system, maintain control, and support clearer and more efficient software architecture.

Events are the fundamental element of our architecture. You can think of events as data classes or data structures. They can contain a lot of information like the event type, timestamp, specific data, etc.

Example:

class Event(object):
    #base class
    pass

class AccountUpdateEvent(Event):
    def __init__(self, timestamp, username, total):
        self.type = 'ACCOUNT_UPDATE'
        self.timestamp = timestamp
        self.username = username
        self.total = float(total)

Now we will build a simple application that can produce and process events in less than 100 lines of python code!

Our first module is going to make API calls and produce events.

Example code(whole class is available in my GitHub repo):

class GetAccountData:
    def create_event(self, timestamp, username, total):
        self.events.put(AccountUpdateEvent(timestamp=timestamp, username=username, total=total))

The second module processes events.

Example code(whole class is available in my GitHub repo):

class ProcessAccountData:
    def update_account_data(self, event):
        print('Account update received at: ', event.timestamp, 'Username: ', event.username, 'Total balance: ', event.total)

And, finally, our app. It will contain both GetAccountData and ProcessAccountData and a queue of events.

Example code(whole class is available in my GitHub repo):

class App:
    def __init__(self):
        self.events = Queue()
        self.get_data_account = GetAccountData(self.events)
        self.process_data_account = ProcessAccountData()

This app contains a loop to process events. Once a new event is in our queue, we take it out and process it.

Example code(whole class is available in my GitHub repo):

def event_loop(self):
       while True:
            try:
                event = self.events.get(False)
            except Exception as e:
                time.sleep(1)
                break

            else:
                try:
                    if event is not None:
                        if event.type == 'ACCOUNT_UPDATE':
                            self.process_data_account.update_account_data(event)

                except Exception as e:
                    print('event_loop error:', e)

This is an example of a simple event-driven application. You can find the source code in my GitHub repo.

Microservices

Microservice software architecture is something opposite of monolithic architecture. For small projects, monolithic architecture is the most straightforward way to think about its design in general. But any mature project cannot be developed and supported easily if it is a monolith.

The microservice pattern forces developers to divide the project not just into separate modules and classes but into individual tiny apps called microservices. They can be deployed in different data centers, supported by other teams, etc.

Now we will build a simple microservice for our trading software. It will receive our order data via API and save it to the local database. Of course, it is possible to save that data to a local disk. Still, suppose we want to build high-frequency trading software. In that case, we don't want to waste any local resources on logging or statistics, or maybe we want to outsource the development of a feature to a different developers team.

First of all, we will build a basic database connector. We will use the SQLite database for educational purposes. We will create a table of Orders containing six fields: timestamp, username, market, side, size, and price.

Example code:

class Database:
    #this is a DB connector
    #we will use SQLite in this example for simplicity
    def init(self):
        #filename and path to the database are hardcoded for simplicity
        self.connect_to = 'test.db'
    def create_table_orders(self):
        #a func to create our database
        conn = sqlite3.connect(self.connect_to)
        conn.execute('''CREATE TABLE if not exists Orders
        (timestamp TEXT NOT NULL,
        username TEXT NOT NULL,
        market TEXT NOT NULL,
        side TEXT NOT NULL,
        size FLOAT NOT NULL,
        price FLOAT NOT NULL
        );''')
        conn.close()

Database connector will have only one method implemented to save order data to the database.

Example code:

def add_data_orders(self, timestamp, username, market, side, size, price):
    #a func to save orders data
    conn = sqlite3.connect(self.connect_to)
    conn.execute("INSERT INTO Orders (timestamp, username, market, side, size, price) VALUES (?, ?, ?, ?, ?, ?)", (timestamp,
    username, market, side, size, price));
    conn.commit()
    conn.close()

Second, we need an API server. Creating a simple API server with the Flask module in less than 30 lines of code is possible. It will be able to receive HTTP POST requests with order data and save it to the database.

Example code:

@app.post("/API/orders")
def save_orders():
    if request.is_json:
        response = request.get_json()
        DB.add_data_orders(response['timestamp'], response['username'], response['market'], response['side'],  
        response['size'], response['price'])
        return response, 201
        return {"error": "Request must be JSON"}, 415

You can find the complete database connector code and API SERVER code on GitHub.

And finally, we need an API connector for our service. Our API connector will use the requests library to make POST HTTP requests to our API server.

Example code:

def generate_request(self, order):
    try:
        response = requests.post(self.api_url, json=order)
        print(response)
    except Exception as e:
        print('generate_request - Exception', e)

You can find the complete API CLIENT code on GitHub.

In ~100 lines of code, we created a database connector, API server, and API client to save order data to the database on the remote server.

The code is available in this Github repo.

In this article, we have discussed the basics of asynchronous programming, event-driven architecture, and microservices - three concepts you need to know about being a software developer. I hope you will use this knowledge in your next projects!  I'll be adding more content to this github repo. And you can follow my twitter as well.