There are a variety of technical challenges associated with designing an Ethereum-based decentralized exchange that settles trades to the blockchain. In this 4 part series, we will discuss front-running, griefing, and the various ways we can choose to address these issues while focusing on the user experience, centralization and security tradeoffs associated with each solution.
Race Conditions on the Blockchain
There is an inescapable time delay between the moment a blockchain transaction is broadcast to the network and the moment that that same transaction is mined into a block. During the intervening period, transactions sit in a pending transaction pool (mempool) waiting to be mined. When the frequency of newly created transactions for a particular blockchain exceeds that blockchain’s maximum throughput (for Ethereum, throughput is dictated by the block gas limit), the backlog of pending transactions will grow, increasing the average length of time that a transaction will remain in the mempool before being mined into a block. This time delay makes blockchain transactions vulnerable to race conditions. In the context of blockchain-based decentralized exchange, race conditions manifest as
- accidental trade collisions, which occur when two or more parties attempt to fill the same order simultaneously (to be more precise: both trades occur within approximately one block time of each other),
- accidental cancel collisions, which occur when a trader attempts to fill an order at the same time that the order originator is attempting to cancel that order,
- other collisions that may be specific to a decentralized exchange implementation.
When these types of collisions occur, one transaction will be executed successfully, one will fail, but both transactions will consume gas (a necessary requirement to make DDoS attacks against the Ethereum network expensive). As a greater number of trades occur within a given market, a greater number of accidental collisions can generally be expected to occur, increasing the average friction costs for all market participants.
Another vulnerability is created when blockchain transactions are both asynchronous and transparent to outside observers such that outside observers can intentionally effect the outcomes of pending transactions. In the context of blockchain-based decentralized exchange, this can manifest as front-running: the practice of intercepting transactions from the network mempool that will influence the market price of a blockchain-based asset and then stepping in front of these transactions to gain a price advantage at the expense of others.
How does a front-runner ensure their transaction will be mined into a block prior to the transaction that they are attempting to front-run? The front-runner sets the gas price for their transaction to be higher than that of the target transaction. Given the inherent limit on gas consumption for each Ethereum block, a rational miner will prioritize transactions that pay a higher price per unit of gas to maximize the value of their block reward. It follows that by setting a higher gas price, a front-runner can generally expect miners to give their transaction higher priority than the transaction they are attempting to front-run.
Worse still, a front-runner that also happens to be a miner can order transactions arbitrarily and at their own discretion each time they mine a block. This means that a front-running miner can ignore gas prices, place their own transactions into a block wherever they please and censor the transactions of others. Given the opportunity, a rational miner will monitor the mempool for market moving trades and methodically front-run each trade such that they maximize their own profit. Once there is sufficient volume moving through decentralized exchanges, and assuming there is no mechanism in place to prevent miner front-running, it is conceivable that front-running could provide miners a greater source of revenue than the block subsidy.
In traditional markets, front-running is not only considered unethical, it is illegal. As designers of trustless distributed systems we do not get to indulge in assumptions about ethics or an underlying justice system; we must assume that we are operating in an adversarial environment where each market participant acts rationally in their own self interest. Therefore, it is our job to place constraints in our system that make undesirable behaviors unprofitable or prevent them altogether.
Off-chain Order Relay, On-chain Settlement
While there are a range of designs for decentralized exchange systems, this article will focus on a broad class of systems that utilize off-chain order relay with on-chain settlement. In these systems, orders consist of cryptographically signed messages that signal the signer’s intent to trade at a specific exchange rate. These orders can be communicated through any arbitrary off-chain transport layer (email, twitter, API, etc…). Most importantly, an Ethereum transaction is only created once a trade is ready for settlement, at which point an order is injected into an Ethereum smart contract that is responsible for trade execution and settlement. Please refer to the 0x white paper for an in-depth explanation of off-chain order relay with on-chain settlement.
The following discussion will use terminology from the 0x white paper to describe the various actors involved in a trade:
- Makers: originate and cryptographically sign orders, providing liquidity.
- Takers: inject maker orders into an Ethereum smart contract for settlement, consuming liquidity.
- Relayers: aggregate orders from numerous makers and present them in the form of an order book for consumption by takers.
- Exchange contract: the Ethereum contract responsible for processing orders that have been passed in by takers, settling trades. All makers, takers and relayers use the same single instance of the Exchange contract.
Open Order Books
An EtherDelta-style open order book uses limit orders that are not assigned to a specific counter party, meaning that the Exchange smart contract accepts these orders from any Ethereum address on a first come first serve basis. Figure 1 shows the off-chain message passing scheme used for open order book exchanges and the process of front-running.
Figure 1: the process of front-running EtherDelta and similar off-chain order book systems.
We performed a (very) rough analysis of EtherDelta trades over a sample of 2,000 blocks during which approximately 90% of EtherDelta trades executed successfully. Groups that have studied this issue in more detail have stated that the percentage of successful EtherDelta trades is closer to 79% (not sure the source wants to be quoted on this). Trade failures can result from collisions, front-running and user error; distinguishing between these failure modes is challenging based upon the information that is available to us. That being said, this data indicates that trade collisions and front-running are issues in practice.
Front-running solutions with 0x protocol
The 0x white paper presented two ways that 0x protocol may be used to facilitate exchange: (1) point-to-point orders for private over-the-counter trades and (2) broadcast orders that sit on an EtherDelta style open order book. Since publishing the 0x white paper, we have frequently encountered confusion around what falls within the scope of the 0x protocol specification and what does not.
What is included within the 0x protocol specification is an order schema — which defines how to populate, package together and cryptographically sign orders — and a system of smart contracts — which contain the business logic responsible for processing orders and settling trades to the blockchain.
Details that do not fall within the scope of the 0x protocol specification are the network transport layer — the medium used to transmit orders off-chain — as well as the message passing semantics — the series of steps followed by market participants to find a counter party and negotiate a trade. It follows that there are a variety of ways one can choose to use 0x protocol and we encourage developers to take advantage of this flexibility by experimenting with their own novel approaches.
Before discussing front-running solutions, we will first inspect the 0x order schema shown in Figure 3. In particular, we are interested in the
taker parameter. If the
taker parameter is populated, the Exchange contract will reject any attempt to fill the associated order when
r (the entity passing in the order) is not equal to
taker can be set to an externally owned account (EOA) or to an Ethereum smart contract that contains any arbitrary set of rules. This may seem like a small detail but it has extremely powerful implications that will become clear in the following sections.
Figure 2: the 0x order schema. Note that the t
aker parameter is optional, but can be populated with any Ethereum address. When the taker parameter is populated, the Exchange contract will only accept an order if it has been passed in by the specified taker.
taker to an Externally Owned Account (EOA)
Peer-to-peer Negotiation Schemes
The simplest way to avoid front-running is to avoid public markets altogether by finding a counter party and negotiating a trade in private. The setting used to find a counter party isn’t important, one can use a messaging app, forum or a subreddit (/r/tokentrade). What is important is that — once a suitable counter party is found and price agreed upon—the resulting order’s
taker parameter is set to the counter party’s Ethereum address before the order is cryptographically signed, locking out potential front-runners from accessing the order. 0x Portal provides an interface that can be using to manually generate and cryptographically sign such an order using 0x protocol.
The overhead associated with negotiating each over-the-counter (OTC) trade makes this approach unsuitable for moving small amounts of value. However, OTC trades are frequently made by institutions that want to liquidate or purchase large amounts of an asset without adversely influencing market prices. AirSwap is attempting to streamline the process of finding a counter party and negotiating a trade by offering the services of an indexer and a price oracle. The indexer maintains a list of market participants that are interested in finding a counter party and initiates a connection between two parties that are deemed to be a good match. The price oracle pulls exchange rate data from a centralized exchange API and suggests this price as a starting point for negotiation between the two parties. A similar strategy that doesn’t rely on a price oracle and that we refer to as the Quote Provider strategy may be employed using 0x protocol.
- Eliminates front-running by traders and miners.
- Eliminates trade collisions.
- Liquidity providers must be online and available to negotiate each trade. Cannot fire and forget orders.
- Cannot source liquidity on demand. There is no guarantee that a counter party is available and interested in trading the same size or at the same price point and the only way to find out is to engage.
- No networked liquidity. Results in a closed liquidity pool that isn’t directly accessible to other exchanges or dApps.
- The process of generating an order is different than the process of consuming an order, which could lead to a confusing UX.
The order matching strategy eliminates front-running by setting the
taker parameter to an EOA while avoiding the overhead associated with peer-to-peer negotiation schemes. In this approach, a relayer hosts a publicly viewable order book that consists of cryptographically signed orders, however, the
taker parameter for each order is set to an EOA that is controlled by the relayer. It follows that the relayer is the only entity capable of filling the orders displayed in the order book, locking out potential front-runners from accessing them. When orders on opposite sides of the market overlap in price, the relayer batch fills these orders simultaneously, matching two or more counter parties together. We refer to a relayer that uses the order matching strategy as a matcher. A similar approach was first proposed in the Hallex white paper and first implemented using 0x protocol by the
Figure 4: In the order matching strategy, the matcher maintains a publicly viewable order book. While the orders are publicly viewable, they can only be filled by the matcher.
- Eliminates front-running from traders and miners.
- Eliminates trade collisions.
- Can provide a trading experience that feels lower latency than the block time as the matcher has some degree of certainty around trade execution.
- “Fire and forget” orders. On-demand liquidity.
- The process of generating an order is identical to the process of consuming an order, which provides a less confusing UX.
- The matcher pays the gas costs (from the perspective of a trader, not having to pay for gas is likely considered a good thing).
- The matcher can censor traders (from the perspective of a matcher, having the ability to blacklist malicious traders could be considered a good thing).
- No networked liquidity. Results in a closed liquidity pool that isn’t directly accessible to other relayers and dApps.
- The matcher can front-run trades.
- The matcher must hold a trading balance due to the current lack of support for atomic order matching in 0x protocol version 1. Support for atomic order matching is planned for version 2 of 0x protocol.
- Each trade consumes twice as much gas as a trade on an open order book because the matcher must batch fill two orders.
This post presented blockchain race conditions, front-running and how they can be a challenge for existing approaches to blockchain-based decentralized exchange. We explored how front-running works for an EtherDelta-style open order book and established that a considerable percentage of EtherDelta trades fail, in part, due to front-running. Finally, we explored two strategies for using 0x protocol that eliminate trade collisions and front-running by setting the
taker parameter to an externally owned account.
In part 2, we will explore front-running solutions that can be implemented by setting the
taker parameter to an Ethereum smart contract address. As we will see, the flexibility we gain by feeding orders through a smart contract that we can design and into the Exchange contract will allow us to avoid some of the downsides of the front-running solutions presented in part 1. We believe that smart contracts should be designed as modular, unopinionated building blocks that can be assembled and reconfigured; part 2 should highlight the extensibility of the 0x protocol’s system of smart contracts and hopefully inspire developers to experiment with their own novel solutions.