Page tree
Skip to end of metadata
Go to start of metadata

Introduction

dxFeed market data feeds (real-time, delayed or historical) allow clients to reconstruct order books, price level aggregations, and aggregations by Market Maker or a bank. In this document, we describe an order book reconstruction. For reconstruction, you can:

  1. Use OrderBookModel class in dxFeed API.

  2. Check out Order event attributes and implement the reconstruction algorithm.

We recommend using dxFeed API, because it correctly implements a reconstruction protocol.

dxFeed API

To get the best and smoothest solution for the order book reconstruction, we recommend using our dxFeed Java API. It provides an OrderBookModel class which builds a high-quality order book from a set of market events. This class keeps track of transactions and snapshots, filters data, etc. Learn more about events and classes in dxFeed API here.

Reconstructing an order book without the OrderBookModel class of dxFeed API

You can reconstruct an order book without using the OrderBookModel class in dxFeed API. For this, do the following:

  1. Learn about Indexes and EventFlags fields to know how to track snapshots, updates and transactions.

  2. Follow the algorithm.

Indexes

An order book consists of an undefined number of orders. Each exchange specifies ID for each order. Since each exchange use different IDs, we assign our own indexes to each order (we put orders in special slots that have indexes assigned).

The index of an order distinguishes different entries in the order book.

Info

An order may move to another index, so this identifier is not permanent. Every event can be a new order, an order modification, or an order cancellation.

Regional events

Since every regional event has an indication of its exchange (non-zero source ID or exchange code in QD-model), you need to reconstruct an order book for each source ID separately. For example, you need to create two order books, if you have both order#bzx and order#byx events in your market data feed.

Event flags applicable to Order event

Event flags define the borders of snapshots and transactions retransmissions.

  1. TX_PENDING indicates a pending transaction update. When TX_PENDING is true, it means that an ongoing transaction update, that spans multiple events, is in process

  2. REMOVE_EVENT indicates that the event with the corresponding index has to be removed

  3. SNAPSHOT_BEGIN indicates when the loading of a snapshot starts. Snapshot load starts with a new subscription and the first indexed event that arrives for each non-zero source ID on a new subscription may have SNAPSHOT_BEGIN set to true. It means, that an ongoing snapshot consisting of multiple events is incoming

  4. SNAPSHOT_END indicates the end of a snapshot. It indicates that the data source has sent all the data pertaining to the subscription for the corresponding indexed event

Snapshots

Order event stream consists of snapshots and updates for these snapshots. Snapshots start with SNAPSHOT_BEGIN flag and end with SNAPSHOT_END flag.

Info

Due to possible reconnections and retransmissions, the snapshots can overlap each other, so in the event stream the flags can intersect: it is possible to find SNAPSHOT_END before SNAPSHOT_BEGIN, or it is possible to find SNAPSHOT_BEGIN after SNAPSHOT_BEGIN, and so on. Therefore, the reconstruction algorithm should extract a complete snapshot ignoring the wrong residues of other snapshots.

If you catch the SNAPSHOT_BEGIN flag, you have to dismiss the previous order book.

Transaction model

Updates of order books happen as a sequence of transactions. If a transaction consists of several events, then all these events except the last one have TX_PENDING flag. If an event does not have TX_PENDING flag, it means that this events finishes the current transaction or, if there is no current transaction, then this event is operated as a single-event transaction.

Snapshots can overlap with order book modifications and transactions. While the snapshot is being transmitted, updates of the order book can occur. There are two consequences:

  1. The snapshot can have modifications of the order book.

  2. A snapshot update during the transfer can create a transaction until the end of the snapshot (we will add TX_PENDING flag until the end of the snapshot). It means that it will create an artificial transaction until the end of this snapshot.

Examples

One transaction:

Order#NTV,AAPL,20180926-100107.858-0400,0,1219,20180926-100107-0400,854:1,223.31,17,1307,NSDQ,EventFlags=TX_PENDING


Order#NTV,AAPL,20180926-100107.858-0400,0,1623,20180926-100107-0400,854:2,223.29,59,1303,NSDQ


Single-event transaction:

Order#NTV,AAPL,20180926-100107.826-0400,0,723,20180926-100107-0400,822:0,223.07,100,1303,NSDQ


Two single-event transactions:

Order#NTV,AAPL,20180926-100107.441-0400,0,2449,20180926-100107-0400,437:0,223.3,100,1307,NSDQ


Order#NTV,AAPL,20180926-100107.441-0400,0,525,20180926-100107-0400,437:1,223.26,100,1303,NSDQ


Algorithm

The proper Order Book Reconstruction Algorithm implies the use of a pending queue, which accumulates events and is applied transactionally to the reconstructed integral model of an order book. Order Book Reconstruction Algorithm tracks incoming event flags and the current accumulation status of the pending queue to determine the next action.

Info

In the case of non-zero source IDs, you need to reconstruct an order book for each source id separately. For example, if you have both order#bzx and order#byx events in your market data feed, you need to distinguish these events and create separate pending queues and consistent order books.

Order Book Reconstruction Algorithm has:

  • A pending queue

  • A consistent order book that we reconstruct

  • Three flags: isSnapshot, txPending, hasSnapshot

    The algorithm is as follows:

  1. If event has SNAPSHOT_BEGIN flag, then clear the pending queue, set isSnapshot = true, hasSnapshot = false.

  2. If event has SNAPSHOT_END flag and isSnapshot is true, then set isSnapshot = false and hasSnapshot = true.

  3. If event has TX_PENDING, then set txPending = true.

  4. If event does not have TX_PENDING, then set txPending = false.

  5. Add event to the pending queue.

  6. If isSnapshot is false and txPending is false, then apply (see below) pending queue to the consistent order book:

    • If hasSnapshot is true, then clear (remove all orders) consistent order book and set hasSnapshot = false

    • Go through pending queue and apply each order to consistent order book:

      • If this order has REMOVE_EVENT flag, then remove event with this index from consistent order book

      • Else add or update event with this index to order book

    • Clear pending queue

  7. Wait for the next order event.

This algorithm also works in case you need to reconstruct price levels or aggregations by Market Maker.

Write a comment…