Instrument Profile Format

Introduction

This document describes an open format to represent and exchange basic profile information about market instruments or set of field values that define corresponding attributes of the instrument. It covers all types of tradable and indicative instruments, including stocks, funds, bonds, indices, futures, options, and others. The format provides a common framework to represent necessary information, defines the data model, and describes file formats for data distribution and exchange.

Data model

Every market instrument is represented by a single profile record as a set of field values that define corresponding attributes of the instrument. The set of defined fields and interpretations of their values form the abstraction layer of the format. Each field is characterized by its name, meaning, applicability to specific instrument types, interpretation rules, and data format. For the purpose of readability and interoperability, all values are represented in textual form even if their primary use is non-textual (e.g. numbers and dates). All values must use a Unicode standard for representation of their textual form.

Data availability

Except for a few identification fields, many fields are either optional or simply not applicable for a given instrument type. Inapplicable fields shall be ignored if present and they shall be left empty when exported. Also, in many cases, the value of certain fields is not known by the data source. Such fields are considered to have an empty value and they are usually represented by a text of length 0. Empty or missing fields shall be interpreted as undefined or unknown.

Field types

Text – value is textual information, such as a company name or exchange code.
Formatted text – complex fields use proprietary formats, such as a list of exchanges.
Number – value is a number, such as a contract size or strike price; value can be either integer or floating point; floating point values must use the . character as a decimal point; a numeric value of 0 is often considered an empty value.
Date – value is a date, such as the last trading day or expiration date; value must be formatted using YYYY-MM-DD; date 1970-01-01 is often considered an empty value.

Format extensibility

The format can be extended by the addition of new fields and new literals (enumerated values) for existing fields. All parties working with this format shall be prepared to deal with such extensions. Standard method is to ignore unknown fields as if they were not there. If some known and important field uses an unknown literal then the application can either ignore the profile altogether, replace the unknown literal with some default one or show an error to an operator and ask him to resolve the situation.

Symbology

This format assumes that instrument profiles use symbology as defined on corresponding exchanges and national markets. Extensions and augmentations of native symbology are formally beyond the scope of this format. However, there are several common notations that are currently in use:

Future symbols use the prefix / for convenience, e.g. /ESZ22:XCME
Option symbols use the prefix . for convenience, e.g. .GOOG220527C1500
Future Option symbols use the prefix ./, e.g. ./ESM22P4300:XCME
Spread symbols use the prefix =, e.g. =/AFRJ19:IFEU-/AFRM19:IFEU

Learn more about Symbology Guide

List of fields

In order to simplify interoperability, the syntax of field names is restricted: names may use only capital Latin letters, decimal digits and underscore character, and they shall start with a Latin letter.

TYPE, text – type of instrument; takes precedence in conflict cases with other fields; mandatory field; may not be empty; example: STOCK, FUTURE, OPTION
SYMBOL, text – identifier of instrument; preferable to an international one in the Latin alphabet; mandatory field; may not be empty; example: GOOG, /YGM9, .ZYEAD
DESCRIPTION, text – description of instrument; preferable to an international one in the Latin alphabet; example: Google Inc., Mini Gold Futures,Jun-2009,ETH
LOCAL_SYMBOL, text – identifier of instrument in national language; shall be empty if it is the same as the SYMBOL field
LOCAL_DESCRIPTION, text – description of instrument in national language; shall be empty if it is the same as the DESCRIPTION field
COUNTRY, text – country of origin (incorporation) of corresponding company or parent entity; shall use a two-letter country code from ISO 3166-1 standard; see ISO 3166-1 on Wikipedia; example: US, RU
OPOL, text – Official Place Of Listing, the organization that has listed this instrument; instruments with multiple listings shall use separate profiles for each listing; shall use Market Identifier Code (MIC) from ISO 10383 standard (see ISO 10383 on Wikipedia or MIC homepage) or custom dxFeed values (see Custom OPOL values chapter); example: XNAS, RTSX
EXCHANGE_DATA, text – exchange-specific data required to properly identify an instrument when communicating with an exchange; uses exchange-specific format
EXCHANGES, formatted text – list of exchanges where instrument is quoted or traded; shall use the following format:
<VALUE> ::= <empty> | <LIST>
<LIST> ::= <MIC> | <MIC> <semicolon> <LIST>
The list shall be sorted by MIC; example: ARCX;CBSX;XNAS;XNYS
CURRENCY, text – currency of quotation, pricing and trading. For Forex data, mainly shall use a three-letter currency code from ISO 4217 standard; see ISO 4217 on Wikipedia; example: USD, RUB; exclusions: MUKO, M5P, etc. For cryptocurrencies, shall use the format transferred by exchange.
BASE_CURRENCY, text – base currency of currency pair. For Forex data, shall use a three-letter currency code with some exclusions (see CURRENCY field). For cryptocurrencies, shall use the format transferred by exchange.
CFI, text – Classification of Financial Instruments code; mandatory field for OPTION instruments as it is the only way to distinguish Call/Put type, American/European exercise, Cash/Physical delivery; shall use a six-letter CFI code from ISO 10962 standard; allowed to use 'X' extensively and to omit trailing letters (assumed to be 'X'); see ISO 10962 on Wikipedia; example: ESNTPB, ESXXXX, ES , OPASPS
ISIN, text – International Securities Identifying Number; shall use a twelve-letter code from ISO 6166 standard; see ISO 6166 on Wikipedia or ISIN on Wikipedia; example: DE0007100000, US38259P5089
SEDOL, text – Stock Exchange Daily Official List; shall use a seven-letter code assigned by the London Stock Exchange; see SEDOL on Wikipedia or SEDOL on LSE; example: 2310967, 5766857
CUSIP, text – Committee on Uniform Security Identification Procedures code; shall use a nine-letter code assigned by CUSIP Services Bureau; see CUSIP on Wikipedia; example: 38259P508
ICB, number – Industry Classification Benchmark; shall use a four-digit number from ICB catalog; see ICB on Wikipedia or ICB homepage; example: 9535
SIC, number – Standard Industrial Classification; shall use a four-digit number from SIC catalog; see SIC on Wikipedia or SIC structure. Note that IPF doesn't display leading zeros in SIC numbers; example: 52 (stands for 0052), 7371
AUTOCOMPLETE_WEIGHT, number – determines the ordering of result profiles in full-text search/symbol-lookup mode (mode=ui), with higher weights leading to higher rankings and appearing first in the results. The instruments are usually weighted depending on their traded volume; example: 1, 100
MULTIPLIER, number – market value multiplier; the weight that is multiplied by the contracted price when calculating the contracted value. E.g., for HSI and H-Shares Index futures, the contract multiplier is $50 per index point, whereas in a mini-HSI futures contract, it is $10 per index point. For HKEx stock futures contracts, this is one board lot of the underlying stock; example: 100, 33.2
PRODUCT, text – an instrument which represents the series of futures contracts with the same parameters (underlying, settlement style, expiration period); example: /YG
UNDERLYING, text – primary underlying symbol for options; example: C, /YGM9
SPC, number – shares per contract for options; example: 1, 100
ADDITIONAL_UNDERLYINGS, formatted text – additional underlyings for options, including additional cash; shall use the following format:
<VALUE> ::= <empty> | <LIST>
<LIST> ::= <AU> | <AU> <semicolon> <space> <LIST>
<AU> ::= <UNDERLYING> <space> <SPC>
the list shall be sorted by UNDERLYING; example: SE 50, FIS 53; US$ 45.46
MMY, text – maturity month-year as provided for corresponding FIX tag (200); can use several different formats depending on the data source:
YYYYMM – if only year and month are specified
YYYYMMDD – if full date is specified
YYYYMMwN – if week number (within a month) is specified
EXPIRATION, date – date of expiration; example: 2009-01-17
LAST_TRADE, date – date of last trading day; example: 2009-01-16
STRIKE, number – strike price for options; example: 80, 22.5
OPTION_TYPE, text – type of option; shall use one of the following values:
STAN = Standard Options
LEAP = Long-term Equity AnticiPation Securities
SDO = Special Dated Options
BINY = Binary Options
FLEX = FLexible EXchange Options
VSO = Variable Start Options
RNGE = Range
EXPIRATION_STYLE, text – expiration cycle style. Possible values are Daily, Weeklys, EOM, Quarterlys, SemiAnnuallys, Yearlys
SETTLEMENT_STYLE, text – settlement price determination style such as Open, Close
PRICE_INCREMENTS, formatted text – minimum allowed price increments with corresponding price ranges; shall use the following format:
<VALUE> ::= <empty> | <LIST>
<LIST> ::= <INCREMENT> | <RANGE> <semicolon> <space> <LIST>
<RANGE> ::= <INCREMENT> <space> <UPPER_LIMIT>
the list shall be sorted by UPPER_LIMIT; example: 0.01 3; 0.05
TRADING_HOURS, formatted text – trading schedule; example: NewYorkUS(rt=0300;0=p04000930r09301600a16002000)
FIRST_INTEREST_DATE, date – for fixed coupon rate bonds: the periodic interest payment that the issuer makes during the life of the bond; example: 2009-01-17
INTEREST_RATE, number – the periodic interest payment that the issuer makes during the life of the bond, denominated in %; example: 2.68
ISSUE_DATE, date – the date on which an instrument is issued and begins to accrue interest; example: 2009-01-27
ANNOUNCEMENT_DATE, date – date of announcement; example: 2010-01-16
AUCTION_DATE, date – date of auction; example: 2009-01-16
PRICE_TYPE, formatted text. Possible values are PerUnit, Yield, YieldSpread
ISSUED_AS_BENCHMARK, number – <number><period: Y or M>; example: 30Y, 20Y, 12M, 1M
BENCHMARK_STATUS, formatted text. Possible values are WhenIssued, Benchmark, OffTheRun

Note

ISSUED_AS_BENCHMARK and BENCHMARK_STATUS fields shall be both filled or both empty.

List of types

BOND – debt instruments, excluding money market funds; see Bond on Wikipedia
CATEGORY:FRED — economic data time series from Federal Reserve Economic Data
CERTIFICATE — investment certificate that allows investors to participate in the price movements of specific securities. Certificates are bonds from the legal prospective, but function similarly to funds, with investor lending money to the issuer and repayment depending on the underlying security's price at the end of the term. Learn more
CFD — contract for differences, an arrangement where the differences in the settlement between the open and closing trade prices are cash-settled; see CFD on Wikipedia
ETF — exchange-traded fund; see ETF on Wikipedia
FOREX — foreign exchange market or cryptocurrency; see Forex on Wikipedia and Cryptocurrency on Wikipedia
FUTURE — futures contract, derivative instrument; see Futures on Wikipedia
INDEX — non-tradable market performance indicators
MONEY_MARKET_FUND — funds that invest in short-term debt instruments; see Money market fund on Wikipedia
MUTUAL_FUND — investment funds, excluding ETFs and money market funds; see Mutual fund on Wikipedia
OPTION — option contract, derivative instrument; see Option on Wikipedia
OTHER — non-tradable miscellaneous instruments
PRODUCT — grouping instrument for futures, aka futures product
REMOVED — special instrument type indicating instrument removal (see below)
STOCK — tradable equities, excluding ETFs and mutual funds; see Stock on Wikipedia
SPREAD — composite virtual instrument consisting of two or several individual instruments that represent multileg order. Learn more about spread instrument format
WARRANT — derivative that gives the right, but not the obligation, to buy or sell a security at a certain price before expiration

Field applicability

	FOREX	BOND	CERTIFICATE	INDEX	STOCK	ETF	MUTUAL_FUND	MONEY_MARKET_FUND	PRODUCT	FUTURE	OPTION	SPREAD	OTHER	WARRANT	CFD	CATEGORY:FRED
TYPE	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M
SYMBOL	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M
DESCRIPTION	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
LOCAL_SYMBOL	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
LOCAL_DESCRIPTION	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
COUNTRY	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
OPOL	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
EXCHANGE_DATA	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
EXCHANGES	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+
CURRENCY	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M
BASE_CURRENCY	M	-	-	-	-	-	--	-	-	-	-	-	+	-	-	-
CFI	+	+	+	+	+	+	+	+	+	+	M	+	+	+	-	-
ISIN*	+	+	+	+	+	+	+	+	-	+	+	+	+	-	-	-
SEDOL*	-	+	+	-	+	+	+	+	-	-	-	-	+	-	-	-
CUSIP*	-	+	+	-	+	+	+	+	-	-	-	-	+	-	-	-
ICB	-	+	+	+	+	+	+	+	+	+	+	-	+	+	-	-
SIC	-	+	+	+	+	+	+	+	+	+	+	-	+	+	-	-
MULTIPLIER	+	-	+	-	-	-	-	-	-	M	M	M	+	+	-	-
PRODUCT	-	-	-	-	-	-	-	-	-	M	+	+	+	-	-	-
UNDERLYING	-	-	+	-	-	-	-	-	-	-	M	-	+	M	-	-
SPC	-	-	+	-	-	-	-	-	-	-	M	-	+	+	-	-
ADDITIONAL_UNDERLYINGS	-	-	-	-	-	-	-	-	-	-	+	-	+	-	-	-
MMY	-	+	+	-	-	-	-	-	-	+	+	+	+	+	-	-
EXPIRATION	-	+	+	-	-	-	-	-	-	M	M	+	+	+	-	-
LAST_TRADE	-	+	+	-	-	-	-	-	-	M	M	+	+	+	-	+
STRIKE	-	-	-	-	-	-	-	-	-	-	M	-	+	+	-	-
OPTION_TYPE	-	-	-	-	-	-	-	-	-	-	+	-	+	-	-	-
EXPIRATION_STYLE	-	-	-	-	-	-	-	-	-	-	+	-	+	-	-	-
SETTLEMENT_STYLE	-	-	+	-	-	-	-	-	-	-	+	-	+	+	-	-
PRICE_INCREMENTS	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	-
TRADING_HOURS	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M	M
FIRST_INTEREST_DATE	-	+	+	-	-	-	-	-	-	-	-	-	+	-	-	-
INTEREST_RATE	-	+	+	-	-	-	-	-	-	-	-	-	+	-	-	-
ISSUE_DATE	-	+	+	-	-	-	-	-	-	-	+	-	+	-	-	-
ANNOUNCEMENT_DATE	-	+	-	-	-	-	-	-	-	-	-	-	+	-	-	-
AUCTION_DATE	-	+	-	-	-	-	-	-	-	-	-	-	+	-	-	-
PRICE_TYPE	-	M	-	-	-	-	-	-	-	-	-	-	+	-	-	-
ISSUED_AS_BENCHMARK	-	+	-	-	-	-	-	-	-	-	-	-	+	-	-	-
BENCHMARK_STATUS	-	+	-	-	-	-	-	-	-	-	-	-	+	-	-	-

Check values for the table above:

Value	Description
M	Mandatory field
+	Optional field is specified if available
-	Inapplicable field is empty
*	For some markets, ISIN, CUSIP and SEDOL fields are available only with the corresponding license

Snapshot and live updates model

Instrument profiles are usually received as a "snapshot" of all available instruments. Yet, profiles may change, new market instruments may appear at any time (even during trading hours), and existing ones may be removed (after their expiration or maturity). So another streaming (live updates) mode can be used. You can receive an initial snapshot followed by a stream of live updates of changed instrument profiles and additional signals.

To work with the live updates model, do the following:

Maintain the set of instrument profiles identified by the unique SYMBOL.
Consider a set of instruments as incomplete until the initial snapshot is fully received. This is indicated by the special signal marking Snapshot Complete state.
Consider all other profiles as live updates after receiving the initial snapshot.
Add the profile to the set (add action) after you receive it for the unknown symbol.
Override the previous profile (update action) after you receive it for an existing symbol.
Remove the profile from the set (delete action) after you receive it with the special REMOVED instrument type.
Send a Heartbeat signal to keep the stream alive, depending on the implementation.

For consistency reasons, a static snapshot must also contain the Snapshot Complete signal for integrity. Depending on the implementation, update and remove actions can appear in any part of a snapshot or live updates stream. Simply process them by the rules above. More than one Snapshot Complete signal can occur in a snapshot or live updates stream.

File format

Instrument profiles can be written to a file in the format, which is based on a CSV (Comma-Separated Values) format as defined in RFC 4180 - see CSV on Wikipedia or RFC 4180 homepage. The basic CSV format is extended to support several alternating record formats which formally violates CSV restrictions. To avoid possible confusion files in this format shall use an IPF extension.

The similarities and differences are:

Each record is located on a separate line. Each line ends with a line break (CRLF). The last line with a record also ends with a line break. Empty lines (parsing artifacts) shall be ignored.
Records contain fields, separated by commas. The last field must not be followed by a comma.
Header line is not used. Instead, records can be of 2 variants: metadata and profile.
Metadata records start with # (hash) character and can be either type records or commands (tags) or comments
Metadata type records define a list of fields used by profile records for a given instrument type.
Metadata commands represent signalling (e.g. Snapshot Complete) in the file format.
All other metadata records are considered comments.
Fields use standard CSV quotation rules by enclosing them in double-quotes.
UTF-8 encoding is used throughout the entire file.

A metadata type record can be distinguished by the special format of its first field: it starts with # character and ends with ::=TYPE text, and the text in the middle specifies the instrument type. The rest of the metadata type record defines a list of fields (in addition to TYPE) that are used by subsequent profile records for a specified instrument type. Thus, a metadata type record looks like the BNF definition of a profile record. Note that the TYPE field must be the first field for each profile record and it determines what format is used by this record.

The ##COMPLETE metadata command represents the Snapshot Complete signal. Note that at least one ##COMPLETE must be present in the file: usually it is located at the end of the file, but can occur in some other place separating the "initial snapshot" part from the live update part of the file. The ## metadata command represents the Heartbeat signal.

If some line starts with a hash character but does not form a correct metadata record, as described above then this line is considered a comment line and can be safely ignored. Note that a comment line must follow the CSV format in regard to the correct field separation and quotation. In this regard, commenting valid but unused CSV lines works perfectly, but plain comments shall avoid double quote characters or they can easily violate CSV specification. These extensions allow for a single IPF file to contain profiles for different instrument types with different record formats. They also allow a user to merge several files with different record formats for the same instrument types into a single one because each new metadata record redefines the record format.

Sample IPF file

#STOCK::=TYPE,SYMBOL,DESCRIPTION,CURRENCY
STOCK,GOOG,"Alphabet Inc. - Class C Capital Stock",USD
#FUTURE::=TYPE,SYMBOL,DESCRIPTION,CURRENCY,MULTIPLIER,PRODUCT,LAST_TRADE
FUTURE,/YGM23:IFUS,"Mini Gold Futures - ICUS - Jun23",USD,32.15,/YG:IFUS,2023-06-28
##COMPLETE

Sample IPF from live updates stream

# Comment (comments here and below are added for documentation purposes only)
# Metadata type records
#STOCK::=TYPE,SYMBOL,DESCRIPTION,CURRENCY 
#FUTURE::=TYPE,SYMBOL,DESCRIPTION,CURRENCY,MULTIPLIER,PRODUCT,LAST_TRADE 
# Instrument profile records
STOCK,GOOG,"Alphabet Inc. - Class C Capital Stock",USD 
FUTURE,/YGM23:IFUS,"Mini Gold Futures - ICUS - Jun23",USD,32.15,/YG:IFUS,2023-06-28
# Initial snapshot is complete
##COMPLETE
# Profile update: changed description
FUTURE,/YGM23:IFUS,"Mini Gold Futures ICUS, June 2023",USD,32.15,/YG:IFUS,2023-06-28
# Heartbeat command
## 
# FUTURE type is updated: new field CFI is added
#FUTURE::=TYPE,SYMBOL,DESCRIPTION,CURRENCY,CFI,MULTIPLIER,PRODUCT,LAST_TRADE 
# Profile update: new value for CFI field is added
FUTURE,/YGM23:IFUS,"Mini Gold Futures ICUS, June 2023",USD,FXXPSX,32.15,/YG:IFUS,2023-06-28
# Profile removal: profile for /YGM23:IFUS symbol must be removed
REMOVED,/YGM23:IFUS
## 
# etc.

In this section: