How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading

Bloomberg recently introduced Intraday BVAL (IBVAL) Front Office, a pricing service featuring a machine learning-based system capable of delivering pricing for fixed income securities every 15 seconds. The solution, which initially delivers intraday pricing for around 30,000 TRACE eligible USD, high yield and investment-grade corporate securities, represents a major advance in the traditionally fragmented world of fixed income pricing. IBVAL Front Office is available to Bloomberg Terminal customers, as well as users of Bloomberg’s real-time streaming market data feed, B-PIPE.

IBVAL Front Office is the culmination of years of work and a decade of innovation in independent and accurate market pricing. According to the team that built it, IBVAL Front Office is a necessary tool for traders to operate effectively in increasingly electronic and automated fixed income markets.

“There are cost constraints, operational constraints, and regulatory overhead that make this a market where efficiency matters greatly,” explains David Krein, Head of Central Pricing Research in Bloomberg’s Office of the CTO. “Having access to intraday pricing to reduce time to trade and to improve accuracy — essentially, to improve economic outcomes — is top of mind.” IBVAL Front Office, he says, has immediate applications for the development of credit algorithms, portfolio trading and ETF strategies, as well as workflow automation, including trade execution.

A “critical mass” of data and demand

IBVAL Front Office is distinct from, but complementary to, Bloomberg’s award-winning evaluated pricing service, BVAL. BVAL covers fixed income instruments and over-the-counter derivatives, providing pricing market snapshots at several specified times each day. It was first launched in 2008, as the financial crisis resulted in a need for more robust independent pricing.

Though the service started relatively small, notes Timothy O’Brien, the Global COO of Bloomberg’s BVAL business in the company’s Enterprise Data department, it has grown into a leading evaluated pricing service that today monitors 2.8 million securities, 10 times per day.

Over time, says O’Brien, Bloomberg saw clients taking multiple snapshots of BVAL to track pricing throughout the day. “With the electronification of fixed income markets, plus internal needs, we recognized there was a clear demand for intraday pricing — that’s how IBVAL Front Office came about,” he says.

The IBVAL Front Office project was the result of a multi-year collaboration across Bloomberg’s Enterprise Data, the CTO’s Office, the AI Engineering group, and the Fixed Income Pricing Engineering group, along with market experts.

Fixed income pricing, in which negotiations and trades are bilateral, was a particularly important challenge to overcome. It has traditionally been “difficult, time consuming, and expensive to find good pricing information,” says Krein. Whereas equities trade on exchanges and price information is widely available in real time, over-the-counter trades require buyers and sellers to negotiate directly, making pricing notoriously difficult to predict. Typically, investors and traders would review information from disparate sources and estimate prices based on their own expertise, recent data, and current market conditions.

But, the combination of client demand and a broader shift towards electronification made it clear that the industry had reached a “critical mass,” says Krein. Fixed income trading, along with the broader financial market, has become more digital. He notes this trend has allowed more data to be collected in concentrated venues, that is, in places “that allow this information to be aggregated and assembled in a way where it can be recycled to help inform the next transaction.”

While others had begun to attempt to solve the problem of intraday pricing for fixed income bonds, few institutions possessed the data and resources to get it right. Bloomberg, with its vast troves of historical financial data and a legacy of creating innovative pricing technologies, was well positioned for this task.

“Under the hood,” says O’Brien, “it’s really the scale of our data that enables us to lead in this category and be best-in-class.”

An intraday solution built to handle billions of data points

IBVAL Front Office takes into account a tremendously high volume of data that changes by the millisecond. Developing the platform required years of work by engineers and financial experts as they sought to most effectively sort and manage this data, and train a machine learning model to accurately analyze it all.

“We didn’t design the system as a monolith,” says Junling Wang, whose Fixed Income Pricing Platforms Engineering team built the IBVAL Front Office infrastructure. “It’s not one component that does everything. We broke down the whole pricing system pipeline into multiple microservice components.”

For the engineers who built IBVAL Front Office, there were three central priorities when considering the architecture of the tool. First, the system would need to scale to handle ever-increasing volumes of information. This would be made possible by Bloomberg’s long-term investment in its private cloud infrastructure. The system would also need to be built using microservices, allowing different components to function independently so multiple teams of engineers could work on the product simultaneously without interfering with each other. Resiliency was another key priority.

The centralized data lake, from which the pricing is generated, is built atop the open source Apache Iceberg format, and contains petabytes of data. It holds a combination of market data, reference data (such as ratings and bond issuers), and internally generated metadata related to each trade. Each day, the system processes approximately 6-7 billion fixed income market data ticks via its data pipelines, ingesting a peak of 300,000 ticks per second. Pricing queries are routed using the Trino query engine.

According to Camilo Ortiz, the manager of the AI Finance Engineering division of Bloomberg’s AI Engineering group, the team tried a number of ML and traditional quantitative approaches before landing on an ensemble approach that includes regression tree models.

“While several of the modern machine learning architectures were competitive in our research process, we ultimately settled on using tree-based methods as these methods could scale better when compared to the alternatives,” he says, “even though we’re dealing with tens of thousands of trees.”

The ML model was trained over a significant period of time, which was deliberately selected to contain different market regimes in order to ensure the model learned the relationship between highly specific data sources, and it is re-trained frequently to ensure it learns new relationships. It currently takes less than six hours to re-train, conduct regression tests, and deploy an updated model. And the company’s financial experts who collaborated with his engineers, Ortiz notes, were crucial in helping to define the best practices that became codified into real-time and offline evaluations.

To support the data pipelines that make up the backbone of IBVAL Front Office, Bloomberg “rebuilt infrastructure from the ground up,” says O’Brien. After years of testing and tweaking the architecture, the system is purpose-built to scale as additional asset classes are added.

Designing with a future product set in mind

“When the fixed income trading community starts their day, they start it with Bloomberg,” says Krein. And deploying intraday fixed income pricing to over 350,000 Terminal users has already generated tangible results. “Clients are demanding this type of insight,” he says, “They’re building new applications on top of it, everything from portfolio trading and ETFs, to algorithmic pricing and automation.”

O’Brien adds that the development of the tool is an ongoing process that will see more asset classes added and continuous finetuning to ensure it is producing the most accurate pricing data.

Ortiz notes, “The ML model at the heart of the system will also get better over time as it is constantly re-trained using new data and as new model architectures emerge.”

The team also expects to price bonds from Europe and Asia in the near future.

“We built IBVAL Front Office to be able to support a growing product set,” says Krein. “We designed it knowing what today’s expectations were. But we’re also mindful that it will continue to evolve and grow in the months and years to come.”