{"id":34045,"date":"2023-12-04T10:05:51","date_gmt":"2023-12-04T15:05:51","guid":{"rendered":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/"},"modified":"2023-12-05T17:05:36","modified_gmt":"2023-12-05T22:05:36","slug":"how-bloomberg-uses-machine-learning-intraday-pricing-transparency","status":"publish","type":"post","link":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/","title":{"rendered":"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading"},"content":{"rendered":"
\n \n\t\n\t\n\t
\n\t\t\n\t\t\t
\n\t
\n \n

Bloomberg recently introduced<\/a> Intraday BVAL (IBVAL) Front Office, a pricing service featuring a machine learning-based system capable of delivering pricing for fixed income securities every 15 seconds. The solution, which initially delivers intraday pricing for around 30,000 TRACE eligible USD, high yield and investment-grade corporate securities, represents a major advance in the traditionally fragmented world of fixed income pricing. IBVAL Front Office<\/a> is available to Bloomberg Terminal customers, as well as users of Bloomberg\u2019s real-time streaming market data feed, B-PIPE.<\/p>\n

IBVAL Front Office is the culmination of years of work and a decade of innovation in independent and accurate market pricing. According to the team that built it, IBVAL Front Office is a necessary tool for traders to operate effectively in increasingly electronic and automated fixed income markets.<\/p>\n

\u201cThere are cost constraints, operational constraints, and regulatory overhead that make this a market where efficiency matters greatly,\u201d explains David Krein, Head of Central Pricing Research in Bloomberg\u2019s Office of the CTO. \u201cHaving access to intraday pricing to reduce time to trade and to improve accuracy \u2014 essentially, to improve economic outcomes \u2014 is top of mind.\u201d IBVAL Front Office, he says, has immediate applications for the development of credit algorithms, portfolio trading and ETF strategies, as well as workflow automation, including trade execution.<\/p>\n\n<\/div>\n

\n \n

A \u201ccritical mass\u201d of data and demand<\/h3>\n

IBVAL Front Office is distinct from, but complementary to, Bloomberg\u2019s award-winning evaluated pricing service, BVAL<\/a>. BVAL covers fixed income instruments and over-the-counter derivatives, providing pricing market snapshots at several specified times each day. It was first launched in 2008, as the financial crisis resulted in a need for more robust independent pricing.<\/p>\n

Though the service started relatively small, notes Timothy O\u2019Brien, the Global COO of Bloomberg\u2019s BVAL business in the company\u2019s Enterprise Data department, it has grown into a leading evaluated pricing service that today monitors 2.8 million securities, 10 times per day.<\/p>\n

Over time, says O\u2019Brien, Bloomberg saw clients taking multiple snapshots of BVAL to track pricing throughout the day. \u201cWith the electronification of fixed income markets, plus internal needs, we recognized there was a clear demand for intraday pricing \u2014 that\u2019s how IBVAL Front Office came about,\u201d he says.<\/p>\n

The IBVAL Front Office project was the result of a multi-year collaboration across Bloomberg\u2019s Enterprise Data, the CTO\u2019s Office, the AI Engineering group, and the Fixed Income Pricing Engineering group, along with market experts.<\/p>\n

Fixed income pricing, in which negotiations and trades are bilateral, was a particularly important challenge to overcome. It has traditionally been \u201cdifficult, time consuming, and expensive to find good pricing information,\u201d says Krein. Whereas equities trade on exchanges and price information is widely available in real time, over-the-counter trades require buyers and sellers to negotiate directly, making pricing notoriously difficult to predict. Typically, investors and traders would review information from disparate sources and estimate prices based on their own expertise, recent data, and current market conditions.<\/p>\n

But, the combination of client demand and a broader shift towards electronification made it clear that the industry had reached a \u201ccritical mass,\u201d says Krein. Fixed income trading, along with the broader financial market, has become more digital. He notes this trend has allowed more data to be collected in concentrated venues, that is, in places \u201cthat allow this information to be aggregated and assembled in a way where it can be recycled to help inform the next transaction.\u201d<\/p>\n

While others had begun to attempt to solve the problem of intraday pricing for fixed income bonds, few institutions possessed the data and resources to get it right. Bloomberg, with its vast troves of historical financial data and a legacy of creating innovative pricing technologies, was well positioned for this task.<\/p>\n

\u201cUnder the hood,\u201d says O\u2019Brien, \u201cit\u2019s really the scale of our data that enables us to lead in this category and be best-in-class.\u201d<\/p>\n\n<\/div>\n

\n \n

An intraday solution built to handle billions of data points<\/h3>\n

IBVAL Front Office takes into account a tremendously high volume of data that changes by the millisecond. Developing the platform required years of work by engineers and financial experts as they sought to most effectively sort and manage this data, and train a machine learning model to accurately analyze it all.<\/p>\n

\u201cWe didn\u2019t design the system as a monolith,\u201d says Junling Wang, whose Fixed Income Pricing Platforms Engineering team built the IBVAL Front Office infrastructure. \u201cIt\u2019s not one component that does everything. We broke down the whole pricing system pipeline into multiple microservice components.\u201d<\/p>\n\n<\/div>\n

\n \"A\"A\n
A diagram of the IBVAL Front Office pricing service pipeline<\/span><\/i><\/figcaption>\n<\/figure>\n
\n \n

For the engineers who built IBVAL Front Office, there were three central priorities when considering the architecture of the tool. First, the system would need to scale to handle ever-increasing volumes of information. This would be made possible by Bloomberg\u2019s long-term investment in its private cloud infrastructure. The system would also need to be built using microservices, allowing different components to function independently so multiple teams of engineers could work on the product simultaneously without interfering with each other. Resiliency was another key priority.<\/p>\n

The centralized data lake, from which the pricing is generated, is built atop the open source Apache Iceberg format, and contains petabytes of data. It holds a combination of market data, reference data (such as ratings and bond issuers), and internally generated metadata related to each trade. Each day, the system processes approximately 6-7 billion fixed income market data ticks via its data pipelines, ingesting a peak of 300,000 ticks per second. Pricing queries are routed using the Trino query engine.<\/p>\n

According to Camilo Ortiz, the manager of the AI Finance Engineering division of Bloomberg\u2019s AI Engineering group, the team tried a number of ML and traditional quantitative approaches before landing on an ensemble approach that includes regression tree models.<\/p>\n

\u201cWhile several of the modern machine learning architectures were competitive in our research process, we ultimately settled on using tree-based methods as these methods could scale better when compared to the alternatives,\u201d he says, \u201ceven though we\u2019re dealing with tens of thousands of trees.\u201d<\/p>\n

The ML model was trained over a significant period of time, which was deliberately selected to contain different market regimes in order to ensure the model learned the relationship between highly specific data sources, and it is re-trained frequently to ensure it learns new relationships. It currently takes less than six hours to re-train, conduct regression tests, and deploy an updated model. And the company\u2019s financial experts who collaborated with his engineers, Ortiz notes, were crucial in helping to define the best practices that became codified into real-time and offline evaluations.<\/p>\n

To support the data pipelines that make up the backbone of IBVAL Front Office, Bloomberg \u201crebuilt infrastructure from the ground up,\u201d says O\u2019Brien. After years of testing and tweaking the architecture, the system is purpose-built to scale as additional asset classes are added.<\/p>\n\n<\/div>\n

\n \"\"\"\"<\/a>\n \n<\/figure>\n
\n \n

Designing with a future product set in mind<\/h3>\n

\u201cWhen the fixed income trading community starts their day, they start it with Bloomberg,\u201d says Krein. And deploying intraday fixed income pricing to over 350,000 Terminal users has already generated tangible results. \u201cClients are demanding this type of insight,\u201d he says, \u201cThey\u2019re building new applications on top of it, everything from portfolio trading and ETFs, to algorithmic pricing and automation.\u201d<\/p>\n

O\u2019Brien adds that the development of the tool is an ongoing process that will see more asset classes added and continuous finetuning to ensure it is producing the most accurate pricing data.<\/p>\n

Ortiz notes, \u201cThe ML model at the heart of the system will also get better over time as it is constantly re-trained using new data and as new model architectures emerge.\u201d<\/p>\n

The team also expects to price bonds from Europe and Asia in the near future.<\/p>\n

\u201cWe built IBVAL Front Office to be able to support a growing product set,\u201d says Krein. \u201cWe designed it knowing what today\u2019s expectations were. But we\u2019re also mindful that it will continue to evolve and grow in the months and years to come.\u201d<\/p>\n\n<\/div>\n\n<\/div>\n\n\n\t\t\n\t<\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"

A behind-the-scenes look at the technical work that went into building Bloomberg’s IBVAL Front Office intraday pricing solution<\/p>\n","protected":false},"author":184,"featured_media":34051,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1466,88],"tags":[1498,1578,1572,28,27,2124,1485],"class_list":["post-34045","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-at-bloomberg","category-technology","tag-ai","tag-artificial-intelligence","tag-cto","tag-engineering","tag-innovation","tag-intraday-pricing","tag-machine-learning"],"yoast_head":"\nHow Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading | Bloomberg<\/title>\n<meta name=\"description\" content=\"Here's a behind-the-scenes look at the technical work that went into building Bloomberg's IBVAL Front Office intraday pricing solution.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading | Bloomberg\" \/>\n<meta property=\"og:description\" content=\"Here's a behind-the-scenes look at the technical work that went into building Bloomberg's IBVAL Front Office intraday pricing solution.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/\" \/>\n<meta property=\"og:site_name\" content=\"Bloomberg L.P.\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/bloomberglp\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-12-04T15:05:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-05T22:05:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"720\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"chaas30\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png\" \/>\n<meta name=\"twitter:creator\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:site\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"chaas30\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/\",\"url\":\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/\",\"name\":\"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading | Bloomberg\",\"isPartOf\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\"},\"datePublished\":\"2023-12-04T15:05:51+00:00\",\"dateModified\":\"2023-12-05T22:05:36+00:00\",\"author\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e\"},\"description\":\"Here's a behind-the-scenes look at the technical work that went into building Bloomberg's IBVAL Front Office intraday pricing solution.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":\"1\",\"name\":\"Home\",\"item\":\"https:\/\/www.bloomberg.com\/company\/\"},{\"@type\":\"ListItem\",\"position\":\"2\",\"name\":\"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\",\"url\":\"https:\/\/www.bloomberg.com\/company\/\",\"name\":\"Bloomberg L.P.\",\"description\":\"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e\",\"name\":\"Bloomberg L.P.\",\"url\":\"https:\/\/www.bloomberg.com\/company\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading | Bloomberg","description":"Here's a behind-the-scenes look at the technical work that went into building Bloomberg's IBVAL Front Office intraday pricing solution.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/","og_locale":"en_US","og_type":"article","og_title":"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading | Bloomberg","og_description":"Here's a behind-the-scenes look at the technical work that went into building Bloomberg's IBVAL Front Office intraday pricing solution.","og_url":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/","og_site_name":"Bloomberg L.P.","article_publisher":"https:\/\/www.facebook.com\/bloomberglp\/","article_published_time":"2023-12-04T15:05:51+00:00","article_modified_time":"2023-12-05T22:05:36+00:00","og_image":[{"width":1280,"height":720,"url":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png","type":"image\/png"}],"author":"chaas30","twitter_card":"summary_large_image","twitter_image":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png","twitter_creator":"@bloomberg","twitter_site":"@bloomberg","twitter_misc":{"Written by":"chaas30","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/","url":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/","name":"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading | Bloomberg","isPartOf":{"@id":"https:\/\/www.bloomberg.com\/company\/#website"},"datePublished":"2023-12-04T15:05:51+00:00","dateModified":"2023-12-05T22:05:36+00:00","author":{"@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e"},"description":"Here's a behind-the-scenes look at the technical work that went into building Bloomberg's IBVAL Front Office intraday pricing solution.","breadcrumb":{"@id":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.bloomberg.com\/company\/stories\/how-bloomberg-uses-machine-learning-intraday-pricing-transparency\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":"1","name":"Home","item":"https:\/\/www.bloomberg.com\/company\/"},{"@type":"ListItem","position":"2","name":"How Bloomberg uses machine learning to bring intraday pricing transparency to fixed income trading"}]},{"@type":"WebSite","@id":"https:\/\/www.bloomberg.com\/company\/#website","url":"https:\/\/www.bloomberg.com\/company\/","name":"Bloomberg L.P.","description":"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e","name":"Bloomberg L.P.","url":"https:\/\/www.bloomberg.com\/company"}]}},"featured_image_rendered":"<img srcset='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png 280w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png 300w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png 1024w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png 768w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png 1280w' src='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2023\/12\/IBVAL-blog-post-cover-image.png' alt='' \/>","category_info":{"name":"Tech At Bloomberg","blog_landing_name":"Tech At Bloomberg"},"_links":{"self":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/34045","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/users\/184"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/comments?post=34045"}],"version-history":[{"count":8,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/34045\/revisions"}],"predecessor-version":[{"id":34071,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/34045\/revisions\/34071"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media\/34051"}],"wp:attachment":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media?parent=34045"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/categories?post=34045"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/tags?post=34045"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}