There’s often a moment when a relatively new Bloomberg Terminal user, having sampled the numerous kinds of data that can be found on it, pauses and wonders: Where does all this information come from? And who are the people whose systems curate the streams of data and news chronicling the prices of equities, bonds and other securities from New York to New Zealand, every second of every day?
The answers lie with the little-known Realtime Feeds team operating out of New York, London, Tokyo and more recently, Hong Kong. The team is responsible for managing more than 5,000 individual data feeds from stock exchanges, banks, brokers, financial institutions and news services around the world.
Anything you’ve ever read on a Bloomberg Terminal has, at one time or another, passed through the domain of this elite team of data wranglers, programmers and networking specialists. Every other part of Bloomberg – and practically every customer – utilizes the data they touch.
The scale of the team’s job is nothing less than enormous: Keep all the data – the lifeblood of the company — flowing, all the time. The technical challenges are huge, owing largely to the sheer volume of data in question. To wit, the data flowing in from a single stock exchange can amount to 10 million messages per second at peak times of the trading day. Multiply that by 530 individual exchanges and all the other data sources and the combined total quickly amounts to billions of messages every day.
Khawar Farooqi manages the team. “All the data comes through this one team and the impact on Bloomberg is tremendous,” he says. “In the event of an unexpected and unlikely issue with a single feed, such as Nasdaq or the S&P, the results could be detrimental. However, our team is building sophisticated monitoring and alarm systems for early detection and remediation.”
And more data sources are coming online all the time, as emerging markets like Pakistan and Bangladesh want the data from their exchanges and financial institutions added to the platform. Meanwhile, existing data sources change their communications infrastructure frequently, forcing the Feeds team to respond with changes of its own.
Every one of the data feeds must be normalized into structured formats and then sent to back-end systems like ticker plant, history and other databases. Engineers on the Feeds team also collaborate with business, product and engineering teams to ensure that data can be used downstream in other products and Terminal functions.
“The amount of work that we have before us is huge,” he said. “Although we have ample work before us, we remain determined in our efforts to complete all high priority projects on time.”
And that’s creating new job opportunities, Farooqi says. The team is growing. By the end of the year, Farooqi expects the team to grow by 20%, and by the middle of next year, he’d like to see that number rise by another 10%.
What makes someone a good member of the Feeds team? Familiarity with C++, the team’s primary language, is helpful, but not required, Farooqi said. Awareness of object-oriented languages is a good skill to have, he said. So is familiarity with communications protocols like FIX and SOAP. “We’ll train you in what you need to know,” he said. “On this team, you’re required to learn new languages and techniques.”
Farooqi, who is a 15-year Bloomberg veteran, says that learning along the way has been the fun part. “I’ve had to learn how to understand all the different asset classes and protocols that we deal with,” he said. “It makes us very visible to the senior managers and executives of the company. What we do here matters to all of Bloomberg.”
Learn more about the technology driving the world’s financial and business markets on our Tech at Bloomberg blog.
Interested in joining our engineering teams? Explore our open roles here.