Meet the Team: Data Technologies Engineering
April 20, 2022
Bloomberg’s Data Technologies Engineering team is responsible for the data collection systems that onboard all of the referential data that drive the company’s applications and enterprise solutions. This team utilizes state-of-the-art technologies to constantly improve the accuracy, quality, coverage, timeliness, and accessibility of Bloomberg’s data. Their systems’ scalable, extensible storage also tracks data provenance.
First, let’s meet Prashanth Bethi, the head of Data Technologies Engineering.
Tell us about your role and what the team is responsible for.
Data is at the heart of everything we do at Bloomberg, and it’s what Bloomberg is built on. The Data Technologies Engineering group builds systems that onboard high-quality referential and transactional data — from various sources around the globe — to power the Bloomberg Terminal and the company’s enterprise products. We do this by developing and leveraging a variety of data acquisition techniques; building complex data processing pipelines; developing cutting-edge machine learning (ML) and natural language processing (NLP) techniques to extract key information from unstructured data; and processing terabytes of data on a daily basis. This group includes more than 300 engineers across offices in Princeton, N.J., New York City, London, San Francisco, and Singapore.
What are some of the unique technical challenges your team needs to tackle?
Solving for breadth, volume, speed, and quality are obvious and interesting technical challenges we face, especially due to the amount and variety of data that we onboard into the Bloomberg ecosystem. A unique challenge for our group is using AI and data science to derive actionable insights from data at scale and building systems that adhere to metadata management principles.
Briefly tell us about your career path.
After completing my master’s degree in computer science, I worked as a management consultant for three years, and then joined Bloomberg’s Internal Applications Engineering group in 2002. First, I implemented Bloomberg’s Procure-to-Pay and Order-to-Cash systems. Within ten years, I had become the head of the Internal Applications organization and subsequently helped it grow to about 400 engineers over the next five years.
In 2017, I had a compelling internal-mobility opportunity to lead the Enterprise Technology Engineering group, which is responsible for supporting our Enterprise Data offerings, including Bloomberg’s Real-time Market Data Feed (B-PIPE), Data License, and Connectivity & Integration (CIS). I thoroughly enjoyed developing low-latency market data products and highly performant and resilient Enterprise Data systems.
Then, in 2021, I became the head of the Data Technologies group.
What’s your strategy for choosing team members?
To build great products, it’s very important to build a great team that has the right skills, and – even more importantly – the right attitude.
Most of the work we do requires collaboration with other engineers, product teams, and end-users. Teamwork and collaboration is critical to our success. We look for candidates who are smart, empathetic, have a positive attitude, and will be good teammates.
Assessing the strengths and weaknesses of the current team also plays into the strategy for choosing team members. I strongly believe that diverse teams make smarter decisions, and creating an inclusive environment continues to be my top priority. We work towards ensuring everyone in the team understands the importance of diversity and overcoming unconscious biases so we can build high-performing teams.
What skills do you look for when hiring engineers for your team?
We are looking for engineers with strong analytical and problem-solving skills who are curious and seek continuous improvement. We have several roles (e.g., full-stack developers, web developers, data engineers, and ML engineers) across our group, each with different technical requirements. Full-stack engineering roles typically require coding fluency in Python, C++, JavaScript/TypeScript, and Java; web developers use Node.js and front-end development frameworks and tools such as React, Angular, and Vue..
What are some of the factors driving the rapid growth/expansion of your team?
My group partners with all Terminal and enterprise product lines within Bloomberg, and we are constantly researching and innovating to add incremental value to our products. As these products grow and expand coverage, they rely on my team to provide the systems to onboard data and the infrastructure and the expertise to derive real-time insights. This requires a consistent investment of resources to meet current business needs and evolve systems for the future.
The majority of your team is based in Bloomberg’s Princeton office, but you also manage people in New York, London, San Francisco, and Singapore. What should people know about the Princeton office?
Our group started in Princeton to support our Data organization, which is based there. Our office buildings were recently renovated to have the look and feel of a suburban research campus, with an open and collaborative environment that enables teams in multiple departments to work very closely with each other. Plus, this campus expanded its on-site solar installation earlier this year, so the two office buildings we work in are now totally powered by renewable energy.
The work culture in Princeton is very similar to any of our offices around the globe, and you’ll often find team members hopping on a shuttle bus to meet stakeholders in our New York City offices or vice versa. Plus, this more suburban location tends to feel more like a “family” environment, where everyone knows one another.
“Solving for breadth, volume, speed, and quality are obvious and interesting technical challenges we face, especially due to the amount and variety of data that we onboard into the Bloomberg ecosystem.”
– Prashanth Bethi
Sang Beom Kim is the Team Lead for the Company and Segment Fundamentals team in Data Technologies. He joined Bloomberg in 2017, after completing his Ph.D. in Chemical and Biological Engineering at Princeton University.
Tell us about what you’re working on now and what your biggest challenge is. What inspires you most about it?
We support the collection, standardization, and quality control of company-disclosed financial content to produce a data set of financial metrics that can be compared across different companies. The end goal is to generate a standardized product from this data that will provide comprehensive financial information for a company. We acquire financial documents from markets around the world and extract, enrich, and publish data to the Terminal. We are constantly challenged to ensure the accuracy and timeliness of the data. We achieve this through highly scalable systems for automated extraction and data processing at low latency. And we build tools that help our analyst partners in the Data department add to their domain expertise and intelligence. I’m most inspired by how we are always challenging ourselves to deliver better client experiences (both internal and external to Bloomberg) by expanding our data/market coverage, reducing time-to-market, and enhancing efficiencies in both automated and manual processing tools.
What do you feel is an advantage to hiring more people with career experience outside of software engineering and financial analysis?
The key to our success here at Bloomberg in building technology solutions that address complex problems that impact hundreds of thousands of clients is our unique combination of technical and domain knowledge, as well as “big-picture” thinking. Doing this requires our teams to have as broad a perspective as possible, so it is incredibly helpful to have people with a wide range of career experiences working here.
I personally have a background in chemical and biological engineering, not one in computer science or financial analysis. In these disciplines, I honed my skills in designing large-scale chemical plants, optimizing processing efficiency and costs, the computational modeling of biological systems, and more. Here at Bloomberg, I apply these same fundamental skills to design, optimize, and model our complex data processing platforms. But, my unique viewpoint helps me approach these problems from different angles. This diversity in backgrounds and skills of each team member brings us closer to developing the most optimal solutions. After all, the substantial impact we are having on the world is the result of our collective efforts, driven by our unique talents, experiences, and passions.
In addition to giving us the opportunity to use our strong technical and problem-solving skills to take on challenging problems, Bloomberg also provides us with the resources to learn new skills and technical domains, regardless of one’s prior experience.
How do you foster a collaborative, inclusive environment at work?
I believe the most important part of it is creating an environment that makes people comfortable to openly be “themselves.” For that, I find it important to be an active listener, which allows one to become aware of many things that one has not noticed before. By listening, you can encourage others to speak up. In addition, I constantly learn new facts and perspectives by doing so, which allows me to reflect on any unconscious biases that I might have had on other people or opinions.
How do you ensure the effectiveness of your team?
Bloomberg’s open layout of desks and conference rooms has been a great factor in promoting and enabling collaboration and communication among team members. I regularly have quick discussions with my co-workers simply by walking over to their desks without any appointments and I enjoy casual chats in the pantry over coffee.
“We are constantly challenged to ensure the accuracy and timeliness of the data. We achieve this through highly scalable systems for automated extraction and data processing at low latency.”
– Sang Beom Kim
Farzaneh Tabataba is a Senior Data Scientist in the Data Technologies Community Products team. She joined Bloomberg four years ago, after finishing her Ph.D. program in Computer Science at Virginia Tech.
Tell us about what you’re working on now and what your biggest challenge is. What inspires you most about it?
Our team is responsible for providing data about people on the Terminal. Some of our functions include BIO <GO> (people profiles), PEOP <GO> (the search engine for people), and MGMT <GO> (company management), and person cards. I have been leading our team’s ML, NLP, and knowledge graph projects, including Learning to Rank (LTR) for the people search engine, graph analysis to find relationships between companies, and our people recommendation system. The biggest challenges we face in our ML projects are getting labeled data for assessment and being unclear about the expected outputs from an ML algorithm.
Extracting meaningful relationships and concepts from our pool of data using ML inspires me the most. It always amazes me how theoretical algorithms solve practical problems when applied to real data.
Bloomberg has a distinctive culture. What attracted you to it?
I like the team spirit at Bloomberg. We help each other solve problems in a faster and better way. Our team leads are role models who help guide people onto a better path. Also, Bloomberg is well-known for offering work-life balance, such as providing vacation days, respecting family time, offering flexible schedules to work from home after the pandemic, and providing reimbursement for the cost of fitness equipment and classes.
How does your involvement with Bloomberg’s D&I Communities impact your life? What are some of the activities you’re involved with?
Being an active member of Data Technologies’ D&I workstream, I have learned a lot about different cultures. For example, I initiated an activity to honor cultural holidays from various countries, religions, and communities. We send an email out across the department that provides a short educational background and facts about those events. People shared that they loved this message and feel touched when their holidays are highlighted through this effort.
What advice do you have for people from underrepresented groups who are pursuing a career in tech?
Besides improving their technical background to be qualified for a job, I encourage them to learn soft skills to raise their voice whenever they realize they are excluded from something. Speaking up at the right time with the right language helps them to establish their roles in the company and grow better without being undermined.
How does the collaborative environment at Bloomberg create opportunities to learn new skills and expand your expertise?
Bloomberg employees share their knowledge with others through seminars and training sessions. There are also mentorship programs in which experienced engineers and managers lead mentees by teaching them technical and leadership skills. All of these collaborative efforts encourage people to expand their knowledge and expertise in an active-learning environment.
“Extracting meaningful relationships and concepts from our pool of data using ML inspires me the most. It always amazes me how theoretical algorithms solve practical problems when applied to real data.”
– Farzaneh Tabataba
Kathleen Bligh is the Team Lead for the Data Technologies Government Issuance team. She joined Bloomberg in 1999 after graduating from Rutgers University, starting in the company’s junior engineer training program in the New York office.
Tell us about what you’re working on now and what your biggest challenge is. What inspires you most about it?
Our team is responsible for the reference data related to U.S. Treasury securities and U.S. Municipal securities. These are some of the oldest and most critical data sets at Bloomberg. It is an exciting time for both of these products, as we are building an expanded dataset and modernizing the pipeline for both products. To do this, we are using a producer/consumer data model, where the producer is composed of a temporal database store powered by Apache HBase, Apache Solr, and MySQL, while the consumer is connected through messaging queues powered by Apache Kafka. Both of these systems are part of the Bloomberg Data Services (BBDS) platform for hosting datasets, which is managed by the DataHub Engineering team.
For the U.S. Treasury data set, timing and accuracy are extremely important. Certain data announcement events are well-known in the market across the globe, and our system needs to capture this data as soon as possible to enable downstream applications and clients to consume it in a timely manner. In order to provide reliable service, the new pipeline captures data from multiple sources in varying formats, as well as including expanded data points that are now captured in a point-in-time dataset. This entire pipeline will be completely automated with a multitude of monitors to identify potential issues as soon as possible. Despite being automated, the system includes tools to remediate the data quickly.
The U.S. Municipal product’s new pipeline is undergoing a major shift. Data is obtained from many sources – some of which include both structured and unstructured formats. To support faster automation and a wider range of data capture, ML models are being used to extract some of the data points. In other cases, new workflow tools have been built to streamline the data capture process and improve the quality of the data by using a business rules engine. This new data pipeline – built using a microservices platform – is also a point-in-time store of well-defined, expanded data points.
How do you keep things interesting after so many years at the same company?
Bloomberg and its products are constantly growing and adapting, so there is always a new product, technology, or opportunity in the company. Plus, Data Technologies is an exciting team because we get exposure to a variety of departments and engineers across the globe. Most importantly, my colleagues are trusting, supportive, curious, knowledgeable, creative, collaborative, passionate about doing a good job, and fun-loving.
You’re very involved in a variety of D&I initiatives. Tell us about why this is important to you and some of the things you’ve played a role in making our workplace more inclusive.
I have been a part of the Data Technologies department’s D&I Agile team. It is an innovative approach to D&I where members volunteer for a rotation lasting three months. They then choose D&I initiatives for the group to focus on during the team sprint. Joining the D&I Agile team also provides a space where people can freely express their concerns or different perspectives and take tangible steps to make improvements. Rotations in the D&I Agile team are not considered extracurricular, as those who volunteer have their workloads adjusted to accommodate their participation in this important effort.
As a D&I Agile team member, I have been part of an early effort to identify our areas of focus. Working in collaboration with another group member and Bloomberg’s D&I team, we created a D&I survey for our group, which continues to be conducted annually. The results led us to develop D&I initiatives such as a networking pairing program that provides the opportunity for individual contributors to meet for casual chats with three department leaders – both technical and people leaders. We also developed a Better Remote Meeting Workshop that focused on the steps that both meeting leaders and participants could take to make remote meetings more inclusive for all participants.
What do you think a team must have to be effective?
Effective teams need trust and empathy. A team should be able to trust their teammates and their manager. Trust and empathy will allow the team to express ideas more freely and honestly which can lead to healthy discussion and innovation. It also allows people to be more honest and not be afraid to ask a question or ask for help because they know the team will support and respect them.
What else would you like to share about your Bloomberg experience?
Bloomberg constantly has new challenging projects. However, working in the Princeton office offers a lot of unique opportunities. Princeton is the headquarters of our Data department. When trying to solve some product-specific problems, having close access to the subject matter experts in Data is invaluable, as it enables us to make better design choices. In addition, the Princeton campus has a cafeteria that offers many chances to run into colleagues throughout the day, facilitating communication and collaboration.
“Trust and empathy will allow the team to express ideas more freely and honestly which can lead to healthy discussion and innovation.”
– Kathleen Bligh
Christopher John is a Senior Software Engineer with the Data Technologies Non-Securitised Data (NSD) team, and has been with Bloomberg since 2011.
Tell us about what you’re working on now and what your biggest challenge is. What inspires you most about it?
I’m working on a project called Data Flow Recipe (DFR), where we aim to simplify the data onboarding process by providing users, such as analysts, engineers, and vendors, with a simple web interface to define their data pipelines. This approach drastically reduces the technical barriers to data ingestion, regardless of the complexity of the workflow.
Our biggest challenges are abstracting away the integration know-how and pain points between the different components in the ingestion (ETL) pipeline and giving users a one-stop-shop UI in which to create their workflow, even while using different components under the hood. As data can have many forms and sources, we’re always striving to add more integration points, while providing users with distributed tracing across the entire tech stack.
I’m inspired by our goal for DFR to be the de facto ingestion pipeline, initially within all of Data and DT Engineering, and then beyond. This will require us to expand beyond our current knowledge and identify new and existing open source tools that would further improve our DFR offering.
How do you keep things interesting after so many years at the same company?
In my time at Bloomberg, I have been working with the same group of engineers. The team has evolved over the years, with different products along the way, and split into different infrastructure teams. I have been fortunate that my team lead and manager keep challenging us as engineers, pushing us out of our comfort zone, and actively encouraging us to grow. For example, I came into the company as a C# and .NET developer. Over the years, I have learned to develop extensively in C++, JavaScript, and Python. These new skills enabled me to improve my user interface, back-end, and architecture designs.
How do you foster a collaborative, inclusive environment at work?
Everyone in the team has an equal voice, whether it be a college graduate or a seasoned engineer. Any sizable piece of work is first discussed from a design standpoint, with everyone giving feedback where applicable. All of this is then collated to produce a final design that the entire team is happy with.
As the project on which I’m working requires multiple integration points with other systems, we engage with other teams, glean their insights, and understand any concerns they may have. I personally strive to maintain a healthy relationship with every stakeholder in the project I interact with, as well as collate their input to help drive the product forward.
“I have been fortunate that my team lead and manager keep challenging us as engineers, pushing us out of our comfort zone, and actively encouraging us to grow.”
– Christopher John
Some open roles with our Data Technologies Engineering team in Princeton