{"id":20665,"date":"2020-07-15T10:24:42","date_gmt":"2020-07-15T14:24:42","guid":{"rendered":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/"},"modified":"2022-02-17T17:52:01","modified_gmt":"2022-02-17T22:52:01","slug":"icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases","status":"publish","type":"post","link":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/","title":{"rendered":"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases"},"content":{"rendered":"
\n \n\t\n\t\n\t
\n\t\t\n\t\t\t
\n\t
\n \n

Learning how to predict future events from patterns of past events is a critical challenge in the field of artificial intelligence. As machine learning pioneer Yann LeCun writes, \u201cprediction is the essence of intelligence<\/a>.\u201d Researchers have recently experienced breakthroughs with neural networks, which have led to more accurate predictions in a wide variety of situations.<\/p>\n

Despite these advances, neural models may \u201coverfit,\u201d picking up on spurious patterns during the training process due to significant over-parameterization. Overfitting remains a chronic weakness of neural systems, particularly in situations where abundant neural links in the network outweigh patterns found in the training examples. Under these circumstances, relying purely on neural models may lead to bad predictions on novel inputs. Machine learning systems designed to make predictions about creditworthiness<\/a>, for instance, may erroneously end up identifying businesses with high debt levels and low profits as \u201csafe.\u201d Overfitting remains a barrier in applying machine learning to a range of high-stakes domains like medicine.<\/p>\n

This week, a joint team of researchers from the Department of Computer Science<\/a> at Johns Hopkins University and Bloomberg are presenting their paper \u201cNeural Datalog Through Time: Informed Temporal Modeling via Logical Specification<\/a>\u201d at the Thirty-seventh International Conference on Machine Learning (ICML 2020)<\/a>. The team includes Hongyuan Mei<\/a>, Guanghui Qin<\/a>, and Professor<\/a> Jason Eisner<\/a> from Johns Hopkins, and AI\/ML researcher Minjie Xu from Bloomberg.<\/p>\n\n<\/div>\n

\n \"\"\"\"\n \n<\/figure>\n
\n \n

The paper attacks the problem of event prediction through a unique method: a neural-symbolic hybrid. \u201cNeural Datalog\u201d is built around a deductive database based in the Datalog programming language that defines and dynamically reconfigures a neural generative model.<\/p>\n

The paper proposes a specific modeling language for encoding the database of facts and their mutual influences, thereby allowing practitioners to define facts and rules that are relevant to a problem space. The resulting structural neural network architectures work to prevent the model from overfitting on irrelevant patterns in the data.<\/p>\n

For a system that predicts future travel to Chicago, for instance, the database might pre-define that a specific individual\u2019s travel plans are determined by certain facts about Chicago, such as the weather. This prevents the model from identifying irrelevant factors that may be incidentally correlated with, but do not determine travel behavior, like sports scores.<\/p>\n

Datalog<\/a> itself harkens back to a different era in artificial intelligence research. Originally developed in the 1970s, Datalog was designed to enable researchers to build deductive databases, systems that can make deductions based on sets of formal facts and rules that are stored in the database. The researchers discovered that the language was ideally suited to the problem space and was a straightforward interface for encoding domain-specific knowledge.<\/p>\n

This unusual combination of new and old makes a major impact. The researchers tested the neural Datalog approach against two very different real-world problems: the prediction of TV viewership behavior, and the passing behavior of robot soccer players during the annual Robocup<\/a> competition. In both cases, the neural Datalog approach showed significantly better performance against strongly competitive neural methods like Know-Evolve<\/a> and DyRep<\/a>.<\/p>\n\n<\/div>\n

\n \"Bloomberg\"Bloomberg\n
Bloomberg Ph.D. Fellow Hongyuan Mei (left) with his advisor, Professor Jason Eisner of Johns Hopkins University (Photographer: Lori Hoffman\/Bloomberg)<\/figcaption>\n<\/figure>\n
\n\t
\n<\/div>\n
\n \n

This paper involved extensive work over two years. As Mei describes it, the publication of the paper is \u201ca rest stop in the middle of a long journey.\u201d The research originated in Mei\u2019s experience as a member of the <\/a>inaugural<\/a> class of Bloomberg Data Science Ph<\/a>.<\/a>D<\/a>.<\/a> Fellows<\/a>. Launched in 2018, the program<\/a> provides support and encouragement for groundbreaking publications in both academic journals and conference proceedings by exceptional Ph.D. students who may be early in their careers in broadly-construed fields of data science, including natural language processing, machine learning, and artificial intelligence.<\/p>\n

Fellows participate in a 12-week summer internship with Bloomberg data scientists in the Office of the CTO and\/or AI Engineering group, collaborating with the teams to bring cutting-edge research and engineering firepower to bear on tough problems. The fellowship provides exposure to the finance industry and Bloomberg\u2019s broad, high-quality data. As Minjie Xu, Mei\u2019s co-author and fellowship program mentor remarks, \u201cthe Fellows program is a great opportunity for top-tier researchers to share their insights with industry and apply the latest techniques to some of the hardest data challenges out there.\u201d<\/p>\n\n<\/div>\n

\n \"Senior\"Senior\n
Senior AI researcher and engineer Minjie Xu<\/figcaption>\n<\/figure>\n
\n\t
\n<\/div>\n
\n \n

The method developed by Mei and his collaborators was inspired during the fellowship by, among other technical motivations, the complex challenge of modeling market events, such as earning reports, news articles, and stock movements. These events are both numerous and have complex dependencies among them, making it difficult to address with pure neural models that do not incorporate domain-specific knowledge.<\/p>\n

Mei believes this paper provides practitioners with a useful tool to automatically configure and reconfigure a neural network based on their domain knowledge written in Datalog. For example, medical experts need only specify their knowledge about pharmaceuticals and clinical measurements to obtain a neural sequential model that can predict future patient outcomes, without bothering learning how to code the model.<\/p>\n

The paper also opens up many future directions for research. For example, the framework may be adapted to enforce logic constraints to a natural language generator to ensure that its output is meaningful and coherent.<\/p>\n

In the current framework, however, the knowledge is still hand-specified by humans. Mei is hopeful that, at some point in the future, one interesting \u2013 but very challenging \u2013 task would be to learn to generate useful and meaningful Datalog programs from the data automatically. This would massively expand the areas where the hybrids explored by him and his co-authors could be applied.<\/p>\n\n<\/div>\n

\n \n

Watch a <\/em>video presentation<\/em><\/a> of the “Neural Datalog” research, learn more about the paper during the <\/em>ICML poster session<\/em><\/a>, (9:00-9:45 AM or 8-8:45 PM EDT on Thursday, July 16th), and find out more about <\/em>Bloomberg’s Data Science Ph.D. Fellowship<\/em><\/a>.<\/em><\/p>\n\n<\/div>\n

\n \n

Also at ICML 2020, senior software engineer Dan Sun co-authored a paper \u201cServerless Inferencing on Kubernetes<\/a>\u201d about KFServing<\/a>, which Seldon CTO Clive Cox will be discussing in an invited talk during the Workshop on Challenges in Deploying and Monitoring Machine Learning Systems<\/a> on Friday, July 17th. During this same workshop, senior software engineer Yuzhui Liu will participate in a panel about open problems in machine learning systems, addressing the unsolved issue of deploying ML models serverlessly at scale.<\/p>\n

Finally, machine learning quant researcher Achintya Gopal will present a paper \u201cQuasi-autoregressive residual flows (QuAR flows)<\/a>\u201d during the ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models (INNF+ 20)<\/a> on Saturday, July 18th. Watch his poster presentation<\/a> during the workshop\u2019s poster session (1:00-1:40 PM EDT).<\/p>\n\n<\/div>\n\n<\/div>\n\n\n\t\t\n\t<\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"

Research from Bloomberg’s AI researchers & ML quants look to improve neural networks, machine learning model serving, and the learning\/modeling of probability distributions<\/p>\n","protected":false},"author":184,"featured_media":19059,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1466],"tags":[1607,1418,1472,1711,1709,1710,1713,1535,1714,1587,1708,1536,1571,1477,1570,1712,1594],"class_list":["post-20665","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-at-bloomberg","tag-computer-science","tag-data","tag-data-science","tag-databases","tag-datalog","tag-dynamic-databases","tag-dyrep","tag-fellowship","tag-know-evolve","tag-neural-network","tag-ph-d","tag-phd","tag-rd","tag-research","tag-research-and-development","tag-robocup","tag-technology"],"yoast_head":"\nICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases | Bloomberg LP<\/title>\n<meta name=\"description\" content=\"At ICML 2020, Bloomberg's AI researchers & ML quants look to improve neural networks, ML model serving, and the learning of probability distributions\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases | Bloomberg LP\" \/>\n<meta property=\"og:description\" content=\"At ICML 2020, Bloomberg's AI researchers & ML quants look to improve neural networks, ML model serving, and the learning of probability distributions\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/\" \/>\n<meta property=\"og:site_name\" content=\"Bloomberg L.P.\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/bloomberglp\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-15T14:24:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-02-17T22:52:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/07\/ICML-2020-Neural-Datalog-paper_Page_011-e1594817613480.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1699\" \/>\n\t<meta property=\"og:image:height\" content=\"1281\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"chaas30\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/07\/ICML-2020-Neural-Datalog-paper_Page_011-e1594817613480.png\" \/>\n<meta name=\"twitter:creator\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:site\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"chaas30\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/\",\"url\":\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/\",\"name\":\"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases | Bloomberg LP\",\"isPartOf\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\"},\"datePublished\":\"2020-07-15T14:24:42+00:00\",\"dateModified\":\"2022-02-17T22:52:01+00:00\",\"author\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e\"},\"description\":\"At ICML 2020, Bloomberg's AI researchers & ML quants look to improve neural networks, ML model serving, and the learning of probability distributions\",\"breadcrumb\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":\"1\",\"name\":\"Home\",\"item\":\"https:\/\/www.bloomberg.com\/company\/\"},{\"@type\":\"ListItem\",\"position\":\"2\",\"name\":\"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\",\"url\":\"https:\/\/www.bloomberg.com\/company\/\",\"name\":\"Bloomberg L.P.\",\"description\":\"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e\",\"name\":\"Bloomberg L.P.\",\"url\":\"https:\/\/www.bloomberg.com\/company\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases | Bloomberg LP","description":"At ICML 2020, Bloomberg's AI researchers & ML quants look to improve neural networks, ML model serving, and the learning of probability distributions","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/","og_locale":"en_US","og_type":"article","og_title":"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases | Bloomberg LP","og_description":"At ICML 2020, Bloomberg's AI researchers & ML quants look to improve neural networks, ML model serving, and the learning of probability distributions","og_url":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/","og_site_name":"Bloomberg L.P.","article_publisher":"https:\/\/www.facebook.com\/bloomberglp\/","article_published_time":"2020-07-15T14:24:42+00:00","article_modified_time":"2022-02-17T22:52:01+00:00","og_image":[{"width":1699,"height":1281,"url":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/07\/ICML-2020-Neural-Datalog-paper_Page_011-e1594817613480.png","type":"image\/png"}],"author":"chaas30","twitter_card":"summary_large_image","twitter_image":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/07\/ICML-2020-Neural-Datalog-paper_Page_011-e1594817613480.png","twitter_creator":"@bloomberg","twitter_site":"@bloomberg","twitter_misc":{"Written by":"chaas30","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/","url":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/","name":"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases | Bloomberg LP","isPartOf":{"@id":"https:\/\/www.bloomberg.com\/company\/#website"},"datePublished":"2020-07-15T14:24:42+00:00","dateModified":"2022-02-17T22:52:01+00:00","author":{"@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e"},"description":"At ICML 2020, Bloomberg's AI researchers & ML quants look to improve neural networks, ML model serving, and the learning of probability distributions","breadcrumb":{"@id":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.bloomberg.com\/company\/stories\/icml-2020-bloomberg-ph-d-fellow-combines-datalog-and-neural-networks-to-model-dynamic-databases\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":"1","name":"Home","item":"https:\/\/www.bloomberg.com\/company\/"},{"@type":"ListItem","position":"2","name":"ICML 2020: Bloomberg Ph.D. Fellow combines Datalog and neural networks to model dynamic databases"}]},{"@type":"WebSite","@id":"https:\/\/www.bloomberg.com\/company\/#website","url":"https:\/\/www.bloomberg.com\/company\/","name":"Bloomberg L.P.","description":"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e","name":"Bloomberg L.P.","url":"https:\/\/www.bloomberg.com\/company"}]}},"featured_image_rendered":"<img srcset='' src='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/07\/ICML-2020-Neural-Datalog-paper_Page_011-e1594817613480.png' alt='' \/>","category_info":{"name":"Tech At Bloomberg","blog_landing_name":"Tech At Bloomberg"},"_links":{"self":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20665","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/users\/184"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/comments?post=20665"}],"version-history":[{"count":2,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20665\/revisions"}],"predecessor-version":[{"id":21095,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20665\/revisions\/21095"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media\/19059"}],"wp:attachment":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media?parent=20665"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/categories?post=20665"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/tags?post=20665"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}