{"id":20694,"date":"2020-01-07T08:22:39","date_gmt":"2020-01-07T13:22:39","guid":{"rendered":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/"},"modified":"2022-02-17T17:59:47","modified_gmt":"2022-02-17T22:59:47","slug":"a-careful-and-principled-study-in-active-learning","status":"publish","type":"post","link":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/","title":{"rendered":"A Careful and Principled Study in Active Learning"},"content":{"rendered":"
\n \n\t\n\t\n\t
\n\t\t\n\t\t\t
\n\t
\n \n

Some of today\u2019s most successful applications of machine learning are based on supervised learning, a type of machine learning algorithm that relies on labelled training data encoding a priori<\/em> known ground truths to reproduce a human decision-making process. Collecting and generating such datasets can be very expensive.<\/p>\n

For example, if we were to apply machine learning in the context of medical diagnosis, then the data we wish to make predictions about– something like X-Ray images or analysis results \u2013 must be reviewed and judged by a panel of medical experts. In the finance domain, if a news editor is covering equities, monitoring the entire stream of Twitter messages for breaking news can be very time consuming. Machine learning models can be trained to monitor the stream for specific tweets about a particular topic, like \u201cearnings.\u201d This way, a reporter can be alerted only when a related news story \u201cbreaks\u201d on Twitter, enabling them to be the first to cover it. Training the models to do that is also typically expensive \u2013 each of the potentially thousands or even millions of tweets used in the model\u2019s training data set must first be annotated by hand, usually by multiple judges.<\/p>\n

Good labels and annotations are therefore key to training models to provide accurate output. Active learning allows us to select the new samples to be labelled strategically, minimizing the amount of samples needed; the goal is for it to perform as if it was trained with a much larger dataset.<\/p>\n

The Bloomberg team conducts machine learning research in-house. In a recent paper, Minjie Xu, a senior ML\/NLP researcher and engineer in London, collaborated with Gary Kazantsev, Head of Quant Technology Strategy in the Office of the CTO, to examine how to use analytical tools recently employed for interpretability research to conduct more efficient and effective active learning. Gary presented the paper, “Understanding Goal-Oriented Active Learning via Influence Functions<\/a>,\u201d at the NeurIPS 2019 Workshop on Machine Learning with Guarantees<\/a> in Vancouver, Canada on Saturday, December 14, 2019.<\/p>\n

\u201cUsing the analytical tool recently resurfaced for interpretability studies to analyze active learning algorithms, we discovered some interesting insights that shed light on a suite of popular active learning strategies, and we hope this will help people better understand their underlying mechanisms and make the right choice in practice,\u201d said Xu. \u201cIt was a natural fit for this workshop, where they focus on theoretical analyses of machine learning algorithms.\u201d<\/p>\n\n<\/div>\n

\n \"Gary\"Gary\n
Gary Kazantsev presented this poster on behalf of corresponding author Minjie Xu during the poster session of the NeurIPS 2019 Workshop on Machine Learning with Guarantees (click on the image to download a PDF of the poster).<\/figcaption>\n<\/figure>\n
\n\t
\n<\/div>\n
\n \n

Active Learning Strategies<\/strong><\/p>\n

\u201cActive learning helps you to figure out which data samples to choose for annotation so you get the most benefit from this training data,\u201d said Xu. \u201cIt helps you prioritize your annotation budget \u2014 ideally, if you have 1 million unlabelled samples, you want to annotate them all, but this takes time and energy. If you only have a budget to label 100 samples, active learning helps you identify which 100 will provide the most benefit.\u201d<\/p>\n

The most popular active learning strategies are based on the concept of \u201cuncertainty.\u201d Very often, a trained model makes a prediction using a certainty, or confidence value, of its prediction being correct. In this class of active learning, those samples that the model is most \u201cuncertain\u201d about are the ones to be selected next for annotation.<\/p>\n

Another type of active learning, called \u201cgoal-oriented active learning\u201d by Xu and Kazantsev, is guided by an explicitly chosen \u201cgoal function\u201d imposed on the trained model. The \u201cutility\u201d of each training sample is then measured by its potential influence on helping the model achieve the designated goal. The next best sample to label is the one having the highest \u201cutility.\u201d<\/p>\n

\u201cYou ask your goal function what is the goal for the new model given I have one more sample.\u201d explains Xu, \u201cAnd you want to annotate those samples so that the goal potentially increases the most.\u201d<\/p>\n\n<\/div>\n

\n \"(L-R):\"(L-R):\n
(L-R): Senior ML\/NLP researcher and engineer Minjie Xu and Gary Kazantsev, Head of Quant Technology Strategy in the Office of the CTO<\/figcaption>\n<\/figure>\n
\n \n

Cost of Goal-Oriented Active Learning from a Pool<\/strong><\/p>\n

However, this is not as straightforward as it may sound, as the ground truth label of those samples in the pool are unknown at this stage. To use them for training purposes, you have to first \u201cguess\u201d their labels in some fashion.<\/p>\n

\u201cTo carry out this goal-oriented utility evaluation, you have to do some computation for each one in this big pool of unlabelled samples \u2013 if you have one million of them, you need to carry out the computation one million times,\u201d said Xu. \u201cIf you have a quick way to compute this utility for each sample, it might still be fine. But in such goal-oriented active learning paradigm, computing the utility for even just one unlabelled sample can already take a long time.\u201d<\/p>\n

For example, if you have a thousand possible categories a label can take, and a million samples in the pool to choose from, you may end up doing the model retraining one million times one thousand times, which amounts to one billion times, just to determine one best next sample. Thus, goal-oriented active learning can be expensive. Accurately evaluating these utilities over a large pool of samples becomes almost impossible in practice.<\/p>\n

One of the contributions of the paper is to efficiently approximate the calculation of such goal-based utilities using a method from robust statistics called \u201cinfluence functions\u201d, which provide an estimate of the updated model (and accordingly the updated goal) without needing to actually retrain the model. As a result, massive model retraining costs can be replaced with much cheaper gradient computations. More importantly, the approximation also naturally extends to the batch mode setting, allowing the algorithm to select multiple new samples at each step, further saving on computational costs, which otherwise scale exponentially with the batch size.<\/p>\n\n<\/div>\n

\n \n

The Unresolved Questions<\/strong><\/p>\n

Along their journey, Xu and Kazantsev also made some unexpected discoveries that call into question some common practices and beliefs about active learning strategies.<\/p>\n

For example, if one were to “guess\u201d the label of the sample directly using current model beliefs (i.e., taking the expectation without any modifications), the approximation would yield exactly the same utility for all of the samples, making it useless. On the other hand, even after doing the expensive computation of the true utilities, they would still be fairly close and therefore have limited distinguishability. In addition, even if one were allowed to \u201cpeek\u201d at the true label of the samples, which in principle should give us an advantage over any other possible \u201cguesses,\u201d it was empirically found that many popular goal-oriented active learning strategies actually perform much worse.<\/p>\n

While Xu and Kazantsev have preliminary results, being able to refine their research further will move their work forward.<\/p>\n

\u201cThe main purpose of this paper is to share our findings with the community so fellow researchers in the machine learning field can become aware of these issues and make more exciting discoveries to carry this line of research forward,\u201d said Xu. \u201cWe are very interested in this topic and will continue looking into it.\u201d<\/p>\n\n<\/div>\n\n<\/div>\n\n\n\t\t\n\t<\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"

NeurIPS 2019 paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs through influence function approximations.<\/p>\n","protected":false},"author":184,"featured_media":19141,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1466],"tags":[1720,1703,1572,1418,1472,1485,1624],"class_list":["post-20694","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-at-bloomberg","tag-active-learning","tag-algorythms","tag-cto","tag-data","tag-data-science","tag-machine-learning","tag-ml"],"yoast_head":"\nA Careful and Principled Study in Active Learning | Bloomberg LP<\/title>\n<meta name=\"description\" content=\"NeurIPS paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs using influence function approximations.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"A Careful and Principled Study in Active Learning | Bloomberg LP\" \/>\n<meta property=\"og:description\" content=\"NeurIPS paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs using influence function approximations.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/\" \/>\n<meta property=\"og:site_name\" content=\"Bloomberg L.P.\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/bloomberglp\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-01-07T13:22:39+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-02-17T22:59:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"633\" \/>\n\t<meta property=\"og:image:height\" content=\"397\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"chaas30\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:site\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"chaas30\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/\",\"url\":\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/\",\"name\":\"A Careful and Principled Study in Active Learning | Bloomberg LP\",\"isPartOf\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\"},\"datePublished\":\"2020-01-07T13:22:39+00:00\",\"dateModified\":\"2022-02-17T22:59:47+00:00\",\"author\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e\"},\"description\":\"NeurIPS paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs using influence function approximations.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":\"1\",\"name\":\"Home\",\"item\":\"https:\/\/www.bloomberg.com\/company\/\"},{\"@type\":\"ListItem\",\"position\":\"2\",\"name\":\"A Careful and Principled Study in Active Learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\",\"url\":\"https:\/\/www.bloomberg.com\/company\/\",\"name\":\"Bloomberg L.P.\",\"description\":\"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e\",\"name\":\"Bloomberg L.P.\",\"url\":\"https:\/\/www.bloomberg.com\/company\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"A Careful and Principled Study in Active Learning | Bloomberg LP","description":"NeurIPS paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs using influence function approximations.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/","og_locale":"en_US","og_type":"article","og_title":"A Careful and Principled Study in Active Learning | Bloomberg LP","og_description":"NeurIPS paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs using influence function approximations.","og_url":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/","og_site_name":"Bloomberg L.P.","article_publisher":"https:\/\/www.facebook.com\/bloomberglp\/","article_published_time":"2020-01-07T13:22:39+00:00","article_modified_time":"2022-02-17T22:59:47+00:00","og_image":[{"width":633,"height":397,"url":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg","type":"image\/jpeg"}],"author":"chaas30","twitter_card":"summary_large_image","twitter_image":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg","twitter_creator":"@bloomberg","twitter_site":"@bloomberg","twitter_misc":{"Written by":"chaas30","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/","url":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/","name":"A Careful and Principled Study in Active Learning | Bloomberg LP","isPartOf":{"@id":"https:\/\/www.bloomberg.com\/company\/#website"},"datePublished":"2020-01-07T13:22:39+00:00","dateModified":"2022-02-17T22:59:47+00:00","author":{"@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e"},"description":"NeurIPS paper shows how to make goal-oriented active learning practical by saving massive machine learning model retraining costs using influence function approximations.","breadcrumb":{"@id":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.bloomberg.com\/company\/stories\/a-careful-and-principled-study-in-active-learning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":"1","name":"Home","item":"https:\/\/www.bloomberg.com\/company\/"},{"@type":"ListItem","position":"2","name":"A Careful and Principled Study in Active Learning"}]},{"@type":"WebSite","@id":"https:\/\/www.bloomberg.com\/company\/#website","url":"https:\/\/www.bloomberg.com\/company\/","name":"Bloomberg L.P.","description":"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/4d4a18aae79d6fcc1ea98181a906905e","name":"Bloomberg L.P.","url":"https:\/\/www.bloomberg.com\/company"}]}},"featured_image_rendered":"<img srcset='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg 633w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg 300w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg 170w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg 140w' src='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2020\/01\/NeurIPS-2019-paper-header-e1578403455478.jpg' alt='' \/>","category_info":{"name":"Tech At Bloomberg","blog_landing_name":"Tech At Bloomberg"},"_links":{"self":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20694","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/users\/184"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/comments?post=20694"}],"version-history":[{"count":1,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20694\/revisions"}],"predecessor-version":[{"id":21100,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20694\/revisions\/21100"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media\/19141"}],"wp:attachment":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media?parent=20694"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/categories?post=20694"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/tags?post=20694"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}