{"id":20556,"date":"2018-06-01T13:53:49","date_gmt":"2018-06-01T17:53:49","guid":{"rendered":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/"},"modified":"2025-06-03T08:35:21","modified_gmt":"2025-06-03T12:35:21","slug":"bloombergs-amanda-stent-natural-language-processing-research-ethics","status":"publish","type":"post","link":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/","title":{"rendered":"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics"},"content":{"rendered":"
\n \n\t\n\t\n\t
\n\t\t\n\t\t\t
\n\t

\n \"\"\n \n<\/figure>\n
\n \n

When Amanda Stent, a Natural Language Processing (NLP) Architect at Bloomberg, took on the role of program co-chair for a prestigious conference in computational linguistics, she knew it would be a lot of work. \u201cI had been an area chair several times and I\u2019d developed a lot of ideas about what I wanted to do,\u201d she says. As Stent approaches the culmination of this effort \u2013 the opening of 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)<\/a> today in New Orleans \u2013 she says, \u201cOf course, you never really know what you\u2019re getting yourself into.\u201d Her colleagues from Bloomberg will be presenting three papers at the event.<\/p>\n

Stent had a number of goals going into the conference \u2013 goals she says were shared by her \u201cfantastic\u201d co-chair, computer science professor Heng Ji of Rensselaer Polytechnic Institute (RPI). Stent wanted to increase the diversity of the reviewers. She wanted it to be easier for reviewers to give good and useful reviews of the papers they were asked to look at. And she wanted to use the conference as a platform to help data scientists, computational linguists, and others reconsider the way they view data.<\/p>\n

Stent will be giving a talk on these topics at the Second ACL Workshop on Ethics in Natural Language Processing (Ethics-NLP18)<\/a> during the conference. \u201cInformed consent at the point of data collection and the point of data sharing is rare,\u201d says Stent. \u201cThe scientific community has to jointly adopt principles for ethical data sharing, or we\u2019re going to be regulated out of existence.\u201d Stent\u2019s co-chair Ji is involved in one research community project around data science ethics<\/a>. Bloomberg sponsors another that Stent is involved in – the Community Principles on Ethical Data Sharing (CPEDS)<\/a>.<\/p>\n

In the past, says Stent, NAACL HLT has not collected demographics about its reviewers. But last year, Stent read a CRA-W conference best practices guideline<\/a> that recommended this and thought it would be wise for NAACL to follow suit. The results were not what she expected. \u201cIt used to be the case that 40 percent of people in ACL were women,\u201d she says. But only 20 percent of reviewers were female<\/a> (even less than the 22 percent of women authors at recent ACL-sponsored conferences). She\u2019s not sure exactly why there\u2019s been such a change, but she does note that graduate programs focusing on neural networks and deep learning seem to have fewer women. \u201cAnd those areas of study have pretty much taken over our field,\u201d Stent says.<\/p>\n

Stent also found only 1.1 percent of reviewers, and less than 1 percent of authors, came from Central and South America. \u201cMaybe they\u2019re publishing at other conferences, or maybe we\u2019re just not plugged in there,\u201d says Stent.<\/p>\n

Stent and Ji also wanted to make it easier for reviewers to provide good feedback to authors. That was a bit challenging because the field is growing so quickly, and many of the reviewers are less experienced. She says that more than 850 reviewers had completed their Ph.D. four or fewer years ago, compared to fewer than 250 who completed their Ph.D. five or more years ago. Stent thought a longer and more structured review form would help.<\/p>\n

On that form, she also included new questions about the use of data. One question asked reviewers whether the authors documented that they had the rights to use the data. Another asked whether the authors had documented informed consent for use of any data from human participants. \u00a0A plurality of reviewers noted that the authors of papers they looked at didn\u2019t say how they got the data, \u201cbut they scraped it from the web, because, who cares?\u201d says Stent. \u201cNot only do many authors not care, many reviewers don\u2019t care.\u201d<\/p>\n

She had expected the problem to be worse in industry, because academics are supposed to submit their planned research to an Institutional Review Board (IRB), which is charged with ensuring that research is conducted ethically. But in many cases, the academics didn\u2019t even mention IRB review in their papers.<\/p>\n

Sometimes, says Stent, researchers will use “fair use” as a shield without understanding what it means. Reddit, she says, \u201cis explicitly a free-for-all,\u201d while Twitter most definitely is not. If someone deletes one of their own posts on Twitter, anyone else with a copy of that post is supposed to delete it as well. \u201cAs a field, we have policies around appropriate citation,\u201d says Stent. \u201cWe need a policy around data.\u201d She notes that other associations, such as the Linguistics Society of America, already have policies around data.<\/p>\n

Stent\u2019s not sure why researchers in linguistics would be ahead of computer scientists, but she says that engineers tend to worry more about the system they\u2019re building than the data they use to run it. \u201cData is just something they need to feed the system,\u201d says Stent. \u201cPeople in linguistics are used to thinking about data as a first class object.\u201d<\/p>\n

When Stent and her co-chair make their confidential report to the Association\u2019s leadership, they\u2019ll have to count the few submissions that had to be rejected or withdrawn because their use of data was inappropriate. But not everything gets caught in time.<\/p>\n

\u201cComputational linguistics cares deeply about open science, which is great,\u201d says Stent. \u201cI want the field to care just as deeply about informed consent and ethical data sharing.\u201d<\/p>\n\n<\/div>\n\n<\/div>\n\n\n\t\t\n\t<\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"

A conversation with the NAACL HLT 2018 Program Co-Chair<\/p>\n","protected":false},"author":313,"featured_media":20008,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1466],"tags":[1577,1418,1676,1472,1344,1486,1580,1478,1673],"class_list":["post-20556","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-at-bloomberg","tag-big-data","tag-data","tag-data-analysis","tag-data-science","tag-inclusion","tag-natural-language-processing","tag-nlp","tag-women-in-tech","tag-women-in-technology"],"yoast_head":"\nBloomberg\u2019s Amanda Stent on NLP, Research, and Ethics | Bloomberg LP<\/title>\n<meta name=\"description\" content=\"A conversation with NAACL HLT 2018 Program Co-Chair about NLP, research, and ethics\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics | Bloomberg LP\" \/>\n<meta property=\"og:description\" content=\"A conversation with NAACL HLT 2018 Program Co-Chair about NLP, research, and ethics\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/\" \/>\n<meta property=\"og:site_name\" content=\"Bloomberg L.P.\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/bloomberglp\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-01T17:53:49+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-03T12:35:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2000\" \/>\n\t<meta property=\"og:image:height\" content=\"1333\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"akelber5\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:site\" content=\"@bloomberg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"akelber5\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/\",\"url\":\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/\",\"name\":\"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics | Bloomberg LP\",\"isPartOf\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\"},\"datePublished\":\"2018-06-01T17:53:49+00:00\",\"dateModified\":\"2025-06-03T12:35:21+00:00\",\"author\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/09c9e1a38b7f345ce5c0b4bbde1656a6\"},\"description\":\"A conversation with NAACL HLT 2018 Program Co-Chair about NLP, research, and ethics\",\"breadcrumb\":{\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":\"1\",\"name\":\"Home\",\"item\":\"https:\/\/www.bloomberg.com\/company\/\"},{\"@type\":\"ListItem\",\"position\":\"2\",\"name\":\"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#website\",\"url\":\"https:\/\/www.bloomberg.com\/company\/\",\"name\":\"Bloomberg L.P.\",\"description\":\"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/09c9e1a38b7f345ce5c0b4bbde1656a6\",\"name\":\"Bloomberg L.P.\",\"url\":\"https:\/\/www.bloomberg.com\/company\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics | Bloomberg LP","description":"A conversation with NAACL HLT 2018 Program Co-Chair about NLP, research, and ethics","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/","og_locale":"en_US","og_type":"article","og_title":"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics | Bloomberg LP","og_description":"A conversation with NAACL HLT 2018 Program Co-Chair about NLP, research, and ethics","og_url":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/","og_site_name":"Bloomberg L.P.","article_publisher":"https:\/\/www.facebook.com\/bloomberglp\/","article_published_time":"2018-06-01T17:53:49+00:00","article_modified_time":"2025-06-03T12:35:21+00:00","og_image":[{"width":2000,"height":1333,"url":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg","type":"image\/jpeg"}],"author":"akelber5","twitter_card":"summary_large_image","twitter_image":"https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg","twitter_creator":"@bloomberg","twitter_site":"@bloomberg","twitter_misc":{"Written by":"akelber5","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/","url":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/","name":"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics | Bloomberg LP","isPartOf":{"@id":"https:\/\/www.bloomberg.com\/company\/#website"},"datePublished":"2018-06-01T17:53:49+00:00","dateModified":"2025-06-03T12:35:21+00:00","author":{"@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/09c9e1a38b7f345ce5c0b4bbde1656a6"},"description":"A conversation with NAACL HLT 2018 Program Co-Chair about NLP, research, and ethics","breadcrumb":{"@id":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.bloomberg.com\/company\/stories\/bloombergs-amanda-stent-natural-language-processing-research-ethics\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":"1","name":"Home","item":"https:\/\/www.bloomberg.com\/company\/"},{"@type":"ListItem","position":"2","name":"Bloomberg\u2019s Amanda Stent on NLP, Research, and Ethics"}]},{"@type":"WebSite","@id":"https:\/\/www.bloomberg.com\/company\/#website","url":"https:\/\/www.bloomberg.com\/company\/","name":"Bloomberg L.P.","description":"Bloomberg L.P. is the leader in global business and financial information, enabling customers to make smarter, faster, more informed business decisions.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.bloomberg.com\/company\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.bloomberg.com\/company\/#\/schema\/person\/09c9e1a38b7f345ce5c0b4bbde1656a6","name":"Bloomberg L.P.","url":"https:\/\/www.bloomberg.com\/company"}]}},"featured_image_rendered":"<img srcset='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg 2000w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg 300w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg 768w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg 1024w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg 170w, https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg 140w' src='https:\/\/assets.bbhub.io\/image\/v1\/resize?width=auto&type=webp&url=https:\/\/assets.bbhub.io\/company\/sites\/51\/2017\/02\/Amanda-Stent-523A5582-select.jpg' alt='' \/>","category_info":{"name":"Tech At Bloomberg","blog_landing_name":"Tech At Bloomberg"},"_links":{"self":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20556","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/users\/313"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/comments?post=20556"}],"version-history":[{"count":1,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20556\/revisions"}],"predecessor-version":[{"id":21865,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/posts\/20556\/revisions\/21865"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media\/20008"}],"wp:attachment":[{"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/media?parent=20556"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/categories?post=20556"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bloomberg.com\/company\/wp-json\/wp\/v2\/tags?post=20556"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}