After a transitional period I have taken up the position of Squibs & Discussions editor for the Computational Linguistics journal, and the first squib for which I’ve served as editor is now available early access online. It’s a meta-review of effectiveness of BLEU, by Ehud Reiter, where he concludes that there is insufficient evidence for using BLEU beyond diagnostic evaluation of MT systems — a conclusion drastically at odds with much current usage.
Keep submitting your interesting squibs!
Ever thought there should be a game for teaching about structural ambiguity and why it’s so hard for computers? Done that, as noted in a previous post. And now you’ll soon be able to read about the motivation and technical details of Madly Ambiguous in our demo paper to appear at NAACL HLT 2018 next week!
Lifeng Jin, David King, Amad Hussein, Doug Danforth and I have found that to tackle the long tail of relatively infrequently asked questions in a virtual patient dialogue system, it pays to combine paraphrasing for data augmentation with neural memory-based classification, as together the two methods yield a nearly 10% absolute improvement in accuracy on the least frequently asked questions. The paper will appear next week at the 13th Workshop on Innovative Use of NLP for Building Educational Applications at NAACL HLT 2018 in New Orleans.
Ever thought there should be a game for teaching about structural ambiguity and why it’s so hard for computers? A big thanks to Ajda, Ethan and everyone who has helped make Madly Ambiguous a success at COSI (see full credits in the github repo).
Delighted to be presenting our work on dynamic continuized CCG at the upcoming workshop on TAG and related formalisms — arguably the first paper to implement an explanatory approach to the exceptional scope of indefinites!
It was a pleasure working with the OSU Linguistics team to break current sentiment analysis systems in every which linguistically-informed way we could think of. See our paper at the upcoming EMNLP Workshop Build It, Break It: The Language Edition for details.
My Robots Podcast interview about the poster Evan Jaffe and I presented at Midwest Speech and Language Days 2017 on question interpretation in a virtual patient dialogue system has appeared. Since then, Lifeng Jin has led our subsequent work on using a CNN to improve upon a strong logistic regression baseline, yielding a remarkable 47% error reduction when used in combination with an existing pattern-matching system. The paper will appear at the 12th Workshop on Innovative Use of NLP for Building Educational Applications at EMNLP 2017 in Copenhagen.
Evan Jaffe and I presented a poster (with Laura Zimmerman and Douglas Danforth) on question interpretation in a virtual patient dialogue system at Midwest Speech and Language Days 2017 in Chicago demonstrating a remarkable error reduction when combining a pattern-matching system with one using logistic regression, also motivating the need to use paraphrasing for data augmentation.
I’m thrilled to announce that I’ll have the opportunity to continue collaborating with Eric Fosler-Lussier, Douglas Danforth, William Schuler, Evan Jaffe and others here on question-answering dialogue systems (e.g. with virtual patients) thanks to new NSF funding. The idea is to use paraphrasing to semi-automatically expand the set of known question variants in order to enhance the robustness of interpretation and enable more accurate speech recognition via discriminative training techniques.
We’ve received NSF funding for our workshop on Uphill Battles in Language Technology! Check out the amazing line-up of invited speakers.