Our INLG-19 workshop on Discourse Structure in Neural NLG was a success, with fantastic invited talks, thought-provoking discussion and interesting posters. Thanks to all who participated! The proceedings are now in the ACL Anthology.
I’m looking to hire multiple postdoctoral scholars to work on a new project whose aim is to investigate ways of adding linguistically informed structure to neural natural language generation (NNLG) models in order to enhance both the quality and controllability of NNLG in dialogue systems. The scholar will have significant freedom to create a research agenda in this area as well as opportunities to advise students and collaborate with other linguistics and AI faculty in the department and across the university. The official job posting is not yet available but informal enquiries may be sent by email in the meanwhile!
Thrilled to note that our forthcoming ACL-19 paper (also here) on using constrained decoding together with hierarchical discourse structure in neural NLG is now out! The paper shows that constrained decoding can help achieve more controllable and semantically correct output in task-oriented dialog with neural sequence-to-sequence models. While the battle to rein in neural generators continues, we’d like to think this work at least represents a quite successful skirmish!
Our paper on LSTM Hypertagging at INLG-18 shows that (partially) neuralizing a traditional grammar-based surface realizer can achieve substantial performance gains. One might have thought that by now end-to-end neural methods would’ve been shown to work the best on the surface realization task. However, our (unpublished) attempts to train an attentional sequence-to-sequence model on the exact same OpenCCG inputs worked poorly, consistent with the poor performance observed by Marcheggiani and Perez-Beltrachini in their INLG-18 paper where they experimented with both sequence-to-sequence and graph-to-sequence models on the related task of generating from the deep representations from the 2011 shared task on surface realization. Along with recent parsing results showing that grammar-based methods can outperform neural ones, this suggests that the highest quality outputs might still be obtained using grammar-based realizers. See the slides from our talk for further discussion and next steps.
Our entry in the multilingual surface realization shared task at this year’s ACL, which used a neural morphological inflection model together with a linearizer employing novel dependency locality features, achieved the best scores on the automatic metrics for more than half of the shared task languages and also did well on the human evaluation.
After a transitional period I have taken up the position of Squibs & Discussions editor for the Computational Linguistics journal, and the first squib for which I’ve served as editor is now available early access online. It’s a meta-review of effectiveness of BLEU, by Ehud Reiter, where he concludes that there is insufficient evidence for using BLEU beyond diagnostic evaluation of MT systems — a conclusion drastically at odds with much current usage.
Keep submitting your interesting squibs!
Ever thought there should be a game for teaching about structural ambiguity and why it’s so hard for computers? Done that, as noted in a previous post. And now you’ll soon be able to read about the motivation and technical details of Madly Ambiguous in our demo paper to appear at NAACL HLT 2018 next week!
Lifeng Jin, David King, Amad Hussein, Doug Danforth and I have found that to tackle the long tail of relatively infrequently asked questions in a virtual patient dialogue system, it pays to combine paraphrasing for data augmentation with neural memory-based classification, as together the two methods yield a nearly 10% absolute improvement in accuracy on the least frequently asked questions. The paper will appear next week at the 13th Workshop on Innovative Use of NLP for Building Educational Applications at NAACL HLT 2018 in New Orleans.