Skip to main content

Chris Callison-Burch: CV

(Last updated November 28, 2023)

Education

PhD in Informatics
University of Edinburgh, Edinburgh, UK
Thesis: Paraphrasing and Translation
Advisors: Miles Osborne and Mark Steedman 		February 2008
M.S. with Distinction in Computer Science
University of Edinburgh, Edinburgh, UK
Thesis: Co-Training for Statistical Machine Translation
Advisor: Miles Osborne 		October 2002
B.S. with Honors in Symbolic Systems
Stanford University, Palo Alto, CA
Thesis: A Computer Model of a Grammar for English Questions
Advisor: Ivan Sag 		June 2000

Professional Appointments

Part-time Visiting Research Scientist
Allen Institute for Artificial Intelligence (AI2), Seattle, WA		 September 2023-
Visiting Research Scientist
Allen Institute for Artificial Intelligence (AI2), Seattle, WA		 January 2023-August 2023
Associate Professor
University of Pennsylvania, Philadelphia, PA		 June 2017-present
Part-time Visiting Researcher
Google, New York, NY		 December 2018-December 2020
Aravind K. Joshi Term Assistant Professor
University of Pennsylvania, Philadelphia, PA		 September 2013-June 2017
Associate Research Professor
Johns Hopkins University, Baltimore, MD		 June 2010-August 2013
Assistant Research Professor
Johns Hopkins University, Baltimore, MD		 June 2007-June 2010

Publications

My publications have been cited more than 25,000 times. I have an h-index of 61.

Conference papers

  1. Ajay Patel, Delip Rao, Ansh Kothary, Kathleen McKeown, Chris Callison-Burch (2023). Learning Interpretable Style Embeddings via Prompting LLMs. EMNLP Findings 2023. 21 pages.
  2. Bryan Li, Chris Callison-Burch (2023). PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale. EMNLP Findings 2023. 21 pages.
  3. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, Chris Callison-Burch (2023). Faithful Chain-of-Thought Reasoning. AACL-IJCNLP 2023. Area Chair Award (Interpretability and Analysis of Models for NLP). 25 pages.
  4. Alyssa Hwang, Natasha Oza, Andrew Head, Chris Callison-Burch (2023). Rewriting the Script: Adapting Text Instructions for Voice Interaction. DIS 2023. 16 pages.
  5. Liam Dugan, Anshul Wadhawan, Kyle Spence, Chris Callison-Burch, Morgan McGuire, Victor Zordan (2023). Learning When to Speak: Latency and Quality Trade-offs for Simultaneous Speech-to-Speech Translation with Offline Models. Interspeech 2023. 2 pages.
  6. Qing Lyu, Marianna Apidianaki, Chris Callison-Burch (2023). Representation of Lexical Stylistic Features in Language Models’ Embedding Space. StarSEM 2023. 18 pages.
  7. Andrew Zhu, Lara J. Martin, Andrew Head, Chris Callison-Burch (2023). CALYPSO: LLMs as Dungeon Masters' Assistants. AAID 2023. 11 pages.
  8. Pei Zhou, Andrew Zhu, Jennifer Hu, Jay Pujara, Xiang Ren, Chris Callison-Burch, Yejin Choi, Prithviraj Ammanabrolu (2023). I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons. ACL 2023. 18 pages.
  9. Andrew Zhu, Karmanya Aggarwal, Alexander Feng, Lara J. Martin, and Chris Callison-Burch (2023). FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information. ACL 2023. 21 pages.
  10. Zoey Sha Li, Ruining Zhao, Manling Li, Heng Ji, Chris Callison-Burch, Jiawei Han (2023). Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification. ACL 2023. 17 pages.
  11. Josh Magnus Ludan, Yixuan Meng, Tai Nguyen, Saurabh Shah, Qing Lyu, Marianna Apidianaki and Chris Callison-Burch (2023). Explanation-based Finetuning Makes Models More Robust to Spurious Cues. ACL 2023. 17 pages.
  12. Yijiang River Dong, Lara J. Martin, Chris Callison-Burch (2023). CORRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding. ACL Findings 2023. 15 pages.
  13. Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, Mark Yatskar (2023). Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. CVPR 2023. 18 pages.
  14. Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, Chris Callison-Burch (2023). Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text. AAAI 2023. 13 pages.
  15. Shriyash Upadhyay, Etan Ginsberg, Chris Callison-Burch (2023). Learn With Martian: A Tool For Creating Assignments That Can Write And Re-Write Themselves. EACL Demos 2023. 10 pages.
  16. Li Zhang, Hainiu Xu, Yue Yang, Shuyan Zhou, Weiqiu You, Manni Arora, Chris Callison-Burch (2023). Causal Reasoning About Entities and Events in Procedural Texts. Findings of EACL 2023. 14 pages.
  17. Ajay Patel, Bryan Li, Mohammad Sadegh Rasooli, Noah Constant, Colin Raffel, Chris Callison-Burch (2023). Bidirectional Language Models Are Also Few-shot Learners. ICLR 2023. 25 pages.
  18. Chris Callison-Burch, Gaurav Singh Tomar, Lara Martin, Daphne Ippolito, Suma Bailis and David Reitter (2022). Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence. EMNLP 2022. 15 pages.
  19. Jeffrey Young-Min Cho, Harry Li Zhang, Chris Callison-Burch (2022). Unsupervised Entity Linking with Guided Summarization and Multiple-Choice Selection. EMNLP 2022. 9 pages.
  20. Yue Yang, Artemis Panagopoulou, Marianna Apidianaki, Mark Yatskar and Chris Callison-Burch (2022). Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction. Findings of EMNLP 2022. 17 pages.
  21. Daphne Ippolito, Liam Dugan, Emily Reif, Ann Yuan, Andy Coenen, Chris Callison-Burch (2022). The Case for a Single Model that can Both Generate Continuations and Fill-in-the-Blank. NAACL 2022. 12 pages.
  22. Qing Lyu, Hua Zheng, Daoxin Li, Li Zhang, Marianna Apidianaki, Chris Callison-Burch (2022). Is “My Favorite New Movie” My Favorite Movie? Probing the Understanding of Recursive Noun Phrases. NAACL 2022. 12 pages.
  23. Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, Jason Wei (2022). A Recipe For Arbitrary Text Style Transfer with Large Language Models. ACL 2022. 12 pages.
  24. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini (2022). Deduplicating Training Data Makes Language Models Better. ACL 2022. 172 citations. 22 pages.
  25. Shuyan Zhou, Li Zhang, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison-Burch, Graham Neubig (2022). Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data. ACL 2022. 14 pages.
  26. Liam Dugan, Eleni Miltsakaki, Etan Ginsberg, Shriyash Upadhyay, Hannah Gonzalez, Dahyeon Choi, Chuning Yuan, Chris Callison-Burch (2022). A Feasibility Study of Answer-Agnostic Question Generation for Education. ACL 2022. 8 pages.
  27. Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, and Sebastian Gehrmann (2021). SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets. NeurIPS 2021. 24 pages.
  28. Qing Lyu and Li Zhang and Chris Callison-Burch (2021). Goal-Oriented Script Construction. INGL 2021. 17 pages.
  29. Nikzad Khani, Isidora Chara Tourni, Mohammad Sadegh Rasooli, Chris Callison-Burch and Derry Tanti Wijaya (2021). Cultural and Geographical Influences on Image Translatability of Words across Languages. NAACL 2021. 12 pages.
  30. Qing Lyu, Li Zhang, Chris Callison-Burch (2020). Reasoning about Goals, Steps, and Temporal Ordering with WikiHow. EMNLP 2020. 11 pages.
  31. Li Zhang, Qing Lyu, Chris Callison-Burch (2020). Intent Detection with WikiHow. AACL-IJCNLP 2020. 6 pages.
  32. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch and Douglas Eck (2020). Automatic Detection of Generated Text is Easiest when Humans are Fooled. ACL 2020. 158 citations. 15 pages.
  33. Daphne Ippolito, David Grangier, Douglas Eck and Chris Callison-Burch (2020). Toward Better Storylines with Sentence-Level Language Models. ACL 2020. Short papers. 7 pages.
  34. Daphne Ippolito, Reno Kriz, João Sedoc, Maria Kustikova and Chris Callison-Burch (2019). Comparison of Diverse Decoding Methods from Conditional Language Models. ACL 2019. 11 pages.
  35. Reno Kriz, João Sedoc, Marianna Apidianaki, Carolina Zheng, Gaurav Kumar, Eleni Miltsakaki, and Chris Callison-Burch (2019). Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification. NAACL 2019. 10 pages.
  36. Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch and Dan Roth (2019). Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims. NAACL 2019. 16 pages.
  37. Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Benjamin V. Hanrahan, Jeffrey P. Bigham and Chris Callison-Burch (2019). Worker Demographics and Earnings on Amazon Mechanical Turk: An Exploratory Analysis. CHI Late Breaking Work 2019. 5 pages.
  38. Anne Cocos, Skyler Wharton, Ellie Pavlick, Marianna Apidianaki and Chris Callison-Burch (2018). Learning Scalar Adjective Intensity from Paraphrases. EMNLP 2018. 11 pages.
  39. John Hewitt, Daphne Ippolito, Brendan Callahan, Reno Kriz, Derry Wijaya and Chris Callison-Burch (2018). Learning Translations via Images with a Massively Multilingual Image Dataset. ACL 2018. 12 pages.
  40. Reno Kriz, Eleni Miltsakaki, Marianna Apidianaki and Chris Callison-Burch (2018). Simplification Using Paraphrases and Context-based Lexical Substitution. NAACL 2018. 12 pages.
  41. Marianna Apidianaki, Guillaume Wisniewski, Anne Cocos and Chris Callison-Burch (2018). Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation. NAACL 2018. Short papers. 6 pages.
  42. Anne Cocos, Marianna Apidianaki and Chris Callison-Burch (2018). Comparing Constraints for Taxonomic Organization. NAACL 2018. 12 pages.
  43. Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, Jeffrey P. Bigham (2018). A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk. CHI 2018. Honorable Mention Award. 590 citations. 12 pages.
  44. Derry Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki and Chris Callison-Burch (2017). Learning Translations via Matrix Completion. EMNLP 2017. 12 pages.
  45. Anne Cocos, Marianna Apidianaki and Chris Callison-Burch (2017). Mapping the Paraphrase Database to WordNet. STARSEM 2017.
  46. Sneha Rajana, Chris Callison-Burch, Marianna Apidianaki and Vered Shwartz (2017). Learning Antonyms with Paraphrases and a Morphology-aware Neural Network. STARSEM 2017. 10 pages.
  47. Ann Cocos and Chris Callison-Burch (2017). The Language of Place: Semantic Value from Geospatial Context. EACL 2017. Short papers. 5 pages.
  48. Ellie Pavlick, Heng Ji, Xiaoman Pan and Chris Callison-Burch (2016). The Gun Violence Database: A new task and data set for NLP. EMNLP 2016. Short papers. 6 pages.
  49. Ellie Pavlick and Chris Callison-Burch (2016). Tense Manages to Predict Implicative Behavior in Verbs. EMNLP 2016. Short papers. 5 pages.
  50. Ellie Pavlick and Chris Callison-Burch (2016). So-Called Non-Subsective Adjectives. STARSEM 2016. Best Paper Award. 6 pages.
  51. Ellie Pavlick and Chris Callison-Burch (2016). Most babies are little and most problems are huge: Compositional Entailment in Adjective-Nouns. ACL 2016. 11 pages.
  52. Ellie Pavlick and Chris Callison-Burch (2016). Simple PPDB: A Paraphrase Database for Simplification. ACL 2016. Short papers. 108 citations. 6 pages.
  53. Anne Cocos and Chris Callison-Burch (2016). Clustering Paraphrases by Word Sense. NAACL 2016. 10 pages.
  54. Courtney Napoles, Chris Callison-Burch, and Matt Post (2016). Sentential Paraphrasing as Black-Box Machine Translation. NAACL 2016. Short papers. 5 pages.
  55. Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Benjamin Van Durme, and Chris Callison-Burch (2015). Adding Semantics to Data-Driven Paraphrasing. ACL 2015. 10 pages.
  56. Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevich, Ben Van Durme, Chris Callison-Burch (2015). PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. ACL 2015. Short papers. 363 citations. 6 pages.
  57. Ellie Pavlick, Juri Ganitkevich, Tsz Ping Chan, Xuchen Yao, Ben Van Durme, Chris Callison-Burch (2015). Domain-Specific Paraphrase Extraction. ACL 2015. Short papers. 6 pages.
  58. Ellie Pavlick, Travis Wolfe, Pushpendre Rastogi, Chris Callison-Burch, Mark Drezde, Ben Van Durme (2015). FrameNet+: Fast Paraphrastic Tripling of FrameNet. ACL 2015. Short papers. 6 pages.
  59. Mingkun Gao, Wei Xu, and Chris Callison-Burch (2015). Cost Optimization for Crowdsourcing Translation. NAACL 2015. 9 pages.
  60. Heba Elfardy, Mona Diab and Chris Callison-Burch (2015). Ideological Perspective Detection Using Semantic Features. STARSEM 2015. 10 pages.
  61. Ann Irvine and Chris Callison-Burch (2014). Hallucinating Phrase Translations for Low Resource MT. CoNLL 2014. 11 pages.
  62. Rui Yan, Mingkun Gao, Ellie Pavlick, and Chris Callison-Burch (2014). Are Two Heads are Better than One? Crowdsourced Translation via a Two-Step Collaboration between Translators and Editors. ACL 2014. 11 pages.
  63. Jonathan Weese, Juri Ganitkevitch, and Chris Callison-Burch (2014). PARADIGM: Paraphrase Diagnostics through Grammar Matching. EACL 2014. 10 pages.
  64. Ellie Pavlick, Rui Yan, and Chris Callison-Burch (2014). Crowdsourcing for Grammatical Error Correction. CSCW Poster 2014. 4 pages.
  65. Juri Ganitkevitch and Chris Callison-Burch (2014). The Multilingual Paraphrase Database. LREC 2014. 8 pages.
  66. Ann Irvine, Joshua Langfus, and Chris Callison-Burch (2014). The American Local News Corpus. LREC 2014. 4 pages.
  67. Ryan Cotterell and Chris Callison-Burch (2014). A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic. LREC 2014. 105 citations. 5 pages.
  68. Xuchen Yao, Ben Van Durme, Chris Callison-Burch and Peter Clark (2013). Semi-Markov Phrase-based Monolingual Alignment. EMNLP 2013. 11 pages.
  69. Xuchen Yao, Peter Clark, Ben Van Durme and Chris Callison-Burch (2013). A Lightweight and High Performance Monolingual Word Aligner. ACL 2013. Short papers. 6 pages.
  70. Travis Wolfe, Benjamin Van Durme, Mark Dredze, Nicholas Andrews, Charley Beller, Chris Callison-Burch, Jay DeYoung, Justin Snyder, Jonathan Weese, Tan Xu and Xuchen Yao (2013). PARMA: A Predicate Argument Aligner. ACL 2013. Short papers. 6 pages.
  71. Jason Smith, Herve Saint-Amand, Magdalena Plamada, Philipp Koehn, Chris Callison-Burch and Adam Lopez (2013). Dirt Cheap Web-Scale Parallel Text from the Common Crawl. ACL 2013. 168 citations. 10 pages.
  72. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch (2013). PPDB: The Paraphrase Database. NAACL 2013. Short papers. 921 citations. 7 pages.
  73. Ann Irvine and Chris Callison-Burch (2013). Supervised Bilingual Lexicon Induction with Multiple Monolingual Signals. NAACL 2013. Short papers. 6 pages.
  74. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch and Peter Clark (2013). Answer Extraction as Sequence Tagging with Tree Edit Distance. NAACL 2013. 245 citations. 10 pages.
  75. Xuchen Yao, Benjamin Van Durme and Chris Callison-Burch (2012). Expectations of Word Sense in Parallel Corpora. NAACL 2012. Short papers. 5 pages.
  76. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch (2012). Monolingual Distributional Similarity for Text-to-Text Generation. STARSEM 2012. 9 pages.
  77. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan and Chris Callison-Burch (2012). Machine Translation of Arabic Dialects. NAACL 2012. 202 citations. 11 pages.
  78. Alex Klementiev, Ann Irvine, Chris Callison-Burch, and David Yarowsky (2012). Toward Statistical Machine Translation without Parallel Corpora. EACL 2012. 11 pages.
  79. Juri Ganitkevitch, Chris Callison-Burch, Courtney Napoles, and Benjamin Van Durme (2011). Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation. EMNLP 2011. 12 pages.
  80. Omar Zaidan and Chris Callison-Burch (2011). The Arabic Online Commentary Dataset: An Annotated Dataset of Informal Arabic with High Dialectal Content. ACL 2011. Short papers. 206 citations. 5 pages.
  81. Omar Zaidan and Chris Callison-Burch (2011). Crowdsourcing Translation: Professional Quality from Non-Professionals. ACL 2011. 500 citations. 10 pages.
  82. Lane Schwartz, Chris Callison-Burch, William Schuler and Stephen Wu (2011). Incremental Syntactic Language Models for Phrase-based Translation. ACL 2011. 12 pages.
  83. Omar Zaidan and Chris Callison-Burch (2010). Predicting Human-Targeted Translation Edit Rate via Untrained Human Annotators. NAACL 2010. Short papers. 4 pages.
  84. Kathryn Baker, Michael Bloodgood, Chris Callison-Burch, Bonnie Dorr, Scott Miller, Christine Piatko, Nathaniel W. Filardo, and Lori Levin (2010). Semantically-Informed Syntactic Machine Translation: A Tree-Grafting Approach. AMTA 2010. 10 pages.
  85. Ann Irvine, Alex Klementiev, and Chris Callison-Burch (2010). Transliterating From All Languages. AMTA 2010. 8 pages.
  86. Michael Bloodgood and Chris Callison-Burch (2010). Large-Scale, Cost-Focused Active Learning for Statistical Machine Translation. ACL 2010. 11 pages.
  87. Abby Levenberg, Chris Callison-Burch, and Miles Osborne (2010). Stream-based Translation Models for Statistical Machine Translation. NAACL 2010. 9 pages.
  88. Scott Novotney and Chris Callison-Burch (2010). Cheap, Fast and Good Enough: Automatic Speech Recognition with Non-Expert Transcription. NAACL 2010. 211 citations. 9 pages.
  89. Chris Callison-Burch (2009). Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon's Mechanical Turk. EMNLP 2009. Nominated for the ACL 2019 Test of Time Award. 632 citations. 10 pages.
  90. Omar Zaidan and Chris Callison-Burch (2009). Feasibility of Human-in-the-loop Minimum Error Rate Training. EMNLP 2009. 10 pages.
  91. Yuval Marton, Chris Callison-Burch and Philip Resnik (2009). Improved Statistical Machine Translation Using Monolingually-Derived Paraphrases. EMNLP 2009. 198 citations. 10 pages.
  92. Nikesh Garera, Chris Callison-Burch and David Yarowsky (2009). Improving Translation Lexicon Induction from Monolingual Corpora via Dependency Contexts and Part-of-Speech Equivalences. CoNLL 2009. 9 pages.
  93. Chris Callison-Burch (2008). Syntactic Constraints on Paraphrases Extracted from Parallel Corpora. EMNLP 2008. 229 citations. 10 pages.
  94. Chris Callison-Burch, Trevor Cohn, Mirella Lapata (2008). ParaMetric: An Automatic Evaluation Metric for Paraphrasing. CoLing 2008. 8 pages.
  95. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst (2007). Moses: Open source toolkit for statistical machine translation. ACL 2007. 7047 citations.
  96. Chris Callison-Burch, Philipp Koehn and Miles Osborne (2006). Improved Statistical Machine Translation Using Paraphrases. NAACL 2006. 391 citations.
  97. Chris Callison-Burch, Miles Osborne and Philipp Koehn (2006). Re-evaluating the Role of Bleu in Machine Translation Research. EACL 2006. 960 citations. 8 pages.
  98. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2005). Scaling Phrase-Based Statistical Machine Translation to Larger Corpora and Longer Phrases. ACL 2005. 116 citations.
  99. Colin Bannard and Chris Callison-Burch (2005). Paraphrasing with Bilingual Parallel Corpora. ACL 2005. 772 citations.
  100. Chris Callison-Burch, David Talbot and Miles Osborne (2004). Statistical Machine Translation with Word- and Sentence-Aligned Parallel Corpora. ACL 2004. 161 citations.

Journal articles

  1. Aditya Kashyap, Chris Callison-Burch, Mary Regina Boland (2022). A Deep Learning Method to Detect Opioid Prescription and Opioid Use Disorder from Electronic Health Records. International Journal of Medical Informatics 2022. 44 pages.
  2. Monique A Sager, Aditya M Kashyap, Mila Tamminga, Sadhana Ravoori, Chris Callison-Burch and Jules B Lipoff (2021). Identifying and Responding to Health Misinformation on Reddit Dermatology Forums With Artificially Intelligent Bots Using Natural Language Processing: Design and Evaluation Study. JMIR 2021. 7 pages.
  3. Isabel Straw and Chris Callison-Burch (2020). Artificial Intelligence in mental health and the biases of language based models. PLOS One 2020. 19 pages.
  4. Aditya Kashyap, Heather Burris, Chris Callison-Burch, Mary Regina Boland (2020). The CLASSE GATOR (CLinical Acronym SenSE disambiGuATOR): A Method for Predicting Acronym Sense from Neonatal Clinical Notes. International Journal of Medical Informatics 2020.
  5. Benjamin Chrisinger, Eliza Kinsey, Ellie Pavlick, Chris Callison-Burch (2020). SNAP judgments into the digital age: Reporting on food stamps varies significantly with time, publication type, and political leaning. PLOS One 2020. 19 pages.
  6. Anne Cocos and Chris Callison-Burch (2019). Paraphrase-Sense-Tagged Sentences. TACL 2019. 15 pages.
  7. Edidiong Okon, Vishnutheja Rachakonda, Hyo Jung Hong, Chris Callison-Burch and Jules Lipoff (2019). Natural Language Processing of Reddit Data to Evaluate Dermatology Patient Experiences and Therapeutics. Journal of the American Academy of Dermatology 2019. 20 pages.
  8. Anne Cocos, Ting Qiana, Chris Callison-Burch, and Aaron J. Masino (2017). Crowd Control: Effectively Utilizing Unscreened Crowd Workers for Biomedical Data Annotation. Journal of Biomedical Informatics 2017. 22 pages.
  9. Wei Xu, Courtney Napoles, Ellie Pavlick, Jim Chen, and Chris Callison-Burch (2016). Optimizing Statistical Machine Translation for Text Simplification. TACL 2016. 520 citations. 15 pages.
  10. Ann Irvine and Chris Callison-Burch (2016). A Comprehensive Analysis of Bilingual Lexicon Induction. Computational Linguistics 2016. 38 pages.
  11. Ann Irvine and Chris Callison-Burch (2016). End-to-End Statistical Machine Translation with Zero or Small Parallel Texts. Journal of Natural Language Engineering 2016. 34 pages.
  12. Wei Xu, Chris Callison-Burch, and Courtney Napoles (2015). Problems in Current Text Simplification Research: New Data Can Help. TACL 2015. 420 citations. 16 pages.
  13. Omar Zaidan and Chris Callison-Burch (2014). Arabic Dialect Identification. Computational Linguistics 2014. 36 pages.
  14. Wei Xu, Alan Ritter, Chris Callison-Burch, William B. Dolan and Yangfeng Ji (2014). Extracting Lexically Divergent Paraphrases from Twitter. TACL 2014. 130 citations. 14 pages.
  15. Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev, and Chris Callison-Burch (2014). The Language Demographics of Amazon Mechanical Turk. TACL 2014. 13 pages.
  16. Adam Lopez, Matt Post, Chris Callison-Burch, Jonathan Weese, Juri Ganitkevitch, Narges Ahmidi, Olivia Buzek, Leah Hanson, Beenish Jamil, Matthias Lee, Ya-Ting Lin, Henry Pao, Fatima Rivera, Leili Shahriyari, Debu Sinha, Adam Teichert, Stephen Wampler, Michael Weinberger, Daguang Xu, Lin Yang, and Shang Zhao (2013). Learning to translate with products of novices: a suite of open-ended challenge problems for teaching MT. TACL 2013. 13 pages.
  17. Kathryn Baker, Bonnie Dorr, Michael Bloodgood, Chris Callison-Burch, Wes Filardo, Christine Piatko, Lori Levin, and Scott Miller (2012). Use of Modality and Negation in Semantically-Informed Syntactic MT. Computational Linguistics 2012. 28 pages.
  18. Ann Irvine, Mike Kayser, Zhifei Li, Wren Thornton, and Chris Callison-Burch (2010). Integrating Output from Specialized Modules in Machine Translation: Transliteration in Joshua. PBML 2010. 10 pages.
  19. Jonathan Weese and Chris Callison-Burch (2010). Visualizing Data Structures in Parsing-Based Machine Translation. PBML 2010. 10 pages.
  20. Lane Schwartz and Chris Callison-Burch (2010). Hierarchical Phrase-Based Grammar Extraction in Joshua: Suffix Arrays and Prefix Trees. PBML 2010. 10 pages.
  21. Zhifei Li, Chris Callison-Burch, Sanjeev Khudanpur, and Wren Thornton (2009). Decoding in Joshua: Open Source, Parsing-Based Machine Translation. PBML 2009. 10 pages.
  22. Trevor Cohn, Chris Callison-Burch, Mirella Lapata (2008). Constructing Corpora for the Development and Evaluation of Paraphrase Systems. Computational Linguistics 2008. 18 pages.

Workshop papers

  1. Andrew Zhu, Liam Dugan, Alyssa Hwang, Chris Callison-Burch (2023). Kani 🦀: A Lightweight and Highly Hackable Framework for Building Language Model Applications. 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS) 2023. 13 pages.
  2. Li Zhang, Liam Dugan, Hainiu Xu, Chris Callison-Burch (2023). Exploring the Curious Case of Code Prompts. Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE) 2023. 9 pages.
  3. Hannah Gonzalez, Jiening Li, Helen Jin, Jiaxuan Ren, Hongyu Zhang, Ayotomiwa Akinyele, Adrian Wang, Eleni Miltsakaki, Ryan Baker, Chris Callison-Burch (2023). Automatically Generated Summaries of Video Lectures May Enhance Students’ Learning Experience. Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) 2023. 12 pages.
  4. Hannah Gonzalez, Liam Dugan, Eleni Miltsakaki, Zhiqi Cui, Jiaxuan Ren, Bryan Li, Shriyash Upadhyay, Etan Ginsberg, Chris Callison-Burch (2023). Enhancing Human Summaries for Question-Answer Generation in Education. Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) 2023. 11 pages.
  5. Shriyash Upadhyay, Etan Ginsberg, Chris Callison-Burch (2023). Improving Mathematics Tutoring With A Code Scratchpad. Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) 2023. 9 pages.
  6. Harry Li Zhang, Chris Callison-Burch (2023). Language Models are Drummers: Drum Composition with Natural Language Pre-Training. AAAI 2023 Workshop on Creative AI Across Modalities 2023. 8 pages.
  7. Rebecca Iglesias-Flores, Megha Mishra, Ajay Patel, Akanksha Malhotra, Reno Kriz, Martha Palmer and Chris Callison-Burch (2021). TopGuNN: Fast NLP Training Data Augmentation using Large Corpora. Workshop on Data Science with Human in the Loop 2021. 15 pages.
  8. Anietie Andy, Chris Callison-Burch and Derry Wijaya (2020). Resolving Pronouns in Twitter Streams: Context Can Help. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC) 2020. 6 pages.
  9. Anietie Andy, Derry Wijaya and Chris Callison-Burch (2019). Winter is here: Summarizing Twitter Streams related to Pre-Scheduled Events. Proceedings of the Second Workshop on Storytelling 2019. 5 pages.
  10. Aina Garí Soler, Anne Cocos, Marianna Apidianaki, Chris Callison-Burch (2019). A Comparison of Context-sensitive Models for Lexical Substitution. 13th International Conference on Computational Semantics (IWCS) 2019. 12 pages.
  11. Daphne Ippolito, David Grangier, Chris Callison-Burch and Douglas Eck (2019). Unsupervised Hierarchical Story Infilling. 13th International Conference on Computational Semantics (IWCS) 2019. 7 pages.
  12. Bhavna Saluja, Gaurav Kumar, João Sedoc, and Chris Callison-Burch (2019). Anonymization of Sensitive Information in Medical Health Records. Iberian Languages Evaluation Forum 2019. 7 pages.
  13. Joao Sedoc, Daphne Ippolito, Arun Kirubarajan, Jai Thirani, Lyle Ungar, and Chris Callison-Burch (2018). ChatEval: A Tool for the Systematic Evaluation of Chatbots. Workshop on Intelligent Interactive Systems and Language Generation 2018. 4 pages.
  14. Anietie Andy, Mark Dredze, Mugizi Rwebangira, and Chris Callison-Burch (2017). Constructing an Alias List for Named Entities During an Event. Workshop on Noisy User-generated Text 2017. 5 pages.
  15. Courtney Napoles and Chris Callison-Burch (2017). Systematically Adapting Machine Translation for Grammatical Error Correction. 12th Workshop on Innovative Use of NLP for Building Educational Applications (BEA12) 2017. 12 pages.
  16. Anne Cocos, Marianna Apidianaki and Chris Callison-Burch (2017). Word Sense Filtering Improves Embedding-Based Lexical Substitution. Workshop on Sense, Concept and Entity Representations and their Applications 2017. Best Paper Award. 9 pages.
  17. Ellie Pavlick and Chris Callison-Burch (2016). The Gun Violence Database. Bloomberg Data for Good Exchange 2016. 6 pages.
  18. Wei Xu, Chris Callison-Burch, and Bill Dolan (2015). SemEval-2015 Task 1: Paraphrase and Semantic Similarity in Twitter. SemEval 2015. 172 citations. 11 pages.
  19. Anne Cocos, Aaron J. Masino, Ting Qian, Ellie Pavlick, and Chris Callison-Burch (2015). Effectively Crowdsourcing Radiology Report Annotations. Sixth International Workshop on Health Text Mining and Information Analysis 2015. 6 pages.
  20. Courtney Napoles and Chris Callison-Burch (2015). Automatically Scoring Freshman Writing: A Preliminary Investigation. Workshop on Innovative Use of NLP for Building Educational Applications 2015. 10 pages.
  21. Ellie Pavlick and Chris Callison-Burch (2015). Extracting Structured Information via Automatic + Human Computation. HCOMP 2015. 2 pages.
  22. Gaurav Kumar, Yuan Cao, Ryan Cotterell, Chris Callison-Burch, Daniel Povey, and Sanjeev Khudanpur (2014). Translations of the CALLHOME Egyptian Arabic corpus for conversational speech translation. IWSLT 2014. 5 pages.
  23. Quanze Chen, Chenyang Lei, Wei Xu, Ellie Pavlick and Chris Callison-Burch (2014). Poetry of the Crowd: A Human Computation Algorithm to Convert Prose into Rhyming Verse. HCOMP Poster 2014. 3 pages.
  24. Chris Callison-Burch (2014). Crowd-Workers: Aggregating Information Across Turkers To Help Them Find Higher Paying Work. HCOMP Poster 2014. 2 pages.
  25. Ann Irvine and Chris Callison-Burch (2014). Using Comparable Corpora to Adapt MT Models to New Domains. WMT 2014. 8 pages.
  26. Ryan Cotterell, Adithya Renduchintala, Naomi Saphra, and Chris Callison-Burch (2014). An Algerian Arabic-French Code-Switched Corpus. LREC Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools 2014. 4 pages.
  27. Matt Post, Gaurav Kumar, Adam Lopez, Damianos Karakos, Chris Callison-Burch and Sanjeev Khudanpur (2013). Improved Speech-to-Text Translation with the Fisher and Callhome Spanish–English Speech Translation Corpus. IWSLT 2013. 129 citations. 7 pages.
  28. Ondrej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia (2013). Findings of the 2013 Workshop on Statistical Machine Translation. WMT 2013. 44 pages.
  29. Matt Post, Juri Ganitkevitch, Luke Orland, Jonathan Weese, Yuan Cao, and Chris Callison-Burch (2013). Joshua 5.0: Sparser, better, faster, server. WMT 2013. 7 pages.
  30. Ann Irvine and Chris Callison-Burch (2013). Combining Bilingual and Comparable Corpora for Low Resource Machine Translation. WMT 2013. 9 pages.
  31. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia (2012). Findings of the 2012 Workshop on Statistical Machine Translation. WMT 2012. 42 pages.
  32. Matt Post, Chris Callison-Burch, and Miles Osborne (2012). Constructing Parallel Corpora for Six Indian Languages via Crowdsourcing. WMT 2012. 164 citations. 9 pages.
  33. Jonathan Weese, Chris Callison-Burch, and Adam Lopez (2012). Using Categorial Grammar to Label Translation Rules. WMT 2012. 10 pages.
  34. Juri Ganitkevitch, Yuan Cao, Jonathan Weese, Matt Post, and Chris Callison-Burch (2012). Joshua 4.0: Packing, PRO, and Paraphrases. WMT 2012. 9 pages.
  35. Ann Irvine, Jonathan Weese, and Chris Callison-Burch (2012). Processing Informal, Romanized Pakistani Text Messages. the NAACL Workshop on Language in Social Media 2012. 4 pages.
  36. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan (2011). Findings of the 2011 Workshop on Statistical Machine Translation. WMT 2011. 43 pages.
  37. Charley Chan, Chris Callison-Burch, and Benjamin Van Durme (2011). Reranking Bilingually Extracted Paraphrases Using Monolingual Distributional Similarity. GEMS 2011. 10 pages.
  38. Jonathan Weese, Juri Ganitkevitch, Chris Callison-Burch, Matt Post and Adam Lopez (2011). Joshua 3.0: Syntax-based Machine Translation with the Thrax Grammar Extractor. WMT 2011. 7 pages.
  39. Byung Gyu Ahn, Ben Van Durme and Chris Callison-Burch (2011). WikiTopics: What is Popular on Wikipedia and Why. ACL Workshop on Automatic Summarization for Different Genres, Media, and Languages 2011. 8 pages.
  40. Courtney Napoles, Ben Van Durme (2011). Evaluating sentence compression: Pitfalls and suggested remedies. Workshop on Monolingual Text-To-Text Generation 2011. 7 pages.
  41. Courtney Napoles, Chris Callison-Burch, Juri Ganitevitch, Ben Van Durme (2011). Paraphrastic Sentence Compression with a Character-based Metric: Tightening without Deletion. Workshop on Monolingual Text-To-Text Generation 2011. 7 pages.
  42. Rui Wang and Chris Callison-Burch (2011). Paraphrase Fragment Extraction from Monolingual Comparable Corpora. BUCC 2011. 9 pages.
  43. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Ann Irvine, Lane Schwartz, Wren N. G. Thornton, Ziyuan Wang, Jonathan Weese and Omar F. Zaidan (2010). Joshua 2.0: A Toolkit for Parsing-Based Machine Translation with Syntax, Semirings, Discriminative Training and Other Goodies. WMT 2010. 5 pages.
  44. Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, Omar Zaidan (2010). Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. WMT 2010. 254 citations. 33 pages.
  45. Chris Callison-Burch and Mark Dredze (2010). Creating Speech and Language Data With Amazon’s Mechanical Turk. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 430 citations. 12 pages.
  46. Michael Bloodgood and Chris Callison-Burch (2010). Using Mechanical Turk to Build Machine Translation Evaluation Sets. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 4 pages.
  47. Scott Novotoney and Chris Callison-Burch (2010). Crowdsourced Accessibility: Elicitation of Wikipedia Articles. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 4 pages.
  48. Rui Wang and Chris Callison-Burch (2010). Cheap Facts and Counter-Facts. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 5 pages.
  49. Chris Callison-Burch, Philipp Koehn, Christof Monz and Josh Schroeder (2009). Findings of the 2009 Workshop on Statistical Machine Translation. WMT 2009. 834 citations. 28 pages.
  50. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren Thornton, Jonathan Weese and Omar Zaidan (2009). Joshua: An Open Source Toolkit for Parsing-based Machine Translation. WMT 2009. 230 citations. 5 pages.
  51. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz and Josh Schroeder (2008). Further Meta-Evaluation of Machine Translation. WMT 2008. 306 citations. 37 pages.
  52. Delip Rao, David Yarowsky, Chris Callison-Burch (2008). Affinity Measures based on the Graph Laplacian. of the 3rd Textgraphs workshop on Graph-based Algorithms for Natural Language Processing at CoLing 2008. 8 pages.
  53. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz and Josh Schroeder (2007). (Meta-) Evaluation of Machine Translation. WMT 2007. 188 citations.
  54. Philipp Koehn, Nicola Bertoldi, Ondrej Bojar, Chris Callison-Burch, Alexandra Constantin, Brooke Cowan, Chris Dyer, Marcello Federico, Evan Herbst, Hieu Hoang, Christine Moran, Wade Shen, and Richard Zens (2007). Open Source Toolkit for Statistical Machine Translation: Factored Translation Models and Confusion Network Decoding. CLSP Summer Workshop Final Report WS, Johns Hopkins University 2007.
  55. Alexandra Birch, Chris Callison-Burch and Miles Osborne (2006). Constraining the Phrase-Based, Joint Probability Statistical Translation Model. WMT 2006.
  56. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2005). A Compact Data Structure for Searchable Translation Memories. EAMT 2005.
  57. Chris Callison-Burch (2005). Linear B System Description for the 2005 NIST MT Evaluation Exercise. Machine Translation Evaluation Workshop 2005.
  58. Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, Chris Callison-Burch, Miles Osborne, and David Talbot (2005). Edinburgh System Description for the 2005 IWSLT Speech Translation Evaluation. IWSLT 2005. 446 citations.
  59. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2004). Searchable Translation Memories. ASLIB Translating and the Computer 2004.
  60. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2004). Improved Statistical Translation Through Editing. EAMT 2004.
  61. Chris Callison-Burch and Miles Osborne (2003). Bootstrapping Parallel Corpora. NAACL workshop Building and Using Parallel Texts 2003. 6 pages.
  62. Chris Callison-Burch and Miles Osborne (2003). Co-training for Statistical Machine Translation. the 6th Annual CLUK Research Colloquium 2003.
  63. Jochen Leidner and Chris Callison-Burch (2003). Evaluating Question Answering Systems Using FAQ Answer Injection. the 6th Annual CLUK Research Colloquium 2003.
  64. Chris Callison-Burch (2001). Upping the Ante for "Best of Breed" Machine Translation Providers. ASLIB Translating and the Computer 2001.
  65. Chris Callison-Burch and Raymond Flournoy (2001). A program for automatically selecting the best output from multiple machine translation engines. MT Summit 2001.
  66. Raymond Flournoy and Chris Callison-Burch (2001). Secondary Benefits of Feedback and User Interaction in Machine Translation Tools. MT Summit Workshop 2001.

Invited Talks and Panels

  1. Penn Association of Senior and Emeritus Faculty (PASEF). Ask an Expert About ChatGPT and Generative AI. November 8, 2023
  2. Mack Institute Conference on Driving Innovation with Generative AI. Panel Discussion About Generative AI. November 6, 2023
  3. Hopkins Business of Health Initiative (HBHI). Driving Innovation with Generative AI. September 15, 2023
  4. The Society Of Composers and Lyricists. Ask an Expert about ChatGPT and Generative AI. August 14, 2023
  5. NLP Highlights Podcast. Generative AI and Copyright. May 25, 2023
  6. Wharton. Invited Speaker at workshop on Large Language Models: Computer Science meets Social Science. May 19, 2023
  7. The U.S. House of Representatives Judiciary Committee, Subcommittee on Courts, Intellectual Property, and the Internet. Testimony at hearing on "Artificial Intelligence and Intellectual Property: Part I – Interoperability of AI and Copyright Law". May 17, 2023
  8. US Copyright Office. Panelist in the Copyright Office AI Listening Sessions - Literary Works. April 19, 2023
  9. Community College of Philadelphia faculty meeting. Ask an Expert about ChatGPT. April 14, 2023
  10. Penn Engineering Podcast. The Growth and Impact of Generative AI. April 14, 2023
  11. SEAS Board of Advisors Meeting. Panel Discussion about ChatGPT. March 31, 2023
  12. Duolingo. Using Large Language Models to Generate Course Materials. March 16, 2023
  13. Penn Center for Learning Analytics. Smart Textbooks and Course Materials Using Large Language Models. March 13, 2023
  14. AI2 Company-wide Meeting. Is ChatGPT A Sputnik Event?. March 10, 2023
  15. Drexel AI Symposium. Ask An Expert About ChatGPT. February 25, 2023
  16. AI2 Aristo Team. Using Large Language Models to Build Explainable Classifiers. February 24, 2023
  17. IARPA Demo Day. Panel Discussion on Large Language Models. February 23, 2023
  18. Penn’s Data Driven Discovery Initiative. Panel Discussion on ChatGPT and Generative AI. February 23, 2023
  19. ISAT/DARPA PARADIGM Workshop. Participant in Performance and Resilience Arising from Defense-Informed Giant Models (PARADIGM) Workshop. February 22, 2023
  20. Penn Medicine Seminar. Ask An Expert About ChatGPT. February 14, 2023
  21. Penn ASSET Seminar. Using Large Language Models to Build Explainable Classifiers. February 8, 2023
  22. TAC Conference. Panel Discussion on What Large Language Models Cannot Do. February 3, 2023
  23. Keynote Address to AACL 2022 Conference. Reasoning about Goals and Making Plans with Large Language Models. November 22, 2022
  24. Salesforce AI Research. Reasoning about Procedures and Goals with wikiHow and Large Language Models. May 16, 2022
  25. University of Lorraine, Nancy. Crowdsourcing for NLP (with Karën Fort and Christoper Cieri). April 14, 2021
  26. Morgan Stanley. Natural Language Understanding with Paraphrases and Word Embeddings. January 19, 2021
  27. Two Sigma. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. August 4, 2020
  28. University of Pennsylvania. Panelist for CURF's Research & Fellowships Week. November 19, 2019
  29. Temple University. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. November 6, 2019
  30. University of Pennsylvania. Panelist for MindCORE's summer program. June 14, 2019
  31. Undergraduate Program in Cognitive Science (UPenn). Representing Word Meaning with Vectors. June 5, 2019
  32. Talk to online MCIT students (UPenn). Natural Language Understanding with Paraphrases and Word Embeddings. April 25, 2019
  33. Vanguard Data Science. Natural Language Understanding with Paraphrases and Word Embeddings. October 26, 2018
  34. Michigan Institute for Data Science. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. November 29, 2018
  35. Google Research (NYC). Learning Translations Without Parallel Texts. August 14, 2018
  36. NSF Convergence Workshop on Crowdsourcing. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. May 18, 2018
  37. National Academies of Sciences workshop on Challenges in Machine Generation of Analytic Products from Multi-source Data. Crowdsourcing for Natural Language Processing. August 10, 2017
  38. NYU. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. February 2, 2017
  39. Columbia University. Large-scale Paraphrasing for Natural Language Generation. September 19, 2016
  40. Cornell University. Large-scale Paraphrasing for Natural Language Generation. September 9, 2016
  41. University of Alabama at Birmingham. Crowdsourcing Translation. April 17, 2015
  42. Drexel University. Crowdsourcing Translation. November 8, 2015
  43. CMU. Crowdsourcing Translation. April 7, 2015
  44. UC Berkeley. Large-scale Paraphrasing for Natural Language Generation. March 12, 2015
  45. Stanford. Crowdsourcing Translation. March 11, 2015
  46. Facebook. Crowdsourcing Translation. March 10, 2015
  47. Coursera. Crowdsourcing Translation. March 9, 2015
  48. Google. Large-scale Paraphrasing for Natural Language Generation. March 9, 2015
  49. MIT. Large-scale Paraphrasing for Natural Language Generation. January 14, 2015
  50. CMU. Large-scale Paraphrasing for Natural Language Generation. November 21, 2014
  51. Microsoft Research. Large-scale Paraphrasing for Natural Language Generation. October 3, 2014
  52. University of Washington, Seattle. Large-scale Paraphrasing for Natural Language Generation. October 2, 2014
  53. The Allen Institute for Artificial Intelligence (AI2). Large-scale Paraphrasing for Natural Language Generation. October 1, 2014
  54. Yahoo! Research Labs. Large-scale Paraphrasing for Natural Language Generation. July 29, 2014
  55. US Army Research Labs DARPA Computer Science Study Group Applied Research Series: Text and Video Analytics Workshop. Language Understanding with the Help of Images. July 16, 2014
  56. International Conference on Natural Language Generation. Large-scale Paraphrasing for Natural Language Generation. June 21, 2014
  57. LREC Workshop on Building and Using Comparable Corpora. Crowdsourcing Translation. May 27, 2014
  58. University of Maryland. Large-scale Paraphrasing for Natural Language Understanding and Generation. April 23, 2014
  59. Institute for Research in Cognitive Science, University of Pennsylvania. Large-scale Paraphrasing for Natural Language Understanding and Generation. March 21, 2014
  60. 37th Annual Penn Linguistics Colloquium, University of Pennsylvania. The Wisdom of Crowdsourcing. March 22, 2013
  61. Johns Hopkins University. Advances to machine translation and language understanding. February 15, 2013
  62. Columbia University IGERT distinguished speaker series. The Promise of Crowdsourcing for NLP and other data sciences. March 29, 2013
  63. UT Austin. Large-scale Paraphrasing for Natural Language Understanding and Generation. December 7, 2012
  64. IBM TJ Watson Research Center. Large-scale Paraphrasing for Natural Language Understanding and Generation. November 9, 2012
  65. NSF-sponsored Workshop on summarizing speaker's attitude and opinion in conversational speech. When annotation with MTurk works. October 20, 2012
  66. Linguistics Data Consortium 20th Anniversary Workshop. The Promise of Crowdsourcing. September 6, 2012
  67. Human Language Technology Center of Excellence. Machine Translation of Arabic Dialects. April 3, 2012.
  68. Human Language Technology Center of Excellence. Machine Translation at the HLTCOE. March 29, 2012
  69. University of Pennsylvania. Advances to machine translation and language understanding. February 28, 2012
  70. Carnegie Mellon University. Advances to machine translation and language understanding. February 21, 2012
  71. Microsoft Research. Crowdsourcing Translation: Professional Quality from Non-Professionals. June 16, 2011
  72. Human Language Technology Center of Excellence. Statistical Machine Translation and Crowdsourcing. March 24, 2011.
  73. CrowdFlower Meetup, Washington DC. Crowdsourcing Translation with Amazon's Mechanical Turk. July 25, 2010
  74. Human Language Technology Center of Excellence. Crowdsourcing Translation with Amazon's Mechanical Turk. November 23, 2010
  75. AAAI Panel on Common Sense Knowledge. Automatic versus Manual Construction of Common Sense Knowledge. November 13, 2010
  76. UMass Amherst. Crowdsourcing Translation with Amazon's Mechanical Turk. October 18, 2010
  77. Brown University. Syntactic Parsing and Machine Translation. September 9, 2010
  78. IARPA. Crowdsourcing Translation. August 5, 2010
  79. UMD Workshop on Crowdsourcing Translation. Crowdsourcing Translation with Amazon's Mechanical Turk. June 10, 2010
  80. BBN. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. March 11, 2010
  81. BBN. Improvements to Urdu-English: SCALE Summer Workshop Results. March 11, 2010
  82. University of Washington. Syntactic translation models help for low-resource, verb final languages. February 25, 2010
  83. Microsoft Research. Syntactic translation models help for low-resource, verb final languages. February 24, 2010
  84. University of Pennsylvania. Syntactic translation models help for low-resource, verb final languages. February 8, 2010
  85. NIST. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. December 18, 2009
  86. University of Maryland. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. December 2, 2009
  87. OHSU, Center for Spoken Language Understanding. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. October 7, 2009
  88. OHSU, Center for Spoken Language Understanding. Improvements to Urdu-English: SCALE Summer Workshop Results. October 5, 2009
  89. University of Pennsylvania. Syntactic Constraints on Paraphrases Extracted from Parallel Corpora. April 13, 2009
  90. University of Maryland. Paraphrasing and Translation. November 28, 2007.
  91. Johns Hopkins University. Improving Statistical Machine Translation With Paraphrases and Generalization. December 5, 2006
  92. Johns Hopkins University. Factored Translation Models. November 28, 2006
  93. University of Pennsylvania. Factored Translation Models. August 23, 2006
  94. Carnegie Mellon University. Statistical Machine Translation Using Semi-Supervised Learning. April 18, 2005.

Grants

Current grants

Grant Title Awarding Body Amount Dates PI Info
UNCOVER: Cross-lingual question answering to identifying information differences between English and Russian Wikipedia articles AFRL $150k 2023-2024 Chris Callison-Burch - PI, Marianna Apidianaki - co-PI
IARPA HIATUS: PAUSIT: Privacy protection and Authorship attribution Using Style-based Interpretable Transfer IARPA $7m (Penn's portion is $3.1m) 2022-2026 Chris Callison-Burch - PI, Marianna Apidianaki - co-PI, Kathleen McKeown (Columbia), Smaranda Muresan (Columbia), Owen Rambow (Stony Brook University), Niranjan Balasubramanian (Stony Brook University), Andy Schwartz (Stony Brook University)
FFW-HTF-RL: Collaborative Research: Up-skilling and Re-skilling Marginalized Rural and Urban Digital Workers: AI-worker collaboration to access creative work NSF $3m (Penn's portion is $375k) 2019-2024 Jeffrey Bigham (CMU) - PI, Chris Callison-Burch (UPenn), Ben Hanrahan (Penn State), Niki Kittur (CMU), Beibei Li (CMU), Amy Ogan (CMU), Amy Pavel (CMU), Saiph Savage (West Virgina University), Julia Ticona (UPenn)
KAIROS: RESIN: Reasoning about Event Schemas for Induction of kNowledge DARPA $12m (Penn's portion is $2m) 2019-2024 Heng Ji (PI - UIUC), Mohit Bansal (UNC), Chris Callison-Burch (UPenn), Shih-Fu Chang (Columbia), Jiawei Han (UIUC), Martha Palmer (Colorado), Dan Roth (UPenn), Carl Vondrick (Columbia)

Past grants

Grant Title Awarding Body Amount Dates PI Info
Semi-supervised Learning of Multimodal Representations DARPA $428k 2019-2022 Chris Callison-Burch (PI-UPenn) and Derry Wijaya (Boston University)
STTR: Personalized Retrieval-based Simplification NSF $225k 2019-2021 Eleni Miltsakaki (Choosito.com) - PI, Chris Callison-Burch
Alexa Prize TaskBot Challenge Amazon $250k 2021-2022 Chris Callison-Burch (PI), Mark Yatskar (UPenn)
Computing Innovation Postdoctoral Fellow Award NSF/CRA ~$140k 2021-2023 Lara Martin (postdoc), Chris Callison-Burch (faculty advisor)
REU Supplement NSF $16k 2019-2020 co-PI Chris Cieri
SPUR WOMEN: Support and Promote Undergraduate Research for Women Google $15k from Google 2019-2020 PI with Rita Powell
LWLL: FLASH: Fast Learning via Auxiliary signals, Structured knowledge, and Human expertise DARPA $3.3m 2019-2022 Dan Roth (PI - UPenn), Irfan Essa (Georgia Tech), Chris Callison-Burch (UPenn), Zsolt Kira (Georgia Tech), Le Song (Georgia Tech), Mayur Naik (UPenn), Osbert Bastani (UPenn)
BETTER: Task and User-Aware Representation Learning for Fine-Grained Cross-Lingual Information Retrieval IARPA $6m 2019-2023 Ellie Pavlick (PI - Brown), Carsten Eickhoff (Brown), Chris Callison-Burch (UPenn), Wei Xu (OSU), Alan Ritter (OSU)
SPUR WOMEN: Support and Promote Undergraduate Research for Women Google $25k from Google, plus $25k matched by SEAS 2018-2019 PI with Rita Powell
CI-NEW: NIEUW: Novel Incentives and Workflows in Linguistics Data Collection NSF $1.2m 2017-2022 co-PI with Christopher Cieri and Mark Liberman
DEFT Extension DARPA $116,000 2017-2017 PI with Ben Van Durme
CI-P: Planning for Scalable Language Resource Creation through Novel Incentives and Crowdsourcing NSF $100,000 2016-2017 co-PI with Christopher Cieri and Mark Liberman
Learning translations from monolingual texts (LORELEI) DARPA $478,000 2015-2019 PI at Penn
SIREN-IL: Specialized Intra/Interlingual Resources for Emergent News - Incident Language DARPA $3,031,412 2015-2017 co-PI with Stephanie Strassel
Natural Logic Solver for Aristo Allen Institute for Artificial Intelligence (AI2) $95,000 2015-2016 PI
EAGER: Simplification as Machine Translation NSF $100,000 2014-2015 PI
Sloan Research Fellowship Alfred P. Sloan Foundation $50,000 2014
Large-scale Paraphrasing for Natural Language Understanding (DEFT) DARPA $1,600,000 2012-2017 PI with Ben Van Durme
EAGER: Combining natural language inference and data-driven paraphrasing NSF $100,000 2012-2013 co-PI with Ben Van Durme
Crowdsourcing Translation (Computer Science Study Panel phase 3) DARPA $500,000 2012-2015 PI
Improved Arabic dialect translation through Crowdsourcing DARPA $176,000 2012-2013 PI
Acquisition and use of paraphrases in a knowledge-rich setting Vulcan $260,000 2011-2013 co-PI with Ben Van Durme
Google Faculty Research Award (Translate the World: A Unified Framework for Crowdsourcing Translation) Google $150,000 2011 co-PI with Philip Resnik and Ben Bederson
RI:Medium: Semi-supervised Discriminative Training of Sequence Transduction Model NSF $800,000 2011-2015 co-PI with Sanjeev Khudanpur, Brian Roark, Damianos Karakos, Richard Sproat
Translation of informal texts via Mechanical Turk BBN Technologies $144,000 2010-2011 PI
BABEL: Bayesian Architecture Begetting Every Language (Computer Science Study Panel phase 2) DARPA $500,000 2010-2012 PI
EuroMatrixPlus: Bringing Machine Translation for European Languages to the User European Union Framework 7 Programme €4,950,000 2009-2012 PI at JHU (JHU Amount: €516,000)
Global Autonomous Language Exploitation (GALE) project, Periods 3 and 4 DARPA $575,000 2009-2011 co-PI with Sanjeev Khudanpur and Damianos Karakos
Computer Science Study Panel DARPA $93,000 2008-2009 PI
RI:SMALL: Multi-Level Modeling of Language and Translation NSF $400,000 2007-2012 co-PI with David Yarowsky
SMART:Scotland Technology Innovation Grant British Government £45,000 2002-2005

Teaching

Teaching Reviews

You can read my full teaching reviews here. Below are the summary statistics.

Penn teaching reviews are on a 0-4 quality scale: 0=Poor 1=Fair 2=Good 3=Very Good 4=Excellent
Term Course Title (Number) Students Enrolled Course Quality Instructor Quality
Fall 2023 Artificial Intelligence (CIS 4210/5210 - on campus) 384
Fall 2023 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 196
Fall 2023 Natural Language Processing (CIS 5300 - Penn Engineering Online) 159
Fall 2023 Large Language Models and Programming Languages (CIS 8000) 20
Summer 2023 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 107 3.4 3.5
Summer 2023 Natural Language Processing (CIS 5300 - Penn Engineering Online) 45 3.3 3.5
Fall 2022 Artificial Intelligence (CIS 4210/5210 - on campus) 363 3.3 3.5
Fall 2022 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 94 3.4 3.6
Fall 2022 Research Practicum (CIS 8000) 16 3.5 3.6
Summer 2022 Artificial Intelligence (CIS 521 - Penn Engineering Online) 70 3.5 3.7
Spring 2022 Interactive Fiction and Text Generation (CIS 700-001) 53 3.3 3.5
Fall 2021 Artificial Intelligence (CIS 521 - MCIT Online) 234 3.2 3.4
Fall 2021 Artificial Intelligence (CIS 421/521 - on campus - section 1) 180 3.2 3.4
Fall 2021 Artificial Intelligence (CIS 421/521 - on campus - section 2) 138 3.2 3.5
Fall 2021 Artificial Intelligence (CIS 421/521 - online only section for foreign graduate students unable to return to campus due to the pandemic) 11 2.5 2.3
Summer 2021 Artificial Intelligence (CIS 521 - MCIT Online) 49 3.0 3.6
Spring 2021 Crowdsourcing and Human Computation (NETS 213) 146 3.0 3.3
Fall 2020 Artificial Intelligence (CIS 421/521) 197 3.1 3.3
Spring 2020 Computational Linguistics (CIS 530) 125 3.3 3.3
Spring 2020 Interactive Fiction and Text Generation (CIS 700-008) 23 3.1 3.3
Fall 2019 Artificial Intelligence (CIS 421/521) 148 3.1 3.3
Summer 2019 Artificial Intelligence (CIS 421/521) 36 2.9 3.0
Spring 2019 Computational Linguistics (CIS 530) 75 2.8 3.0
Spring 2019 Crowdsourcing and Human Computation (NETS 213) 59 2.5 2.7
Fall 2018 Artificial Intelligence (CIS 421/521) 101 2.5 2.5
Spring 2018 Computational Linguistics (CIS 530) 64 2.8 2.7
Fall 2017 Data Structures and Algorithms (CIS 121) 216 2.1 1.7
Fall 2016 Data Structures and Algorithms (CIS 121) 219 2.5 2.2
Spring 2016 Crowdsourcing and Human Computation (NETS 213) 113 2.4 2.8
Fall 2015 Data Structures and Algorithms (CIS 121) 174 2.2 2.2
Spring 2015 Machine Translation (CIS 526) 51 2.9 3.2
Fall 2014 Crowdsourcing and Human Computation (NETS 213) 48 3.2 3.6
Spring 2014 Machine Translation (CIS 526) 25 3.3 3.5
Fall 2013 Crowdsourcing and Human Computation (CIS 399) 26 3.1 3.5

Awards

Research Awards from Companies (given as gifts to the university for my research group)

Graduate Student Supervision

Current PhD Students

  1. Andrew Zhu. Expected graduation date: Summer 2027
  2. Artemis Panagopoulou. Co-advised by Mark Yatskar. Expected graduation date: Summer 2026
  3. Liam Dugan. Expected graduation date: Summer 2026
  4. Ajay Patel. Expected graduation date: Summer 2025
  5. Alyssa Hwang. Co-advised by Andrew Head. Expected graduation date: Summer 2025
  6. Bryan Li. Expected graduation date: Summer 2025
  7. Samar Haider. Co-advised by Duncan Watts. Expected graduation date: Summer 2025
  8. Yue Yang. Co-advised by Mark Yatskar. Expected graduation date: Summer 2025
  9. Veronica Qing Lyu. Co-advised by Marianna Apidianaki. Expected graduation date: Summer 2024
  10. Harry Zhang. Expected graduation date: Summer 2024
  11. Aditya Kashyap. Co-advised by Mary Regina Boland. Expected graduation date: Summer 2024

PhDs Graduated

  1. Daphne Ippolito, University of Pennsylvania (advisors: Chris Callison-Burch and Doug Eck), "Understanding the Limitations of Using Large Language Models for Text Generation", September 2022. Current position: Assistant Professor at Carnegie Mellon University.
  2. Reno Kriz, University of Pennsylvania (advisors: Chris Callison-Burch and Marianna Apidianaki), "Towards a Practically Useful Text Simplification System", June 2021. Current position: Research Scientist at Human Language Technology Center of Excellence.
  3. Anne Cocos, University of Pennsylvania (advisors: Chris Callison-Burch and Marianna Apidianaki), "Paraphrase-based Models of Lexical Semantics", May 2019. Current position: Senior Research Scientist at Netflix.
  4. Courtney Napoles, Johns Hopkins University (advisors: Chris Callison-Burch and Benjamin Van Durme), "Monolingual Sentence Rewriting as Machine Translation: Generation and Evaluation", June 2018. Current position: Engineering Director, NLP/ML/AI at Grammarly.
  5. Juri Ganitkevitch, Johns Hopkins University (advisor: Chris Callison-Burch), "Large-Scale Paraphrase Extraction and Applications", February 2018. Current position: Chief Scientist & Co-Founder at Espresso AI.
  6. Ellie Pavlick, University of Pennsylvania (advisor: Chris Callison-Burch), "Compositional Lexical Semantics in Natural Language Inference", July 2017. Current position: Assistant Professor at Brown University.
  7. Ann Irvine, Johns Hopkins University (advisor: Chris Callison-Burch), "Using Comparable Corpora to Augment Low Resource Statistical Machine Translation Models", July 2014. Current position: Chief Data Scientist & VP of Product Management at Resilience.
  8. Xuchen Yao, Johns Hopkins University (advisors: Benjamin Van Durme and Chris Callison-Burch), "Feature-Driven Question Answering with Natural Language Alignment", July 2014. Current position: Cofounder/CEO at Seasalt.ai and Vobil.com.
  9. Omar Zaidan, Johns Hopkins University (advisor: Chris Callison-Burch), "Crowdsourcing Annotation for Machine Learning in Natural Language Processing Tasks", April 2012. Current position: Senior Applied Scientist at Amazon.

Postdocs

  1. Lara Martin, PhD from Georgia Tech, postdoc at University of Pennsylvania from 2021 through August 2023. Current position: Assistant Professor at University of Maryland, Baltimore County.
  2. Mohammad Sadegh Rasooli, PhD from Columbia University, postdoc at University of Pennsylvania from January 2020 through July 2021. Current position: Senior Applied Scientist at Microsoft.
  3. Derry Wijaya, PhD from Carnegie Mellon University, postdoc at University of Pennsylvania from 2016 through August 2018. Current position: Assistant Professor at Boston University.
  4. Anietie Andy, PhD from Howard University, postdoc at University of Pennsylvania from January 2017 through June 2018. Current position: Assistant Professor at Howard University.
  5. Wei Xu, PhD from New York University, postdoc at University of Pennsylvania from February 2014 through August 2016. Current position: Assistant Professor at Georgia Tech.
  6. Matt Post, PhD from University of Rochester, postdoc at Johns Hopkins University from September 2010 through June 2012. Current position: Researcher at Microsoft Translator.
  7. Alex Klementiev, PhD from UIUC, postdoc at Johns Hopkins University from November 2009 through July 2011. Current position: Principal Applied Scientist at Amazon.

Master's Students

  1. Yifei Li, University of Pennsylvania (advisors: Chris Callison-Burch and Mark Yatskar), "Improving Text-to-image Diffusion Generation Via Large Language Models", May 2023.
  2. River Yijiang Dong, University of Pennsylvania (advisors: Chris Callison-Burch and Lara Martin), "COTTAGE: Coherent Text Adventure Games Generation", May 2023.
  3. Hainiu Xu, University of Pennsylvania (advisors: Chris Callison-Burch and Harry Li Zhang), "Fine-Grained And Coarse-Grained Causal Reasoning In Procedural Texts", May 2023.
  4. Anshul Wadhawan, University of Pennsylvania (advisors: Chris Callison-Burch and Liam Dugan), "Simultaneous Speech To Speech Translation", May 2023.
  5. Anna Orosz, University of Pennsylvania (advisors: Chris Callison-Burch and Lara Martin), "Generating Text-based Adventure Games", December 2021.
  6. Sri Sanjeevini Devi Ganni, University of Pennsylvania (advisors: Chris Callison-Burch and Lara Martin), "Narratology and Fan Fiction", May 2021.
  7. Jacob Beckerman, University of Pennsylvania (advisor: Chris Callison-Burch), "Graph Algorithms and Visualization of Complex Legal Contracts", December 2020.
  8. Liam Dugan, University of Pennsylvania (advisor: Chris Callison-Burch), "Learning Formality from Japanese-English Parallel Corpora", May 2020.
  9. Yonah Mann, University of Pennsylvania (advisor: Chris Callison-Burch), "A Data Set for Training QA Systems to Answer Questions about Novels", May 2020.
  10. Maria Kustikova, University of Pennsylvania (advisors: Chris Callison-Burch and Kostas Daniilidis), "Clustering Paraphrases by Word Sense Using Textual and Visual Information", May 2019.
  11. Devanshu Jain, University of Pennsylvania (advisor: Chris Callison-Burch), "Machine Transliteration", May 2018.
  12. Aditya Kashyap, University of Pennsylvania (advisor: Chris Callison-Burch), "Generalizable Identity Classifiers from Self-Reporting Statements on Reddit", May 2018.
  13. Brendan Daniel Callahan, University of Pennsylvania (advisor: Chris Callison-Burch), "Image-based Bilingual Lexicon Induction for Low Resource Languages", May 2017.
  14. Sneha Rajana, University of Pennsylvania (advisor: Chris Callison-Burch), "Learning Antonyms with Paraphrases and a Morphology-Aware Neural Network", May 2017.
  15. Mingkun Gao, University of Pennsylvania (advisor: Chris Callison-Burch), "Crowdsourcing Machine Translation", May 2015.

Academic Service

Service to professional associations:

University Service:

Press

  1. A peek into the future of visual data interpretation – Penn Today. November 16, 2023
  2. As OpenAI’s multimodal API launches broadly, research shows it’s still flawed – TechCrunch. November 6, 2023
  3. Why actors are fighting for AI protections – The Hill. October 23, 2023
  4. Why ChatGPT's New Ability to Speak Could Change Everything – LifeWire – Tech for Humans. September 27, 2023
  5. What Amazon's up to $4B commitment to Anthropic could mean for AI space – Yahoo Finance. September 25, 2023
  6. Microsoft Sees Low Risk for Customers in AI Copyright Lawsuits – Bloomberg Law. September 11, 2023
  7. What Teachers and Parents Need to Know about ChatGPT – Bloomberg Businessweek. September 6, 2023
  8. Alien Minds, Immaculate Bullshit, Outstanding Questions – The Pennsylvania Gazette. April 26, 2023
  9. Copyright Law and Generative AI: What a mess – American Bar Association Journal. August 30, 2023
  10. What does Congress need to do amid AI boom? – Fox News. May 20, 2023
  11. Hallucinating ChatGPT finds a role playing Dungeons & Dragons – The Register. August 19, 2023
  12. Bot or not? How to tell when you’re reading something written by AI – CNN. July 11, 2023
  13. NewsChannel 12 Investigates: Artificial Intelligence Part 3 – Channel 12 News - ABC affiliate in North Carolina. May 19, 2023
  14. Real or Fake Text? We can learn to spot the difference – Penn Today. March 10, 2023
  15. A Bot Isn’t Going to Take Your Place, But AI Will Make Your Job Harder – Corporate Compliance Insights. March 8, 2023
  16. How can humans detect AI writing? These Penn researchers have some tips – Technical.ly. March 6, 2023
  17. Newly Unemployed, and Labeling Photos for Pennies: People who've lost jobs and are stuck indoors are turning to crowd work—filling out online surveys and transcribing audio for less than the minimum wage. – Wired Magazine. April 23, 2020
  18. America’s Gig-Based Economy Gets Zero – Full Frontal with Samantha Bee. March 11, 2020
  19. I Found Work on an Amazon Website. I Made 97 Cents an Hour. – New York Times. November 15, 2019