Skip to main content

Chris Callison-Burch: CV

(Last updated May 13, 2025)

Education

PhD in Informatics
University of Edinburgh, Edinburgh, UK
Thesis: Paraphrasing and Translation
Advisors: Miles Osborne and Mark Steedman 		February 2008
M.S. with Distinction in Computer Science
University of Edinburgh, Edinburgh, UK
Thesis: Co-Training for Statistical Machine Translation
Advisor: Miles Osborne 		October 2002
B.S. with Honors in Symbolic Systems
Stanford University, Palo Alto, CA
Thesis: A Computer Model of a Grammar for English Questions
Advisor: Ivan Sag 		June 2000

Professional Appointments

Professor
University of Pennsylvania, Philadelphia, PA		 July 2024-present
Part-time Visiting Research Scientist
Allen Institute for Artificial Intelligence (Ai2), Seattle, WA		 September 2023-August 2024
Visiting Research Scientist
Allen Institute for Artificial Intelligence (Ai2), Seattle, WA		 January 2023-August 2023
Associate Professor
University of Pennsylvania, Philadelphia, PA		 June 2017-July 2024
Part-time Visiting Researcher
Google, New York, NY		 December 2018-December 2020
Aravind K. Joshi Term Assistant Professor
University of Pennsylvania, Philadelphia, PA		 September 2013-June 2017
Associate Research Professor
Johns Hopkins University, Baltimore, MD		 June 2010-August 2013
Assistant Research Professor
Johns Hopkins University, Baltimore, MD		 June 2007-June 2010

Publications

My publications have been cited more than 31,000 times. I have an h-index of 71.

Conference papers

  1. Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, Chris Callison-Burch (2025). StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples. NAACL 2025. 24 pages.
  2. Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Borchardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, Aniruddha Kembhavi (2025). Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models. CVPR 2025. 114 citations. 30 pages.
  3. Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Hancheng Min, Chris Callison-Burch, René Vidal (2025). Concept Lancet: Image Editing with Compositional Representation Transplant. CVPR 2025. 19 pages.
  4. Artemis Panagopoulou, Honglu Zhou, Silvio Savarese, Caiming Xiong, Chris Callison-Burch, Mark Yatskar, Juan Carlos Niebles (2025). ViUniT: Visual Unit Tests for More Robust Visual Programming. CVPR 2025. 41 pages.
  5. Veronica Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar, Niket Tandon, Marianna Apidianaki, Mrinmaya Sachan, Chris Callison-Burch (2025). Calibrating Large Language Models with Sample Consistency. AAAI 2025.
  6. Jenny S Wang, Samar Haider, Amir Tohidi, Anushkaa Gupta, Yuxuan Zhang, Chris Callison-Burch, David Rothschild, Duncan J Watts (2025). Media Bias Detector: Designing and Implementing a Tool for Real-Time Selection and Framing Bias Analysis in News Coverage. CHI 2025. 17 pages.
  7. Zachary Horvitz, Ajay Patel, Kanishk Singh, Chris Callison-Burch, Kathleen McKeown, Zhou Yu (2024). TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings. EMNLP 2024. 14 pages.
  8. Runsheng (Anson) Huang, Liam Dugan, Yue Yang, Chris Callison-Burch (2024). MiRAGeNews: Multimodal Realistic AI-Generated News Detection. EMNLP 2024. 12 pages.
  9. Yue Yang, Mona Gandhi, Yufei Wang, Yifan Wu, Michael S. Yao, Chris Callison-Burch, James C. Gee, Mark Yatskar (2024). A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis. NeurIPS 2024. 23 pages.
  10. Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Darshan Thaker, Aditya Chattopadhyay, Chris Callison-Burch, René Vidal (2024). PaCE: Parsimonious Concept Engineering for Large Language Models. NeurIPS 2024.
  11. Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, Peter Clark (2024). CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization. COLM 2024. 23 pages.
  12. Ajay Patel, Colin Raffel, Chris Callison-Burch (2024). DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows. ACL 2024. 29 pages.
  13. Andrew Zhu, Alyssa Hwang, Liam Dugan, Chris Callison-Burch (2024). FanOutQA: Multi-Hop, Multi-Document Question Answering for Large Language Models. ACL 2024. 18 pages.
  14. Liam Dugan, Alyssa Hwang, Filip Trhlik, Josh Magnus Ludan, Andrew Zhu, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch (2024). RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors. ACL 2024. 26 pages.
  15. Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, Chris Callison-Burch, Mark Yatskar, Aniruddha Kembhavi, Christopher Clark (2024). Holodeck: Language Guided Generation of 3D Embodied AI Environments. CVPR 2024. 118 citations. 20 pages.
  16. Yiming Huang, Weilin Wan, Yue Yang, Chris Callison-Burch, Mark Yatskar, Lingjie Liu (2024). CoMo: Controllable Motion Generation through Language Guided Pose Code Editing. ECCV 2024. 27 pages.
  17. Li Zhang, Peter Jansen, Tianyi Zhang, Peter Clark, Chris Callison-Burch, Niket Tandon (2024). PDDLego: Iterative Planning in Textual Environments. *SEM 2024. 10 pages.
  18. Zachary Horvitz, Ajay Patel, Chris Callison-Burch, Zhou Yu, Kathleen McKeown (2024). ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer. AAAI 2024. 17 pages.
  19. Ajay Patel, Delip Rao, Ansh Kothary, Kathleen McKeown, Chris Callison-Burch (2023). Learning Interpretable Style Embeddings via Prompting LLMs. EMNLP Findings 2023. 21 pages.
  20. Bryan Li, Chris Callison-Burch (2023). PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale. EMNLP Findings 2023. 21 pages.
  21. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, Chris Callison-Burch (2023). Faithful Chain-of-Thought Reasoning. AACL-IJCNLP 2023. Area Chair Award (Interpretability and Analysis of Models for NLP). 266 citations. 25 pages.
  22. Alyssa Hwang, Natasha Oza, Andrew Head, Chris Callison-Burch (2023). Rewriting the Script: Adapting Text Instructions for Voice Interaction. DIS 2023. 16 pages.
  23. Liam Dugan, Anshul Wadhawan, Kyle Spence, Chris Callison-Burch, Morgan McGuire, Victor Zordan (2023). Learning When to Speak: Latency and Quality Trade-offs for Simultaneous Speech-to-Speech Translation with Offline Models. Interspeech 2023. 2 pages.
  24. Bryan Li, Chris Callison-Burch (2023). This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models. NAACL 2023. 9 pages.
  25. Qing Lyu, Marianna Apidianaki, Chris Callison-Burch (2023). Representation of Lexical Stylistic Features in Language Models’ Embedding Space. StarSEM 2023. 18 pages.
  26. Andrew Zhu, Lara J. Martin, Andrew Head, Chris Callison-Burch (2023). CALYPSO: LLMs as Dungeon Masters' Assistants. AAID 2023. 11 pages.
  27. Pei Zhou, Andrew Zhu, Jennifer Hu, Jay Pujara, Xiang Ren, Chris Callison-Burch, Yejin Choi, Prithviraj Ammanabrolu (2023). I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons. ACL 2023. 18 pages.
  28. Andrew Zhu, Karmanya Aggarwal, Alexander Feng, Lara J. Martin, and Chris Callison-Burch (2023). FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information. ACL 2023. 21 pages.
  29. Zoey Sha Li, Ruining Zhao, Manling Li, Heng Ji, Chris Callison-Burch, Jiawei Han (2023). Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification. ACL 2023. 17 pages.
  30. Josh Magnus Ludan, Yixuan Meng, Tai Nguyen, Saurabh Shah, Qing Lyu, Marianna Apidianaki and Chris Callison-Burch (2023). Explanation-based Finetuning Makes Models More Robust to Spurious Cues. ACL 2023. 17 pages.
  31. Yijiang River Dong, Lara J. Martin, Chris Callison-Burch (2023). CORRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding. ACL Findings 2023. 15 pages.
  32. Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, Mark Yatskar (2023). Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. CVPR 2023. 243 citations. 18 pages.
  33. Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, Chris Callison-Burch (2023). Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text. AAAI 2023. 13 pages.
  34. Shriyash Upadhyay, Etan Ginsberg, Chris Callison-Burch (2023). Learn With Martian: A Tool For Creating Assignments That Can Write And Re-Write Themselves. EACL Demos 2023. 10 pages.
  35. Li Zhang, Hainiu Xu, Yue Yang, Shuyan Zhou, Weiqiu You, Manni Arora, Chris Callison-Burch (2023). Causal Reasoning About Entities and Events in Procedural Texts. Findings of EACL 2023. 14 pages.
  36. Ajay Patel, Bryan Li, Mohammad Sadegh Rasooli, Noah Constant, Colin Raffel, Chris Callison-Burch (2023). Bidirectional Language Models Are Also Few-shot Learners. ICLR 2023. 25 pages.
  37. Chris Callison-Burch, Gaurav Singh Tomar, Lara Martin, Daphne Ippolito, Suma Bailis and David Reitter (2022). Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence. EMNLP 2022. 15 pages.
  38. Jeffrey Young-Min Cho, Harry Li Zhang, Chris Callison-Burch (2022). Unsupervised Entity Linking with Guided Summarization and Multiple-Choice Selection. EMNLP 2022. 9 pages.
  39. Yue Yang, Artemis Panagopoulou, Marianna Apidianaki, Mark Yatskar and Chris Callison-Burch (2022). Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction. Findings of EMNLP 2022. 17 pages.
  40. Daphne Ippolito, Liam Dugan, Emily Reif, Ann Yuan, Andy Coenen, Chris Callison-Burch (2022). The Case for a Single Model that can Both Generate Continuations and Fill-in-the-Blank. NAACL 2022. 12 pages.
  41. Qing Lyu, Hua Zheng, Daoxin Li, Li Zhang, Marianna Apidianaki, Chris Callison-Burch (2022). Is “My Favorite New Movie” My Favorite Movie? Probing the Understanding of Recursive Noun Phrases. NAACL 2022. 12 pages.
  42. Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, Jason Wei (2022). A Recipe For Arbitrary Text Style Transfer with Large Language Models. ACL 2022. 12 pages.
  43. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini (2022). Deduplicating Training Data Makes Language Models Better. ACL 2022. 666 citations. 22 pages.
  44. Shuyan Zhou, Li Zhang, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison-Burch, Graham Neubig (2022). Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data. ACL 2022. 14 pages.
  45. Liam Dugan, Eleni Miltsakaki, Etan Ginsberg, Shriyash Upadhyay, Hannah Gonzalez, Dahyeon Choi, Chuning Yuan, Chris Callison-Burch (2022). A Feasibility Study of Answer-Agnostic Question Generation for Education. ACL 2022. 8 pages.
  46. Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, and Sebastian Gehrmann (2021). SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets. NeurIPS 2021. 24 pages.
  47. Qing Lyu and Li Zhang and Chris Callison-Burch (2021). Goal-Oriented Script Construction. INGL 2021. 17 pages.
  48. Nikzad Khani, Isidora Chara Tourni, Mohammad Sadegh Rasooli, Chris Callison-Burch and Derry Tanti Wijaya (2021). Cultural and Geographical Influences on Image Translatability of Words across Languages. NAACL 2021. 12 pages.
  49. Qing Lyu, Li Zhang, Chris Callison-Burch (2020). Reasoning about Goals, Steps, and Temporal Ordering with WikiHow. EMNLP 2020. 11 pages.
  50. Li Zhang, Qing Lyu, Chris Callison-Burch (2020). Intent Detection with WikiHow. AACL-IJCNLP 2020. 6 pages.
  51. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch and Douglas Eck (2020). Automatic Detection of Generated Text is Easiest when Humans are Fooled. ACL 2020. 411 citations. 15 pages.
  52. Daphne Ippolito, David Grangier, Douglas Eck and Chris Callison-Burch (2020). Toward Better Storylines with Sentence-Level Language Models. ACL 2020. Short papers. 7 pages.
  53. Daphne Ippolito, Reno Kriz, João Sedoc, Maria Kustikova and Chris Callison-Burch (2019). Comparison of Diverse Decoding Methods from Conditional Language Models. ACL 2019. 11 pages.
  54. Reno Kriz, João Sedoc, Marianna Apidianaki, Carolina Zheng, Gaurav Kumar, Eleni Miltsakaki, and Chris Callison-Burch (2019). Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification. NAACL 2019. 10 pages.
  55. Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch and Dan Roth (2019). Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims. NAACL 2019. 16 pages.
  56. Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Benjamin V. Hanrahan, Jeffrey P. Bigham and Chris Callison-Burch (2019). Worker Demographics and Earnings on Amazon Mechanical Turk: An Exploratory Analysis. CHI Late Breaking Work 2019. 5 pages.
  57. Anne Cocos, Skyler Wharton, Ellie Pavlick, Marianna Apidianaki and Chris Callison-Burch (2018). Learning Scalar Adjective Intensity from Paraphrases. EMNLP 2018. 11 pages.
  58. John Hewitt, Daphne Ippolito, Brendan Callahan, Reno Kriz, Derry Wijaya and Chris Callison-Burch (2018). Learning Translations via Images with a Massively Multilingual Image Dataset. ACL 2018. 12 pages.
  59. Reno Kriz, Eleni Miltsakaki, Marianna Apidianaki and Chris Callison-Burch (2018). Simplification Using Paraphrases and Context-based Lexical Substitution. NAACL 2018. 12 pages.
  60. Marianna Apidianaki, Guillaume Wisniewski, Anne Cocos and Chris Callison-Burch (2018). Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation. NAACL 2018. Short papers. 6 pages.
  61. Anne Cocos, Marianna Apidianaki and Chris Callison-Burch (2018). Comparing Constraints for Taxonomic Organization. NAACL 2018. 12 pages.
  62. Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, Jeffrey P. Bigham (2018). A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk. CHI 2018. Honorable Mention Award. 762 citations. 12 pages.
  63. Derry Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki and Chris Callison-Burch (2017). Learning Translations via Matrix Completion. EMNLP 2017. 12 pages.
  64. Anne Cocos, Marianna Apidianaki and Chris Callison-Burch (2017). Mapping the Paraphrase Database to WordNet. STARSEM 2017.
  65. Sneha Rajana, Chris Callison-Burch, Marianna Apidianaki and Vered Shwartz (2017). Learning Antonyms with Paraphrases and a Morphology-aware Neural Network. STARSEM 2017. 10 pages.
  66. Ann Cocos and Chris Callison-Burch (2017). The Language of Place: Semantic Value from Geospatial Context. EACL 2017. Short papers. 5 pages.
  67. Ellie Pavlick, Heng Ji, Xiaoman Pan and Chris Callison-Burch (2016). The Gun Violence Database: A new task and data set for NLP. EMNLP 2016. Short papers. 6 pages.
  68. Ellie Pavlick and Chris Callison-Burch (2016). Tense Manages to Predict Implicative Behavior in Verbs. EMNLP 2016. Short papers. 5 pages.
  69. Ellie Pavlick and Chris Callison-Burch (2016). So-Called Non-Subsective Adjectives. STARSEM 2016. Best Paper Award. 6 pages.
  70. Ellie Pavlick and Chris Callison-Burch (2016). Most babies are little and most problems are huge: Compositional Entailment in Adjective-Nouns. ACL 2016. 11 pages.
  71. Ellie Pavlick and Chris Callison-Burch (2016). Simple PPDB: A Paraphrase Database for Simplification. ACL 2016. Short papers. 108 citations. 6 pages.
  72. Anne Cocos and Chris Callison-Burch (2016). Clustering Paraphrases by Word Sense. NAACL 2016. 10 pages.
  73. Courtney Napoles, Chris Callison-Burch, and Matt Post (2016). Sentential Paraphrasing as Black-Box Machine Translation. NAACL 2016. Short papers. 5 pages.
  74. Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Benjamin Van Durme, and Chris Callison-Burch (2015). Adding Semantics to Data-Driven Paraphrasing. ACL 2015. 10 pages.
  75. Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevich, Ben Van Durme, Chris Callison-Burch (2015). PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. ACL 2015. Short papers. 400 citations. 6 pages.
  76. Ellie Pavlick, Juri Ganitkevich, Tsz Ping Chan, Xuchen Yao, Ben Van Durme, Chris Callison-Burch (2015). Domain-Specific Paraphrase Extraction. ACL 2015. Short papers. 6 pages.
  77. Ellie Pavlick, Travis Wolfe, Pushpendre Rastogi, Chris Callison-Burch, Mark Drezde, Ben Van Durme (2015). FrameNet+: Fast Paraphrastic Tripling of FrameNet. ACL 2015. Short papers. 6 pages.
  78. Mingkun Gao, Wei Xu, and Chris Callison-Burch (2015). Cost Optimization for Crowdsourcing Translation. NAACL 2015. 9 pages.
  79. Heba Elfardy, Mona Diab and Chris Callison-Burch (2015). Ideological Perspective Detection Using Semantic Features. STARSEM 2015. 10 pages.
  80. Ann Irvine and Chris Callison-Burch (2014). Hallucinating Phrase Translations for Low Resource MT. CoNLL 2014. 11 pages.
  81. Rui Yan, Mingkun Gao, Ellie Pavlick, and Chris Callison-Burch (2014). Are Two Heads are Better than One? Crowdsourced Translation via a Two-Step Collaboration between Translators and Editors. ACL 2014. 11 pages.
  82. Jonathan Weese, Juri Ganitkevitch, and Chris Callison-Burch (2014). PARADIGM: Paraphrase Diagnostics through Grammar Matching. EACL 2014. 10 pages.
  83. Ellie Pavlick, Rui Yan, and Chris Callison-Burch (2014). Crowdsourcing for Grammatical Error Correction. CSCW Poster 2014. 4 pages.
  84. Juri Ganitkevitch and Chris Callison-Burch (2014). The Multilingual Paraphrase Database. LREC 2014. 8 pages.
  85. Ann Irvine, Joshua Langfus, and Chris Callison-Burch (2014). The American Local News Corpus. LREC 2014. 4 pages.
  86. Ryan Cotterell and Chris Callison-Burch (2014). A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic. LREC 2014. 105 citations. 5 pages.
  87. Xuchen Yao, Ben Van Durme, Chris Callison-Burch and Peter Clark (2013). Semi-Markov Phrase-based Monolingual Alignment. EMNLP 2013. 11 pages.
  88. Xuchen Yao, Peter Clark, Ben Van Durme and Chris Callison-Burch (2013). A Lightweight and High Performance Monolingual Word Aligner. ACL 2013. Short papers. 6 pages.
  89. Travis Wolfe, Benjamin Van Durme, Mark Dredze, Nicholas Andrews, Charley Beller, Chris Callison-Burch, Jay DeYoung, Justin Snyder, Jonathan Weese, Tan Xu and Xuchen Yao (2013). PARMA: A Predicate Argument Aligner. ACL 2013. Short papers. 6 pages.
  90. Jason Smith, Herve Saint-Amand, Magdalena Plamada, Philipp Koehn, Chris Callison-Burch and Adam Lopez (2013). Dirt Cheap Web-Scale Parallel Text from the Common Crawl. ACL 2013. 168 citations. 10 pages.
  91. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch (2013). PPDB: The Paraphrase Database. NAACL 2013. Short papers. 988 citations. 7 pages.
  92. Ann Irvine and Chris Callison-Burch (2013). Supervised Bilingual Lexicon Induction with Multiple Monolingual Signals. NAACL 2013. Short papers. 6 pages.
  93. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch and Peter Clark (2013). Answer Extraction as Sequence Tagging with Tree Edit Distance. NAACL 2013. 252 citations. 10 pages.
  94. Xuchen Yao, Benjamin Van Durme and Chris Callison-Burch (2012). Expectations of Word Sense in Parallel Corpora. NAACL 2012. Short papers. 5 pages.
  95. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch (2012). Monolingual Distributional Similarity for Text-to-Text Generation. STARSEM 2012. 9 pages.
  96. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan and Chris Callison-Burch (2012). Machine Translation of Arabic Dialects. NAACL 2012. 202 citations. 11 pages.
  97. Alex Klementiev, Ann Irvine, Chris Callison-Burch, and David Yarowsky (2012). Toward Statistical Machine Translation without Parallel Corpora. EACL 2012. 11 pages.
  98. Juri Ganitkevitch, Chris Callison-Burch, Courtney Napoles, and Benjamin Van Durme (2011). Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation. EMNLP 2011. 12 pages.
  99. Omar Zaidan and Chris Callison-Burch (2011). The Arabic Online Commentary Dataset: An Annotated Dataset of Informal Arabic with High Dialectal Content. ACL 2011. Short papers. 206 citations. 5 pages.
  100. Omar Zaidan and Chris Callison-Burch (2011). Crowdsourcing Translation: Professional Quality from Non-Professionals. ACL 2011. 535 citations. 10 pages.
  101. Lane Schwartz, Chris Callison-Burch, William Schuler and Stephen Wu (2011). Incremental Syntactic Language Models for Phrase-based Translation. ACL 2011. 12 pages.
  102. Omar Zaidan and Chris Callison-Burch (2010). Predicting Human-Targeted Translation Edit Rate via Untrained Human Annotators. NAACL 2010. Short papers. 4 pages.
  103. Kathryn Baker, Michael Bloodgood, Chris Callison-Burch, Bonnie Dorr, Scott Miller, Christine Piatko, Nathaniel W. Filardo, and Lori Levin (2010). Semantically-Informed Syntactic Machine Translation: A Tree-Grafting Approach. AMTA 2010. 10 pages.
  104. Ann Irvine, Alex Klementiev, and Chris Callison-Burch (2010). Transliterating From All Languages. AMTA 2010. 8 pages.
  105. Michael Bloodgood and Chris Callison-Burch (2010). Large-Scale, Cost-Focused Active Learning for Statistical Machine Translation. ACL 2010. 11 pages.
  106. Abby Levenberg, Chris Callison-Burch, and Miles Osborne (2010). Stream-based Translation Models for Statistical Machine Translation. NAACL 2010. 9 pages.
  107. Scott Novotney and Chris Callison-Burch (2010). Cheap, Fast and Good Enough: Automatic Speech Recognition with Non-Expert Transcription. NAACL 2010. 669 citations. 9 pages.
  108. Chris Callison-Burch (2009). Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon's Mechanical Turk. EMNLP 2009. Nominated for the ACL 2019 Test of Time Award. 632 citations. 10 pages.
  109. Omar Zaidan and Chris Callison-Burch (2009). Feasibility of Human-in-the-loop Minimum Error Rate Training. EMNLP 2009. 10 pages.
  110. Yuval Marton, Chris Callison-Burch and Philip Resnik (2009). Improved Statistical Machine Translation Using Monolingually-Derived Paraphrases. EMNLP 2009. 407 citations. 10 pages.
  111. Nikesh Garera, Chris Callison-Burch and David Yarowsky (2009). Improving Translation Lexicon Induction from Monolingual Corpora via Dependency Contexts and Part-of-Speech Equivalences. CoNLL 2009. 9 pages.
  112. Chris Callison-Burch (2008). Syntactic Constraints on Paraphrases Extracted from Parallel Corpora. EMNLP 2008. 229 citations. 10 pages.
  113. Chris Callison-Burch, Trevor Cohn, Mirella Lapata (2008). ParaMetric: An Automatic Evaluation Metric for Paraphrasing. CoLing 2008. 8 pages.
  114. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst (2007). Moses: Open source toolkit for statistical machine translation. ACL 2007. 7358 citations.
  115. Chris Callison-Burch, Philipp Koehn and Miles Osborne (2006). Improved Statistical Machine Translation Using Paraphrases. NAACL 2006. 391 citations.
  116. Chris Callison-Burch, Miles Osborne and Philipp Koehn (2006). Re-evaluating the Role of Bleu in Machine Translation Research. EACL 2006. 1135 citations. 8 pages.
  117. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2005). Scaling Phrase-Based Statistical Machine Translation to Larger Corpora and Longer Phrases. ACL 2005. 116 citations.
  118. Colin Bannard and Chris Callison-Burch (2005). Paraphrasing with Bilingual Parallel Corpora. ACL 2005. 801 citations.
  119. Chris Callison-Burch, David Talbot and Miles Osborne (2004). Statistical Machine Translation with Word- and Sentence-Aligned Parallel Corpora. ACL 2004. 161 citations.

Journal articles

  1. Aditya M Kashyap, Delip Rao, Mary Regina Boland, Li Shen, Chris Callison-Burch (2025). Predicting Explainable Dementia Types with LLM-Aided Feature Engineering. Bioinformatics 2025.
  2. Qing Lyu, Marianna Apidianaki, Chris Callison-Burch (2024). Towards Faithful Model Explanation in NLP: A Survey. Computational Linguistics 2024. 65 pages.
  3. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, ... (450 authors including Chris Callison-Burch), Ziyi Wu (2023). Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. TMLR 2023. 1526 citations. 95 pages.
  4. Abi Adams-Prassl, Kotaro Hara, Kristy Milland, Chris Callison-Burch (2023). The Gender Wage Gap in an Online Labor Market: The Cost of Interruptions. The Review of Economics and Statistics 2023. 23 pages.
  5. Aditya Kashyap, Chris Callison-Burch, Mary Regina Boland (2022). A Deep Learning Method to Detect Opioid Prescription and Opioid Use Disorder from Electronic Health Records. International Journal of Medical Informatics 2022. 44 pages.
  6. Monique A Sager, Aditya M Kashyap, Mila Tamminga, Sadhana Ravoori, Chris Callison-Burch and Jules B Lipoff (2021). Identifying and Responding to Health Misinformation on Reddit Dermatology Forums With Artificially Intelligent Bots Using Natural Language Processing: Design and Evaluation Study. JMIR 2021. 7 pages.
  7. Isabel Straw and Chris Callison-Burch (2020). Artificial Intelligence in mental health and the biases of language based models. PLOS One 2020. 19 pages.
  8. Aditya Kashyap, Heather Burris, Chris Callison-Burch, Mary Regina Boland (2020). The CLASSE GATOR (CLinical Acronym SenSE disambiGuATOR): A Method for Predicting Acronym Sense from Neonatal Clinical Notes. International Journal of Medical Informatics 2020.
  9. Benjamin Chrisinger, Eliza Kinsey, Ellie Pavlick, Chris Callison-Burch (2020). SNAP judgments into the digital age: Reporting on food stamps varies significantly with time, publication type, and political leaning. PLOS One 2020. 19 pages.
  10. Anne Cocos and Chris Callison-Burch (2019). Paraphrase-Sense-Tagged Sentences. TACL 2019. 15 pages.
  11. Edidiong Okon, Vishnutheja Rachakonda, Hyo Jung Hong, Chris Callison-Burch and Jules Lipoff (2019). Natural Language Processing of Reddit Data to Evaluate Dermatology Patient Experiences and Therapeutics. Journal of the American Academy of Dermatology 2019. 20 pages.
  12. Anne Cocos, Ting Qiana, Chris Callison-Burch, and Aaron J. Masino (2017). Crowd Control: Effectively Utilizing Unscreened Crowd Workers for Biomedical Data Annotation. Journal of Biomedical Informatics 2017. 22 pages.
  13. Wei Xu, Courtney Napoles, Ellie Pavlick, Jim Chen, and Chris Callison-Burch (2016). Optimizing Statistical Machine Translation for Text Simplification. TACL 2016. 736 citations. 15 pages.
  14. Ann Irvine and Chris Callison-Burch (2016). A Comprehensive Analysis of Bilingual Lexicon Induction. Computational Linguistics 2016. 38 pages.
  15. Ann Irvine and Chris Callison-Burch (2016). End-to-End Statistical Machine Translation with Zero or Small Parallel Texts. Journal of Natural Language Engineering 2016. 34 pages.
  16. Wei Xu, Chris Callison-Burch, and Courtney Napoles (2015). Problems in Current Text Simplification Research: New Data Can Help. TACL 2015. 561 citations. 16 pages.
  17. Omar Zaidan and Chris Callison-Burch (2014). Arabic Dialect Identification. Computational Linguistics 2014. 438 citations. 36 pages.
  18. Wei Xu, Alan Ritter, Chris Callison-Burch, William B. Dolan and Yangfeng Ji (2014). Extracting Lexically Divergent Paraphrases from Twitter. TACL 2014. 130 citations. 14 pages.
  19. Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev, and Chris Callison-Burch (2014). The Language Demographics of Amazon Mechanical Turk. TACL 2014. 13 pages.
  20. Adam Lopez, Matt Post, Chris Callison-Burch, Jonathan Weese, Juri Ganitkevitch, Narges Ahmidi, Olivia Buzek, Leah Hanson, Beenish Jamil, Matthias Lee, Ya-Ting Lin, Henry Pao, Fatima Rivera, Leili Shahriyari, Debu Sinha, Adam Teichert, Stephen Wampler, Michael Weinberger, Daguang Xu, Lin Yang, and Shang Zhao (2013). Learning to translate with products of novices: a suite of open-ended challenge problems for teaching MT. TACL 2013. 13 pages.
  21. Kathryn Baker, Bonnie Dorr, Michael Bloodgood, Chris Callison-Burch, Wes Filardo, Christine Piatko, Lori Levin, and Scott Miller (2012). Use of Modality and Negation in Semantically-Informed Syntactic MT. Computational Linguistics 2012. 28 pages.
  22. Ann Irvine, Mike Kayser, Zhifei Li, Wren Thornton, and Chris Callison-Burch (2010). Integrating Output from Specialized Modules in Machine Translation: Transliteration in Joshua. PBML 2010. 10 pages.
  23. Jonathan Weese and Chris Callison-Burch (2010). Visualizing Data Structures in Parsing-Based Machine Translation. PBML 2010. 10 pages.
  24. Lane Schwartz and Chris Callison-Burch (2010). Hierarchical Phrase-Based Grammar Extraction in Joshua: Suffix Arrays and Prefix Trees. PBML 2010. 10 pages.
  25. Zhifei Li, Chris Callison-Burch, Sanjeev Khudanpur, and Wren Thornton (2009). Decoding in Joshua: Open Source, Parsing-Based Machine Translation. PBML 2009. 10 pages.
  26. Trevor Cohn, Chris Callison-Burch, Mirella Lapata (2008). Constructing Corpora for the Development and Evaluation of Paraphrase Systems. Computational Linguistics 2008. 18 pages.

Workshop papers

  1. Liam Dugan, Andrew Zhu, Firoj Alam, Preslav Nakov, Marianna Apidianaki, Chris Callison-Burch (2025). GenAI Content Detection Task 3: Cross-Domain Machine-Generated Text Detection Challenge. GenAIDetect @ COLING 2025. 12 pages.
  2. Bryan Li, Samar Haider, Fiona Luo, Adwait Agashe, Chris Callison-Burch (2024). BORDIRLINES: A Dataset for Evaluating Cross-lingual Retrieval-Augmented Generation. First Workshop on Advancing Natural Language Processing for Wikipedia 2024. 13 pages.
  3. Bryan Li, Aleksey Panasyuk, Chris Callison-Burch (2024). Uncovering Differences in Persuasive Language in Russian versus English Wikipedia. First Workshop on Advancing Natural Language Processing for Wikipedia 2024. 15 pages.
  4. Artemis Panagopoulou, Coby Melkin, Chris Callison-Burch (2024). Evaluating Vision-Language Models on Bistable Images. Workshop on Cognitive Modeling and Computational Linguistics 2024. Best paper award at the Workshop on Cognitive Modeling and Computational Linguistics (CMCL). 22 pages.
  5. Tianyi Zhang, Harry Li Zhang, Zhaoyi Hou, Ziyu Wang, Yuling Gu, Peter Clark, Chris Callison-Burch, Niket Tandon (2024). PROC2PDDL: Open-Domain Planning Representations from Texts. Natural Language Reasoning and Structured Explanations Workshop 2024.
  6. Samuel Thudium, Federico Cimini, Rut Vyas, Kyle Sullivan, Louis Petro, Andrew Zhu and Chris Callison-Burch (2024). Outwit, Outplay, Out-Generate: A Framework for Designing Strategic Generative Agents in Competitive Environments. Wordplay Workshop 2024. 24 pages.
  7. Jaewoo Song, Andrew Zhu and Chris Callison-Burch (2024). You Have Thirteen Hours in Which to Solve the Labyrinth: Enhancing AI Game Masters with Function Calling. Wordplay Workshop 2024. 26 pages.
  8. Chris Callison-Burch, Ajay Patel, James Dennis, and Andrew Zhu (2024). DAGGER: Data Augmentation for Generative Gaming in Enriched Realms. Wordplay Workshop 2024. 23 pages.
  9. Meiqing Jin, Manvi Kaul, Shriya Ramakrishanan, Hardik Jain, Samarth Chandrawat, Ishita Agarwal, Tianyi Zhang, Andrew Zhu, and Chris Callison-Burch (2024). WorldWeaver: Procedural World Generation for Text Adventure Games using Large Language Models. Wordplay Workshop 2024. 18 pages.
  10. Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, Peter Clark (2023). CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization. Agent Learning in Open-Endedness (ALOE) Workshop 2023. 20 pages.
  11. Andrew Zhu, Liam Dugan, Alyssa Hwang, Chris Callison-Burch (2023). Kani 🦀: A Lightweight and Highly Hackable Framework for Building Language Model Applications. 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS) 2023. 13 pages.
  12. Li Zhang, Liam Dugan, Hainiu Xu, Chris Callison-Burch (2023). Exploring the Curious Case of Code Prompts. Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE) 2023. 9 pages.
  13. Hannah Gonzalez, Jiening Li, Helen Jin, Jiaxuan Ren, Hongyu Zhang, Ayotomiwa Akinyele, Adrian Wang, Eleni Miltsakaki, Ryan Baker, Chris Callison-Burch (2023). Automatically Generated Summaries of Video Lectures May Enhance Students’ Learning Experience. Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) 2023. 12 pages.
  14. Hannah Gonzalez, Liam Dugan, Eleni Miltsakaki, Zhiqi Cui, Jiaxuan Ren, Bryan Li, Shriyash Upadhyay, Etan Ginsberg, Chris Callison-Burch (2023). Enhancing Human Summaries for Question-Answer Generation in Education. Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) 2023. 11 pages.
  15. Shriyash Upadhyay, Etan Ginsberg, Chris Callison-Burch (2023). Improving Mathematics Tutoring With A Code Scratchpad. Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) 2023. 9 pages.
  16. Harry Li Zhang, Chris Callison-Burch (2023). Language Models are Drummers: Drum Composition with Natural Language Pre-Training. AAAI 2023 Workshop on Creative AI Across Modalities 2023. 8 pages.
  17. Rebecca Iglesias-Flores, Megha Mishra, Ajay Patel, Akanksha Malhotra, Reno Kriz, Martha Palmer and Chris Callison-Burch (2021). TopGuNN: Fast NLP Training Data Augmentation using Large Corpora. Workshop on Data Science with Human in the Loop 2021. 15 pages.
  18. Anietie Andy, Chris Callison-Burch and Derry Wijaya (2020). Resolving Pronouns in Twitter Streams: Context Can Help. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC) 2020. 6 pages.
  19. Anietie Andy, Derry Wijaya and Chris Callison-Burch (2019). Winter is here: Summarizing Twitter Streams related to Pre-Scheduled Events. Proceedings of the Second Workshop on Storytelling 2019. 5 pages.
  20. Aina Garí Soler, Anne Cocos, Marianna Apidianaki, Chris Callison-Burch (2019). A Comparison of Context-sensitive Models for Lexical Substitution. 13th International Conference on Computational Semantics (IWCS) 2019. 12 pages.
  21. Daphne Ippolito, David Grangier, Chris Callison-Burch and Douglas Eck (2019). Unsupervised Hierarchical Story Infilling. 13th International Conference on Computational Semantics (IWCS) 2019. 7 pages.
  22. Bhavna Saluja, Gaurav Kumar, João Sedoc, and Chris Callison-Burch (2019). Anonymization of Sensitive Information in Medical Health Records. Iberian Languages Evaluation Forum 2019. 7 pages.
  23. Joao Sedoc, Daphne Ippolito, Arun Kirubarajan, Jai Thirani, Lyle Ungar, and Chris Callison-Burch (2018). ChatEval: A Tool for the Systematic Evaluation of Chatbots. Workshop on Intelligent Interactive Systems and Language Generation 2018. 4 pages.
  24. Anietie Andy, Mark Dredze, Mugizi Rwebangira, and Chris Callison-Burch (2017). Constructing an Alias List for Named Entities During an Event. Workshop on Noisy User-generated Text 2017. 5 pages.
  25. Courtney Napoles and Chris Callison-Burch (2017). Systematically Adapting Machine Translation for Grammatical Error Correction. 12th Workshop on Innovative Use of NLP for Building Educational Applications (BEA12) 2017. 12 pages.
  26. Anne Cocos, Marianna Apidianaki and Chris Callison-Burch (2017). Word Sense Filtering Improves Embedding-Based Lexical Substitution. Workshop on Sense, Concept and Entity Representations and their Applications 2017. Best Paper Award. 9 pages.
  27. Ellie Pavlick and Chris Callison-Burch (2016). The Gun Violence Database. Bloomberg Data for Good Exchange 2016. 6 pages.
  28. Wei Xu, Chris Callison-Burch, and Bill Dolan (2015). SemEval-2015 Task 1: Paraphrase and Semantic Similarity in Twitter. SemEval 2015. 172 citations. 11 pages.
  29. Anne Cocos, Aaron J. Masino, Ting Qian, Ellie Pavlick, and Chris Callison-Burch (2015). Effectively Crowdsourcing Radiology Report Annotations. Sixth International Workshop on Health Text Mining and Information Analysis 2015. 6 pages.
  30. Courtney Napoles and Chris Callison-Burch (2015). Automatically Scoring Freshman Writing: A Preliminary Investigation. Workshop on Innovative Use of NLP for Building Educational Applications 2015. 10 pages.
  31. Ellie Pavlick and Chris Callison-Burch (2015). Extracting Structured Information via Automatic + Human Computation. HCOMP 2015. 2 pages.
  32. Gaurav Kumar, Yuan Cao, Ryan Cotterell, Chris Callison-Burch, Daniel Povey, and Sanjeev Khudanpur (2014). Translations of the CALLHOME Egyptian Arabic corpus for conversational speech translation. IWSLT 2014. 5 pages.
  33. Quanze Chen, Chenyang Lei, Wei Xu, Ellie Pavlick and Chris Callison-Burch (2014). Poetry of the Crowd: A Human Computation Algorithm to Convert Prose into Rhyming Verse. HCOMP Poster 2014. 3 pages.
  34. Chris Callison-Burch (2014). Crowd-Workers: Aggregating Information Across Turkers To Help Them Find Higher Paying Work. HCOMP Poster 2014. 2 pages.
  35. Ann Irvine and Chris Callison-Burch (2014). Using Comparable Corpora to Adapt MT Models to New Domains. WMT 2014. 8 pages.
  36. Ryan Cotterell, Adithya Renduchintala, Naomi Saphra, and Chris Callison-Burch (2014). An Algerian Arabic-French Code-Switched Corpus. LREC Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools 2014. 4 pages.
  37. Matt Post, Gaurav Kumar, Adam Lopez, Damianos Karakos, Chris Callison-Burch and Sanjeev Khudanpur (2013). Improved Speech-to-Text Translation with the Fisher and Callhome Spanish–English Speech Translation Corpus. IWSLT 2013. 129 citations. 7 pages.
  38. Ondrej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia (2013). Findings of the 2013 Workshop on Statistical Machine Translation. WMT 2013. 44 pages.
  39. Matt Post, Juri Ganitkevitch, Luke Orland, Jonathan Weese, Yuan Cao, and Chris Callison-Burch (2013). Joshua 5.0: Sparser, better, faster, server. WMT 2013. 7 pages.
  40. Ann Irvine and Chris Callison-Burch (2013). Combining Bilingual and Comparable Corpora for Low Resource Machine Translation. WMT 2013. 9 pages.
  41. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia (2012). Findings of the 2012 Workshop on Statistical Machine Translation. WMT 2012. 42 pages.
  42. Matt Post, Chris Callison-Burch, and Miles Osborne (2012). Constructing Parallel Corpora for Six Indian Languages via Crowdsourcing. WMT 2012. 164 citations. 9 pages.
  43. Jonathan Weese, Chris Callison-Burch, and Adam Lopez (2012). Using Categorial Grammar to Label Translation Rules. WMT 2012. 10 pages.
  44. Juri Ganitkevitch, Yuan Cao, Jonathan Weese, Matt Post, and Chris Callison-Burch (2012). Joshua 4.0: Packing, PRO, and Paraphrases. WMT 2012. 9 pages.
  45. Ann Irvine, Jonathan Weese, and Chris Callison-Burch (2012). Processing Informal, Romanized Pakistani Text Messages. the NAACL Workshop on Language in Social Media 2012. 4 pages.
  46. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan (2011). Findings of the 2011 Workshop on Statistical Machine Translation. WMT 2011. 43 pages.
  47. Charley Chan, Chris Callison-Burch, and Benjamin Van Durme (2011). Reranking Bilingually Extracted Paraphrases Using Monolingual Distributional Similarity. GEMS 2011. 10 pages.
  48. Jonathan Weese, Juri Ganitkevitch, Chris Callison-Burch, Matt Post and Adam Lopez (2011). Joshua 3.0: Syntax-based Machine Translation with the Thrax Grammar Extractor. WMT 2011. 7 pages.
  49. Byung Gyu Ahn, Ben Van Durme and Chris Callison-Burch (2011). WikiTopics: What is Popular on Wikipedia and Why. ACL Workshop on Automatic Summarization for Different Genres, Media, and Languages 2011. 8 pages.
  50. Courtney Napoles, Ben Van Durme (2011). Evaluating sentence compression: Pitfalls and suggested remedies. Workshop on Monolingual Text-To-Text Generation 2011. 7 pages.
  51. Courtney Napoles, Chris Callison-Burch, Juri Ganitevitch, Ben Van Durme (2011). Paraphrastic Sentence Compression with a Character-based Metric: Tightening without Deletion. Workshop on Monolingual Text-To-Text Generation 2011. 7 pages.
  52. Rui Wang and Chris Callison-Burch (2011). Paraphrase Fragment Extraction from Monolingual Comparable Corpora. BUCC 2011. 9 pages.
  53. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Ann Irvine, Lane Schwartz, Wren N. G. Thornton, Ziyuan Wang, Jonathan Weese and Omar F. Zaidan (2010). Joshua 2.0: A Toolkit for Parsing-Based Machine Translation with Syntax, Semirings, Discriminative Training and Other Goodies. WMT 2010. 5 pages.
  54. Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, Omar Zaidan (2010). Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. WMT 2010. 258 citations. 33 pages.
  55. Chris Callison-Burch and Mark Dredze (2010). Creating Speech and Language Data With Amazon’s Mechanical Turk. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 449 citations. 12 pages.
  56. Michael Bloodgood and Chris Callison-Burch (2010). Using Mechanical Turk to Build Machine Translation Evaluation Sets. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 4 pages.
  57. Scott Novotoney and Chris Callison-Burch (2010). Crowdsourced Accessibility: Elicitation of Wikipedia Articles. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 4 pages.
  58. Rui Wang and Chris Callison-Burch (2010). Cheap Facts and Counter-Facts. NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk 2010. 5 pages.
  59. Chris Callison-Burch, Philipp Koehn, Christof Monz and Josh Schroeder (2009). Findings of the 2009 Workshop on Statistical Machine Translation. WMT 2009. 868 citations. 28 pages.
  60. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren Thornton, Jonathan Weese and Omar Zaidan (2009). Joshua: An Open Source Toolkit for Parsing-based Machine Translation. WMT 2009. 230 citations. 5 pages.
  61. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz and Josh Schroeder (2008). Further Meta-Evaluation of Machine Translation. WMT 2008. 324 citations. 37 pages.
  62. Delip Rao, David Yarowsky, Chris Callison-Burch (2008). Affinity Measures based on the Graph Laplacian. of the 3rd Textgraphs workshop on Graph-based Algorithms for Natural Language Processing at CoLing 2008. 8 pages.
  63. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz and Josh Schroeder (2007). (Meta-) Evaluation of Machine Translation. WMT 2007. 188 citations.
  64. Philipp Koehn, Nicola Bertoldi, Ondrej Bojar, Chris Callison-Burch, Alexandra Constantin, Brooke Cowan, Chris Dyer, Marcello Federico, Evan Herbst, Hieu Hoang, Christine Moran, Wade Shen, and Richard Zens (2007). Open Source Toolkit for Statistical Machine Translation: Factored Translation Models and Confusion Network Decoding. CLSP Summer Workshop Final Report WS, Johns Hopkins University 2007.
  65. Alexandra Birch, Chris Callison-Burch and Miles Osborne (2006). Constraining the Phrase-Based, Joint Probability Statistical Translation Model. WMT 2006.
  66. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2005). A Compact Data Structure for Searchable Translation Memories. EAMT 2005.
  67. Chris Callison-Burch (2005). Linear B System Description for the 2005 NIST MT Evaluation Exercise. Machine Translation Evaluation Workshop 2005.
  68. Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, Chris Callison-Burch, Miles Osborne, and David Talbot (2005). Edinburgh System Description for the 2005 IWSLT Speech Translation Evaluation. IWSLT 2005. 446 citations.
  69. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2004). Searchable Translation Memories. ASLIB Translating and the Computer 2004.
  70. Chris Callison-Burch, Colin Bannard and Josh Schroeder (2004). Improved Statistical Translation Through Editing. EAMT 2004.
  71. Chris Callison-Burch and Miles Osborne (2003). Bootstrapping Parallel Corpora. NAACL workshop Building and Using Parallel Texts 2003. 6 pages.
  72. Chris Callison-Burch and Miles Osborne (2003). Co-training for Statistical Machine Translation. the 6th Annual CLUK Research Colloquium 2003.
  73. Jochen Leidner and Chris Callison-Burch (2003). Evaluating Question Answering Systems Using FAQ Answer Injection. the 6th Annual CLUK Research Colloquium 2003.
  74. Chris Callison-Burch (2001). Upping the Ante for "Best of Breed" Machine Translation Providers. ASLIB Translating and the Computer 2001.
  75. Chris Callison-Burch and Raymond Flournoy (2001). A program for automatically selecting the best output from multiple machine translation engines. MT Summit 2001.
  76. Raymond Flournoy and Chris Callison-Burch (2001). Secondary Benefits of Feedback and User Interaction in Machine Translation Tools. MT Summit Workshop 2001.

Invited Talks and Panels

  1. WorldQuant Bridgestone. Large Language Models and Generative AI. May 13, 2025
  2. Penn Conference on AI Infrastructure: Foundations for Energy Efficiency and Scalability. AI, Energy, and Sustainability. April 14, 2025
  3. AIRFoundry Annual Meeting. Harnessing Artificial Intelligence and Language Modeling for Enhancing Innovation and Evaluating Research Claims. March 31, 2025
  4. Cleaver Magazine Workshop. Demystifying AI: Threat or Tool?. March 30, 2025
  5. James Brister Society Spring Conference. History of NLP + Intro to Large Language Models. March 28, 2025
  6. Penn Dental Medicine. Ask an Expert About ChatGPT and Generative AI. February 4, 2025
  7. Learning Analytics M.S.Ed. Online at UPenn GSE. Undetectable AI∷ Risks, Challenges, and Policy Implications. November 14, 2024
  8. Penn State NLP - Communication and Intelligence conference. Using Large Language Models to Build Explainable Classifiers. November 1, 2024
  9. UChicago - Communication and Intelligence conference. Using Large Language Models to Build Explainable Classifiers. October 18, 2024
  10. Penn Alumni Association Event in Texas. Is AI coming for our jobs?. October 16, 2024
  11. William Penn Foundation. Ask an Expert About ChatGPT and Generative AI. September 26, 2024
  12. Global Conference on Frontier AI at the Brookings Instituion. Undetectable AI: Risks, Challenges, and Policy Implications. September 16, 2024
  13. Informatics in the Round podcast. Chatbots in Healthcare: The Ultimate Turing Test. August 28, 2024
  14. Summer Institutes in Computational Social Science. Large Language Models. July 11, 2024
  15. Bi-annual Conference at the Philadelphia Fed. A Primer on Large Language Models. June 27, 2024
  16. Penn Law School. Fundamentals of AI for Regulators. June 6, 2024
  17. Penn Engineering - AI for Executives. A Primer on Large Language Models. May 30, 2024
  18. Penn IT Staff Convention. Ask an Expert About ChatGPT and Generative AI. May 23, 2024
  19. NRT Soft AE Annual Symposium. State of the Art in Large Language Models. May 2, 2024
  20. Penn Engineering Podcast. Exploring AI in Engineering. April 24, 2024
  21. Howard University NLP Course. Using Large Language Models to Build Explainable Classifiers. April 8, 2024
  22. The Nancy Ide Annual Lecture in Computer Science, Vassar College. Using Large Language Models to Build Explainable Classifiers. April 5, 2024
  23. Penn Law Center for Technology, Innovation and Competition. Law + Engineering Panel. March 26, 2024
  24. AIs Wide Open Podcast. The relationship between AI and copyright. February 21, 2024
  25. CIBC. Using Large Language Models to Build Explainable Classifiers. January 24, 2024
  26. University of Pennsylvania Faculty Senate. Generative AI in Your Teaching. December 6, 2023
  27. Penn Association of Senior and Emeritus Faculty (PASEF). Ask an Expert About ChatGPT and Generative AI. November 8, 2023
  28. Mack Institute Conference on Driving Innovation with Generative AI. Panel Discussion About Generative AI. November 6, 2023
  29. Hopkins Business of Health Initiative (HBHI). Driving Innovation with Generative AI. September 15, 2023
  30. The Society Of Composers and Lyricists. Ask an Expert about ChatGPT and Generative AI. August 14, 2023
  31. NLP Highlights Podcast. Generative AI and Copyright. May 25, 2023
  32. Wharton. Invited Speaker at workshop on Large Language Models: Computer Science meets Social Science. May 19, 2023
  33. The U.S. House of Representatives Judiciary Committee, Subcommittee on Courts, Intellectual Property, and the Internet. Testimony at hearing on "Artificial Intelligence and Intellectual Property: Part I – Interoperability of AI and Copyright Law". May 17, 2023
  34. US Copyright Office. Panelist in the Copyright Office AI Listening Sessions - Literary Works. April 19, 2023
  35. Community College of Philadelphia faculty meeting. Ask an Expert about ChatGPT. April 14, 2023
  36. Penn Engineering Podcast. The Growth and Impact of Generative AI. April 14, 2023
  37. SEAS Board of Advisors Meeting. Panel Discussion about ChatGPT. March 31, 2023
  38. Duolingo. Using Large Language Models to Generate Course Materials. March 16, 2023
  39. Penn Center for Learning Analytics. Smart Textbooks and Course Materials Using Large Language Models. March 13, 2023
  40. AI2 Company-wide Meeting. Is ChatGPT A Sputnik Event?. March 10, 2023
  41. Drexel AI Symposium. Ask An Expert About ChatGPT. February 25, 2023
  42. AI2 Aristo Team. Using Large Language Models to Build Explainable Classifiers. February 24, 2023
  43. IARPA Demo Day. Panel Discussion on Large Language Models. February 23, 2023
  44. Penn’s Data Driven Discovery Initiative. Panel Discussion on ChatGPT and Generative AI. February 23, 2023
  45. ISAT/DARPA PARADIGM Workshop. Participant in Performance and Resilience Arising from Defense-Informed Giant Models (PARADIGM) Workshop. February 22, 2023
  46. Penn Medicine Seminar. Ask An Expert About ChatGPT. February 14, 2023
  47. Penn ASSET Seminar. Using Large Language Models to Build Explainable Classifiers. February 8, 2023
  48. TAC Conference. Panel Discussion on What Large Language Models Cannot Do. February 3, 2023
  49. Keynote Address to AACL 2022 Conference. Reasoning about Goals and Making Plans with Large Language Models. November 22, 2022
  50. Salesforce AI Research. Reasoning about Procedures and Goals with wikiHow and Large Language Models. May 16, 2022
  51. University of Lorraine, Nancy. Crowdsourcing for NLP (with Karën Fort and Christoper Cieri). April 14, 2021
  52. Morgan Stanley. Natural Language Understanding with Paraphrases and Word Embeddings. January 19, 2021
  53. Two Sigma. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. August 4, 2020
  54. University of Pennsylvania. Panelist for CURF's Research & Fellowships Week. November 19, 2019
  55. Temple University. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. November 6, 2019
  56. University of Pennsylvania. Panelist for MindCORE's summer program. June 14, 2019
  57. Undergraduate Program in Cognitive Science (UPenn). Representing Word Meaning with Vectors. June 5, 2019
  58. Talk to online MCIT students (UPenn). Natural Language Understanding with Paraphrases and Word Embeddings. April 25, 2019
  59. Vanguard Data Science. Natural Language Understanding with Paraphrases and Word Embeddings. October 26, 2018
  60. Michigan Institute for Data Science. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. November 29, 2018
  61. Google Research (NYC). Learning Translations Without Parallel Texts. August 14, 2018
  62. NSF Convergence Workshop on Crowdsourcing. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. May 18, 2018
  63. National Academies of Sciences workshop on Challenges in Machine Generation of Analytic Products from Multi-source Data. Crowdsourcing for Natural Language Processing. August 10, 2017
  64. NYU. The Promise of Crowdsourcing for Natural Language Processing and Other Data Sciences. February 2, 2017
  65. Columbia University. Large-scale Paraphrasing for Natural Language Generation. September 19, 2016
  66. Cornell University. Large-scale Paraphrasing for Natural Language Generation. September 9, 2016
  67. University of Alabama at Birmingham. Crowdsourcing Translation. April 17, 2015
  68. Drexel University. Crowdsourcing Translation. November 8, 2015
  69. CMU. Crowdsourcing Translation. April 7, 2015
  70. UC Berkeley. Large-scale Paraphrasing for Natural Language Generation. March 12, 2015
  71. Stanford. Crowdsourcing Translation. March 11, 2015
  72. Facebook. Crowdsourcing Translation. March 10, 2015
  73. Coursera. Crowdsourcing Translation. March 9, 2015
  74. Google. Large-scale Paraphrasing for Natural Language Generation. March 9, 2015
  75. MIT. Large-scale Paraphrasing for Natural Language Generation. January 14, 2015
  76. CMU. Large-scale Paraphrasing for Natural Language Generation. November 21, 2014
  77. Microsoft Research. Large-scale Paraphrasing for Natural Language Generation. October 3, 2014
  78. University of Washington, Seattle. Large-scale Paraphrasing for Natural Language Generation. October 2, 2014
  79. The Allen Institute for Artificial Intelligence (AI2). Large-scale Paraphrasing for Natural Language Generation. October 1, 2014
  80. Yahoo! Research Labs. Large-scale Paraphrasing for Natural Language Generation. July 29, 2014
  81. US Army Research Labs DARPA Computer Science Study Group Applied Research Series: Text and Video Analytics Workshop. Language Understanding with the Help of Images. July 16, 2014
  82. International Conference on Natural Language Generation. Large-scale Paraphrasing for Natural Language Generation. June 21, 2014
  83. LREC Workshop on Building and Using Comparable Corpora. Crowdsourcing Translation. May 27, 2014
  84. University of Maryland. Large-scale Paraphrasing for Natural Language Understanding and Generation. April 23, 2014
  85. Institute for Research in Cognitive Science, University of Pennsylvania. Large-scale Paraphrasing for Natural Language Understanding and Generation. March 21, 2014
  86. 37th Annual Penn Linguistics Colloquium, University of Pennsylvania. The Wisdom of Crowdsourcing. March 22, 2013
  87. Johns Hopkins University. Advances to machine translation and language understanding. February 15, 2013
  88. Columbia University IGERT distinguished speaker series. The Promise of Crowdsourcing for NLP and other data sciences. March 29, 2013
  89. UT Austin. Large-scale Paraphrasing for Natural Language Understanding and Generation. December 7, 2012
  90. IBM TJ Watson Research Center. Large-scale Paraphrasing for Natural Language Understanding and Generation. November 9, 2012
  91. NSF-sponsored Workshop on summarizing speaker's attitude and opinion in conversational speech. When annotation with MTurk works. October 20, 2012
  92. Linguistics Data Consortium 20th Anniversary Workshop. The Promise of Crowdsourcing. September 6, 2012
  93. Human Language Technology Center of Excellence. Machine Translation of Arabic Dialects. April 3, 2012.
  94. Human Language Technology Center of Excellence. Machine Translation at the HLTCOE. March 29, 2012
  95. University of Pennsylvania. Advances to machine translation and language understanding. February 28, 2012
  96. Carnegie Mellon University. Advances to machine translation and language understanding. February 21, 2012
  97. Microsoft Research. Crowdsourcing Translation: Professional Quality from Non-Professionals. June 16, 2011
  98. Human Language Technology Center of Excellence. Statistical Machine Translation and Crowdsourcing. March 24, 2011.
  99. CrowdFlower Meetup, Washington DC. Crowdsourcing Translation with Amazon's Mechanical Turk. July 25, 2010
  100. Human Language Technology Center of Excellence. Crowdsourcing Translation with Amazon's Mechanical Turk. November 23, 2010
  101. AAAI Panel on Common Sense Knowledge. Automatic versus Manual Construction of Common Sense Knowledge. November 13, 2010
  102. UMass Amherst. Crowdsourcing Translation with Amazon's Mechanical Turk. October 18, 2010
  103. Brown University. Syntactic Parsing and Machine Translation. September 9, 2010
  104. IARPA. Crowdsourcing Translation. August 5, 2010
  105. UMD Workshop on Crowdsourcing Translation. Crowdsourcing Translation with Amazon's Mechanical Turk. June 10, 2010
  106. BBN. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. March 11, 2010
  107. BBN. Improvements to Urdu-English: SCALE Summer Workshop Results. March 11, 2010
  108. University of Washington. Syntactic translation models help for low-resource, verb final languages. February 25, 2010
  109. Microsoft Research. Syntactic translation models help for low-resource, verb final languages. February 24, 2010
  110. University of Pennsylvania. Syntactic translation models help for low-resource, verb final languages. February 8, 2010
  111. NIST. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. December 18, 2009
  112. University of Maryland. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. December 2, 2009
  113. OHSU, Center for Spoken Language Understanding. Fast, Cheap and Creative: Evaluating Translation Quality with Amazon’s Mechanical Turk. October 7, 2009
  114. OHSU, Center for Spoken Language Understanding. Improvements to Urdu-English: SCALE Summer Workshop Results. October 5, 2009
  115. University of Pennsylvania. Syntactic Constraints on Paraphrases Extracted from Parallel Corpora. April 13, 2009
  116. University of Maryland. Paraphrasing and Translation. November 28, 2007.
  117. Johns Hopkins University. Improving Statistical Machine Translation With Paraphrases and Generalization. December 5, 2006
  118. Johns Hopkins University. Factored Translation Models. November 28, 2006
  119. University of Pennsylvania. Factored Translation Models. August 23, 2006
  120. Carnegie Mellon University. Statistical Machine Translation Using Semi-Supervised Learning. April 18, 2005.

Grants

Current grants

Grant Title Awarding Body Amount Dates PI Info
DARPA SciFy: Harnessing Artificial Intelligence and Language Modeling for Enhancing Innovation and Evaluating Research Claims (HAILMEIER-C) DARPA $6m (Penn's portion is $3.8m) 2024-2027 Chris Callison-Burch - PI, Andrew Head, Eric Wong, Duncan Watts, Mark Yatskar, Chinedum Osuji, Peter Jansen (University of Arizona), Yulia Tsvetkov (University of Washington), Hanna Hajishirzi (Universityof Washington)
IARPA HIATUS: PAUSIT: Privacy protection and Authorship attribution Using Style-based Interpretable Transfer IARPA $7m (Penn's portion is $3.1m) 2022-2026 Chris Callison-Burch - PI, Marianna Apidianaki - co-PI, Kathleen McKeown (Columbia), Smaranda Muresan (Columbia), Owen Rambow (Stony Brook University), Niranjan Balasubramanian (Stony Brook University), Andy Schwartz (Stony Brook University)

Pending grants

Grant Title Awarding Body Amount Dates PI Info
IARPA BENGAL: TIGER: Trustable Information Generation and Explanation Resilience IARPA $4m René Vidal (UPenn - PI), Chris Callison-Burch, Hamed Hassani, Mark Yaskar and Eric Wong, Rama Chellappa (JHU) and Vishal Patel (JHU)
IIS: AI Institute for Human-AI Cooperation (AI2HAI) NSF Cathy Wu (UDel - PI), Jose Principe (UDel), Austin Brockmeirer (UDel), Hacene Boukari (DESU), Eric Eaton (PI at Penn), Chris Callison-Burch (Penn), and others

Past grants

Grant Title Awarding Body Amount Dates PI Info
UNCOVER: Cross-lingual question answering to identif information differences between English and Russian Wikipedia articles AFRL $150k 2023-2024 Chris Callison-Burch - PI, Marianna Apidianaki - co-PI
Semi-supervised Learning of Multimodal Representations DARPA $428k 2019-2022 Chris Callison-Burch (PI-UPenn) and Derry Wijaya (Boston University)
STTR: Personalized Retrieval-based Simplification NSF $225k 2019-2021 Eleni Miltsakaki (Choosito.com) - PI, Chris Callison-Burch
Alexa Prize TaskBot Challenge Amazon $250k 2021-2022 Chris Callison-Burch (PI), Mark Yatskar (UPenn)
Computing Innovation Postdoctoral Fellow Award NSF/CRA ~$140k 2021-2023 Lara Martin (postdoc), Chris Callison-Burch (faculty advisor)
REU Supplement NSF $16k 2019-2020 co-PI Chris Cieri
SPUR WOMEN: Support and Promote Undergraduate Research for Women Google $15k from Google 2019-2020 PI with Rita Powell
FFW-HTF-RL: Collaborative Research: Up-skilling and Re-skilling Marginalized Rural and Urban Digital Workers: AI-worker collaboration to access creative work NSF $3m (Penn's portion is $375k) 2019-2024 Jeffrey Bigham (CMU) - PI, Chris Callison-Burch (UPenn), Ben Hanrahan (Penn State), Niki Kittur (CMU), Beibei Li (CMU), Amy Ogan (CMU), Amy Pavel (CMU), Saiph Savage (West Virginia University), Julia Ticona (UPenn)
LWLL: FLASH: Fast Learning via Auxiliary signals, Structured knowledge, and Human expertise DARPA $3.3m 2019-2022 Dan Roth (PI - UPenn), Irfan Essa (Georgia Tech), Chris Callison-Burch (UPenn), Zsolt Kira (Georgia Tech), Le Song (Georgia Tech), Mayur Naik (UPenn), Osbert Bastani (UPenn)
KAIROS: RESIN: Reasoning about Event Schemas for Induction of kNowledge DARPA $12m (Penn's portion is $2m) 2019-2024 Heng Ji (PI - UIUC), Mohit Bansal (UNC), Chris Callison-Burch (UPenn), Shih-Fu Chang (Columbia), Jiawei Han (UIUC), Martha Palmer (Colorado), Dan Roth (UPenn), Carl Vondrick (Columbia)
BETTER: Task and User-Aware Representation Learning for Fine-Grained Cross-Lingual Information Retrieval IARPA $6m 2019-2023 Ellie Pavlick (PI - Brown), Carsten Eickhoff (Brown), Chris Callison-Burch (UPenn), Wei Xu (OSU), Alan Ritter (OSU)
SPUR WOMEN: Support and Promote Undergraduate Research for Women Google $25k from Google, plus $25k matched by SEAS 2018-2019 PI with Rita Powell
CI-NEW: NIEUW: Novel Incentives and Workflows in Linguistics Data Collection NSF $1.2m 2017-2022 co-PI with Christopher Cieri and Mark Liberman
DEFT Extension DARPA $116,000 2017-2017 PI with Ben Van Durme
CI-P: Planning for Scalable Language Resource Creation through Novel Incentives and Crowdsourcing NSF $100,000 2016-2017 co-PI with Christopher Cieri and Mark Liberman
Learning translations from monolingual texts (LORELEI) DARPA $478,000 2015-2019 PI at Penn
SIREN-IL: Specialized Intra/Interlingual Resources for Emergent News - Incident Language DARPA $3,031,412 2015-2017 co-PI with Stephanie Strassel
Natural Logic Solver for Aristo Allen Institute for Artificial Intelligence (AI2) $95,000 2015-2016 PI
EAGER: Simplification as Machine Translation NSF $100,000 2014-2015 PI
Sloan Research Fellowship Alfred P. Sloan Foundation $50,000 2014
Large-scale Paraphrasing for Natural Language Understanding (DEFT) DARPA $1,600,000 2012-2017 PI with Ben Van Durme
EAGER: Combining natural language inference and data-driven paraphrasing NSF $100,000 2012-2013 co-PI with Ben Van Durme
Crowdsourcing Translation (Computer Science Study Panel phase 3) DARPA $500,000 2012-2015 PI
Improved Arabic dialect translation through Crowdsourcing DARPA $176,000 2012-2013 PI
Acquisition and use of paraphrases in a knowledge-rich setting Vulcan $260,000 2011-2013 co-PI with Ben Van Durme
Google Faculty Research Award (Translate the World: A Unified Framework for Crowdsourcing Translation) Google $150,000 2011 co-PI with Philip Resnik and Ben Bederson
RI:Medium: Semi-supervised Discriminative Training of Sequence Transduction Model NSF $800,000 2011-2015 co-PI with Sanjeev Khudanpur, Brian Roark, Damianos Karakos, Richard Sproat
Translation of informal texts via Mechanical Turk BBN Technologies $144,000 2010-2011 PI
BABEL: Bayesian Architecture Begetting Every Language (Computer Science Study Panel phase 2) DARPA $500,000 2010-2012 PI
EuroMatrixPlus: Bringing Machine Translation for European Languages to the User European Union Framework 7 Programme €4,950,000 2009-2012 PI at JHU (JHU Amount: €516,000)
Global Autonomous Language Exploitation (GALE) project, Periods 3 and 4 DARPA $575,000 2009-2011 co-PI with Sanjeev Khudanpur and Damianos Karakos
Computer Science Study Panel DARPA $93,000 2008-2009 PI
RI:SMALL: Multi-Level Modeling of Language and Translation NSF $400,000 2007-2012 co-PI with David Yarowsky
SMART:Scotland Technology Innovation Grant British Government £45,000 2002-2005

Teaching

Teaching Reviews

You can read my full teaching reviews here. Below are the summary statistics.

Penn teaching reviews are on a 0-4 quality scale: 0=Poor 1=Fair 2=Good 3=Very Good 4=Excellent
Term Course Title (Number) Students Enrolled Course Quality Instructor Quality
Summer 2025 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 132
Summer 2025 Natural Language Prorcessing (CIS 5300 - Penn Engineering Online) 138
Spring 2025 Intro to Artificial Intelligence (CIS 3990) 57
Spring 2025 Natural Language Prorcessing (CIS 5300 - Penn Engineering Online) 236
Fall 2024 Research Practicum (CIS 8000) 54 2.9 3.3
Fall 2024 Artificial Intelligence (CIS 4210/5210 - on campus) 373 3.1 3.3
Fall 2024 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 159 3.3 3.5
Summer 2024 Natural Language Prorcessing (CIS 5300 - Penn Engineering Online) 131 3.5 3.7
Summer 2024 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 115 3.1 3.4
Spring 2024 Interactive Fiction and Text Generation (CIS 7000) 53 3.4 3.7
Fall 2023 Artificial Intelligence (CIS 4210/5210 - on campus) 373 3.1 3.4
Fall 2023 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 186 3.0 3.5
Fall 2023 Natural Language Processing (CIS 5300 - Penn Engineering Online) 152 3.2 3.6
Fall 2023 Large Language Models and Programming Languages (CIS 8000) 12 3.5 3.5
Summer 2023 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 107 3.4 3.5
Summer 2023 Natural Language Processing (CIS 5300 - Penn Engineering Online) 45 3.3 3.5
Fall 2022 Artificial Intelligence (CIS 4210/5210 - on campus) 363 3.3 3.5
Fall 2022 Artificial Intelligence (CIS 5210 - Penn Engineering Online) 94 3.4 3.6
Fall 2022 Research Practicum (CIS 8000) 16 3.5 3.6
Summer 2022 Artificial Intelligence (CIS 521 - Penn Engineering Online) 70 3.5 3.7
Spring 2022 Interactive Fiction and Text Generation (CIS 700-001) 53 3.3 3.5
Fall 2021 Artificial Intelligence (CIS 521 - MCIT Online) 234 3.2 3.4
Fall 2021 Artificial Intelligence (CIS 421/521 - on campus - section 1) 180 3.2 3.4
Fall 2021 Artificial Intelligence (CIS 421/521 - on campus - section 2) 138 3.2 3.5
Fall 2021 Artificial Intelligence (CIS 421/521 - online only section for foreign graduate students unable to return to campus due to the pandemic) 11 2.5 2.3
Summer 2021 Artificial Intelligence (CIS 521 - MCIT Online) 49 3.0 3.6
Spring 2021 Crowdsourcing and Human Computation (NETS 213) 146 3.0 3.3
Fall 2020 Artificial Intelligence (CIS 421/521) 197 3.1 3.3
Spring 2020 Computational Linguistics (CIS 530) 125 3.3 3.3
Spring 2020 Interactive Fiction and Text Generation (CIS 700-008) 23 3.1 3.3
Fall 2019 Artificial Intelligence (CIS 421/521) 148 3.1 3.3
Summer 2019 Artificial Intelligence (CIS 421/521) 36 2.9 3.0
Spring 2019 Computational Linguistics (CIS 530) 75 2.8 3.0
Spring 2019 Crowdsourcing and Human Computation (NETS 213) 59 2.5 2.7
Fall 2018 Artificial Intelligence (CIS 421/521) 101 2.5 2.5
Spring 2018 Computational Linguistics (CIS 530) 64 2.8 2.7
Fall 2017 Data Structures and Algorithms (CIS 121) 216 2.1 1.7
Fall 2016 Data Structures and Algorithms (CIS 121) 219 2.5 2.2
Spring 2016 Crowdsourcing and Human Computation (NETS 213) 113 2.4 2.8
Fall 2015 Data Structures and Algorithms (CIS 121) 174 2.2 2.2
Spring 2015 Machine Translation (CIS 526) 51 2.9 3.2
Fall 2014 Crowdsourcing and Human Computation (NETS 213) 48 3.2 3.6
Spring 2014 Machine Translation (CIS 526) 25 3.3 3.5
Fall 2013 Crowdsourcing and Human Computation (CIS 399) 26 3.1 3.5

Awards

Research Awards from Companies (given as gifts to the university for my research group)

Graduate Student Supervision

Current PhD Students

  1. Grace Kim. Expected graduation date: Summer 2030
  2. Maxine Liu. Expected graduation date: Summer 2030
  3. Delip Rao. Expected graduation date: Summer 2029
  4. Andrew Zhu. Expected graduation date: Summer 2027
  5. Liam Dugan. Expected graduation date: Summer 2026
  6. Ajay Patel. Expected graduation date: Summer 2025
  7. Bryan Li. Expected graduation date: Summer 2025
  8. Samar Haider. Co-advised by Duncan Watts. Expected graduation date: Summer 2025

PhDs Graduated

  1. Aditya Kashyap, University of Pennsylvania (advisors: Chris Callison-Burch and Mary Regina Boland), "Applying Language Models to Patient Health Records: Acronym Expansion, Long Document Classification and Explainable Predictions", January 2025. Current position: Biomedical NLP Engineer at Selfii.
  2. Veronica Qing Lyu, University of Pennsylvania (advisors: Chris Callison-Burch and Marianna Apidianaki), "Faithful and Useful Explanations by Large Language Models", August 2024. Current position: Researcher at Databricks.
  3. Harry Li Zhang, University of Pennsylvania (advisor: Chris Callison-Burch), "Structured Event Reasoning with Large Language Models", May 2024. Current position: Assistant Professor at Drexel University.
  4. Daphne Ippolito, University of Pennsylvania (advisors: Chris Callison-Burch and Doug Eck), "Understanding the Limitations of Using Large Language Models for Text Generation", September 2022. Current position: Assistant Professor at Carnegie Mellon University.
  5. Reno Kriz, University of Pennsylvania (advisors: Chris Callison-Burch and Marianna Apidianaki), "Towards a Practically Useful Text Simplification System", June 2021. Current position: Research Scientist at Human Language Technology Center of Excellence.
  6. Anne Cocos, University of Pennsylvania (advisors: Chris Callison-Burch and Marianna Apidianaki), "Paraphrase-based Models of Lexical Semantics", May 2019. Current position: Senior Research Scientist at Netflix.
  7. Courtney Napoles, Johns Hopkins University (advisors: Chris Callison-Burch and Benjamin Van Durme), "Monolingual Sentence Rewriting as Machine Translation: Generation and Evaluation", June 2018. Current position: Engineering Director, NLP/ML/AI at Grammarly.
  8. Juri Ganitkevitch, Johns Hopkins University (advisor: Chris Callison-Burch), "Large-Scale Paraphrase Extraction and Applications", February 2018. Current position: Chief Scientist & Co-Founder at Espresso AI.
  9. Ellie Pavlick, University of Pennsylvania (advisor: Chris Callison-Burch), "Compositional Lexical Semantics in Natural Language Inference", July 2017. Current position: Assistant Professor at Brown University.
  10. Ann Irvine, Johns Hopkins University (advisor: Chris Callison-Burch), "Using Comparable Corpora to Augment Low Resource Statistical Machine Translation Models", July 2014. Current position: Chief Data Scientist & VP of Product Management at Resilience.
  11. Xuchen Yao, Johns Hopkins University (advisors: Benjamin Van Durme and Chris Callison-Burch), "Feature-Driven Question Answering with Natural Language Alignment", July 2014. Current position: Cofounder/CEO at Seasalt.ai and Vobil.com.
  12. Omar Zaidan, Johns Hopkins University (advisor: Chris Callison-Burch), "Crowdsourcing Annotation for Machine Learning in Natural Language Processing Tasks", April 2012. Current position: Senior Applied Scientist at Amazon.

Postdocs

  1. Lara Martin, PhD from Georgia Tech, postdoc at University of Pennsylvania from 2021 through August 2023. Current position: Assistant Professor at University of Maryland, Baltimore County.
  2. Mohammad Sadegh Rasooli, PhD from Columbia University, postdoc at University of Pennsylvania from January 2020 through July 2021. Current position: Senior Applied Scientist at Microsoft.
  3. Derry Wijaya, PhD from Carnegie Mellon University, postdoc at University of Pennsylvania from 2016 through August 2018. Current position: Assistant Professor at Boston University.
  4. Anietie Andy, PhD from Howard University, postdoc at University of Pennsylvania from January 2017 through June 2018. Current position: Assistant Professor at Howard University.
  5. Wei Xu, PhD from New York University, postdoc at University of Pennsylvania from February 2014 through August 2016. Current position: Assistant Professor at Georgia Tech.
  6. Matt Post, PhD from University of Rochester, postdoc at Johns Hopkins University from September 2010 through June 2012. Current position: Researcher at Microsoft Translator.
  7. Alex Klementiev, PhD from UIUC, postdoc at Johns Hopkins University from November 2009 through July 2011. Current position: Principal Applied Scientist at Amazon.

Master's Students

  1. Yifei Li, University of Pennsylvania (advisors: Chris Callison-Burch and Mark Yatskar), "Improving Text-to-image Diffusion Generation Via Large Language Models", May 2023.
  2. River Yijiang Dong, University of Pennsylvania (advisors: Chris Callison-Burch and Lara Martin), "COTTAGE: Coherent Text Adventure Games Generation", May 2023.
  3. Hainiu Xu, University of Pennsylvania (advisors: Chris Callison-Burch and Harry Li Zhang), "Fine-Grained And Coarse-Grained Causal Reasoning In Procedural Texts", May 2023.
  4. Anshul Wadhawan, University of Pennsylvania (advisors: Chris Callison-Burch and Liam Dugan), "Simultaneous Speech To Speech Translation", May 2023.
  5. Anna Orosz, University of Pennsylvania (advisors: Chris Callison-Burch and Lara Martin), "Generating Text-based Adventure Games", December 2021.
  6. Sri Sanjeevini Devi Ganni, University of Pennsylvania (advisors: Chris Callison-Burch and Lara Martin), "Narratology and Fan Fiction", May 2021.
  7. Jacob Beckerman, University of Pennsylvania (advisor: Chris Callison-Burch), "Graph Algorithms and Visualization of Complex Legal Contracts", December 2020.
  8. Liam Dugan, University of Pennsylvania (advisor: Chris Callison-Burch), "Learning Formality from Japanese-English Parallel Corpora", May 2020.
  9. Yonah Mann, University of Pennsylvania (advisor: Chris Callison-Burch), "A Data Set for Training QA Systems to Answer Questions about Novels", May 2020.
  10. Maria Kustikova, University of Pennsylvania (advisors: Chris Callison-Burch and Kostas Daniilidis), "Clustering Paraphrases by Word Sense Using Textual and Visual Information", May 2019.
  11. Devanshu Jain, University of Pennsylvania (advisor: Chris Callison-Burch), "Machine Transliteration", May 2018.
  12. Aditya Kashyap, University of Pennsylvania (advisor: Chris Callison-Burch), "Generalizable Identity Classifiers from Self-Reporting Statements on Reddit", May 2018.
  13. Brendan Daniel Callahan, University of Pennsylvania (advisor: Chris Callison-Burch), "Image-based Bilingual Lexicon Induction for Low Resource Languages", May 2017.
  14. Sneha Rajana, University of Pennsylvania (advisor: Chris Callison-Burch), "Learning Antonyms with Paraphrases and a Morphology-Aware Neural Network", May 2017.
  15. Mingkun Gao, University of Pennsylvania (advisor: Chris Callison-Burch), "Crowdsourcing Machine Translation", May 2015.

Academic Service

Service to professional associations:

University Service:

Press

  1. When ChatGPT Broke an Entire Field: An Oral History – Quanta Magazine. April 30, 2025
  2. ‘WithdrarXiv’ database of 14,000 retracted preprints launches – Nature News. January 6, 2025
  3. ‘WithdrarXiv’ database of 14,000 retracted preprints launches – Nature News. January 6, 2024
  4. What you need to know about AI, misinformation and the election – PBS NewsHour. November 1, 2024
  5. Training Medical AI with Knowledge, Not Shortcuts – Penn Engineering. October 15, 2024
  6. AI detectors are easily fooled, researchers find – EdScoop. September 9, 2024
  7. Teachers still can't trust AI text checkers – Axios. September 3, 2024
  8. Most AI text detectors aren’t as reliable as advertised, study finds – Emerging Tech Brew. August 22, 2024
  9. Recreating ‘Star Trek’ Virtual Environments: Holodeck generates a virtually limitless range of indoor environments, using AI to interpret users’ requests. – Tech Briefs. June 11, 2024
  10. Artificial Intelligence 101: Understanding the power of AI and its dangers – ABC 6 Action News WPVI-TV Philadelphia. June 4, 2024
  11. Should We Trust AI? A Penn Expert Weighs In – Philadelphia Magazine. June 8, 2024
  12. New Penn AI master’s program aims to prep students for ‘jobs that we can’t yet imagine’ – Philadelphia Inquirer. May 2, 2024
  13. Penn Engineering rolls out an online master’s degree in AI, first in Ivy League – Technical.ly. April 30, 2024
  14. Training Medical AI with Knowledge, Not Shortcuts – Penn Engineering. October 15, 2024
  15. Teachers still can't trust AI text checkers – Axios. September 3, 2024
  16. Detecting Machine-Generated Text: An Arms Race With the Advancements of Large Language Models – Penn Engineering Today. August 12, 2024
  17. Recreating ‘Star Trek’ Virtual Environments: Holodeck generates a virtually limitless range of indoor environments, using AI to interpret users’ requests. – Tech Briefs. June 11, 2024
  18. Copyright Law and Generative AI: What a mess – American Bar Association Journal. August 30, 2023
  19. What does Congress need to do amid AI boom? – Fox News. May 20, 2023
  20. Hallucinating ChatGPT finds a role playing Dungeons & Dragons – The Register. August 19, 2023
  21. Bot or not? How to tell when you’re reading something written by AI – CNN. July 11, 2023
  22. NewsChannel 12 Investigates: Artificial Intelligence Part 3 – Channel 12 News - ABC affiliate in North Carolina. May 19, 2023
  23. Real or Fake Text? We can learn to spot the difference – Penn Today. March 10, 2023
  24. A Bot Isn’t Going to Take Your Place, But AI Will Make Your Job Harder – Corporate Compliance Insights. March 8, 2023
  25. How can humans detect AI writing? These Penn researchers have some tips – Technical.ly. March 6, 2023
  26. Newly Unemployed, and Labeling Photos for Pennies: People who've lost jobs and are stuck indoors are turning to crowd work—filling out online surveys and transcribing audio for less than the minimum wage. – Wired Magazine. April 23, 2020
  27. America’s Gig-Based Economy Gets Zero – Full Frontal with Samantha Bee. March 11, 2020
  28. I Found Work on an Amazon Website. I Made 97 Cents an Hour. – New York Times. November 15, 2019