Research Teaching Publications Advisees & Collaborators DB Group

Zachary G. Ives

Adani President's Distinguished Professor and Department Chair
Computer & Information Science Department
University of Pennsylvania

Warren Center for Network and Data Science
Center for Neuroengineering and Therapeutics
Center for Health, Devices & Technology
Distinguished Research Fellow, Annenberg Center for Public Policy


Contact Information

305 Levine Hall
Computer and Information Science Department
University of Pennsylvania
3330 Walnut Street
Philadelphia, PA 19104-6389
(215) 746-2789    Fax: (215) 898-0587
Twitter: @zgives

Teaching Big Data Analytics
Office Hours for the Spring: Mon 3:00pm-4:00pm Eastern or book via Google Calendar

Biographical Sketch

Zachary Ives is the Department Chair and Adani President's Distinguished Professor of Computer and Information Science at the University of Pennsylvania. Zack's research interests include data integration and sharing, data provenance and trustworthiness, and machine learning systems. He is a recipient of the NSF CAREER award, and an alumnus of the DARPA Computer Science Study Panel and Information Science and Technology advisory panel.  He has also been awarded the Christian R. and Mary F. Lindback Foundation Award for Distinguished Teaching and an IEEE Technical Committee on Data Engineering Education Award, and he is a Fellow of the ACM. He is a co-author of the textbook Principles of Data Integration, and has received an ICDE 2013 ten-year Most Influential Paper award as well as the 2017 SWSA Ten-Year Award at the International Semantic Web Conference. He has served as the Program Co-Chair and General Chair for the ACM SIGMOD conference, and has been an Associate Editor for the Proceedings of the VLDB Endowment and the VLDB Journal.

CIS@Penn... And Amy Gutmann Hall and the IDEAS Initiative

Exciting things are happening in Computer and Information Science at Penn, as we continue to start revolutionary new projects, recruit new students and faculty, and continue on our period of high-speed growth. We have a new building on the way and a major commitment from the university towards expansion in AI for data science (with the new ASSET Center in Safe and Trustworthy AI as one of the first research initiatives). We are welcoming 10 new assistant professors between July 2022 and July 2023, and we are just getting started! Want to know more about the great things happening in the department? Please check out our Highlights site, and follow me on Twitter (@zgives)! Please also see our faculty and lecturer ads.


My research interests lie in building data science platforms, using techniques at the intersection of databases, machine learning, and distributed systems. I am interested in applications both to the Web and question answering, and to conducting data science. I often work with life scientists (especially in genetics and neuroscience) to evaluate our techniques with real data and real hypotheses.

The power of the Web and conventional search is limited, because Web search does not reason about relationships between facts. Question answering and data analysis systems need better techniques for integrating data from multiple sources, and reasoning about certainty. Similarly, we are at still in the early stages of building the "right" tools for data science, that let us link data, rapidly pose and evaluate hypotheses, and ensure we have trustworthy results. I'm interested in questions such as:

I am a member of the database and systems research groups, the Warren Center for Network and Data Science, and the Center for Health, Devices, and Technology at Penn. My research projects relate to making it easier to exchange, locate, and analyze networked information.

Automatically Structuring and Searching Data Lakes and Data Corpora. As we collect large sets of related, multi-versioned data and documents --- what are the mechanisms by which we can automatically order, organize, integrate, and process them? Can we use machine learning (and modern notions like embeddings) to help with this process? What indexing and incremental update mechanisms can we develop?

Understanding Claims, Quotes, and Discussion in Documents and Social Media. When someone makes a statement in an article, what (in the article, in the Web at large) backs up that claim? In collaboration with Prof. Dan Roth and Dr. Yi Zhang, we are studying the questions of the provenance of claims, both in terms of sources and text, as well as in tabular data. Looking more broadly (with Dr. Wang-Chiew Tan at Meta), we are also interested in understanding the discussion revolving around a claim or a quote in an article, as it happens in social media.

Facilitating Data Management and Reuse in Data Science. Today the predominant mode of interacting with data has changed: rather than working with highly controlled, regularized databases, data scientists tend to work with a variety of different data sources within computational notebook software such as Jupyter Notebook and JupyterLab. Such software allows for ad hoc discovery as well as for the creation of sophisticated data analyses and machine learning models. A key issue becomes the management of the many data products (tables, dataframes, models) produced; and there is a key opportunity to help new users understand prior best-practices in using, importing, cleaning, extracting, and analyzing datasets. The Juneau project addresses these issues. Funded by NSF III-1910108.

Our collaborations with neuroscientists (esp. Profs Brian Litt in Bioengineering and Neurology, and Joost Wagenaar in Biostatistics, Informatics, and Epidemiology) has received a good deal of notice for its impact on data science:

Several prior projects have resulted in building blocks towards our ongoing work in supporting large-scale data integration and analysis. These projects are no longer directly active, but their core ideas (and code) are part of our more recent projects:

Trustworthy Data Science. For any type of data science computation, the "glue" that links results to how they were derived is data provenance. Provenance explains the steps involved in the results, as well as what facts went into which conclusion. However, we need to develop better tools for collecting provenance in a convenient way; for reasoning about data's value given its provenance; for recommending related data; and broadly to assess trustworthiness of data analysis results. Funded by NSF (CiCi) and NIH (BD2K Targeted Software) and in collaboration with biologists at Penn, clinicians at UCSF, and computer scientists and computer engineers at U Memphis, Georgia Tech, and UCLA.

Question Answering Over Integrated Data. The Q query system addresses the challenges of querying in a system like Orchestra, when one does not know apriori where to find the most relevant data.  Q takes as input a keyword query, which it matches against schema elements to produce potential data integration queries.  The system returns answers from the most promising queries and takes user feedback on the results.  This feedback is used to learn which sources are most relevant to the information need that motivated the query.  Funded by NSF CAREER #IIS-0477972, SEIII #IIS-0513778, and grants from Google. We are now applying the lessons of the Q System to "the real world" with scientific data.

Developing a Testbed for Data Science. The IEEG Web Portal, in collaboration with Prof. Brian Litt of Bioengineering and Neurology, and Prof. Greg Worrell at Mayo Clinic, seeks to enable community-scale data integration and cloud-hosted science for epileptic seizure prediction (and beyond). Beyond its scientific applications, IEEG serves as a testbed for technologies from the Q System and other data integration research. As of Oct 2014 we have over 1200 datasets and 450 users. We have also hosted competitions for epileptic seizure detection and epileptic seizure prediction. Funded by NIH as well as grants from Amazon.

ORCHESTRA focuses on the problem of collaborative data sharing:  exchanging data and updates among loose confederations of databases, when the different database owners have different schemas and different ideas of what is the "right" content. We have developed techniques to map data and updates among different sites, maintain data provenance, and use the data provenance as the basis of assessing trust and ultimately to resolve conflicts.  We specifically target biological data sharing applications.  See here for an overview paper. Funded by NSF CAREER #IIS-0477972.
Aspen addresses the problem of programming and integrating large-scale and complex sensor networks. The system focuses on a setting in which large numbers of distributed sensors, with varying capabilities, must be coordinated in order to manage and reason about collections of physical entities and phenomena. My focus is on sensor data integration, i.e., integration of data streams from multiple sensor (and other) sources. A target application is data center monitoring for energy, temperature, load, and other factors. Different aspects of the research are funded by NSF III #IIS-0713267, NOSS #CNS-0721541, and a University Research Initiative grant from Lockheed Martin.

Acknowledgments: I have also received grants from DARPA CSSG (#HRO011-06-1-0016 and HRO1107-1-0029), Penn ISTAR, the State of Pennsylvania, Amazon, Google, and Lockheed Martin, and software donations from MarkLogic, Electric Software, and IBM Corp.


I was the first Undergraduate Curriculum Chair for Penn's Singh Program on Networked and Social Systems Engineering, NETS, which was formerly known as MKSE. This Internet-centered degree program looks at how people and systems interact over networks. It combines computer science (algorithms, distributed systems) with sociology, incentives (game theory), and dynamic systems. The overall program is directed by Ali Jadbabaie. New NETS courses I co-developed include NETS (MKSE) 212 "Scalable and Cloud Computing" and NETS (MKSE) 150 "Market and Social Systems on the Internet".

Selected recent courses and seminars:

Detailed information is here.

Textbooks and Monographs

Principles of Data Integration, with AnHai Doan and Alon Halevy. This textbook gives a comprehensive academic treatment of the wide range of topics related to research in data integration: mappings and data transformations, query rewriting, adaptive query processing, XML and streaming data, probabilistic mappings, keyword search, data provenance, and much more. We also describe research challenges, real systems, and implementation techniques. Lecture slides are available from Elsevier. Available from Amazon in hardcopy or Kindle form; from Google Play store in e-book form; from Barnes & Noble in hardcopy or Nook form. Thanks to Xiaofeng Meng, there is also now a Chinese translation of the book.
Adaptive Query Processing, with Amol Deshpande and Vijayshankar Raman. Foundations and Trends in Databases, Vol. 1 No. 1, 2007. Hardcopy available at a discount from Now Publishers; see here.

Selected Publications

A complete list is here.

Current Postdoc, PhD, and MS Research Advisees


Frequent Collaborators

Tips on Interviewing

Finishing your PhD and going on the job market? I have previously compiled a list of reverences on interviewing, which you can find here.