Workshop on Human-In-the-Loop Data Analytics
Co-located with SIGMOD 2019 (5 July 2019, Amsterdam, Netherlands)

Past workshops: HILDA 2018 | HILDA 2017 | HILDA 2016


Any data management system needs to work together with people, whose needs determine the goals for the system, and who must provide the input and who need to work effectively with the output. Data management systems will work much better when they take account of the cognitive and physiological characteristics of the people involved. Recent technology trends (such as touch screens, motion detection, and voice recognition) are widening the possibilities for users to interact with systems, and many information-provision industries are shifting to personalized processing to better target their services to the users’ wishes. HILDA is a workshop that will allow researchers and practitioners to exchange ideas and results relating to how data management can be done with awareness of the people who form part of the processes. A sample of topics that is are in the spirit of this workshop includes, but is not limited to: novel query interfaces, interactive query refinement, data exploration and analysis, data visualization, human-assisted data integration and cleaning, perception-aware data processing, database systems designed for highly interactive use cases, empirical studies of database use, and crowd-powered data infrastructure.

HILDA intends to be a forum where people from varied communities engage with one another's ideas. We are keen to have submissions that present initial ideas and visions, just as much as reports on early results, or reflections on completed projects. The workshop will focus on discussion and interaction, rather than static presentations of what is in the paper.


Authors are invited to submit either of the following: (1) original, unpublished research papers that are not being considered for publication in any other forum. These will be published in the HILDA proceedings, or (2) abstracts of previous work presented in other venues ideally outside the database community.

Abstracts will be considered for discussion at the workshop only and not be included in the HILDA proceedings. Submissions of abstracts must be marked in the title as (Abstract). Research papers must follow the latest ACM Proceedings format (2017). Research papers submitted can be between four and six pages in length, excluding references. Abstracts should be maximally one page long.

Submissions will be handled through EasyChair.

Keynote Speakers

  • Juliana Freire
  • Jean-Daniel Fekete


  • All HILDA papers are available as clickable links below.

  • Important Dates

    • Workshop Date: July 5, 2019
    • Submissions (Research Papers): March 22 16, 2019, 11:59PM US PDT
    • Notification of outcome: April 13, 2019 11:59PM US PDT (before SIGMOD early registration deadline)
    • Camera-ready due: April 27, 2019 11:59PM US PDT

    Related Workshops

Program Chairs

Program Committee

  • Aditya Parameswaran, University of Illinois
  • Adriane Chapman, University of Southampton
  • Anil Bahuman, Reliance Jio Infocomm
  • Carl-Christian Kanne, Platfora
  • Beth Trushkowsky, Harvey Mudd College
  • Cagatay Demiralp, Megagon Labs
  • Carlos Scheidegger, University of Arizona
  • Chengkai Li, University of Texas at Arlington
  • Danyel Fisher,
  • Dana Groff, AWS
  • Dominik Moritz, University of Washington
  • Giuseppe Santucci, Sapienza University of Rome
  • Guoliang Li, Tsinghua University
  • Harish Doraiswamy, NYU Data Science Center
  • Iddo Drori, NYU and Columbia University
  • James Terwilliger, Microsoft Research
  • Jean-Daniel Fekete, Equipe Aviz, INRIA Saclay - Île-de-France
  • Mohamed Sarwat, Arizona State University
  • Nikos Bikakis, University of Ioannina
  • Oliver Kennedy, University at Buffalo
  • Remco Chang, Tufts University
  • Rick Cole, Tableau Software
  • Stratos Idreos, Harvard University
  • Thibault Sellam, Columbia University
  • Tiziana Catarci, Sapienza Universit di Roma
  • Yash Govind, University of Wisconsin-Madison
  • Yunyao Li, IBM Research
  • Zhe Cui, University of Maryland

Steering Committee

  • Carsten Binnig (TU Darmstadt)
  • Juliana Freire (New York University)
  • Joseph M. Hellerstein (University of California, Berkeley)
  • Aditya Parameswaran (University of Illinois)


8:50-9:00 Opening

9:00-9:45 Keynote: "Towards Usability, Transparency, and Trust in Data-Driven Exploration " — Juliana Freire

9:45-10:30 Keynote: "Multidimensional Projection at Scale on the Web: Tips and Tricks" — Jean-Daniel Fekete

10:30-11:00 Coffee Break

11:00-12:20 Session 1: Data Cleaning and Entity Resolution (Session Chair: Thibault Sellam)

  • Reinforcement Learning for Data Cleaning and Data Preparation (Invited Talk) [pdf] Laure Berti-Equille
  • Towards an End-to-End Human-Centric Data Cleaning Framework [pdf] El Kindi Rezig, Mourad Ouzzani, Ahmed Elmagarmid, Walid Aref and Michael Stonebraker
  • Effective and Efficient Data Cleaning for Entity Matching [pdf] Jing Ao and Rada Chirkova
  • UserDEV: A Mixed-Initiative System for User Group Analyticsrescheduled [pdf] Behrooz Omidvar-Tehrani, Sihem Amer-Yahia, Eric Simon, Fabian Colque Zegarra, Joao Comba and Viviane Moreira

12:20-14:00 Lunch

14:00-15:20 Session 2: Human-in-the-loop Learning (Session Chair: Richard Lenz)

  • Learning to Validate the Predictions of Black Box Machine Learning Models on Unseen Data [pdf] Sergey Redyuk, Sebastian Schelter, Felix Biessmann, Tammo Rukat and Volker Markl
  • OPaPi: Optimized Parts Pick-up routing for efficient manufacturing [pdf] Chidansh Bhatt, Jian Zhao, Hideto Oda, Francine Chen and Matthew Lee
  • Visus: An Interactive System for Automatic Machine Learning Model Building and Curation [pdf] Aécio Santos, Sonia Castelo Quispe, Cristian Felix, Jorge Piazentin Ono, Bowen Yu, Sungsoo Hong, Cláudio Silva, Enrico Bertini and Juliana Freire
  • Knowledge Graph Programming with a Human-in-the-Loop: Preliminary Results [pdf] Yuze Lou, Mahfus Uddin, Nathaniel Brown and Michael Cafarella

15:20-16:30 Poster Session (All Workshops)

16:30-17:50 Session 3: Text, Graphs, and Groups (Session Chair: Carsten Binnig)

  • Visualizing Session-Based Data Profiles (Invited Talk) [pdf] Andreas M. Wahl, Christian Sauerhammer and Richard Lenz
  • A Collaborative Framework for Structure Identification over Print Documents [pdf] Maeda Hanafi, Miro Mannino and Azza Abouzied
  • Interactive Summarization of Large Document Collections [pdf] Benjamin Hättasch, Christian Meyer and Carsten Binnig
  • Explaining Entity Resolution Predictions : Where are we and What needs to be done?rescheduled [pdf] Saravanan Thirumuruganathan, Mourad Ouzzani and Nan Tang

17:50-18:00 Closing Remarks



For questions, email us at

Follow us

Join us on Twitter.