8:00-8:10 Opening

8:10-9:00 Keynote

9:00-10:30 Session 1: Visual Interactive Exploration (Session Chair: Alan Fekete)

• Human-in-the-Loop Data Analysis: A Personal Perspective. Anhai Doan (University of Wisconsin-Madison) (toggle abstract) In the past few years human-in-the-loop data analysis (HILDA) has received significant growing attention. Most HILDA works have focused on concrete problems. In this paper I take a step back and discuss several ``big picture'' questions regarding HILDA. First, I discuss problems that I believe should fall under the scope of the field, including some that have received little attention, such as fostering user communities that develop data repositories and tools. Next, I discuss important aspects in developing HILDA solutions that I believe should receive more attention. These include solving problems that real users care about, developing how-to guides to users, building end-to-end systems (such as extending the ``Pandas system''), developing benchmarks and challenges, and developing a theory of human data interaction. Finally, I speculate about the future of the field, and discuss the dangers it can face, given that many other communities are also working on related problems. I argue that a focus on end-to-end problems and system building is important for us to thrive and make significant impacts.
• ViDeTTe Interactive Notebooks. Costas Zarifis (University of California San Diego) (toggle abstract) Interactive notebooks allow the use of popular languages, such as python, for composing data analytics projects. The interface they provide, enables data scientists to import data, analyze them and compose the results into easily readable report-like web pages, that can contain re-runnable code, visualizations and textual description of the entire process, all in one place. Scientists can then share such pages with other users in order to present their findings, collaborate and further explore the underlying data. However, as we show in this work, interactive notebooks lack in interactivity and ease of use, both for the data scientist that composes the notebook and, even more so, for the reader of the published notebook. First, utilizing libraries that obtain, process and visualize data, requires technical expertise that often exceeds the skill-set of a typical data scientist. Second, while the user interface allows readers to rerun or extend the code included in the notebook, it does not allow them to directly interact with the generated visualizations in order to trigger additional computation and further explore the underlying data. This means that only code-literate readers can further interact with and extend such notebooks, while the rest can only passively read the provided report. To address these issues, we propose ViDeTTe, an engine that enhances interactive notebooks with capabilities that benefit both data scientists and non-technical notebook readers. ViDeTTe uses a declarative language to simplify data retrieval and data visualization for analysts. The generated visualizations are capable of collecting the reader's input and reacting to it. Additionally, the user input is utilized by ViDeTTe's propagation algorithm, to identify subsequent parts of the notebook that depend on it and cause their reevaluation. By doing this, ViDeTTe offers enhanced data exploratory capabilities to readers, without requiring any coding skills.
• Towards a Unified Representation of Insight in Human-in-the-Loop Analytics: A User Study. Eser Kandogan (IBM) and Ulrich Engelke (CSIRO) (toggle abstract) Understanding what insights people draw from data visualizations is critical for human-in-the loop analytics systems to facilitate mixed-initiative analysis. In this paper we present results from a large user study on insights extracted from commonly used charts. We report several patterns of insights we observed and analyze their semantic structure to identify key considerations towards a unified formal representation of insight, human or computer generated. We also present a model of insight generation process, where humans and computers work cooperatively, building on each other's knowledge, where a common representation acts as the currency of interaction. While not going as far as proposing a formalism, we pointed to a few potential directions for representing insight. We believe our findings could also inform the design of novel human-in-the-loop analytics systems.
• Evaluating Visual Data Analysis Systems: A Discussion Report. Leilani Battle (University of Washington), Marco Angelini (Sapienza University of Rome), Carsten Binnig (TU Darmstadt), Tiziana Catarci (Sapienza University of Rome), Philipp Eichmann (Brown University), Jean-Daniel Fekete (INRIA), Giuseppe Santucci (Sapienza University of Rome), Michael Sedlmair (Jacobs University Bremen) and Wesley Willett (University of Calgary) (toggle abstract) Visual data analysis is a key tool for helping people to make sense of and interact with massive data sets. However, existing evaluation methods (e.g., database benchmarks, individual user studies) fail to capture the key points that make systems for visual data analysis (or visual data systems) challenging to design. In November 2017, members of both the Database and Visualization communities came together in a Dagstuhl seminar to discuss the grand challenges in the intersection of data analysis and interactive visualization. In this paper, we report on the discussions of the working group on the evaluation of visual data systems, which addressed questions centered around developing better evaluation methods, such as "How do the different communities evaluate visual data systems?" and "What we could learn from each other to develop evaluation techniques that cut across areas?". In their discussions, the group brainstormed initial steps towards new joint evaluation methods and developed a first concrete initiative - a trace repository of various real-world workloads and visual data systems - that enables researchers to derive evaluation setups (e.g., performance benchmarks, user studies) under more realistic assumptions, and enables new evaluation perspectives (e.g., broader meta analysis across analysis contexts, reproducibility and comparability across systems).
• (Vis Abstract) Visual Exploration of Deep Learning Models in Industry. Minsuk Kahng (Georgia Institute of Technology), Pierre Andrews (Facebook), Aditya Kalro (Facebook) and Duen Horng Chau (Georgia Institute of Technology) (toggle abstract) While deep learning has led to major breakthroughs in various domains, understanding these models remains a challenge. Despite the increasing interest in developing visualization tools for deep learning interpretation, the complexity of large-scale models and datasets used in industry pose unique design challenges. It motivates us to design and develop ActiVis, a visual analytics system for deep neural network models, now deployed on Facebook's machine learning platform. With ActiVis, users can get a high-level overview of a model and drill down into its activations at both instance- and subset-level. Our one-page abstract has been uploaded to the system. This abstract paper is a summary for the paper presented at the IEEE VIS'17 conference, titled "ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models."

10:30-11:00 Coffee Break (Location: Houston Foyer)

11:10-12:30 Session 2: Engines and Languages (Session Chair: Oliver Kennedy)

• DIVE: A Mixed-Initiative System Supporting Integrated Data Exploration Workflows. Kevin Hu (Massachusetts Institute of Technology), Diana Orghian (Massachusetts Institute of Technology) and César Hidalgo (Massachusetts Institute of Technology) (toggle abstract) Generating knowledge from data is an increasingly important activity. This process of data exploration consists of multiple tasks: data ingestion, visualization, statistical analysis, and storytelling. Though these tasks are complementary, analysts often execute them in separate, standalone tools. Moreover, these tools have steep learning curves due to their reliance on manual query specification. Here, we describe the design and implementation of DIVE, a web-based system that integrates state-of-the-art data exploration features into a single tool. DIVE contributes a mixed-initiative interaction scheme that combines recommendation with point-and-click manual specification, and a consistent visual language that unifies different stages of the data exploration workflow.
• Querying Videos Using DNN Generated Labels by Designing DSL driven UIs. Wu Yifan (University of California, Berkeley), Steven Drucker (Microsoft), Matthai Philipose (Microsoft) and Lenin Ravindranath (Microsoft) (toggle abstract) Massive amounts of videos are generated for entertainment, security, and science, powered by a growing supply of user-produced video hosting services. Unfortunately, searching for videos is complicated due to the lack of content annotations. While many have tackled this problem before, recent breakthroughs in image labeling with deep neural networks (DNNs) create a unique opportunity for re-imagining the querying of videos. To exploit this opportunity, We introduce a new query language that operates on the output labels of DNNs trained for object recognition. By applying visualization techniques, we also create a graphical query interface that supports fast refinement of queries, which helps to overcome limitations in label consistency and accuracy.
• Optimally Leveraging Density and Locality for Exploratory Browsing and Sampling. Albert Kim (Massachusetts Institute of Technology), Liqi Xu (University of Illinois Urbana-Champaign), Tarique Siddiqui (University of Illinois Urbana-Champaign), Silu Huang (University of Illinois Urbana-Champaign), Samuel Madden (Massachusetts Institute of Technology) and Aditya Parameswaran (University of Illinois Urbana-Champaign) (toggle abstract) Exploratory data analysis often involves repeatedly browsing a small sample of records that satisfy certain predicates. However, existing database systems are not optimized for such queries. We propose a fast query evaluation engine, called NeedleTail, aimed at letting analysts browse a subset of the query result on large datasets as quickly as possible, independent of the overall size of the result. NeedleTail introduces density maps, a lightweight in-memory indexing structure, and a set of efficient and theoretically sound algorithms to quickly locate promising blocks, trading of locality and density. In settings where the samples are used to compute aggregates, we extend techniques from survey sampling to mitigate the bias in our samples. Our experimental results demonstrate that NeedleTail returns results 7× faster on average and can achieve the same error rate as random sampling but in much less time.
• Source Selection Languages: A Usability Evaluation. Ixent Galpin (Universidad Jorge Tadeo Lozano), Edward Abel (The University of Manchester) and Norman Paton (The University of Manchester) (toggle abstract) When looking to obtain insights from data, and given numerous possible data sources, there are certain quality criteria that retrieved data from selected sources should exhibit so as to be most fit-for-purpose. An effective source selection algorithm can only provide good results in practice if the requirements of the user have been suitably captured, and therefore, an important consideration is how users can effectively express their requirements. In this paper, we carry out an experiment to compare user performance in two different languages for expressing user requirements in terms of data quality characteristics, pairwise comparison of criteria values, and single objective constrained optimization. We employ crowd-sourcing to evaluate, for a set of tasks, user ability to choose effective formulations in each language. The results of this initial study show that users were able to determine more effective formulations for the tasks using pairwise comparisons. However, user usability evaluations show that the languages were evaluated similarly in terms of subjective user preference.
• Provenance for Interactive Visualizations. Fotis Psallidas (Columbia University) and Eugene Wu (Columbia University) (toggle abstract) We highlight the connections between data provenance and interactive visualizations. To do so, we first incrementally add interactions to a visualization and show how these interactions are readily expressible in terms of provenance. We then describe how an interactive visualization system that natively supports provenance can be easily extended with novel interactions.
• (Vis Abstract) A Visual Interaction Framework for Dimensionality Reduction Based Data Exploration. Marco Cavallo (IBM) and Cagatay Demiralp (IBM) (toggle abstract) Dimensionality reduction is a common method for analyzing and visualizing high-dimensional data. However, reasoning dynamically about the results of a dimensionality reduction is difficult. Dimensionality-reduction algorithms use complex optimizations to reduce the number of dimensions of a dataset, but these new dimensions often lack a clear relation to the initial data dimensions, thus making them difficult to interpret. Here we propose a visual interaction framework to improve dimensionality-reduction based exploratory data analysis. We introduce two interaction techniques, forward projection and backward projection, for dynamically reasoning about dimensionally reduced data. We also contribute two visualization techniques, prolines and feasibility maps, to facilitate the effective use of the proposed interactions. We apply our framework to PCA and autoencoder-based dimensionality reductions. Through data-exploration examples, we demonstrate how our visual interactions can improve the use of dimensionality reduction in exploratory data analysis.

12:30-14:00 Lunch (Location: Houston 123)

14:00-15:30 Session 3: Data Curation & Quality (Session Chair: Carlos Scheidegger)

• Beaver: Towards a Declarative Schema Mapping. Zhongjun Jin (University of Michigan), Christopher Baik (University of Michigan), Michael Cafarella (University of Michigan) and H V. Jagadish (University of Michigan). (toggle abstract) Schema mapping describes how to transform data of the source schemas into the data of the target schema. Manually writing com- plete schema mapping speci cations requires a deep understanding of the source and target schema which can be burdensome for the user. Programming By Example (PBE) schema mapping systems have attempted to reduce this burden by allowing the user to declar- atively specify the schema mapping using a few sample records. However, specifying real records can sometimes be di cult for the user too, whereas the user might have other useful insights about the desired schema mapping. In this paper, we develop a new schema mapping technique, Beaver, that enables an interaction model that gives the user far more exibility in describing the de- sired schema mapping. The end user is not limited to providing target schema examples but may also provide other types of de- scriptions, such as the data type or value range of target schema columns. Moreover, beyond constraining the target schema itself, the user can provide constraints for the schema mapping process. We propose a search algorithm with on-line pruning that e ciently solves the schema mapping problem. We implemented a prototype of our schema mapping technique and experimentally evaluated the e ciency of the system in handling traditional PBE schema map- ping test cases, as well as our newly-proposed declarative schema mapping test cases. The experiment results suggest that the end user can provide declarative queries which we believe are easier to input and retain a reasonable system e ciency as they have for traditional PBE queries.
• SchemaDrill: Interactive Semi-Structured Schema Design. William Spoth (University at Buffalo), Ting Xie (University at Buffalo), Oliver Kennedy (University at Buffalo), Dieter Gawlick (Oracle), Ying Yang (Oracle), Zhen Hua-Liu (Oracle) and Beda Hammerschmidt (Oracle) (toggle abstract) Ad-hoc data models like Json make it easy to evolve schemas and to multiplex different data-types into a single stream. This flexibility makes Json great for generating data, but also makes it much harder to query, ingest into a database, and index. In this paper, we explore the first step of Json data loading: schema design. Specifically, we consider the challenge of designing schemas for existing Json datasets as an interactive problem. We present SchemaDrill, a roll-up/drill-down style interface for exploring collections of Json records. SchemaDrill helps users to visualize the collection, identify relevant fragments, and map it down into one or more flat, relational schemas. We describe and evaluate two key components of SchemaDrill: (1) A summary schema representation that significantly reduces the complexity of Json schemas without a meaningful reduction in information content, and (2) A collection of schema visualizations that help users to qualitatively survey variability amongst different schemas in the collection.
• What Type of a Matcher Are You? - Human and Algorithmic Matchers Coordination. Roee Shraga (Technion - Israel Institute of Technology), Avigdor Gal (Technion - Israel Institute of Technology) and Haggai Roitman (IBM Research - AI) (toggle abstract) In this work we explore relationships between human and algorithmic schema matchers. We provide a novel approach to similar schema matchers termed \emph{coordinated matchers} and use it to predict future human matching choices. We show throughout a comprehensive analysis that human matchers are usually coordinated with intuitive algorithms, \emph{e.g.,} based on attribute name similarity, and frequently do not assign lower confidence levels, which indicates over confidence in their choices. Finally, we show that human choices can be reasonably predicted using collaborative algorithmic opinions based on matchers coordination.
• Draining the Data Swamp: A Similarity-based Approach. Will Brackenbury (University of Chicago), Rui Liu (University of Chicago), Mainack Mondal (University of Chicago), Aaron Elmore (University of Chicago), Blase Ur (University of Chicago), Kyle Chard (University of Chicago) and Michael Franklin (University of Chicago) (toggle abstract) While hierarchical namespaces such as filesystems and repositories have long been used to organize data, the rapid increase in data production places increasing strain on users who wish to make use of the data. So called “data lakes” embrace the storage of data in its natural form, integrating and organizing in a Pay-as-you-go fashion. While this model defers the upfront cost of integration, the result is that data is unusable for discovery or analysis until it is processed. Thus, data scientists are forced to spend significant time and energy on mundane tasks such as data discovery, cleaning, integration, and management – when this is neglected, “data lakes” become “data swamps.” Prior work suggests that pure computational methods for resolving issues with the data discovery and management components are insufficient. Here, we provide evidence to confirm this hypothesis, showing that methods such as automated file clustering are unable to extract the necessary features from repositories to provide useful information to end-user data scientists, or make effective data management decisions on their behalf. We argue that the combination of frameworks for specifying file similarity and human-in-the-loop interaction is needed to aid automated organization. We propose an initial step here, classifying several dimensions by which items may be considered similar: the data, its origin, and its current characteristics. We initially consider this model in the context of identifying data that can be integrated or managed collectively. We additionally explore how current methods can be used to automate decision making using real-world data repository and file systems, and suggest how an online user study could be developed to further validate this hypothesis.
• (Vis Abstract) Interactive Visual Analytics for Simpson’s Paradox Detection. Chenguang Xu (University of oklahoma), Sarah Brown (University of California, Berkeley), Christan Grant (University of oklahoma) and Chris Weaver (University of Oklahoma) (toggle abstract) When a trend is observed in aggregate data, Simpson’s paradox is the phenomenon that some or all of the partitions of a data set exhibit the opposite trend. Detecting Simpson’s paradox uncovers the anomaly in a data set, and helps us to make prudent data-driven decisions after looking deeper into the data. The analysis of Simpson’s paradox often requires the comparison of the trends between the whole population and the subpopulations. This comparison is typically the most complicated step of the analysis process, especially for data with high dimensions. We use the bivariate color scheme to accurately illustrate the relationship between the trend of the association in the whole population and within the subpopulations. We provide a novel visual analytics approach to facilitate the exploration of the data to detect Simpson’s paradox. The detection and analysis are tightly coupled by multiple coordinated views. Our multiple views provide users with not only an overview of the entire data set but also details when needed. The interactive features empowered user to do analyses effectively and efficiently. We will demonstrate that using our application, users are able to detect Simpson’s paradox and draw insights from data sets with confidence.

15:30-16:30 Poster Session (Location: Houston 56)

16:30-17:30 Panel (Moderator: Eugene Wu)

17:30-18:00 Closing Remarks

Sponsors