Vogue or Visionary? Current Challenges and Future Opportunities in Situated Visualizations

Situated visualizations are visual data representations that are deeply integrated with the spaces, objects, and activities in a physical environment. Situated visualizations enable people to take advantage of data to support their work or daily activities, while minimizing the cognitive effort of accessing and using that data within physical environments. They display data in close proximity to physical referents, or physical objects to which the data refer. Situated visualizations may be created with a variety of technologies including small displays and mobile devices, augmented reality goggles, handheld projectors, and data physicalizations. Evidence suggests that situated visualization tools can reduce the friction of interacting with data in the context of physical world activities, serve as reminders for data actions, and encourage engagement and reflection.

Yet situated visualizations have not be widely adopted, in part because there is little design guidance. Numerous challenges remain, including context switching (e.g., between primary physical activities and secondary visualization tasks), integration with existing systems (e.g., manufacturing equipment), alignment with physical referents (which may move), potential interference or distraction from primary tasks (which in some contexts could be dangerous), the need for people to learn and adopt new technology into well ingrained workflows and the need to accommodate individual differences and accessibility. New technology is enabling, but is it just a fad? This panel will discuss and debate these questions to clarify the current challenges as well as future opportunities for situated visualizations.

Schedule

The schedule for the panel session will be as follows:

Time Title Speaker
12:00-12:05 Introduction by the Organizers Michelle Borkin, Melanie Tory
12:06-12:13 Presentation 1: Putting the 'Situated' Back into Situated Visualization Nathalie Bressa
12:13-12:20 Presentation 2: Anytime, Anywhere vs. Right Here, Right Now: Rethinking Situated Analytics Niklas Elmqvist
12:20-12:27 Presentation 3: Let's not miss the boat Petra Isenberg
12:27-12:34 Presentation 4: AR displays as a new medium for visualization Michael Sedlmair
12:34-12:41 Presentation 5: Situated Visualization isn't just the Future of Vis, it's the Future of Computing. Wesley Willett
12:41-13:10 Q&A from audience and moderators for an active panel discussion Panelists
13:10-13:15 Wrap-up by the Organizers Michelle Borkin, Melanie Tory
The live-stream for this panel will be accessible via the IEEE VIS 2024 virtual program page.

Panelists

Natahlie's Bio

Nathalie Bressa is an assistant professor at the Institut polytechnique de Paris and Télécom Paris in France. She received her PhD from Aarhus University in Denmark on situated visualization design. Her research lies at the intersection of human-computer interaction and information visualization. She is particularly interested in situated visualization, participatory visualization design, and how visualizations can be used for data input.

Putting the 'Situated' Back into Situated Visualization

We are not yet fully mobilizing the concept of situatedness and what it can offer for information visualization. While much of the current research focuses on the development of enabling technologies, the real potential of situated visualization lies in integrating visualizations into people's actual environments, taking into account their lived experiences in meaningful ways. To fully leverage this potential, we need to rethink how we can situate visualizations—shifting from a technology-driven approach to one that prioritizes social and material circumstances while incorporating a broader understanding of situatedness. We have to critically question why situatedness matters, to what end, and whether it is even desirable at all. Drawing from critical perspectives in visualization, I argue for putting the 'situated' back into situated visualization to make situated visualization more impactful, relevant, and responsive to the complexities of everyday life.

Niklas's Bio

Niklas Elmqvist (he/him/his) is a Villum Investigator, a Fellow of the IEEE, and a full professor in the Department of Computer Science at Aarhus University in Aarhus, Denmark. Prior to joining Aarhus, he was faculty at University of Maryland in College Park, MD, USA from 2014 to 2023, and at Purdue University in West Lafayette, IN, USA from 2008 to 2014. He is the recipient of Villum Investigator and NSF CAREER grants. He was papers chair for IEEE InfoVis 2016, 2017, and 2020, subcommittee chair for ACM CHI 2020 and 2021, and papers chair for IEEE PacificVis and IEEE VIS in 2024. He is a member of the ACM SIGCHI Executive Board as Adjunct Chair for Awards as well as an elected member of the VIS Steering Committee. He was elevated to IEEE Fellow in 2024 and ACM Distinguished Scientist in 2018.

Anytime, Anywhere vs. Right Here, Right Now: Rethinking Situated Analytics

Situated analytics offers significant promise and exciting scientific challenges in data visualization, providing novel ways to integrate and embed data analysis with physical environments. However, I argue that its limited widespread adoption stems primarily from poor everyday applicability. The specialized nature of situated analytics, combined with hardware constraints and the often abstract nature of data, restricts its practical use beyond niche scenarios. As an alternative, I propose a ubiquitous analytics approach, encompassing a broader spectrum of data engagement methods across various mobile devices, novel computer hardware, and more general and varied physical settings. This perspective positions situated analytics as a valuable subset within a larger framework of ubiquitous and immersive analytics. By optimizing for common use-cases rather than specialized scenarios, we can benefit a wider range of users and applications in interactive data analysis while still pursuing the unique opportunities that situated analytics presents.

Petra's Bio

Petra Isenberg (petra.isenberg@inria.fr) is a research director(DR) at the Inria Saclay Centre at Université Paris-Saclay, France in the Aviz team and part of the Computer Science Laboratory (LISN) of the University Paris-Saclay. Prior to joining Inria, she received her PhD from the University of Calgary in 2010 on collaborative information visualization. Petra also holds a Diplom-engineer degree in Computational Visualistics from the University of Magdeburg. Her main research areas are visualization and visual analytics with a focus on visualization for non-desktop devices, interaction, and evaluation. She is particularly interested in exploring how people can most effectively work together when analyzing large and complex data sets on novel display technology such as small touch-screens, wall displays, or tabletops. Petra is associate editor-in-chief at IEEE CG&A and the vice-chair of the IEEE VIS Steering Committee.

Let's not miss the boat

Display technology is evolving quickly and new display form factors offer, next to AR, new ways to represent and explore data embedded in everyday environments, to communicate it, and share it. For a possible future where situated visualizations become ubiquitous, however, fundamental research is missing on what visualizations should look like on these displays, how we would interact with them, how people would engage with them, and where they might become "too much". Now is a good time to prepare for this future and engage in the necessary fundamental research on perceptional, user-experience, design, and evaluation-related challenges. Situated visualization is more than "XR" and physical displays have many benefits as they equip the environment rather than the viewer. Let's not miss the boat and begin to shape how data will be used to empower people in their environments.

Michael's Bio

Michael Sedlmair is a professor at the University of Stuttgart and leads the research group for Visualization and Virtual/Augmented Reality. His research interests focus on immersive analytics and situated visualization, novel interaction technologies, visual and interactive machine learning, perceptual modeling for visualization, as well as the methodological and theoretical foundations underlying them.

AR displays as a new medium for visualization

Augmented Reality (AR) holds immense potential for (situated) visualizations by seamlessly embedding digital content into the physical world. However, despite the progress in AR technology, building effective and practical AR applications remains highly challenging. Core issues like tracking, registration, and limited field of view persist. Ironically, media representations of AR, such as in advertisements, paint a misleadingly perfect picture, setting unrealistic expectations. This disconnect is problematic, as building functional and accurate situated visualizations in AR is far more complex than what these idealized portrayals suggest. Unlike visualizations on screens, which have been optimized for decades, AR visualization is still in its infancy, and we must be willing to accept its imperfections as the field matures. As a consequence, (accepted) papers on situated visualization at IEEE VIS seem to often prioritize theoretical approaches or simulated concepts over the practical building of real AR systems, due to the difficulties in meeting these inflated expectations. This same situation hindered research in mobile technology 15–20 years ago, only for it to later explode into mainstream use. My position is that, if we don't take a more open and forgiving approach now, we risk missing the opportunity to advance AR visualization when it becomes a dominant medium. We must shift our thinking. AR is not a replacement for display-based visualization, but an entirely new medium with distinct challenges—such as adapting to dynamic physical environments like changing light conditions. To prepare for the future, we need to embrace these challenges now, learning how to design visualizations that are effective within AR’s unique constraints.

Wesley's Bio

Wesley Willett is an Associate Professor of Computer Science at the University of Calgary where he holds a Canada Research Chair in Visual Analytics. His interests span information visualization, social computing, human-computer interaction, and data science, while his research focuses on pairing data and interactivity to support collaboration, learning, and discovery. At Calgary, Wesley leads the Data Experience Lab and co-directs the Interactions Lab, the university's human-computer interaction research collective. He is also faculty in the University's Computational Media Design and Data Science programs. He received his BS from the University of Colorado at Boulder and his PhD from UC Berkeley.

Situated Visualization isn't just the Future of Vis, it's the Future of Computing.

The history of visualization has been defined primarily by the limitations of our display technologies and our tool chains. While most visualizations currently live on medium-sized non-situated screens, it seems deeply unlikely that our current "rectangles on desks" paradigm reflects the long-term future of visualization, or of computing more generally. Extrapolating contemporary computing trends forward into the next few decades, I posit that: (1) our computers and environments will increasingly blur together, (2) visually-encoded information will make up an growing proportion of our computing experiences, and (3) for a variety of practical reasons, we'll tend situate many of those visual representations near the things in our environments that they're related to. If true, this makes the visualization community (and situated visualization research in particular) uniquely positioned to help lay the groundwork for the next generations of computing systems — especially if we're willing to broaden our view of visualization to include more diverse tasks, users, and technologies.

Organizers