Cracks in the Success Narrative: Rethinking Failure in Design Research through a Retrospective Trioethnography

2.1 Reflection, Critique, and Seeking Justice

We draw from reflective design's capacity to question the limits of design practice itself—not only users, but also designers. Two key principles of reflective design as put forth by Sengers et al. are that designers should use reflection to “uncover and alter the limitations of design practice” (p. 7) and “re-understand their own role in the technology design process” (p. 7) [95]. They define reflection as, “bringing unconscious aspects of experience to conscious awareness, thereby making them available for conscious choice” (p. 2) [95]. As designers, we draw from reflective design by engaging “failure” in design research as a prompt for us to reflect on and re-understand the specific situated roles we played in our past projects. Eschewing a specific definition of “failure” invites us to explore vague yet sticky feelings of failure, surfacing areas of our own design practice for increased conscious scrutiny, articulation, and choice. Both pragmatic and emotional ways of knowing work together in our reflections.

Many efforts expand approaches to designer reflection and reflexivity throughout design processes, such as during field work [70, 103] or for design futuring [71]. Especially pertinent to our approach, feminist HCI leverages critical reflection to “analyze designs and design processes in order to expose their unintended consequences” (p. 1308) [10]. Haraway explains how all human knowledge comes from a situated human perspective [50]; this suggests designer reflexivity can lead to more rigorous knowledge production. Critical technical practice calls for “one foot planted in the craft work of design and the other foot planted in the reflexive work of critique” [2]. Our first-person method of retrospective trioethnography (detailed in the next section) leverages this reflexivity to provide rigorous, richly contextualized accounts of design practice. Wong et al. bring an infrastructural turn to speculative design, expanding beyond the technological artifact to critical reflective consideration of the surrounding lifeworlds [106]. Drawing from these approaches, we contribute a unique reflective focus on “failure” not only in design artifacts or technical processes but also and especially in the social lifeworlds and design processes surrounding our work.

By engaging in critical reflection, as proposed by these calls, recent works critique specific aspects of design research practice. Balaam et al. critique how experience-centered design does not typically report on the emotion work done by design researchers and how this limits effective training and knowledge sharing in design research [8]. Examining not only at the emotion work conducted by researchers but also the emotional burden on participants, Hirsch raises ethical concerns stemming from similarities between qualitative interviews and therapy for vulnerable participants, calling for trauma-informed research practices [55]. Furthermore, other works highlight and critique the solutionist bias of HCI by instead exploring when designing technology may not be appropriate [11], offering an account of inaction as a design decision [56] or proposing inquiry and commemoration as alternative design goals [28]. Our work adds to these critiques of design research practice by offering vulnerable reflections of our own instances of failure to make legible the beneficial insights that can stem from reflecting and reporting on failure in design research.

Underlying many such critiques of design research practice are morals of justice in research and design. Research justice emphasizes self-determination for marginalized people and communities, in part by turning critical attention to power imbalances between researchers and participants [5, 7, 92]. Focusing on justice in the design process, design justice critically reworks design processes to center those marginalized by design along axes of systemic oppression such as white supremacy, ableism, heteropatriarchy, and so on [18, 20]. Combining research justice and design justice, recent work calls for more just practices in HCI. Asad offers prefigurative design as a framework for more just research practices [5]. Asad et al. call on HCI to recognize how standard academic research practices can reify systemic oppression and offer alternative strategies for research justice [7]. Dombrowski et al. outline strategies for social justice-oriented interaction design [30]. Harrington et al. call for more equitable participatory design engagements in community-based collaborative design [52]. Liboiron calls for care and solidarity in interventionist research [75]. Attending to these calls, we offer methodological reflections on using retrospective trioethnography as a technique for engaging difficult ethical and methodological questions around design justice.

2.2 Failure in Design Research and Beyond

Engaging failure in design research has ethical and epistemological implications. What counts as failure or success in design research? If research success is related to knowledge production, what counts as knowledge production in design research? In this article, we only dip our toes into these difficult questions, but we ask these questions of ourselves and our readers to hint at the stakes of engaging failure.

Notions of failure in design research partially stem from functionalist engineering evaluation. A technical system may succeed or fail to achieve specific functional objectives in engineering evaluation, but this notion of failure does not neatly transfer to evaluating designs with human interaction. Design practice often structures itself around cycles of iteration, analyzing failures at each cycle to iteratively improve [96]. Human-centered design consultancy IDEO advises iteration and “fail early and often” [63]. As usually adopted by design practice, failures get folded into cycles of iterative improvement leading to a successful design, reifying typical success narratives.

Design researchers have begun to explore failures that do not directly iterate into “success” in a single project's story arc. Gaver argues that one-way design research produces knowledge is by describing “what works” about specific design artifacts, synthesizing these characteristics into key suggestions or considerations for future design research [43]. Learning from design failure can also contribute to this mode of knowledge production, as failings of prior iterations inform future iterations or future directions potentially by other researchers working on other projects. In this vein, Gaver et al. reflect on the failure of Home Health Monitor, yielding insight on how to design with data for open-ended interpretation [44]. Their analysis of this design's failures, and Gaver's agenda-setting design research more broadly (e.g., [41–43, 46]), help shift design evaluation beyond functionalist engineering standards of task completion or efficiency. The analysis of the Home Health Monitor's failures centers how participants interacted with the design artifact, locating failure in the design artifact itself [44]. In another example, Torres et al. propose strategies for mitigating and productively working through feelings of failure around laser cut objects [105]. Heinzel et al. exhibited failed e-textiles and show how reflecting on failures not only helps others learn from those mistakes but can also stem new directions for the designers themselves [53]. Additionally, it is fairly common in design publications to report on flaws of selected prior design iterations to help explain key design decisions for the final “successful” design. In a sense, reflecting on failure is already integral to much design research practice. Our work builds on and expands this practice of reflecting on failure in design works.

We expand this approach of reflecting on failure in design research in two ways. First, we add nuance to the notion of success vs. failure. Although Gaver et al. describe “an obvious, incontrovertible, and multidimensional flop” (p. 2222) [44], we reflect on projects that entangled aspects of both success and failure, as many projects do. In many ways the prior projects we reflect on were successful, as indicated by resultant design insights and publications, yet we still choose to turn self-critical attention to where these projects held lingering feelings of unease or aspects of failure. Second, we expand consideration beyond the design artifact and participants’ interactions with it to additionally include considerations of multiple stakeholders, participant labor, researcher labor, and so on. Our reflections tease out aspects of success and failure in prior projects and then move away from this binary toward more situated recounting of our process and broader accountability in terms of design justice commitments.

Trouble is a related term inviting design researchers to challenge the status quo. For example, Haraway's call to “stay with the trouble” [51] has been taken up in design research. Søndergaard and Hansen engage this by staying with the trouble of digital personal assistants [101] and Søndergaard by troubling design for women's health [100]. In 2019, the Design Trouble symposium challenged hegemonic notions of successful design by positioning non-teleological notions of the design (including failure) at the center of discussions between design practitioners, thinkers and researchers in landscape architecture, media art, design, anthropology and more [3]. By focusing on non-teleological aspects of design—moving away from design narratives that exclusively focus on the purpose, end, goal, or function—the symposium allowed participants to emphasize process which includes moments of reflection, temporary successes and points of failures. In a sense, our retrospective trioethnography on failures of our past projects serves to trouble the tidy success narratives of our prior publications.

Beyond design research, science and technology studies (STS) finds great analytic utility in studying moments of breakdown for revealing what might otherwise be left unexamined, particularly as infrastructure “becomes visible upon breakdown” (p. 5) [102]. STS and HCI scholar Jackson draws from phenomenology, Vygotskian activity theory, and the concept of invisible work to argue it is “in moments of breakdown that we learn to see and engage our technologies in new and sometimes surprising ways” (p. 230) that resist and rework dominant narratives of technological innovation and open up possibilities for building anew [65]. Jackson argues for an ontological shift toward centering maintenance and repair within the design and study of technological artifacts [65]. Yet, in this article we choose to linger on failures or breakdowns in our own design research practices to leverage their analytic utility for revealing what prior success narratives of our work left uninterrogated.

Moving away from success makes way for critiques of normative notions of advancement too often steeped in hegemony. Shorey et al., for example, call for a departure from monolithic narratives of progress toward singular goals to value friction and breakdowns [97]. Bell et al.’s [12] retrospective autoethnography describes how Black and Brown scholars in academia are often expected to fail, and invites marginalized researchers to embrace failure as part of the radical decolonial dreaming of the “New University.” By making space to think more deeply, care for students, and fight for social justice, they argue, “failure allows for a retreat to think anew about what went wrong” and for being “more deliberate in pursuing social justice” (p. 8) [12]. In The Queer Art of Failure, Halberstam argues “success in a heteronormative, capitalist society equates too easily to specific forms of reproductive maturity combined with wealth accumulation” (p. 2). As an alternative, “under certain circumstances failing, losing, forgetting, unmaking, undoing, unbecoming, not knowing may in fact offer more creative, more cooperative, more surprising ways of being in the world” (p. 2, 3) [48]. As design research and third wave HCI move away from an uncritical embrace of capitalist production toward more cooperative ways of being in the world, a focus on failure helps surface and critically re-evaluate underlying values at work in design research practice. We take up these calls to challenge success narratives and embrace failure, specifically within our own domain of design research.

3.1 Duoethnography and Trioethnography

In duoethnography and trioethnography, the emphasis is on the dialogical relationship between the lived experiences of the researchers [94], a key distinction from other first-person methods. By juxtaposing multiple voices, the researchers can reflect on the similarities and differences between their experiences, opening doors for learning [94]. Researchers have used duoethographies to inquire into personal aspects of life such as sexual orientation, race, friendship, and feminism (as exemplified in the 2015 special issue of the International Review of Qualitative Research [93]), often sharing vulnerable experiences where they have been hurt or where they might have hurt others. This suggests that it is a well-suited methodological approach for us to explore the uncomfortable, sticky and sometimes shameful personal experiences of failure in design. Furthermore, Norris and Sawyer, who coined the term duoethnography, state that in duoethnography “the intent [is] not to profess but rather to learn and change as the result of the conversation” (p. 2) [93]. This resonates strongly with our process, in which we not only analyzed our own experiences to share back with the HCI community, but we also genuinely learned from our exchanges, or as Huckaby and Weinburgh say “we [came] to know the other and ourselves through the space between our experiences” (p. 62) [61]. Duoethnography and trioethnography are frequently retrospective, including reflections reaching back into childhood (e.g., [61, 62]). Revisiting the past enables the dialogic juxtaposition of different life experiences when the researchers were apart. Employing retrospective trioethnography enabled us to reflect on our failures across several past design research projects spanning several years. The process allowed us to share and learn from these vulnerable, uncomfortable experiences that were glossed over in publications or other project documentation.

3.2 Our Trioethnographic Process

We came together around a shared interest in the generative potential of failure in design research. To start the project, Noura reached out to Audrey for an initial conversation. Both of us engage in critically oriented design research as a central methodology to our work and share an interest in examining the assumptions and potentials of data in everyday life. While Noura focuses on data and the body, Audrey's interest centers around data in home settings. Together, we decided to reach out to Sarah who is also a design researcher critically working with data, but at the scale of infrastructures. This combination of shared methodology, variations on data, and familiarity with each other and each other's work seemed like good starting points for a collaboration. In terms of exploring failure, Noura wanted to generate something insightful from a 2-year long project that failed to deploy at the end. Audrey welcomed the idea of pausing to look back in order to look forward to future projects. By examining repeating patterns of things not working in her practice, she was interested in finding ways to do things differently. Sarah was in the mid of moving institutions and saw an opportunity to reflect on the body of work that she had done thus far. Creating a generous space to examine instances of failure felt like a way forward in terms of outlining a more considered and responsible research practice.

After a few meetings and email exchanges to discuss the goal of the project and to agree on our process, we each selected two design research projects to discuss with the group. By leaving the definition of failure open, we were able to choose projects that had left us with lingering feelings of stickiness, shame, or guilt. We also could look back at projects in which we had left out details to fit in the success narratives in our previous publications. Before we turn to a description of our method, we share our own positionality as a way to ground our work, an important part of any trioethnography. We are three white, cisgender women assistant professors based in the United States. Noura was trained in information science and is now in a communication department, Sarah was trained in human centered design is now in an HCI department, and Audrey was trained in design and is now in an art and design school, all at research universities. From these different backgrounds, we have each developed careers in critically oriented design research in the broad field of design and HCI.

Our reflective process went as follows. We wrote individual narratives for each project (between 1,300 and 2,100 words each), focusing on points of failure in our past work. We used past emails, notebooks, project documentation, photos, and previous publications (and their drafts) to jog our memory when writing, a state-of-the-art practice in duoethnography [94]. We knew we would share these narratives only between us, a generous and caring audience, encouraging us to be open, vulnerable, and honest, taking hints from Devendorf et al.’s work on vulnerable design memoirs [28]. We shared them via Google doc to start our dialogue. Garcia and Cifor emphasize the importance of “practicing engaged and interactive dialogue as a primary approach for the shared probing of a theme, activity, event, or problem” (p. 6) [40]; we planned for two modes of dialogue. First, we asynchronously commented on each other's narratives with questions, requests for clarification, and comments emphasizing a point, or sharing a similar anecdote. Second, for three weeks in a row, we met twice a week to interview each other, using a different project as a starting point for each session of approximately 90 minutes. We recorded each Zoom session and downloaded the transcripts. In each session, we first checked in with each other about life in general, continuing to build a generous, open and welcoming space for discussion. Then, one of us would talk through one of our narratives, while the other two were listening, taking notes. We then discussed the failures brought up in that narrative. We often started by asking questions or offering statements of meaning (inspired by Lerman's critical response process [19]), but often moved into three-way discussions around issues we all encountered in our own projects. Orienting questions during those discussions included the following:

• What are challenges or issues in the project and to what extent, how, in what ways, were they addressed, mitigated, solved, or not?

• How does the researcher/participant feel about this aspect, stage, artifact, and so on, and why—offering particular attunement to frustration, surprise, “not working,” “failure”?

• If there is a sense of failure, who is the design or process failing, in what ways, how, to what extent? What are the stakes of failure, and to whom?

• Describe the experience of reaching failure. Was it a particular moment? Or did it bubble up over time? How did you come to identify it as failure?

• How has the failure stayed with you? How does it “haunt” your work? What lessons or cautionary tales have you carried forward? How does it affect the ways you think about your research or how you teach?

After each session, we each wrote a memo to note down points that stuck out during the discussion, questions we wanted to investigate deeper, and our own feelings during these meetings. Our collective memos added up to 8,000 words.

Once we had concluded our deep dives into the six projects, we read back through each narrative, the memos, and the transcripts from our six sessions. We open coded the documents and then performed a thematic analysis to start organizing the findings. Before we turn to those findings below, we note that our process of writing this article is also part of conducting the research itself, as Sawyer and Norris state: “in duoethnography, then, writing is simultaneously a form of data generation, data interpretation, and data dissemination, in not so nearly as linear a fashion as the finished text suggests” (p. 76) [94].

5.1 Deployment as Relational, Situated

We reflect on the subtle but persistent pull that we all felt as HCI researchers to deploy our design artifacts in “real world” settings. Among many instances of this pull, we juxtapose two illustrative examples here. Our narrative first-person accounts relate efforts for Sarah to deploy Riot and Noura to deploy the Heart Sounds Bench:

• Sarah: The Riot team and I felt the need to realize the project within a large organizational setting to demonstrate that it could “work” beyond existing as a prototype. We found an early champion in an upper level management contact who quickly enrolled others within the organization to support our efforts and met with us to define the length of the pilot and subsequent deployments at partner sites. He arranged for us to meet members of the janitorial staff early in the morning before opening hours to calibrate the device and sort out the engineering needs associated with a long-term installation. But a few months into our collaboration, a new manager took the helm and there was an immediate change in tone. Though he initially replied to our inquiries about meeting times and other logistical details, he often left off key portions of the response (e.g., “no” to a question requiring specific dates or times). Eventually, it became more difficult to organize times to stop by and members of the janitorial staff conveyed confusion about the ongoingness of the project. After several weeks of disjointed conversation, we began to have the sense that we were being sidelined.

  In an effort to remedy any miscommunication, I reached out to the new manager to set up a time to meet in person. After I didn't hear back, I offered to answer questions he might have over the phone. He replied briskly that he understood the former manager had approved the collaboration, but that he had no intention of supporting it. After a few more attempts at connection, he stopped returning our calls and emails altogether. It is difficult to say whether he took issue with the project due to the topic or whether he simply did not want to bother with the complication that came with coordinating with researchers. Access left much more quickly than it came. Perhaps we should have planned for a quicker pilot? Yet, this would have gone against our aim of a long-term in-situ deployment. Might we have been able to engage a broader set of stakeholders? Without the blessing of the manager, a deployment within this setting would have been impossible. In discussing the situation with others in the organization, they told us that all decisions related to facilities and infrastructure would still ultimately lead back to him.

Listening to Sarah describe how her project found an early champion for site access that was later gradually rescinded prompted Noura to reflect on her own varied and extended efforts for gaining access:

• Noura: It took a year of communicating with the City to get permits to deploy the Heart Sounds Bench in public. I sent many emails and went to multiple committee meetings, trying to make the project appeal to the varied interests of city stakeholders to gain their approval. I measured sidewalk widths all over town to make sure there would still be at least six feet wide sidewalk access even with the bench there. A few weeks later I unexpectedly got a call from an unknown number and was overjoyed to learn I was speaking with the permits office and could make an appointment with them. During the appointment, I tried to be ingratiating to this “keeper of the keys,” while steering clear of anything that might seem overdone such as bringing homemade baked goods. I tried to foster an interpersonal connection and make pleasant conversation. I tried to fill out as many permits as possible; they limited it to a month and then said to come back if I needed more. I remember worrying at the time that they might not remember me and it might be another months long delay.

  The deployment duration and locations were largely set by the city's requirements. A year of delay and very little control over the location of deployment were difficulties for the design process, but I understand their requirements are to protect public access and am grateful they worked through the whole process with me.

Our trioethnographic process enabled us to recount frictions of past projects, frictions that had been glossed over in prior “success story” narratives of these projects. Reflecting in conversation, rather than alone, sparked renewed attention to recounting details of our personal experiences and, through juxtaposition and further collaborative discussion, led to higher level reflections on design research practice and culture.

5.2 The Limits of Design Artifacts: Mismatches in Imaginaries

In the work of deploying design research artifacts, we also encountered difficulties navigating mismatches in sociotechnical imaginaries [68, 69] between researchers’ intentions and participants’ ideas of technology or design. Each of us in different ways sought to open up reflection with participants on alternative possibilities with technology, yet were struck by the degree to which participants’ ideas aligned with the very same dominant narratives of technological progress that our work seeks to resist and reimagine. For example, in Noura's Ripple project, participants seemed to grant data authority regarding lived experiences and emotions. However, Noura's goal was to infuse enough ambiguity in the design of the technology to push back against the authority of data:

• Noura: Despite my design efforts to the contrary, for some participants Ripple's display seemed to have the potential to foster or aggravate insecurities, as previously published [59]. For example, as previously described in publication [59], one participant seemed worried the display meant she was “broken and unfeeling.” When this concern arose during the post interview, although my study's setup could not gauge whether there may have been pre-existing insecurities at play, I felt responsible for dispelling whatever discomfort Ripple might have caused. I emphasized that the design could not indicate whether someone was “feeling” or “unfeeling” and explained again how the display only responds to particular kinds of intense excitement related to skin conductance.

  A PhD student at the time, I was relatively inexperienced at qualitative interviewing. My decision to emphasize a particular interpretation of the artifact felt against the grain of probing participant reactions and experiences. After this point, the participant dismissed the display as completely random. My spur-of-the-moment reasoning behind this decision was simply, interview technique be damned, interpersonal care feels more important. In hindsight the fact that research technique could ever feel at odds with interpersonal care stands out as deeply problematic. I need to more deeply unlearn the “neutral observer” assumption.

  This participant's experience, analyzed alongside others, “yielded insight about how it can be difficult for a design artifact to resist the perceived authority of data in our present societal imaginary. I distilled her experience into an anecdote, a touchstone point in the paper, a laugh-out-loud yet sobering moment in the conference presentation and job talks. As I paraded her experience over and over again, each time I persuaded an audience of my cleverness, whatever point I was trying to make rang more hollow. I am left mainly with a feeling that I extracted or harvested others” emotional experiences for my own career advancement. After this project, I shifted gears away from designing for open-ended emotional interpretation toward designing for more affirmative experiences.

We return to issues of extractive storytelling described above in Sections 5.3 and 6.3, but hold our focus here on mismatches in imaginaries. Listening to Noura describe navigating participants’ expectations with data resonated with Audrey, who also encountered mismatches with participants’ imaginaries around data. Yet, Audrey engaged these imaginaries differently, seeking to invite participants into a shared space of co-speculation. The central tenet of the Alternative Avenues for IoT project was to move away from solutionist one-size-fits-all concepts in IoT. In a form of discursive design, Audrey and her team purposefully used post-functional, humorous, and whimsical IoT concepts to provoke participants’ imaginations and pivot to co-speculate “what else” could be possible in terms of IoT. Yet, participants’ imaginaries around IoT were often heavily influenced by existing IoT narratives—the exact narratives we as designers sought to move away from—and these utilitarian IoT narratives prominently came through in the ideas they were contributing to our co-speculation:

• Audrey: The bespoke booklets included five concepts made by us, and five by co-speculators. Our studio made a conscious effort to come up with concepts that had humor, a sense of whimsy, a clear relationship to the nature of this specific home, and a criticality against surveillance capitalism in home data. We believed this was the best way to open up alternative avenues for IoT and sever the tie to current trends in IoT. With the goal of imagining together, we wanted to recognize and celebrate this intimate knowledge of our co-speculators’ homes, but also direct their imagination toward new paths for IoT. We thought that by offering them a photo to draw on top of, and by filling in the first half of the booklet to show examples of whimsy and humorous concepts, our co-speculators would imagine creative and surprising concepts about another IoT.

  We had a strong agenda, we had design and speculative design training, and we had been thinking about this space for a while: in retrospect it looked like we had “an advantage.” When we conducted our analysis, many of the participants’ concepts were not particularly surprising or novel. When we wrote about the work, our own drawings/concepts were inadvertently used in majority to illustrate our points. In the CHI paper reporting on the project [24], out of 21 concepts that served as examples, only one was designed by the co-speculators... (a count I only made while reflecting on the project during our trioethnography). Yes, those concepts built on what we learned from co-speculators guiding us through home tours and the photos that were taken, but they were mainly our studio's interpretation of the visits (combined with our own agenda to dramatically reimagine IoT in other homes). I am still proud of the avenues we proposed, and I think that there is no way we could have come up with these five themes if it had not been for our co-speculators, yet, I feel like we misreported on the method by letting it appear as a “successful” co-speculation.

  In hindsight, I reflect on how much work it took our design team to iterate toward the concepts that felt right. If it took that much work for us—trained in design and discursive design—was it fair to expect co-speculators to completely forget what they know about IoT and become “creative” on the spot? In our team, we talked a lot about how hard it is to scaffold creativity. We talked about how uneven the playing field is, about how we will always have a kind of advantage since we are framing the study. So, I am not sure where the failure is here: is it in the fact that I feel there is a level of dishonesty in how we chose to report on the project? Or is it that we actually failed at truly supporting participants in being creative? Or is it that they were creative, but it was not a creativity that was aligned with our expectations or standards? Is the failure in how we exercised too much authorial power in a project we had called a co-speculation?

Throughout our research efforts, each party (e.g., researcher and participant) brings to the table their intention, goals, and current understanding of the topic at hand, be it IoT, data, or the goals of design. Our reflections uncovered instances where researcher and participant understandings were mismatched: While Noura designed Ripple to challenge the authority of data, participants brought an expectation that this data display would yield “truthful” insight. While Audrey sought to explore beyond utilitarian home IoT in co-speculating with participants, participants frequently offered utilitarian design ideas. As design researchers, we are often pushing against common narratives surrounding technology, for instance that data is clear, clean and perfect; that IoT is supposed to serve functional and solution-oriented goals; or that design is always human-centered design. With our work, we are often aiming at finding alternatives, playing with ambiguous or alternative visions for data, technology, and design. This intention is often at odds with the normalized imaginaries often presented to the public.

This mismatch between common narratives around technology and design in the popular imagination and a desire to critique these narratives creates a challenging space within which to work, with a high potential for failure. When recruiting participants for a study with new technologies, the common assumption is one where research is positioned at the forefront of innovation and technological prowess. Participants may come in with the mindset of pushing technology forward (whatever “forward” means), while our goal as design researchers is so often to stop, reflect, critique, and explore alternatives that do not go “forward” (and may, in fact, mean moving sideways, backwards, or not at all).

5.3 The Labor and Burden of Participation in Design Research

While in the previous sections we focused on the practice of artifact deployment and participation in HCI and design research, here we take a critical look at the labor and burden of participation. Participation is work [99], and carries with it affective demands [31]. While some participants are enthusiastic and may not feel a heavy burden in participation, we each shared stories where the weight of research on participants’ shoulders became evident. In those moments, we are reminded of how research can be extractive, and of how much we need participants’ labor for certain types of research to work.

For example, Audrey reflects on one participant's experience of the labor and burden of participation, and a participant's difficulty in removing herself from a study:

• Audrey: In the data-human entanglements project [22], we worked with participants in a sequence of phases over a few months. One participant did great in the first phase of the project: her contribution was creative, poetic, surprising, and the research team invited her to join the second phase of the project. She promptly said she would be interested in continuing to work with us, so we sent her our next activity: a set of blank postcards for her to draw data from her everyday life. At that point, her personal life and work life became very busy and she was only able to complete 2 out of our 10 cards. This should have been a red flag, but she kept telling us she was interested in seeing the next phase. For phase three, our team needed to fabricate three props for her to use in making a short 60-second video. We delivered the probes, a couple of months after phase two. While she was still responding to our messages, she did not make the videos. She kept saying she was going to do it, and it stayed on her to-do list for months.

  Eventually, I went out for dinner with her. During that dinner, I realized how much pressure this had been for her, it was like a small cloud hovering over her. Even if in our messaging with her we had been clear that she could stop the project at any time, and that it would be OK, our language was not clear enough to make her feel comfortable in saying no. At that dinner, she finally told me that she did not want to continue the project. Her body language changed, she felt relieved. Her partner, who was also there at the dinner table, also was relieved; he said she had been agonizing over this for weeks.

  This story stayed with me for a long time. How could I have missed her signs? Why did she feel like she could not say no before that dinner? Why was a short video of 60 seconds putting so much pressure on her? If we had not met for dinner, would I have ever known? Or maybe another participant would have felt more comfortable saying no earlier? While consent forms state that participation is voluntary at all times, social pressure and personal commitment can be very strong.

Listening and responding to one another's recountings spurred us to reflect on how, across our different vignettes we find varied instances of the labor and burden of participation in design research. The Ripple vignette presented above (Section 5.2) illustrates an acute instance of upsetting emotional labor, in which a participant had to wrestle with the emotions of feeling “broken and unfeeling.” This participant's experience presented above was perhaps less acute but more drawn out, not wanting to break her “promise” to “help” Audrey even though she felt burdened and stressed by participating. In a different instance, toward the end of an ethnographic engagement, a long-time interlocutor asked Sarah if she “got a good grade,” assuming that her participation was meant to contribute to a course project for which Sarah would be evaluated. This often felt like a failure on our end in seeking ongoing, active consent, explaining how research works, and conveying the broader goals for research. These examples raise different issues of unexpected worries triggered by a design, proactive consent or opting out, and participant altruism. The vignettes above reflect on how participation, as commonly done in HCI research, is broken.

5.4 Temporality of Failures and Researcher Labor

Project timelines rarely unfold as neatly as research publications might suggest. Our discussions allowed us to revisit the messiness and many varied attempts comprising our projects before they became tidy post hoc narratives. The scale and timing of our attempts in tandem with our shifting assessments of attempts’ outcomes were key to project management. As design researchers, when, how and why do we decide if an attempt is not/working, un/finished, success/failure, a bit of both, or something else entirely? Our discussions also surfaced unrecognized research labor as a risk with the potential to harm both sustainable work practices and the research output. We argue that greater consideration and more candid reporting of the scales and temporalities of research efforts, and greater recognition of researcher labor, can both enrich the insights gained from design research projects and help make legible design research practice.

6.1 Methodological Considerations for Using Trioethnography to Reflect on Failure in Design Research

We offer methodological considerations on engaging retrospective trioethnography. We put forth our own process, our particular way of doing retrospective trioethnography, to assist others who may wish to use retrospective trioethnography as a means of attending to and learning from failures in their own design research practices. We found this method well-suited for reflecting on thorny issues, spurring insights for our own practice. We emphasize here that our method is not only centered on finding ways to reflect on failure, but to do so via duo- or trioethnography. Our considerations include carefully choosing co-conspirators, generous opening questions, following trains of thoughts to rearticulate meaning, acknowledging ourselves as instruments for research, and paying attention to forms of materializing dialogical exchanges.

6.2 Letting Go as a Mode of Research Care

Across our reflections, we noted instances of letting go, where moments of departure led to broader learnings about how our research initiatives develop and evolve. Responding to issues with the labor and burden of participation (as we described in Section 5.3), and the need to rethink participation in critically oriented design research (Section 5.3.1), we explore possibilities for participants to “let go” by creating openings for dialogue and checkpoints along the way. It is insufficient to rely solely on institutional ethical protocols or existing standardized methods for research engagement. We argue that design research practice needs to more openly acknowledge the relational nature of this work of participation, and allow for more ongoingness in our consideration of what constitutes an ethical encounter. For example, Harrington et al. provide guidelines for more equitable community-engaged design research [52], and Asad offers prefigurative design as a framework for more equitable community-engage design research [5]. Stemming from our reflections, one tactic could be an active affirmation approach, where participants are offered opportunities to explore their shifting relationship to the project and affirm their continued interest. For design researchers, this would mean acknowledging the authority that we possess in creating research encounters and actively seeking ways to ease the stress of departure for participants. Rather than loss, leaving can offer valuable insight on the project as a whole. These considerations ask that as researchers we rework participation to be more equitable, a key goal of design justice, by caring about and exploring tactics for letting go.

A second form of letting go may help cope when external factors derail or halt project timelines (responding to issues raised in Section 5.4.2). In some cases, external factors such as extreme weather events and the COVID-19 pandemic can prevent work from proceeding as planned. The Heart Sounds Bench embodied this form of letting go, where a compounding set of circumstances made public deployment untenable. More mundane or bureaucratically imposed standstills, as with the library deployment of Riot, offer a subtler sense of closure, where departure is more like slowly closing a tap than halting to a stop (Sections 5.1, 5.4.2). Under the tight deadlines of conference submissions, push-backs and pivots such as these can feel devastating, particularly for early career scholars for whom publication records might be crucial for advancement in graduate education or in securing career opportunities. Here then, the type of collective letting go we might need to do as a research community is of the expectation of twice-yearly publications. When doing community-based and ethnographic design research projects, such rapid markers of research progress obscure the contingencies we face in conducting this work and often do little to advance our methodological understandings of how projects unfold over time.

Collectively, these examples point to acts of letting go as a mode of research care. The first form of letting go expresses care for participants by employing tactics to ease the burden of exit for participants. The second form consists of a combination of self-care, team-care, and collective research community care for the researcher by moving beyond guilt or shame associated with “failure,” letting go of project plans and moving toward reworking methods and modes of engagement. In this way, letting go aligns with recent efforts to outline more just research practices for HCI [5, 30, 52, 75]. Asad and others describe the need to transform the social relationships of research, and the forms of extraction perpetrated by existing structural hierarchies inherent within current arrangements [5–7]. Practicing justice means attending to harm and healing, by creating fairer conditions and continually seeking out ways of mitigating the exploitative tendencies of research labor. In trioethnography, we find a technique for examining such ethical and methodological questions in our own research practices, drawing connections between our personal experiences and broader systemic issues.

6.3 Translation Work, Extraction, and (Dis)Honesty in Reporting

We began this article motivated by a desire to challenge success narratives and tell richer stories of design research. Here, we critically reflect on how tidy success narratives glossed over important issues in our own work, and point to possibilities for more honest reporting in design research. For example, Noura reflected on ways she leveraged participant experiences in formulating an engaging narrative for the purposes of publication and research talks. Through collaborative retrospective trioethnography, she was able to recognize the extractive nature of such storytelling (Section 5.2). Similarly, considering what forms of labor went (un)reported in research publications led us to broader reflections on exploitation within research labor, as well as the academic hierarchy of power relations within which we in academia are all embedded (Section 5.4). In reflecting on the Bespoke Booklets project, Audrey realized that only one of the 21 concepts described in her publication was designed by a co-speculator (as opposed to members of the research team). Through our process, she was able to recognize and contend with the shortcomings of the work in scaffolding creativity, and critically explore further avenues for participatory speculation techniques (Section 5.2). With Riot, Sarah sought to honor the existing advocacy work happening around menstrual access. Across these accounts, we consider the need to take care when claiming impact, to attribute credit widely and place fault narrowly.

These varied considerations and points of contention on labor, attribution, and credit underscore calls within the design research community to move away from the narrative of the “heroic designer” toward more modest and relational accounts. Irani and Silberman [64], for example, note the tendency of news reporting on HCI work to hone in on stories of design saviorism, rather than recognize collaborative efforts or celebrate ongoing acts of maintenance over time. Parvin similarly discusses how design storytelling can risk being extractive, and how reciprocity, responsiveness, and communion are crucial in doing justice to the narratives we generate and retell [86]. Alongside such critiques, there has also been a recent push within HCI to further consider the role of the researcher in shaping the work done, with positionality statements becoming a hallmark of qualitative writing. In extending such efforts, we recognize the need to look at the margins of typical accounts, or what we might ordinarily leave out of our research publications—the ambivalent, untidy stories of failure.

All projects have shortcomings. Normalizing this and more openly reporting on the ways we respond to and move forward from misalignments or missteps is offered here as one partial way forward in formulating more responsive and responsible design research methods and reporting. Retrospective trioethnography gave us the space to confront the ways in which we contribute to systems of power within and through our work. In grappling with our own feelings of disquiet and frustration collectively, we also began to envision research done differently or how we might take forward the lessons learned. For example, moving away from typical deployment or extractive storytelling opens possibilities for more just design practices, such as bringing research results back to the field [36], more thoughtfully scaffolding dialogue with participants to more directly talk about differing sociotechnical imaginaries, more candidly reporting on participant and researcher labor, and questioning the necessity of deployment.

Broadening a reworking of research communication to the field writ large would take not only the commitment of individual researchers, but also reviewers, book editors, and audience members who contribute to publishing norms in myriad ways. Citational practices that give extensive credit to prior work may seem to diminish a sense of authorial contribution in the perception of reviewers or book editors. Yet, as Rankin and Thomas make clear [90], citational practices are a key form of recognition and career advancement in academia. Alongside the need for a broader shift in design research reporting practices, we re-iterate calls by Noble, D'Ignazio, Klein, and many others to cite junior scholars and scholars of color as one small way to move away from “authorial saviorism” in research reporting narratives [29, 72, 83, 84].

6.4 Trioethnography as an Approach for Engaging Design Justice

As Asad describes, “it can be difficult to make sense of how to incorporate the more abstract concept of justice into our research practices” (p. 1) [5]. Our retrospective trioethnography helped surface and articulate ways in which our design projects “failed” according to values of design justice that were insufficiently carried through yet deeply important to us. The goal of this article is not to provide a list of such values, though we have drawn connections to calls for design ethics and justice throughout. Rather, this article foregrounds the reflexivity of our trioethnographic process to offer a technique for deeply and practically engaging such ethical and methodological questions, hinting at moments where reflection and critique turn to realignment and action.

As examples, our reflections led us to question and critique our impulse to deploy, and to hold ourselves accountable to ways our design work matters (or does not) to particular contexts and communities with social and political histories (Section 5.1). Recounting uncomfortable moments with participants helped us recognize the burden of participation and articulate the value of equitable exchange with participants (5.3), a key value of design justice, and to see how this value was insufficiently embodied in our own past research practices. Responding to this in the discussion, we offer letting go as a strategy for mitigating harm and providing more care in research practices (Section 6.2). By articulating invisibilities of researcher labor (5.4.2), we underscore calls to rework academic labor relations to be more equitable. Responding to broader issues of inequitable labor for both participants and researcher, we discuss strategies for reworking research communication norms away from the “heroic designer” narrative toward more modest, relational accounts (Section 6.3). Of course, these values are already shared by many in HCI. The unique contribution offered by a retrospective trioethnography is surfacing ways in which these values were deeply at play yet insufficiently carried out throughout our design process or reported in publications. If we had instead initially selected a set of values by which to evaluate our designs, we may not have realized what was deeply important yet missing from consideration. To practice our design values, to do design justice, the process of critical self-reflection is essential to holding ourselves accountable.

6.5 Limitations and Failures

In an article challenging success narratives, we would be remiss not to explicitly acknowledge limitations and failures of this article itself. Although our process of retrospective trioethnography enabled us to generatively engage difficult personal experiences in prior projects, helping us link these difficulties to calling out systemic issues in design research practice throughout the findings and suggest alternative possibilities in the discussion, we have largely imagined alternative tactics that may be taken up by an individual researcher or an individual research group, such as an advisor and their students. This is problematic because such individual tactics, and even the ability to reflect on failures as we have done, may not be practicable for many. Learning from failure requires time, effort, emotional capacity, and the privilege to survive potential stigma surrounding failure [96]. Early career scholars needing to progress to degree completion or tenure on a fixed timeline or scholars facing extra career pressure due to axes of social marginalization may legitimately feel a need to hold to success narratives as they navigate an already challenging and discriminatory academic culture. Furthermore, individual tactics, even if adopted by many, are still insufficient to challenge systemic issues in design research practice. Greater attention and willingness to make structural shifts to address systemic issues is needed among the design research community writ large.