Oublié/trouvé, or Towards a Theory of the Invested Object
The following is my master's thesis paper and project description. It is very long. Links are being added as I have time. The paper with still images can also be downloaded as a PDF.
1. Towards a Theory of the Invested Object
how I got into this in the first place • an overview of the paper
2. The Problem, or How It All Began
the quickest of overviews • computing prehistory (until 1948 or so) • logic, abstraction, and the Platonic backhand • George Boole • Bertrand Russell • the perfectibility of thought • information theory & the cyberneticists (the 1950s and 1960s) • Claude Shannon • Norbert Wiener • mixing with the counterculture • the power of the interdisciplinary • Stewart Brand • Wired • the problem: information today • genes, memes, and information • cybernetic art and design • criticism from the trenches: Ellen Ullman and Jaron Lanier
3. Polluting the Possible
polluting the possible • why make prototypes • the effects of objects on the world • the spheres of communication and production • critical design • A/B manifesto • the futures cone • this is not a critical design project • beyond Dunne & Raby
4. Characteristics of an Invested Object
characteristics of the object • it should be a real thing • it should be a thing you can think with • it should be a private thing • it should be a reflexive thing • it should be a transportable thing • it should be something people can make themselves
5. What an Invested Object Might Contain
on distance and memory • memory is embodied • neurons and scents • memory is reflexive • a return to the autopoeitic • memory is cultural • de Waal’s netsuke • memory belongs to artists • lichen you again • kinds of memory • mystical, documentary, fragmentary, hallucinatory, Nora’s four types, public and private • Proust • the truth about where I started • Proust v. the cyberneticists • stereoscopic memory • optics and errors • depth in time • optical relations: cinematic, montage, stereoscopic • the moment of inflection • instantanés, moments bienhereux, and inflection • an initial test
6. The Idea
oublié/trouvé • a short explanation of the system • design objects • icon sets • storyboards i • reading response film • classifications • system overview • characters • storyboards ii • pitfalls and objections • expanding the empathy circle too far • nostalgia as a waste of our new capabilities • dangers of completionism • perilous duration related works • mylifebits • reporter • moodnotes • memory device • dear data • datacatcher
an unconventional research • research alternatives • Goldsmiths Interaction Research Studio • the phenomenological approach • practice-as-research • autoethnography • output & ducumentation • system test • reflections on the development and design process • self-interviews • other writings • objections • lack of rigor • lack of external input
8. The Project
the object • mold making • size • materials testing • top-shape selection • the zine and polluting the fablab • adding components • reflections on being a domestic manufacturer • the application • revisiting the goals • initial screen designs • developing the data views • screen-by-screen • the code • reflections on being a developer • a website for the project • reflections on the role of the designer
life with oublié/trouvé • through other lenses • phenomenologies and alternate research methods • material culture • characteristics of an invested object • “digital character” • revisiting the watcher object • future plans
Appendix A. Surveys
general population makeup • lovable objects • lost-time.club
Appendix B. Other Writings
self interview 01 • test projections • seeded moments • self interview 02
Appendix C. Resources
collaborators • materials & vendors • technical details
1. Towards a Theory of the Invested Object
In January 2015, Buzzfeed’s Longform section published Anne Helen Petersen’s “Big Mother Is Watching You.” The article reviews fitness trackers, home trackers, baby trackers, car trackers, health trackers — everything trackers — in pursuit of the twinned perils and promises of our headlong rush into surveillance-based quantification. And it is headlong. As Petersen reports,
By 2018, there will be 60 million fitness trackers in use worldwide. In, outside, and around the body and our homes, the devices just keep coming — in part because the funding does as well: As of September 2014, $1.4 billion in venture capital funding has been directed toward the wearable and biosensing market; by 2018, wearable sales are expected to push $30.2 billion.
The perils, of course, are familiar to anyone who has spent time with the dystopian novels or films of the twentieth century. Petersen’s title nods to the most emblematic, 1984, but constant controlling surveillance features in other classic works like The Running Man, We, and Minority Report. If the promise of quantification and self-surveillance is that “capturing data that used to be inaccessible and turning it into something knowable,” (Petersen) then it all goes bad when this knowledge, and thereby control, is turned over to large, unaccountable organizations, be they corporations or totalitarian governments.
Or, as the article puts it,
Fears of what can be done with this data are not unfounded. In 2013, a San Francisco man was convicted of vehicular manslaughter using Strava data concerning his speed on his bike, and an upcoming civil case will be the first to use Fitbit data; in that case, Fitbit is being used to protect the individual — a personal trainer, injured on the job, who claims that her baseline activity levels remain below average for someone of her age and occupation.
It’s not difficult to imagine a future in which similar data sets are wielded by employers, the government, or law enforcement. Instead of liberating the self through data, these devices could only further restrain and contain it. As Walter De Brouwer, co-founder of the health tracker Scanadu, explained to me, “The great thing about being made of data is that data can change.” But for whom — or what — are such changes valuable? (Petersen)
These cases are recent; we are at the precipice. But when these works first appeared — 1948, 1982, 1921, 1956 — we were still decades away from the technology to bring complete surveillance into reality. Yet despite consistently imagining the dangers of surveillance in all their grotesque contours, we still worked to bring these capabilities into reality. Why?
Why did we make Lively, the device that allows you to spy on your elder parents under the aegis of care? Why the Canary Project, ready to spy on your teens whenever they are in the car (you know, for safety!)? Why Point, listening to your home, Dropcam watching it, and Nest deciding exactly how warm it is? Spire to tell you you’re slouching and gloves to determine exactly who is at fault in your worker’s comp claim? Sense will watch you sleep and, the ultimate, Pavlok, the bracelet that allows you to shock yourself into good habits.
Why is the Bill and Melinda Gates Foundation paying for biosensors to track “engagement” in schoolchildren (Simon) — with all the assumptions about the measurability of interest and ability to quantify learning such a project implies?
In January 2015, my answer was simple; it was pure lack of imagination. These objects, which I like to call watcher objects, were the product of an engineering culture where our capabilities had outrun our dreams. Once we had brought Vannevar Bush’s Memex to life, the only big projects left were those circulating in our pessimistic science fictions; what else could we do with our sensors and networks but use them to create objects that noted and reported? (Okay, we could also make robots to shoot and bomb people — but that’s another paper.)
In this way, our relative lack of popular positive technological dreams is as much the fault of the artists as the technologists; after all, when engineers act in concord with the traditional colonial engineering imagination (see, for instance, the “Engineer’s Imperialism” chapter of Michael Ada’s Dominance by Design), artists have to explain why we have not offered a better view of object-data-human relations or why, if we have, the engineers have not heeded it.
Janet Murray, in “Inventing the Medium,” the essay by which she introduces the New Media Reader, surveys the development of both belief in the technical imagination and perspectives critical of it. Those working from the latter she calls humanists. Rather than put forth a better narrative or a more compelling story through which to understand our technical selves, the humanists primarly chose repulsion and refusal, she writes:
One can think of the humanist strand as dramatizing the problem amplifying our discomfort by denaturalizing the rituals by which we deny it. The disciplinary humanists in this volume, whether artists, theorists, or scholars, are all engaged in foregrounding our cultural confusions, tuning up our sense of existential befuddlement before the scientifically revealed world of the twentieth century. (Murray 4)
And yet, if we survey the current state of object-data-human relations, we see the humanist complication has had no effect in creating a better or more-human relationship. This is rooted in the strand’s fundamental oppositionalism:
They find the punchcards of the early information age of little use. They are surveying the wreck of ideologies, coming to terms with the failed promises of print, the horrifying trajectory of the rationalist arrow. They insist that we experience the flickering focus, the slipping away of meaning between the signifier and the signified that is the intellectual predicament of the second half of the twentieth century. (Murray 4)
A refusal to acknowledge the tangible benefits technological developments and engineers have brought and an inability to locate meaning in the ambiguous, however flickering it may be, makes it difficult if not impossible to provide a template and impetus for a more humane bent to technical culture. It leaves the engineering-minded without a concept for better or target for improvement.
By engaging contemporary technical capabilities to eat away at the beliefs sustaining our current rush into intimate surveillance, we can offer ideas that animate and embody more humane beliefs, and thereby offer better possibilities for engineers and technologists to turn their skills towards.
In Towards a Theory of Invested Objects then, I will investigate the types of object-data-human relationships at the foundation of contemporary digital culture, their origin in the prehistory of computing and nineteenth-century conceptions of logic and truth, and the path they took between the two. Focusing on the concept of information and, in particular, its expression in cybernetic theories, we can see how the notion of data as contextless and disembodied won out over more contingent understandings and how that enables a problematic surveillance culture.
Next, I consider ways to pollute the possible with ideas counter to those that promote such a culture. The objects that populate our lives can be considered to be part of a flow of ideas, the technology needed to bring these ideas into being, and the lived experience of their instantiation; polluting the possible means setting loose prototypes of the ideas we want into this flow so they may be noticed, chosen, expanded, and influence future cycles of technological development. The notion is rooted in an understanding of the past filtered through concepts from material culture history and an understanding of the future shaped by Joseph Voros’s futures cone. Polluting the possible is similar to critical design in the way it views the possibilities of speculative design; it diverges in its earnestness.
The following three sections outline the characteristics required from an information theory pollutant and introduce the broad strokes of my project, Oublié/trouvé. O/t is a hardware–software system for saving and reflecting on memories. It is an embodiment of one kind of invested object.
Rather than undertake the traditional user-centered design approach or quantitative research, this project pulls from the second-wave phenomenological research approach outlined by practitioners like Susan Kozel.
She explains this approach to truth:
As a first-person methodology, a phenomenological description is received subjectively. As a purveyor of lived experience with the potential for new knowledge contained within it, one person’s phenomenological account can be received by others within circles of shared truth. Truth according to this model may be objective and verifiable through repeated experiments, but it also may be entirely unrepeatable and subjective. (24)
The same philosophy underlies the studio-based practice-as-research approach written about by Graeme Sullivan, Estelle Barrett and Barbara Bolt. Describing Bolt’s theories, Barrett writes,
Rather than constituting a relationship between image and text …, materialising practices constitute relationships between process and text— of which the first iteration is necessarily the researcher’s own self-reflexive mapping of the emergent work as enquiry. A dialogic relationship between studio practice and the artist’s own critical commentary in writing of the creative arts exegesis is crucial to articulating and harnessing the outcomes of these materialising practices for further application. (Barrett & Bolt 5, emphasis original)
As a project whose aims are to question the positivism that sits at the core of more traditional research methodologies, it only makes sense for the invested objet project to move away from such methodologies and into the more contingent and practice-based. This means the primary goal for the project is to explore my experience of different roles — designer, hardware manufacturer, software developer, tester — in the conception and creation of an artwork that plays with personal data and information. Within this paper, which serves as exegesis to the project, it means using I and foregrounding anecdotes and impressions.
The secondary goal of the project, as a work of pollution as well as as a phenomenology, is to resonate with other technically inclined artists and peers. To investigate the success of this secondary goal, I undertook two short surveys, despite the majority of the project remaining inward-looking.
The last sections cover the process of developing Oublié/trouvé, reflections on my experience in the various roles, a review of related works, and a look at responses to a marketing site explaining the project as a consumer product.
But let’s begin at the beginning.
2. The Problem, or How It All Began
The language and ideas we use today to discuss the relationship between machines, data, and humanity remains that of the engineers who developed digital technology in the years around the Second World War, working for the U.S. government, Bell Labs, and ARPA–funded university departments. The men — and they were pretty much all men — who laid the foundations for computer engineering worked with assumptions and aspirations based in Claude Shannon’s information theory, itself heir to nineteenth-century assumptions about mathematics and the perfectibility of human thought, and Norbert Wiener’s parallel theories of cybernetics. Both of these featured as topics of discussion and elaboration at the ten Macy Conferences, from 1948 through 1953. The conferences were the source of many small and large assumptions about the meaning and uses of computers and their relationships to humans; these assumptions spread to larger cultural venues through the the next few decades as the cyberneticists met the counterculture, often as a result of the ever-present Stewart Brand. Total saturation was reached in the 1990s when Wired, also founded in part by Brand, wove the story of our networked age from these same strands.
As N. Katherine Hayles relates in her thorough and insightful investigation of the evolution of cybernetic theories, the information theory that took hold at the conferences was one that “conceptualized information as an entity distinct from the substrates carrying it” (ix) and finds its essential cybernetic expression in the notion that “humans and machines are brothers under the skin” (50).
The arguments in the case, she explains,
were deployed along three fronts. The first was concerned with the construction of information as a theoretical entity; the second with the construction of (human) neural structures so that they were seen as flows of information; the third with the construction of artifacts that translated information flows into observable operations, thereby making the flows “real.” (50)
The idea that we see expressed in watcher objects today, that data is out there to be gathered and deployed, to be hoovered up and processed into life-changing suggestions, is predicated on this first front, the theoretical construction of information as “a mathematical quantity, weightless as sunshine, moving the rarefied realm of pure possibility, not tied down to bodies or material instantiations” (Hayles 56).
We hear its echo in arguments that “information wants to be free,” as if information were an autonomous creature with its own desires. We see it in suggestions that properly anonymized, data is free of us, and in the notion underlying most watcher objects: that data lives in the aether to be harvested and milled. In order to imagine a project that would work to counter and complicate these claims, it is worth first taking a trip through the historical construction of information.
We are most capable of proposing countermeasures when we understand that the way we discuss data and technology today is a specific point of view with a particular source.
James Gleick takes us there in The Information, a work of popular information hagiography that takes a teleological perspective on the development of our concept of information from prehistory to today. Fortunately, the book’s intellectual flaw is its strength as a source for us, providing history alongside a prime demonstration of the methods used to present particular ideas about information as incontrovertible facts of nature.
Gleick begins by locating pre-information concepts historically in African talking drums and the use of bonfires and lanterns as signals in wars from the Trojan war to the American Revolution. These, however, are not true codes because as unwritten works, they are unable to access the logical powers of abstraction. “Logic might be imagined to exist independent of writing — syllogisms can be spoken as well as written — ,” he writes,
but it did not. Speech is too fleeting to allow for analysis. Logic descended from the written word …. Logic turns the act of abstraction into a tool for determining what is true and what is false: truth can be discovered in words alone, apart from concrete experience. (Gleick 37–38)
The development of writing, whose primary characteristic is that it “separated the speaker from the listener, by so many miles or years” (Gleick 30) is the crucial break between the preliterate and contextual and the abstract, a rupture required to set loose technology. But writing is not pure enough.
The paths of logic into modern thought are roundabout, broken and complex. Since the paradoxes [that arise under close examinations of logic] seem to be in language, or about language, one way to banish them was to purify the medium: eliminate ambiguous words and wooly syntax, employ symbols that were rigorous and pure. To turn, that is, to mathematics. By the beginning of the twentieth century, it seemed that only a system of purpose-built symbols could make logic work properly — free of errors and paradoxes. This dream was to prove illusory; the paradoxes would creep back in, but no one could hope to understand until the paths of logic and mathematics converged. (Gleick 41)
Throughout his history, Gleick repeatedly emphasizes progress as the continued spread of abstraction or rather recognition of the immanent abstract in the world. This is fundamental to the deployment of what Hayles terms the Platonic backhand, one of “two moves in particular that played important roles in constructing the information/materiality hierarchy,” which is to say, the construction of information as weightless. She explains,
The Platonic backhand works from inferring from the world’s noisy multiplicity a simplified abstraction. So far so good: this is what theorizing should do. The problem comes when we move circles around to constitute the abstraction as the originary form from which the world’s multiplicity derives. Then complexity appears as a “fuzzing up” of an essential reality rather than as a manifestation of the world’s holistic nature. (Hayles 12)
Thus in Gleick’s telling, each moment in the history is a step towards recognizing the abstract truth until it emerges all around us. The core of life is information expressed as genes and the core of ideas is information in the guise of memes.
Charles Babbage’s great insight upon the discovery of Joseph-Marie Jacquard’s loom cards is the recognition of abstraction:
What caught Babbage’s fancy was not the weaving but rather the encoding …. The notion of abstracting information away from its physical substrate required careful emphasis. (Gleick 109)
Form remains despite changes in thread or color. Ada Lovelace’s genius is the same — she extends the discovery of essential abstraction into processes themselves. (Gleick 116–17)
As is the case for Hayles, my argument is not that abstraction is useless, incorrect or not crucial to computing. Rather, I’m arguing it is foundational and that is problematic. “The point of highlighting such moments,” she writes, “is to make clear how much had to be erased to arrive at such abstractions as bodiless information” (12). The point here of calling attention to the valorization of abstraction is to consider the stories we tell to make such severing appear natural as well as historical.
To return to history, in this story, the “the paths of logic and mathematics converged” most strikingly in the work of George Boole, who gave his name to Boolean logic, the system of logic that lays at the heart of computers today. For Gleick, Boole is a conqueror: “Until now logic had belonged to philosophy. Boole was claiming possession on behalf of mathematics” (164). In this slippage, logic as a way of describing a search for truth moves from one option among many ways to investigate the world and the meaning of our existence in it — that is from a philosophy — into the claim to factual description mathematics is meant to be.
Boole’s system encoded logical propositions into mathematical equations.
The encoding, the conversion from one modality to the other, served a purpose. … In the case of symbolic logic, the new form was suitable for manipulation by a calculus. The symbols were like little capsules, protecting their delicate cargo from the wind and fog of everyday communication. How much safer to write [an equation] than the real-language proposition for which, in a typical Boolean example it stood ….
The safety came in no small part from draining the words of meaning. …
[Language] was seen distinctly now as an instrument with two separate functions: expression and thought. Thinking came first, or so people assumed. To Boole, logic was thought — polished and purified. (Gleick 165)
In this description, a number of interesting phenomena can be observed. We see the bifurcation of information, described in terms of language, and its carrying case. We see the Platonic backhand, with the transmutation of logic into thought and messy, fuzzy “wind and fog” of language relegated to expression. We see these expressed in terms of safety, defining intelligible meaning as dangerous.
The development of this argument for the perfection of the mathematical finds it apotheosis in the work of Bertrand Russell, in particular his collaboration with Alfred North Whitehead, the Principia Mathematica.
[T]heir ambition was nothing less than the perfection of all mathematics. This was finally possible, they claimed, through the instrument of symbolic logic, with its obsidian signs and implacable rules. Their mission was to prove every mathematical fact. The process of proof, when carried out properly, should be mechanical. In contrast to words, symbolism, they declared, enables “perfectly precise expression.” (Gleick 178, emphasis mine)
However, Gleick recounts, “The more rigorously they built, the more paradoxes they found” (179). Russell’s primary paradox, the one named for him, was to consider the set of all sets that are not members of themselves.
To eliminate Russell’s paradox, Russell took drastic measures. The enabling factor seemed to be the peculiar recursion within the offending statement …. Russell’s paradoxical set relies on the meta-set: a set of sets. So the problem was the crossing of levels, or, as Russell termed it, a mixing of types. His solution: declare it illegal, taboo, out of bounds. No mixing different levels of abstraction. No self-reference; no self-containment. (Gleick 180–81)
This was logic finally pure, contextless and ultimately atomized. The backhand is complete; no messy life is admitted.
It was in this environment of thought that Claude Shannon took up Boole’s logic and eventually developed his theory of information. This work took place alongside mechanical computing developments, such as Vannevar Bush’s Differential Analyzer, on which Shannon had worked at MIT, and the ENIAC, developed during World War II.
Theory and machine would collide after the war and, with the invention of the transistor, send us headlong into the computer age.
Information Theory & the Cyberneticists (the 1950s and 1960s)
Significant development of and discussion around the theory of human-computer-information relationships occurred in the late 1940s and early 1950s at the Macy Conferences. It was there that Shannon brought information theory to his compatriots and there that Norbert Wiener developed cybernetics.
When Claude Shannon first discussed his information theory as a mathematical theory of probability unconcerned with the content of the messages it described, he emphasized its limited applicability to problems of sending messages, as many as possible, through contemporary communications systems (Hayles 54). However, within the context of postwar positivism, the allure of abstract and simple theory was too much to resist.
Or, as J.C.R. Licklider put it,
It is probably dangerous to use this theory of information in fields for which it was not designed, but I think the danger will not keep people from using it. (quoted in Gleick 233)
The culture of post-war technology, metastasizing the true gains of engineering into an assertion of the preeminent value of mathematic description — abstract and simplified over messy and difficult — encouraged the adoption of a theory that refused to consider context and the production of meaning. Rather it claimed information as a free material.
Along with these claims came an emphasis on systems as vessels for homeostasis, the closed-loop alternative to the observer-influenced and subjective reflexivity.
This was Norbert Wiener’s cybernetics. Wiener was a child prodigy who began by studying symbolic logic with Russell and spent the war years working on anti-aircraft systems. He considered information theory and cybernetics to be the same theory under different names, but he suggested the applicability was far wider: “Cybernetics, he wrote in his memoirs, amounted to ’a new interpretation of man, of man’s knowledge in the universe, and of society’” (Gleick 237–38). Or, as Hugh Dubberly and Paul Pangaro put it, “the study of what in a human context is sometimes loosely described as thinking and in engineering is known as control and communication” (Dubberly & Pangaro 130).
Other options were suggested. Reflexivity, Russell’s old outlawed bugaboo and an alternative theory of information promulgated by Donald MacKay, suggests that context and structure are vital to understanding information, that “subjectivity, far from being a morass to be avoided, is precisely what enables information and meaning to be connected.” Reflexivity, however, “lost because specifying and delimiting context quickly ballooned into an unmanageable project” in the minds of researchers (Hayles 56–57).
The conceptual gains and losses in the triumph of Shannon and Wiener over MacKay can be understood by contrasting the implications of these viewpoints. As Hayles puts it,
The price [contextless information] pays for this universality is its divorce from representation…. The price [reflexive information] pays for embodiment is difficulty of quantification and loss of universality…. Making information a thing allies it with homeostasis, for so defined, it can be transported into any medium and maintain a stable quantitative value…. Making information an action links it with reflexivity, for then its effect on the receiver must be taken into account…. (56–57)
She continues to point out that homeostatic systems re-enact the same pathologization of difference as Anglo-American engineering culture.
Carolyn Marvin notes a decontextualized construction of information has important ideological implications, including an Anglo-American ethnocentrism that regards digital information as more important than more context-bound analog information. (19)
Using Norbert Wiener’s electronic rat as an example of the embodiment of homeostatic ideas and the tendency to “construct the human in terms of the machine,” Hayles writes,
Presuppositions embodied in the electronic rat include the idea that both humans and cybernetic machines are goal-seeking mechanisms that learn, through corrective feedback, to reach a stable state. Both are information processors that tend toward homeostasis when they are functioning correctly.
Given these assumptions, it was perhaps predictable that reflexivity should be constructed as neurosis in this model. (65)
Now competing information theories are not merely a sign of different approaches, but context becomes sickness itself. As Hayles relates, cybernetics eventually moved on to incorporate questions of reflexivity and context. This slightly loosened version of cybernetics began to spread into the counterculture. As it did, it carried the primary belief in unmoored information into the mainstream as the rebels of the 1960s became the technical establishment of the 1990s and beyond.
In this way, information has gone from Boole and Russell’s material of perfect knowledge, which has no use for context, to a broader suggestion that context and reflexiveness is destructive.
Mixing With the Counterculture
But how did a theory, even a compelling one, move from the defense department, industrial labs, and universities into wider culture? Fred Turner, in a foundational work of computing history, From Cyberculture to Counterculture, asserts cybernetic ideas spread through interdisciplinary sharing and concomitant legitimacy exchange, both within the confines of the Macy Conferences and in postwar research projects as a whole. He explains,
The power of cybernetics and systems theory to facilitate interdisciplinary collaboration emerged in large part thanks to the entrepreneurship of Norbert Wiener and the research climate of World War II. Wiener did not create the discipline of cybernetics out of thin air; rather he pulled its analytical terms together by bridging multiple, if formerly segregated scientific communities. …
Because of the changes in scientific practice brought about by World War II, specialists in one discipline began to do things that had previously been considered the proper domain of specialists in other areas. They could justify such leaps across disciplinary boundaries by drawing on the rhetoric of cybernetics. If biological principles were at work in machines, then why shouldn’t a physiologist contribute to work on computers? If “information” was the lifeblood of automatons, human beings, and societies alike, why shouldn’t a mechanical engineer become a social critic? With such justifications, Wiener and a string of later cyberneticians and systems theorists reached across disciplinary boundaries and claimed a universal relevance for their new “science.” (Turner 24–25)
While research projects across the country — many funded by Macy attender, futuristic dream weaver (see his “Man-Computer Symbiosis”), and defense department employee Licklider — put these ideas into daily practice, the conferences made sure the ideas were strong and they spread as far as possible.
Over time, the Macy conferences helped refine a number of cybernetic concepts …. They also sent individual participants back to their home disciplines with a deep systems orientation toward their work and a habit of deploying informational and systems metaphors. In this way the Macy meetings helped transform cybernetics into one of the dominant intellectual paradigms of the postwar era. (Turner 26–27)
Once this dominance was established in universities and industrial research communities, it was just one more step from the institutions of the Bay Area and Silicon Valley to the wider counterculture.
In “How Cybernetics Connects Computing, Counterculture & Design,” Dubberly and Pangaro take up the “universal discipline,” as Geoffey Bowker calls cybernetics (quoted in Turner 25), and charts its interlocking paths. The diagram that accompanies the essay literally draws the lines between Macy attendees like Licklider, Wiener, Gregory Bateson, Margaret Mead and Heinz von Forester; scions of personal computer development at places like PARC, SRI and the MIT Media Lab; and counterculture figures like Ken Kesey. At the center stands Stewart Brand, Merry Prankster, founder of the Whole Earth catalog and Wired, and general connector. “Brand traveled between — and connected — several communities,” they write, “cybernetics (Bateson, Mead, and von Forester), computing (Englebart, Kay, Nelson and Negroponte), and of course counterculture (Ken Kesey, the Merry Pranksters, and other communards” (Dubberly & Pangaro 137).
The canonical Brand-facilitated interface between the counterculture and the cyberneticists was of course the Whole Earth Catalog, first published in 1968. Here the supplies needed by tune-in drop-out back-to-the-landers like wood stoves were for sale next to early computers and books from Wiener and von Forester.
However, as Felicity D. Scott recounts it was not the only one. There was “Spacewar,” Brand’s article for Rolling Stone, that
worked to naturalize the connection between a distinctly techno-utopian … image of a better world promised through technology and the alternative ethos of hippie culture, to script a future for “hippie modernism” not within the low-tech, do-it-yourself domain prevailing within the Whole Earth Catalog but rather in the less-than-accessible world of computers that nevertheless haunted its pages. Desublimating this connection to computers, the text focused on the intermingling or cohabitation of the “youthful fervor and firm dis-Establishmentarianism of the freaks who design computer science” with the military and corporate interests funding the computer industry, for whom such freaks often worked…. (Scott 103)
In addition to this popular storytelling, there were also personal encounters staged by Brand.
Brand’s entrepreneurial brokering of “a series of encounters” or “network forums” to bring these groups together was not incidental to the emergence and conception of what Turner calls networked modes of techno-social life, fostering an ethos of feedback-based communication and a common vision of technology as a tool for personal and social change. (Scott 103)
Cybenetic communication is at it again.
While this paper cannot delve deeply into the transformation from computers as instrument of war to computers as personal empowerment (and besides Fred Turner has done such a thorough job before us), the transformation was well underway by the time the Portola Institute was founded in 1966 and completed as Wired — product of Whole Earth-related writers and mainstream voice for the tech industry, rolled out in 1993. As Turner explains, this was all Brand and his compatriots.
In the last two decades of the twentieth century,
ideas born within Whole Earth–derived network forums became key frames through which both public and professional technologists sought to comprehend the potential social impact of information and information technologies. …
At the same time, and by means of the same social processes, members of the Whole Earth network made themselves visible and credible spokesmen for the socio-technical visions they had helped create. …
Brand and other writers and editors of Whole Earth publications developed extraordinary reputations as journalists, [winning prizes for both the Whole Earth Catalog and Wired, the latter cofounded by Brand]. They did so, however, by building the communities on whose activities they were reporting. (Turner 6–7)
As their fame grew and they were able to tell their stories to higher levels of powerful men, the values and stories of cybernetics, including atomized information became as natural facts of life, essential to computing and all technological progress.
The Problem: Information Today
This comes to full bloom in a book like Gleick’s. Not only does the story of technological progress become a teleology of abstraction but the cybernetic understanding of information becomes universal.
In the chapter “Entropy and Its Demons” information and entropy are shown to be the same and physics and universal laws are subject to cybernetic understanding. Then in “Life’s Own Code” RNA and DNA are information technologies on the level of the body. Humans are in fact tiny computers. Even our minds are not spared: “Into the Meme Pool” demonstrates that ideas are just as subject to systems laws “as the neurons they inhabit” (Gleick 311).
Gleick also draws in art technologies, identifying a fundamental difference of type between painting and early photography.
By painting or drawing, an artist — with skill, training, and long labor — reconstructs what the eye might see. By contrast, a daguerreotype is in some sense the thing itself — the information, stored, in an instant. (Gleick 376)
Ignoring the incredibly narrow conception of painting, the idea that a photograph or daguerreotype is more real is also predicated on the cybernetic notion of information transfer, where truth can be sent from reality to a storage medium untainted by the choices and interpretations of a mind is pure cybernetic thinking.
Cybernetic ideas were taken up in more complex and exploratory way by artists throughout the 1960s and 1970s, from Gordon Pask and his Colloquy of Mobiles at ICA London’s Cybernetic Serendipity to Ant Farm and their Truckstop Network and pneumatic structures.
They also found their way into design projects like the Cranbrook design trip undertaken in 1973 by students of a new interdisciplinary design program in “product, graphic, and interior architecture.” Illustrated by Katherine McCoy and Edward Fella, the piece includes “a lexicon of the terms the students encountered during their visits — adhocism, head dudes, design freaks, architecture machines, pneumatic-nomadic environment — which paints a linguistic landscape of emerging design philosophies (Blauvelt 256–57).
Dubberly and Pangaro’s essay ultimately ties cybernetic thoughts from art into the DNA of contemporary interaction design itself.
What Pask said about architecture also applies to design for human-computer interaction. A software program interacts with its “users,” serving them and also constraining their behavior. Software, too, only makes sense when framed as part of larger systems that also include humans. These larger systems are what interaction designers design. …
In many ways the story of cybernetics is the prehistory or backstory of interaction design…. Wiener’s notion of feedback is the very foundation of interaction design, and thus the foundation of any framing of design as engaging people rather than simply as giving form to objects. (140)
Not only do these ideas animate the designs of the technology we use today, but the carved away abstractions animate our programmers — even when it is problematic.
Ellen Ullman, whose memoir of life as a programmer appeared in 1997 — right around the time the first dotcom boom took off and Wired started showing up in everyday homes — tells it like this:
I’d like to think computers are neutral, a tool like, any other, a hammer that can build a house or smash a skull. But there is something in the system itself, in the formal logic of programs and data, that recreates the world in its own image. … We think we are creating the system for our own purposes. We believe we are making it in our own image. We call the microprocessor the “brain”; we say the machine has “memory.” But the computer is not really like us. It is a projection of a very slim part of ourselves: that portion devoted to logic, order, rule, and clarity….
We place this small projection of ourselves all around us and we make ourselves reliant on it. To keep information, buy gas, save money, write a letter — we can’t live without it any longer. The only problem is this: the more we surround ourselves with a narrowed notion of existence, the more narrow existence becomes. We conform to the range of motion the system allows. We must be more orderly, more logical. Answer the question, Yes or No, OK or Cancel. (90)
The systems that Dubberly and Pangaro love to design can become a straightjacket on the ground and the people who are implementing the systems, living with them closely, may be the first to notice.
More inside-the-house criticism comes from Jaron Lanier, a longtime software engineer known for contributing to early virtual reality work. He picks up the argument against the conflation of machines and humanity in his manifesto You Are Not a Gadget. After establishing his engineering-insider bonafides, Lanier hits out at a contemporary culture where we accommodate ourselves to the limitations of machines:
When developers of digital technologies design a program that requires you to interact with a computer as if it were a person, they ask you to accept in some corner of your brains that you might also be conceived of as a program.(4)
That is, as the cyberneticists suggest, human thought and free information are portrayed as one and the same. Lanier ties this to a penchant for self-abdication, in this case expressed as thinking machines will know us better than ourselves, a sickness to which many prominent technologists are prone. The same dreams that Lanier critiques coming from “cybernetic totalists” like Marvin Minsky and Chris Anderson (once editor-in-chief of Wired) — dreams of uploadable minds and science robots who iterate on theories we no longer understand — are those Hayles identifies as the outcome of cybernetic theories. These “post-biological” hopes and all the humanity-usurping developments they support are rooted in the initial division of information from agar, the likeness of neuron and circuit that frees data and divorces it from the analog context — the bodies — in which it arises.
This division goes back to Boole and Russell’s claims for the ultimate truth value of propositions lifted from fuzzy language into rigid symbols and makes possible histories like Gleick’s which assert — with little fear of contradiction — that knowledge is only available when it persists through the removal of context.
Lanier also identifies some of the microaggressions that flow from this conception citing the myriad ways we are asked to accommodate databases and their schema. In fitting ourselves to the standardizing, context-eschewing demands of information and the computers built to contain and manipulate it, we lose some of the value of being known and recognized for our idiosyncrasies. (Lanier 52)
This is the ultimate result of the information imagined by Boole and Russell, transported into machines and programming culture by Shannon and Wiener, and spread into broad cultural expectations and experiences of technology by Brand and his merry band of journalists. This includes not just how we talk about computers but also how we design for and live with them.
Is it possible to work against this? Can information and machines become sources or facilitators of unexpected weirdness, idiosyncrasy, and contextualized information?
3. Polluting the Possible
The cybernetic conception of information as a “free-floating, decontextualized, quantifiable entity” is the result of a centuries-long project tying technological progress to abstraction and this brand of abstraction into the core of our artistic and scientific cultures. It powers watcher objects and a culture that seems to shear off our idiosyncrasies.
Lanier provides one possible avenue for revolt when he describes what technologists do:
Technologists don’t use persuasion to influence you, or at least we don’t do it very well…. We make up extensions to your being, like remote eyes and ears (webcams and mobile phones) and expanded memory (the world of details you can search for online). These become the structures by which you connect to the world and other people. These structures can in turn change how you conceive of yourself and the world. We tinker with your philosophy by direct manipulation of your cognitive experience, not indirectly, through argument (6).
That is, we can propagandize through objects, by making and distributing the means for the relationships we seek.
In “How Things Shape Us,” material culture historian Manuel Charpy makes the historical case for persuasion and expression via objects as he traces the influence of physical objects on Victorian culture and the manner in which the objects reflect the cultural preoccupations of the time.
Charpy directly ties material culture history to technological history, writing,
The history of technology has only recently abandoned the epic story of great inventions and has fostered new links with the history of material culture. Such a convergence between technology and material culture is to be found in the way in which technologies — even relatively modest ones — constantly are seen as key to changing social practices and our daily lives. (200)
By way of example, he goes on to cite not only the common story of the industrialization of time, in which the proliferation of cheap clocks make abstract time and railroad timetables plausible, but also the manner in which objects “imposed a daily discipline through their surfaces” (204). High-pile rugs, delicate cuffs, easy-to-crease fabrics, glass furniture, and white underwear took marks easily and therefore worked to enforce high demands for self-control through the threat of tattletale disorder.
Charpy also charts the role objects in reflecting and supporting modern “self-fashioning.” “The multiplication of personal objects allowed for the affirmation of the individual and acted as ways to acquire consciousness of the self,” he writes, citing the influence of writing desks (fitted out with mirrors for women), dressing gowns, and a proliferation of locked boxes and diaries: ”It was this accumulation of new tools that encouraged a new objectification of the self” (208). This self-objectification blossoms into self-souvenirs and the cabinets to host to them, before finally reaching its apotheosis in the mania for collections, where a system of objects becomes a reflection of the collector.
Collecting objects was an ideal activity of self-fashioning. … Collecting objects implied using one’s knowledge but also reorganizing the world around oneself. While the order mimicked a scientific one, by the second half of the nineteenth century is had become an affirmation of taste and the expression of one’s personal order. … These collections were material autobiographies to be seen as forms of possessive individualism. (Charpy 210)
As descendants of these early modern people, we continue these sorts of relationships with the objects we own and touch. From here, it is only the smallest step to asserting that creating an object that instantiates the relationship we want to see and explore may make this relationship more common in fact and in imagination.
If Charpy — and more broadly, the discipline of material culture history itself — makes the case for the workings of the physical on the cultural, Hayles calls out to the intellectual construction of cultural meanings with her notes on including the literary in her examination of cybernetic concepts in culture: “Literary texts … actively shape what the technologies mean and what the scientific theories signify in cultural contexts,” she writes. “[C]ulture circulates through science no less than science circulates through culture. The heart that keeps this circulatory system flowing is narrative” (21, emphasis mine).
Through the creation of devices and objects that operate on reflexive principles, that value context, we can provide imaginative alternatives to the current watcher objects and thereby a possible alternative narrative of coexistence with machines — one on human terms. Through the deliberate stories we tell, we can influence the reception these objects receive. Rather than attempting to block the circulation of meanings, we can introduce our own small bacteria.
The diagram I use to picture this system of cultural construction into which prototypes should be injected is shown in figure 3.1. Imagination and the world of art and cultural production communicate concepts that find their expression in physical objects, which are of course only made possible by tools resulting from engineering work. The instantiation of the object itself moves ideas from the sphere of culture to the physical world. Here a loop occurs between the owner of the object and the object itself, in which sense experience combines with the owner’s conception to invest meanings in materials. At the same time, this relationship is considered internally and projected outwards both to our contemporaries and to future humans, through the creation of future artifacts. That is, today’s tools and toys are the next millennium’s anthropological grist.
However, as John Styles notes in his study of London Foundling hospital tokens, it is only through the interplay of written culture and physical objects that a full transmission of meaning into the future — or back into the cultural sphere — is possible (Styles 165, 168).
And so, to fight back, to infect this system, we look to the creation of prototypes and their tales.
While my conception of the cultural system may be idiosyncratic, the idea that creating prototypes of what we want to see in the world in order to influence the flow of ideas into objects is not. In fact, as Dubberly and Pangaro write, “Prototyping was common in the cybernetics community — something Turner calls ‘a rhetorical tactic,’ a method of increasing awareness and spreading influence” (Dubberly and Pangaro 136).
More recently, critical design, a movement that surfaced twenty years ago in the work Anthony Dunne and Fiona Raby, instantiatesd this approach in the world of design. It too was born of a dissatisfaction with technological tales:
We coined the term critical design in the mid-nineties when we were researchers in the Computer Related Design Research Studio at the Royal College of Art. It grew out of our concerns with the uncritical drive behind technological progress, when technology is always assumed to be good and capable of solving any problem. Our definition then was that “critical design uses speculative design proposals to challenge narrow assumptions, preconceptions, and givens about the role products play in everyday life.” (Dunne & Raby 34)
They go on to specify that critique is not meant to be only negative but rather that “it can also be a gentle refusal, a turning away from what exists, a longing, wishful thinking, a desire, and even a dream” (34–35).
To understand in more detail what critical design entails, we can look to Dunne & Raby’s A/B Manifesto project, where they oppose the characteristics of affirmative, a.k.a traditional, design and critical design. (See figure 3.2.)
Here we discover a conception of design that uses the approach as a medium for questioning the underlying assumptions of our world. Consider problem solving vs. problem finding, design as process vs. design as medium, and provides answers vs. asks questions: rather than seek to solve given problems, critical design works to unearth and display problems — often by proposing solutions to them. By opposing science fiction and social fiction, futures and parallel worlds, Dunne & Raby indicate their refusal to be swept away by the promises of technical application; rather, critical design looks for technical implication.
They summarize the technique most pithily towards the end of Speculative Everything, asking, “Can design operate in this way, borrowing methods from literature and art and applying them to the real world as thought experiments?” (Dunne & Raby 170).
The approach I take in the invested objects project is superficially similar: In the project I create a dream object using the vernacular of standard design deliverables and software development. Both critical design and polluting the possible insist that a dream must become physical to change possibility. Dunne & Raby suggest this operates significantly through consumption:
In a consumer society like ours, it is through buying goods that reality takes shape. The moment money is exchanged, a possible future becomes real. If it did not sell it would be sent back, becoming a rejected reality. In a consumer society, the moment we part with our money is the moment a little bit of reality is created. Not just physical reality or cultural but psychological, ethical, and behavioral. (37)Polluting the possible operates on a plane slightly removed from the practicality implicit in critical design. Rather than manifesting reality through channels of commodity consumption, polluting the possible expects cultural consumption to be a realistic pressure point. Yet, despite this difference of application, both critical design and the polluting the possible expect to change the world in a similar way — by creating deviations from the most probable future into a slightly preferable one.
I was delighted when I initially came across Tobias Revell’s version of Joseph Voros’s futures cone (see Revell), and unsurprised when I found it again in Speculative Everything (Dunne & Raby 5). The drawings, which I have interpreted in figure 3.3, illustrate categories from Voros’s “A Primer on Futures Studies, Foresight and the Use of Scenarios.” It is a concise depiction of the way in which the future is projected from the present, like a slide onto a wall. The minimal labelling of the diagram, and the way the internet often presents it without context, allows each group to imagine the relationship that suits it.
For Voros the categories are descriptive, based primarily on what is possible given current technical knowledge and probable given current trends. For Revell, the point to highlight, in addition to the working of projection, is the “wildcard” element. These scenarios encourage observers to expand their considerations. The cone describes the work of design fiction and critical design by describing the space in which they operate.
For Dunne & Raby, the cone is less an expression of spaces than in business-soaked definitions of the future. Scenarios as individual points have disappeared. Instead, these Brutalist cones represent social, almost paranoid futures. The probable is what happens without catastrophe. The plausible contains imaginable disasters.
This is the space of scenario planning and foresight, the space of what could happen. In the 1970s companies such as Royal Dutch Shell developed techniques for modeling alternative near-future global situations to ensure that they would survive through a number of large-scale, global, economic, or political shifts. The space of plausible futures is not about prediction but exploring alternative economic and political futures to ensure an organization will be prepared for and thrive in a number of different futures. (4)
The possible includes the not-impossible scenarios that remain difficult to imagine. And the preferable in the hands of Dunne & Raby is the expression of power, it is distinguished by who is able to define it. In the end, this engagement with power is their animating question.
[A]ssuming it is possible to create more socially constructive imaginary futures, could design help people participate more actively as citizen-consumers? And if so, how?
This is the bit we are interested in. Not in trying to predict the future but in using design to open up all sorts of possibilities that can be discussed, debated, and used to collectively define a preferable future for a given group of people: from companies, to cities, to societies. Designers should not define futures for everyone else but working with experts, including ethicists, political scientists, economists, and so on, generate futures that act as catalysts for public debate and discussion about the kinds of futures people really want. Design can give experts permission to let their imaginations flow freely, give material expression to the insights generated, ground these imaginings in everyday situations, and provide platforms for further collaborative speculation. (Dunne & Raby 6)
It is here we part ways and polluting the possible becomes something other than a work of critical design.
This Is Not a Critical Design Project
Polluting the possible takes as its premise the idea that the experts are biased and quite likely wrong. The cone-based question here is How do we make the preferable strong enough to cause deviations from the probable? and the answer is By creating ever more preferable scenarios here on the tiny point of the present. Standing here on the verge of solipsism (without going over), we don’t need to use the tools of design to inquire into definitions of the preferable; we know what the preferable is — it is what we the maker prefer. By enlarging the possible and reflecting on these explorations, we make a preferable future more likely. If we can’t want what we haven’t yet imagined, then the first step is to start imagining a lot more.
Though large, this is not the only difference. Dunne & Raby insist that critical design must present scenarios that one can plausibly imagine emerging from our present circumstances (4, 43). Polluting the possible would say that insisting on plausibility means ideas will consistently be too conservative, too based on current power structures.
And Dunne & Raby do continually circle back to neoliberal values even while claiming to question neoliberalism. In defining critical design, they argue that the critical can be positive, never discussing that demands for positivity above all is a common tactic for shutting down dissent (34–43). Rather they accept this framing as implicitly true: positive is better than negative.
For this project, though the form is positive, in the sense that it is making real a possibility that I want to see in the world, the effects may not be. I may also fail; I may make something that is useless for its purposes or make something with a terrible effect. I am just aiming for interesting.
Conventionality shows through again for Dunne & Raby in their focus on the individual. They write
Change can happen in a number of ways: propaganda, semiotic and subconscious communication, persuasion and argument, art, terrorism, social engineering, guilt, social pressure, changing lifestyles, legislation, punishment, taxation, and individual action. Design can be combined with any of these but it is the last one — individual action — that we value most. We believe change starts with the individual and that the individual needs to be presented with many options to form an opinion. (Dunne & Raby 160, emphasis mine)
It is this emphasis that underlies their commitment to change via consumerism, as quoted above.
Polluting the possible, which focuses on the individual as a generator of dreams and ideas to be injected into the cultural imagination, would say that if you are effecting change by what you buy, you are still letting others define your dreams. This is not to say there is no room for close exploration, for asking what happens when power doesn’t shift. But it is not the best description for our goals here.
Finally, the two approaches differ in terms of why we insist on making the object, the embodiment of the idea, real and physical. Dunne & Raby focus on prop-ness as vehicle to give ideas weight, to make them seem less fictional (79, 90). For this project, the reasons to insist on the realness, the physicality of the manifestation, the reason to take it beyond sketches, is two-fold: It allows us to to investigate the idea in depth and over time by letting us live with the idea. At the same time, the process of creating and use serves as a ground for observation of the various roles involved in bringing it to life — designer, programmer, user, and makes possible a phenomenological approach to the research. This thread will be picked up again in §7.
Beyond Dunne & Raby
Though Dunne & Raby may be the originators of critical design, definitions of the approach and related uses of design for alternate futures are not limited only to those given by the two. Seeking to place polluting the possible in terms of the wider definitions of critical design, I turned to Matt Malpass’s Contextualising Critical Design: Towards a Taxonomy of Critical Practice in Product Design.
In this thesis, Malpass interviews eight key critical design practitioners and reviews the history and products of the related movements found under the critical design umbrella. The final taxonomic sketch identifies two primary axes by which the different varieties may be distinguished: type of satire and type of ambiguity (211–213). The latter category is outlined in Gaver et al.’s “Ambiguity as a Resource for Design,” which makes the case for the use ambiguity “to encourage close personal engagement with systems” (233). It is a technique used by the invested object project and will be discussed in the next section.
The question of satire however does not apply to this project or polluting the possible; in fact, it points to the final break between these and works of critical design. “It is the instrumental use of satire and how it links to criticism, which forms the foundation of critical design practice. Design and satire are integrated to create a rhetorical language, through strangely familiar form and ambiguity that encourages user engagement by forcing a dilemma of interpretation within the user” (Malpass 171).
These rhetorical gambits are in service to “the primary goal of delivering a deliberate message potent enough to spark contemplation, discussion, and debate…” (Malpass 76). To return to the terms of the A/B Manifesto, satire is used to “make us think.”
The goal of this project is less to spark ideas through provocation than to see what it is like to create and use a system that endeavors to embody a new relationship with information. But what does that actually look like?
4. Characteristics of an Invested Object
Characteristics of the Object
If we are not objects or gadgets, what kind of relationship can we have with an object and what sort would it be? Developments in digital computing have changed our world and expectations of objects — and these we should leverage. As Lanier points out in the previous section, we do now have remote eyes and ears, and that is a compelling development. To set technology and humanity at odds is a failing strategy. The objections and complications of Murray’s “humanists” — who seem to urge us to stop playing with fire and instead return to a pastoral Eden — are easily dismissed as an unenlightened Luddism. And then off go the engineers, refusing the critiques.
Likewise, as we can see with critical design, accepting without discrimination the value of experts and the values of the technocratic — in their case by insisting on a plausible path to the imagined situation — can blunt critical sharpness.
Lanier appears to avoid both, finishing You Are Not A Gadget with his own prescription for more humane technical development. Extolling the “morphing” capabilities latent in VR, he suggests that, when paired with human neoteny (that is, our plastic brains), these will allow for us to create an augmented future that keeps the human at fore and thereby remains humane. For this he uses the cephalopod as metaphor and contrast.
For the invested object, I prefer a half-aquatic metaphor: the lichen. Lichen are actually two creatures, algae and fungus, combined as one. The protection of the external fungus allows the algae to live in environments too dry for survival alone. The photosynthetic properties of the algae provide the energy the fungus needs to live. It is this type of symbiotic experience I hope we can mirror with the invested object.
In this case, we can use digital technology and its veneer of the future path as the shell to protect and shepherd an experience of contextualized information. But what features can we expect from a wee investable lichen?
It should be a real thing.
An investable must be a real thing, which in this case implies both depth and materiality.
Just twenty or so years after the domestication of the computer, there exists already a growing desire to rip pixels off the static and distant screen. Put the screen on your face and call it virtual reality; put the sensors on your skin and call it embodiment; put reality back into the pixels as video and call it augmented reality — do something to make it real again. Lanier echoes Walter Benjamin’s construction of the aura as he expresses the lure of the object:
A real painting is a bottomless mystery, like any other real thing. An oil painting changes with time; cracks appear on its face. It has texture, odor, and a sense of presence and history. (133)
Here he is touching on what Benjamin calls the “authenticity of a thing”: “the essence of all that is transmissible from the beginning” (221).
Later Lanier renders a distinction between the digital and the physical, writing,
[A digital object] will be a flat, mute nothing if you ask something of it that exceeds those expectations [of which aspects are important]. If you didn’t specify the weight of a digital painting in the original definition, it is not just weightless, it is less than weightless.
A physical object will be fully rich and fully real whatever you do to it. (Lanier 134)
It can be easy when considering the aura, or authenticity, to forget that “Art in the Age of Mechanical Reproduction” is actually an argument for the value of a new, de-ritualized art. Benjamin asserts that though film and photography work differently from painting, they allow for “the direct, intimate fusion of visual and emotional enjoyment by the public” and allow for “simultaneous collective experience” in a way painting does not, and therefore still possess an artistic value (Benjamin 234).
And so, contra Lanier, I would say it is not an immanent difference between digital and physical that gives one depth and leaves the other dead and lesser. As ethnologist and researcher Sherry Turkle puts it, technology is ”as much an architect of our intimacies as our solitudes” (Devices 29). Intra Lanier, though, there lies a point central to the invested object project: there are strengths in the physical and the digital that complement one another and we can access them when we have objects that are both physical and digital and thereby become real. That is, a fully rich digital object must have a physical manifestation beyond a screen to be a real thing.
In fact, the watcher objects with whom we began are precisely this combination. They are literally digital lichen (though inside-out from our intention to inter the contextual in the digital) wrapping electronics inside plastics and other materials that can be held, that hold on them physical traces much like the Charpy’s Victorian objects in §3.
In this way, realness also opens up for the invested object project a place in material culture analysis. Beyond saving and reflecting the gestures of its time as Charpy describes, the physicality of the object both makes it available as a projection surface for current meanings and as a meaning-infested material to be projected far into the future.
John Styles sketches the working of the first in “Objects of Emotion,” a study of tokens left at the London Foundling hospital with abandoned infants. These tokens served to identify the children, should the mother return to reclaim them under better circumstances. These, like most objects, encode emotion, he notes, but transmission is a complex function and may not always succeeed.
Objects can transmit emotional messages, carry emotional associations, and evoke emotional responses but frequently they do in such a personal way as to defy broader appreciation. Even when objects are emotionally charged in ways that command wide recognition, that recognition is often restricted to very specific circumstances. Things that exhibit emotional power in one setting can lack it in another. Moreover, even when an object’s emotional charge was widely recognized at some period in history, there is no guarantee it can be recaptured by the historian. (165)
He goes on to explore the meanings embedded in the tokens, which were often fabric or ribbon. The cultural ground — that ribbon was cheap and available for poor women, that ribbon was already a tool for emotional communication — is reflected in the tokens. The figuring on the ribbon, often of hearts, reflects eighteenth century emotional beliefs (Styles 166–68). Likewise, the figuring on invested objects may reflect the uses and emotions invested in them. (The implementation details are discussed in §8.)
Styles complicates this communication of emotion by suggesting it may just be a reflection of the emotions mothers were expected to display instead of being known to be authentic, but this is a criticism that could be leveled at most communications that survive beyond their context. At some point, we must take earnestness on faith. On the other hand, keeping in mind some historian’s demands for text corroboration can only help us shape our communications.
Paying attention to the external design and decoration of our physical object, in addition to surrounding it by texts, means we are at least attempting to leave a trail for those who follow us — and this trail creates an opportunity for the ideas embedded in the objects to be re-inscribed into the cultural sphere once we are gone.
If we want the object inscribing long after we are gone, of course, the materials used are also important. David Gaimster, in “Material Culture, Archaeology, and Defining Modernity,” describes how ceramics become an ideal medium for material cultural transmission from Europeans circa 1400 to 1700. “By virtue of its utility at all levels of society, its relatively short lifespan and its durability in the ground, ceramics have proved to be one of the most sensitive and reliable archaeological indicators of social behavior and mobility” (59). The durability combined with mutating decoration creates an expressive and long-lived object of communication.
Altogether, physical-digital objects provide the base desirable in an invested object: they can communicate across cultures, they are capable of embodying emotional and historical experiences. In addition, our experiences with them are rich and complex.
It should be a thing you can think with.
The richness of physical objects does not lie only in their physical presence and concomitant persistence in the world once we are gone; there is also great value in the relationships and connections we are able to find with our objects. We are able to think with them, as Turkle says.
In The Inner History of Devices and Evocative Objects, Turkle collects stories of the transformative relationships between everyday people and the objects around them. There is E. Cabell Hankinson Gathman, who holds her study abroad to Japan frozen inside a cell phone and Barry, the man for whom computers open the world by making math accessible and tactile. He tells Turkle, “that with a computer and calculator, ‘The numbers are in your fingers …. They put mathematics in my hands and I am good with my hands’” (Turkle, Devices 28). Basic digital items can morph our bodies and minds without VR at all.
In her piece on the digitization of Le Corbusier’s digital archives, Susan Yee articulates the connection to the past derived from physical objects, reveling in her ability to touch the paper Le Corbusier touched and to do the calculations he has scrawled into the margins along with him (Turkle, Objects 32–33). She laments the passing to the digital — “It made the drawings feel anonymous and me feel anonymous” (35) — but we know that we can look for a way to keep the values of both working together — instead of letting the algae escape.
“The acknowledgement of the power of objects has not come easy,” writes Turkle, tracing the reluctance to engage with the power of material culture to the same Western predilection for abstraction that gave us free-floating data. Fortunately, through the 1980s and into today, Turkle and others made it their work to pinpoint and relate the value of objects. This was often found in the very heart of abstraction territory (that is, science) (Turkle, Objects 6–7), which made for a compelling case. Turkle has collected these stories and outlined common functions of object-thinking found within: “Objects are able to catalyze self-creation. … Objects bring together thought and feeling. … [W]e often feel at one with our objects” (Objects 9, emphasis original).
The three capabilities work together to make objects part of us and us part of the world. This push and pull can be seen in the relationship between objects and theory Turkle describes:
One role of theory here is to defamiliarize [objects]. Theory enables us, for example, to explore how everyday objects become part of our inner life: how we use them to extend the reach of our sympathies by bringing the world within.
As theory defamiliarizes objects, objects familiarize theory. The abstract becomes concrete, closer to lived experience. (Objects 307)
This is a goal then of the invested object. To bring the abstract theory of contextualized information into a touchable, desirable, expressive form.
As the essays in Evocative Objects demonstrate, the types of evocative objects — in form and function — are vast. There are objects of transition and passage, of memory and mourning. These are trains and suitcases and Fords and wooden radios. Slime molds straddle a border and serve as objects of meditation and new vision. Objects can contain thoughts and experiences that may only ever get out through thinking with them.
We find it familiar to consider objects as useful or aesthetic, as necessities or vain indulgences. We are on less familiar ground when we consider objects as companions to our emotional lives or as provocations to thought. The notion of evocative objects brings together these two less familiar ideas, underscoring the inseparability of thought and feeling in our relationship to things. We think with the objects we love; we love the objects we think with. (Turkle, Objects 5)
We can build on this. Just as objects we think with are invested with thoughts by experience, they can be invested with the emotions they provoke by their very shapes.
Human-computer interaction researchers Jarmo Laaksolahti, Katherine Isbister and Kristina Höök investigate how physical objects can work to express experiences outside words. Together they created and tested the sensual evaluation instrument (SEI), “an instrument previously developed to facilitate nonverbal self-report of emotion, which consists of eight sculpted objects” (Laaksolahti, et al. 165). These sculptural objects, ranging from smoothly stone-like to spiky, to complexly blobby, were deployed in a number of studies, including one that used photos from the International Affective Picture Set, computer games, and conversation to test the expressive values of the objects across two cultures, Swedish and American.
Explaining the project they write,
We wanted to allow for ambiguous and evocative communication between user and designer, to preserve the rich and multi-layered feelings that users might be having and to create sparks in the designer that could lead to powerful design insights — what Gaver and Dunne [our old critical design friend!] might refer to as ‘inspiration cues’ (Isbister 315)
and later emphasize the value of ambiguity in emotional expression
Language is wonderful for summarizing and categorizing and processing one’s emotions after an experience, but might sometimes be a clumsy tool for communicating affect in the fleeting moment of interaction, particularly if one is feeling a jumble of half-resolved emotions-in-progress. We hoped using the visual sense as well as the sense of touch would elicit different kinds of responses from users, and perhaps allow for greater unresolved ambiguity in their expression. (Isbister 317)
The same emotional ambiguity that historians such as Styles might identify as a liability becomes a support for an expressive emotional complexity. And it worked. While participants from both cultures complained some emotions were lacking from the set, all were able to express emotion with the objects and their choices were consistent across most of them. (A few were hard to interpret — sometimes only for one culture.) Although some differences were observed, particularly in terms of combining objects to make complex communications, the overall ability of these shapes to be read across cultures was established — spikiness was used to communicate frustration, roundness to communicate calmness, and asymmetry for feelings of tension (Isbister 319–322).
By holding on to experiences and emotions, the invested object reaches out towards what Frederick Brooks calls intelligence amplification (IA). As Susan Kozel explains,
AI refers to a field of research that seeks to replace the human mind with machines. IA is different since it aims to build systems that amplify the human mind by providing it with computer-based auxiliaries to do the things that it has trouble doing (like enormous sums), thereby freeing it to scale new heights at more creative tasks (Kozel 99; internal citation omitted).
In the case of the invested object, holding onto some aspects of experience and emotion might free us to make more complex observations and insights into the information contained within.
It should be a private thing.
If information will be brought down to live again within its context in an investable object, we must consider how to avoid the pitfall of recreating the surveillance-friendliness of the watcher object. Wanting to pollute the possible with ideas that move away from the inhumanity of contemporary digital hybrid objects requires the development of pieces that make surveillance difficult if not impossible.
Höök, who worked with Isbister and Laaksolahti on the SEI, has also researched what she calls interactional design:
An interactional perspective on design will not aim to detect a singular account of the “right” or “true” emotion of the user and tell them about it, but rather make emotional experiences available for reflection. That is, to create a representation that incorporates people’s everyday experiences that they can later reflect on. Users’ own, richer interpretation guarantees that it will be a more “true” account of what they are experiencing. (647)
The goal of designing from this perspective is to preserve user’s autonomy and privacy. Höök et al. begin by describing privacy in the negative, pointing to un-private systems, like the EmpathyBuddy email system or affective learning tools from Picard’s group, which use different methods to build up an emotional model of the student and use this to shape the learning approach the system takes. Höök describes the problem:
All of these applications regard emotion as something that can be measured, isolated and then used as a basis for how to make a system respond. This makes these kinds of systems potentially vulnerable to privacy protection issues. By that, we do not mean that the problem necessarily lies in what these systems store on the computer and whether that can be properly protected by various security solutions. Our concern lies on the level of what users may feel about systems that claims to know something about their emotional states, perhaps building profiles of them. (648)
When the system proposes to know a user’s state, particularly when it uses methods to which the user has no access, the system can be seen as violating privacy (649). This, too, is one of the more creepy features of the watcher objects — their promise to track your actions and thereby predict and automate system changes, without your needing to have any input at all.
Höök continues to address the second contributor to the creepiness of the watcher object — the question of autonomy:
Overall, these systems may also threaten users autonomy since they do not hand over any control to the user, but instead decides what to communicate to others (be it friends or teachers or the system itself) about the end-user’s emotional state (648).
To counter these issues Höök et al. advocate for designs that “assume that the meaning of an emotional process is created by people and that affective interactive systems should be such that users are encouraged to negotiate these meanings themselves” (649). After reviewing designs they believe to meet this approach, in particular, the eMoto and Affective Diary, the authors offer up six principles based on work from Kirsten Boehner and colleagues:
- The interactional approach recognizes affect as a social, cultural and bodily product
- The interactional approach relies on and supports interpretive flexibility
- The interactional approach is non-reductionist
- The interactional approach supports an expanded range of communication acts
- The interactional approach focuses on people using systems to experience and understand emotions
- The interactional approach focuses on designing systems that stimulate reflection on and awareness of affect (652–53)
which can be summed up in three design approaches:
- Designing open familiar surfaces that can be appropriated by users
- Leaving the interpretation to the user through a balanced ambiguous design elements
- Involving users in affective loop experiences (653)
Openness and balanced ambiguity are approaches that allow users to invest the system with their own meanings and communicate them in the manner they find most true and accurate. The affective loop is a process wherein the system reflects the user’s emotions back to them, creating both a relationship with the system and deeper understanding within the mind of a user.
Or as Hayles herself wrote,
If what is exactly stated can be done by a machine, the residue of the uniquely human becomes coextensive with the linguistic qualities that interfere with precise specification — ambiguity, metaphoric play, multiple encoding and allusive exchanges between one symbol system and another. (Hayles 67)
We see an expression of this sense of ambiguity in the SEI’s non-anthropomorphic, non-singly mapped approach. The sculpted objects are meant to express an emotion without using faces to describe them or being mapped to one exclusively; for instance conflict is often represented by users with the spiky object or the anteater, but neither is said to consistently represent conflict in a direct mapping. Each object can be used with other objects to express multi-layered emotions. They take on meaning culturally, in context and in conversation.
Gaver et al. also make a case for ambiguity in design as a driver of engagement and contextualization:
[Ambiguity can] be intriguing, mysterious, and delightful. By impelling people to interpret situations for themselves, it encourages them to start grappling conceptually with systems and their contexts, and thus to establish deeper and more personal relations with the meanings offered by those systems. (233)
If we keep the suggestions and cautions of interactional design in mind when considering the investable object, in particular ambiguity, we can be sure we are working towards creating objects unlike the watchers — objects that support the privacy and autonomy of the users.
It should be a reflexive thing.
Investable objects ought to be reflexive objects instead of homeostatic ones. That is, rather than reflecting a theory of an objective conservation of stability, the objects will reflect an interpretation of information that is focused on expressive systems.
When Höök et al. describe the interactional point-of-view, they are also describing a reflexive one. Interactional design is design that works to communicate emotions and these:
are not only cognitive phenomena, but are also experienced as physical, bodily processes, and are in turn influenced by our bodily experiences. The way we experience emotions is shaped by the culture we live in and the specific social setting they occur in. (647–48)
Continuing, they write
From an interactional perspective, communication of emotions is not simply an information transfer problem; it is about physically and intellectually experiencing the whole range of emotions that make up a conversation. We name them affective loop experiences, experiences where it is not possible to separate the intellectual from sensual experiences, nor to single out what is my individual experience from the overall experience arising in a dialogue with a friend or in dialogue with a system. (648)
This in itself rhymes with McKay’s alternative information theory as reported by Hayles, which “[a]rguing for a strong correlation between the nature of a representation and its effect, … recognized the mutual constitution of form and content, message and receiver” (Hayles 56).
It further matches with Turkle’s evocative objects. These are, she writes, objects that work “as companions to our emotional lives or as provocations to thought.” They “[underscore] the inseparability of thought and feeling in our relationship to things” (Objects 5).
However, reflexivity, in terms of the second wave cybernetic theories, is about more than just re-situating information into a body — tethering it, as it were. The theory, developed by Humberto Maturana and Francisco Varela, describes how physical systems make possible the production of meaning. With a background in biology, Maturana’s theory of reflexivity was born in a paper, “What the Frog’s Eye Tells the Frog’s Brain,” and various retinal studies. In the former, he looks at the way in which a frog’s nervous system interprets and filters actions in the world to provide images relevant to a frog’s needs. That is, frogs hardly see objects that are not in motion, as these are not relevant to the frog; this is a result of the physical development of the frog’s nervous system. Likewise perceptions of color cannot be mapped in any way to the outside world but are rather determined by an animal’s retinal shape.
From these experiments, Maturana concluded “that perception is not fundamentally representational.” Instead reality, insofar as it exists, “comes in to existence for us, and for all living creatures, only through interactive processes determined solely by the organism’s own organization” (Hayles 136, emphasis hers). This organization, whole and complete to the organism itself, is autopoeitic, self-created and self-organizing. From this comes the assertion that “[e]ach living system thus constructs its environment through the ‘domain of interactions’ made possible” by such organization (Hayles 137). Each organism is therefore capable of expressing that which its organization permits.
To describe communication between various autopoeitic structures, between different people or even perhaps between our current selves and past ones, Maturana refers to “‘languaging,’ a process wherein observers, acting solely within their own domain of interactions provide the triggers that help other observers similarly orient themselves within their domains” (Hayles 147).
Being rooted in physical bodies, in biology rather than mathematics, seeing information as not only tethered to but constructed by our organization, Maturana’s reflexivity provides a radically different and perhaps more humane way to look at message generation and communication. A reflexive object is then part of a system and a manifestation of its organization; it will provide triggers for other autopoeitic systems, which is to say, ourselves. A reflexive object too can be an object that facilitates an observer’s self-conscious review of its own organization.
It should be a transportable thing.
An object we want to develop a relationship with should be an object we can hold close.
In the initial stages of the investable object project, I ran an online survey investigating what participants considered lovable objects, giving the prompt “Tell me about an object you love and why you love it.” In this case, a survey was good tool, giving broad responses to a wide request for information quickly and without geographic limitation.
Because I wanted as much emotional honesty as possible I did not ask for demographic or personally identifying information; however, as the respondents were drawn from my Twitter followers (or followers of followers in the case of retweets), certain facts can be inferred from Twitter’s audience analytics. Namely, they are tech-savvy and design-interested, between 25 and 44 (mostly under 35), and from the coastal US. Although containing slightly more men than my real-life friendship group at 60%, the audience are my peers. This is who we are aiming for when we check for resonance beyond the immediate experience. (See §7 for more on resonance as a goal.)
More than two-thirds reported touching the object they loved sometimes or often. Over a quarter had it in their pocket or hand when answering. Half chose an object of memory, something that reminded them of something and a further fifth chose an object of comfort.
While some objects stayed at home and a hard-to-move invested object is imaginable, for most respondents, an investable object, a loved object that contains history, experience and emotion, was at hand.
It should be something people can make themselves.
As a final corollary to reflexivity, an investable object should be something people can make themselves. In this way the object also pollutes the possible proffered by the maker movement, today a defender of the real object but also a breeding ground for the watcher object.
This state of affairs is in many ways the expression of the construction of information as untethered. As Evgeny Morozov relates in his history of the maker movement for the New Yorker, the flowering of DIY electronics and related products also begins with Brand who has championed makers who “take whatever we’re not supposed to take the back off of, rip the back off and get our fingers in there and mess around. That’s the old impulse of basically defying authority and of doing it your way” (Brand, quoted in Morozov).
And yet, as Morozov points out, the resulting works are about anything but rebellion: “our hackers aren’t smashing the system; they’re fiddling with it so that they can get more work done.” He elaborates, “In this vision, it’s up to individuals to accommodate themselves to the system rather than to try to reform it.” And much in the same way that revolutionary language has produced submissive action, the actual devices produced by these makers tend to be either robot toys, fun but unremarkable, or sensors, made to watch and report back on heat, weather, movement, and other free-floating information around us.
Morozov identifies the core of our sickness again in imagination:
Our tech imagination, to judge from catalogues like “Cool Tools,” is at its zenith. (Never before have so many had access to thermostatically warmed toilet seats.) But our institutional imagination has stalled, and with it the democratizing potential of radical technologies.
But if we were able to disseminate the means of creating reflexive, personal, investable objects, we might pollute the narrative currently living inside the maker movement. One vector might be FabLabs, as developed by Neil Gershenfeld. The labs are “globally connected open workshops where people can meet, collaborate, interact and exchange ideas, machines, tools, materials, and software with the common purpose of making distinctive and digitally designed objects (from scratch) in an accessible and cheap way” (Walter-Hermann and Büching 13). They were born from Gershenfeld’s experience at MIT, where a class he created for advanced physical sciences students, “How to Make (Almost) Anything,” ended up attracting a hundred interested artists, architects, and engineers. Not only was he surprised by the students, but their approach to learning also made a big impression: “the class turned out to be a bit of an intellectual pyramid scheme,” he writes, noting that students learned machines and techniques just-in-time then passed them on to others when others needed them (Gershenfeld 5–7).
From his experience putting fabrication tools in students hands and standing back, Gershenfeld concludes “the killer app for personal fabrication is fulfilling individual desires rather than merely meeting mass-market needs” (9).
What if instead of just sharing machine knowledge, a maker also spread the possibility for a different kind of object? The invested object — a real object that exists in the world, uniting thought and emotion, reflecting them back privately and providing a means to project our concerns into the future — is ready.
5. What an Invested Object Might Contain
The primary characteristics of the investable object itself have been sketched. But what should be invested into the object? Whether or not people make the physical container themselves, whether or not the object is reflexive, there must be personal content, communication, emotion, and it must come from the object’s owner. But how do we begin to talk about content without slipping again into the common construction of information as unmoored entity?
Sara Hendren, in a short reflection on Ted Porter’s Trust in Numbers: The Pursuit of Objectivity in Science and Public Life posted to her blog quotes Porter:
Vernacular languages are also available for communication. What is special about the language of quantity?
My summary answer to this crucial question is that quantification is a technology of distance. (emphasis Hendren’s)
and then provides a response that is perhaps the barest statement of the problem at the heart of this paper:
That last phrase has been especially provocative for me lately: It states at once the convenience of numerical descriptions (percentages, pie charts, ratios) and the immediacy of their appeal. I think it also suggests why artists tend to object to these descriptions of the world and to the comfortable assurance that this kind of “objectivity” will be respected, cited, trusted. Artists so often traffic in drawing close attention, in careful notice, in celebrating outliers and the unexpected. So it can seem like the posture of mathematical purity — in all its applied forms of social and political organization — is so much a technology of distance that it overlooks the subtlety and depth of human experience.
Can the content invested our object work against this technology of distance? Can we have a technology of outliers or a technology that foregrounds “the subtlety and depth of human experience”?
In his information hagiography, Gleick includes a description of the radicalism of the telegraph:
The most fundamental concepts were now in play as a consequence of instantaneous communication between widely separated points. Cultural observers began to say that the telegraph was “annihilating” time and space. … This was an exaggeration that soon became a cliché. The telegraph did seem to vitiate or curtail time in one specific sense: time as an obstacle or encumbrance to human intercourse. (148)
I find a significant amount of inspiration here. If the telegraph, a technology of distance, is also able to become a technology of closeness through what it communicates and who it communicates for, the invested object, too, should be able to transmute distance-creating logics into close and subtle communications. To achieve this, we must pay close attention to what we choose to communicate and to whom.
Höök et al., in their work on interactional empowerment, choose to speak of emotion, using a term already grounded in subjectivity and generally accepted to be contextual, and in particular the emotion generated by interactions with games and other stimuli. Coming from a scientific perspective, this is enough — the researchers are looking for emotionally expressive tools as part of an investigation into a particular set of hypotheses around design.
In the case of attempts to pollute the possible, however, it would be ideal to develop a tool that can be used in a broader context, with a stimulus that is common but ambiguous, vast in its experience and interpretation, and near to our hearts. It should be known already as a continuous phenomenon, emotional, specific, embodied, a target for communication throughout human existence. For this, I choose memory. As Walter Benjamin writes in his essay on Proust, “An experienced event is finite — at any rate confined to one sphere of experience; a remembered event is infinite, because it is only a key to everything that has happened before and after it” (Benjamin 202).
Memory is arguably what makes us ourselves: it is our experiences that shape our ongoing interpretation of continuous phenomena. That is, we understand the present in relation to the past and what we remember of the past in this way determines who we are. It already collapses time and temporal distance and with it we communicate with ourselves.
Memory is embodied.
Memory is embodied in multiple ways: we can only remember the places where we have been, physically or psychically; as we remember, we reconfigure actual synapses in our physical brain; we can be spurred to remember through our body, in particular via olfactory stimulus.
Memory is bound by the limitations of being a creature with a body. We may imagine or project ourselves somewhere far away, somewhere different or better. But we will not remember it. We remember what has happened to us (even if we may shape it with strategic forgettings sometimes).
Memory is also embodied in a more literal way. As Jonah Lehrer charts in Proust Was a Neuroscientist, memories persist in our minds through despite the death of neurons, just under the surface, through the changes wrought by particular prions (Lehrer 76–78).
This physicality may be why, as anyone who ever quoted Proust or typed the word madeleine will tell you, memories may be summoned by our senses — sounds and smells and tastes. Psychologist Rachel Herz and other memory researchers have shown that olfactory-induced memory real and powerful. As Herz reports:
Descriptive autobiographical memory studies have shown that odor-evoked memories are highly emotional as measured by self-report. Several cross-modal laboratory experiments have further demonstrated that memories associated to odors are more emotional than memories associated to cues perceived through other modalities (vision, tactile, verbal) … (217)
She has even charted the geographic and density-related variations in the resonance of particular smells: “for memory vividness, participants who lived in the city had the most vivid memories to popcorn and people who lived in rural/country areas had most vivid memories to fresh-cut-grass” (Herz 220–21).
Memory is physically situated and effected; it is bodies inside and out.
Memory is reflexive.
Memory is also deeply reflexive. It may be the canonical example of a system wherein the observer changes the observed and the observed changes the observer. In reflecting on our reflections, we are able to see the development of self-consciousness and identity in an autopoeitic organism. This action can be described as a palimpsest, with each cycle adding layers, generating complex meaning, much like Proust’s drafts of A la recherche du temps perdu themselves; this action can be a bricolage, cited by Turkle as the practice that inspired her work in material culture — in her case literally as the search for her father’s memories in an assemblage of objects (Turkle, Objects 5).
As Maturana and Varela write in Autopoiesis and cognition: The realization of the living,
An autopoietic machine is a machine organized (defined as a unity) as a network of processes of production (transformation and destruction) of components which: (i) through their interactions and transformations continuously regenerate and realize the network of processes (relations) that produced them; and (ii) constitute it (the machine) as a concrete unity in space in which they (the components) exist by specifying the topological domain of its realization as such a network. (78–79)
When we consider the embodiment of memory, we can see that the process of memory is indeed a process of “production (transformation and destruction) of components” — these components being both neurons and thoughts about who we are — and they do “continuously regenerate and realize the network of processes (relations) that produced them,” that is, ourselves. In this case, the topological domain is the connections between the network of individual moments.
The instantiation of reflexive memory in this project in particular will be discussed later in this section.
Memory is cultural.
Memory, social and personal, is known to live within objects. When it takes up residence in the physical, memory becomes cultural. We have seen how this works in terms of material culture history in §§3 and 4, where we considered Manuel Charpy’s elucidation of the gestural constraints and communications of Victorian objects and John Styles’s investigation into the emotional meanings of London Foundling Hospital tokens.
Maurice Halbwachs, in “Space and the Collective Memory,” depicts the same cultural communication through personal imagination as opposed to professional history:
In an antique shop, the various eras and classes of a society come face to face in the scattered assortment of household belongings. One naturally wonders who would have owned such an armchair, tapestry, dishes or other necessities. Simultaneously (it is basically the same thing), one thinks about the world recognizable in all this, as if the style of furniture, the manner of decor and arrangement, were some language to be interpreted. The picture a Balzac provides of a family lodging or the home of a miser, a Dickens gives of the study of a notary public, already suggests the social type or category of the humans who live in that framework. What is involved is no mere harmony and physical congruence between place and person. Rather, each object appropriately placed in the whole recalls a way of life common to many men. To analyze its various facets is like dissecting a thought compounded of the contributions of many groups. (47–48)
In this way, memory as content doesn’t have to doom our project to solipsism or isolation. The specifics of a single system or framework may be personal and in the interests of privacy they may be limited to complete interpretation in the mind of the user, but they are also always somewhat typical of our type and open to imaginative interpretation by others.
Edmund de Waal’s story in The Hare With Amber Eyes is a compelling example of this principle. In charting the travels of a collection of netsuke that has been in his family since the nineteenth century, de Waal is able to spin us the story of increasing contact between France and Japan and the craze for Japonisme in nineteenth-century Paris, as filtered through newly wealthy Odessan Jews.
The collection begins just at the moment to which Charpy was referring when he considered the mania for collections “as forms of possessive individualism” (Charpy 210).
The netsuke move through the geography and upheavals of the twentieth century, through rarefied spheres and rubble, alternately inhabiting dressing rooms, hiding spots, and places of pride. But it is not the facts of their adventures that make the netsuke potent. It is the memories they were present for — the worlds they organize around themselves — and the imaginations with which de Waal surrounds them.
For me, it was this book that began to make this project feel possible and true.
Memory belongs to artists.
Finally, memory as a cultural field of inquiry resides squarely in the domain of the artist. Engineers have given us the intersection of machine memory and a problematic understanding of knowledge — information — for it to contain. The artist has access to “a body of theory that has considered memory’s intersection with the imagination and the aesthetic,” (Farr 18) and thereby the materials for a thoughtful salve. (None of this is to say that only artists possess or consider memory — for instance Herz takes it on as a chemist and folks from all walks of life have put pen to paper and written memoirs. Charpy too details to uses of personal souvenirs to construct objectified displays of our memories.)
But, by availing ourselves of the artistic memory tradition, we open a conduit for those specific, subtle, intimate technologies tightly-hewed abstraction has removed. When Gleick recounts the cleavage between the oral and the written that he deems necessary for technological progress, he contrasts written literature to earlier oral works, which is to say, poetry.
Logic takes its form in chains: sequences whose members connect to one another. Conclusions follow from premises. These require a degree of constancy. They have no power unless people can examine and evaluate them. In contrast, an oral narrative proceeds by accretion, the words passing by in a line of parade past the viewing stand, briefly present and then gone, interacting with one another via memory and association. (Gleick 38)
Memory is a medium for denigrated poetic knowledge. Which brings us back to the lichen — our hybrid metaphor for the combination of old and new that characterizes a healthier approach to information and countermeasure to our rush into self-surveillance. Memory is that artistic, atavistic, extra-logical algae to smuggle back in, wrapped in protective technical fungus.
More practically, too, the art world has already lived through the return of an embodied memory. Ian Farr explains:
If, from the end of the 1950s onwards, a spectre haunted modernism, decade by decade weighing down on its structures until they collapsed into noble but now incoherent fragments, it was the spectre of memory…. The liberation from tradition and historical contingency, the presentness yet simultaneously abstracted remoteness considered central to mid-century modernism’s aesthetic aspiration was assailed by the return of what it had seemingly repressed.
This return took the form of postmodern approaches to memory or, as he writes, “to be marginally less imprecise, strands of artistic practice increasingly inflected by remembrance and forgetting” (Farr 13).
Kinds of Memory
But let’s step away from lyricality for a moment and to add a little more precision to our questions; to create a pollution-worthy prototype we must be able to state our aims a little more straightforwardly, art or not. First we must consider what we are actually discussing.
Memory can take a multiplicity of forms; this suits its purpose but can sometimes bedevil ours.
It can be the Bergsonian cone interacting with the plane of experience. In this view, memory is the intersection of the individual with the plane of history. This nearly mystical definition suits Bergson’s philosophy, described by Lehrer as a refusal of the mechanization of human thought and insistence on the wonder of human consciousness (Lehrer 66).
In contrast, memory can be documentary — the memory that exists frozen within a photograph — as with the work of Miroslav Tichy (Farr 12) or the interpretations of Gleick (376, quoted in §2).
It can be the antithesis to this documentation which, as Siegfried Kracauer posits in “Memory Images,” is unable to completely contain memory or be made sense of by memory. “Memory is neither the entire spatial appearance of a state of affairs nor its entire temporal course,” and so,
Since what is significant is not reducible to either merely spatial or merely temporal terms, memory images are at odds with photographic representations. From the latter’s perspective, memory images appear to be fragments — but only because photography does not encompass the meaning to which they refer and in relation to which they cease to be fragments. Similarly from the perspective of memory, photography appears as a jumble that consists partly of garbage. (Kracauer 46)
Memory can also be the hallucinatory surrealist manifestations, for instance, in Andre Breton’s Nadja. As Farr writes, connecting the work of Sartre and Breton via Paul Ricoeur,
Whether the trigger is a familiar landmark on an everyday walk suddenly made strange by a chance incident of light or shadow, or a curious object or person emerging into vision for the first time, the état d’attente (state of expectation) of the surrealist viewer is a mode of being that allows an opens up to this semi-hallucinatory pathway in a process of memory-association. (14)
It can be a property moving from the realms of the cultural to the archival and beyond, as suggested by Pierre Nora. He locates true, or traditional, memory in the “spontaneous, social and all-embracing” French conception of a remembrance rooted in a place, and contrasts it with three modern versions.
The first is archival memory, which “relies entirely on the specificity of the trace, the materiality of the vestige, the concreteness of the recording, the visibility of the image” (62). This is the memory of historians, memory circumscribed by logic, where what is true is what is precise. It is also selective.
This is followed by the much-less-selective memory as individual duty, the memory that creates contents to be archived, with each of us a memoirist.
Nowadays who does not feel called upon to record his reminisces or write his memoirs? Everyone has gotten into the act: not just people whose roles in history was minor at best, but also the relatives of such people and their doctors and lawyers and anyone else who happens to be standing about. The less extraordinary the testimony, the more aptly it is taken to illustrate the average mentality. … Within a single generation the imaginary museum of memory has expanded beyond belief. (Nora 63)
The final type of memory to be considered, “alienated memory,” is a modern sort, the result of our discontinuity with true memory, wherein we scramble to recreate the past, to understand the lost country from which we are irreparably separated (Nora 64–66). Nora reports these developments with a sense of a loss and an eye on the absurdity of our attempts to prevent any trace from being lost — our frantic collections nearly burying us in the garbage Kracauer finds in photographs.
Memory can also be plotted along the axis from private to public, or as I like to think of it, from Proust to Perec. In discussing the second type of memory, individual, Nora ties Proust (along with Freud) to the shift of memory into the psychological, “from the social to the individual, from the concrete message to its subjective representation from repetition to remembrance.” The result? “Memory becomes a private affair.” (64) (We do not need to agree with the entire theory, and I do not, to see the resonance of the description.)
In Proust, memory is intensely personal. Though it bears up its global truths through the repetition of motifs and phrases, and tells us it will from the moment Swann’s little phrase is introduced in Swann’s Way, it can do so only through the means of the narrator’s careful reporting on the events of his life.
Perec works with memories that are public, nearly dispassionate, but still need to be “provoked, rescued from oblivion.” This he does by enumerating observations in a public place, as in Attempt to Exhaust a Parisian Space, or by enumerating memories themselves as with Je Me Souviens. The memories Perec is interested in are of a particular sort as well, “memories which truly render the ’tissu du quotidien’ (fabric of everyday) — a body of experience that transcends our own individuality and yet invokes a commonality of experience.” These “cannot be purely personal (what happened to me) or factual (what happened to be the case)” (Sheringham 88).
And yet, though we can oppose Proust and Perec in denotative terms: long, winding sentences entwined around I versus short, dispassionate lists on TV stars and the number of cars in the street, both work to draw information out of the aether and root it again in the particular.
Up to now, absent the overview of the first section, this paper has been working from outside in, in a reversal of the discoveries that brought me to this project. Nearly a year before the watcher objects caught my eye, I found my way to A la recherche du temps perdu, and as happened for scads of Proust fans before me, the work caught my heart. In all of its inconsistencies, rambling philosophies, and hallucinatorily evocative moments, A la recherche described a reality that was far more real, specific, subtle and true than any chart-based insights watcher objects might promise. It was even more vivid than the hallucinations of Nadja, which I recall stumbling through in college.
What began as pleasure in Proust’s recognition and articulation of the pleasures of reading, especially through long childhood afternoons, or of the beach, became a view that was able to stand up to the depredations of shorn information. Here, in Swann’s Way was the narrator depicting specific, lived memory giving vigor to such abstract categories as lilacs, hawthorne, tadpoles and buttercups (188). There, in In the Shadow of Young Girls in Flower was Marcel painting a milkmaid as a concrete embodiment of abstractions like beauty and happiness (234–235) in firm rebuke to the ideas haunting information. Everywhere in the novel memory is encountered, re-encountered, revised and re-experienced.
Just like computing.
Choosing memory as the algal substance to be invested in our technological fungus becomes suddenly obvious. Computers are a technology of memory. In The Information Gleick suggests Babbage’s ideas really became different when they addressed carrying values from one column to another: a work of memory (101). Computers become faster and powerful because they can remember more — better. Computers are positioned as superior to humanity because their memories are less porous. (Until they fail catastrophically, all at once.)
Sometimes we fight back by suggesting that data, information, memory and knowledge are different substances. This may be true but is a difficult way to pollute the possible of computing. Instead, we can slip a different sort of memory into system we already have, add it on top of the strict abstract electric memory.
The right type of memory on which to focus is the memory that haunts A la recherche. But how does it work?
In Proust’s Binoculars, renowned Proust scholar Roger Shattuck analyzes his version of memory in terms of time and optical metaphors.
Right away, he ties optics to error in the novel and outlines how it undermines the notion of a photographic truth.
Regarding a paragraph in which the narrator has “[learned] that M. de Norpois, who has always appeared to have a benevolent attitude toward him has called him ‘a semi-hysterical flatterer,’” Shattuck cites all the optical and photographic terms contained — “milieu, propage (propagates), réfraction, image, épreuve (proof), photographie, radiographie, tirées (taken, drawn out, printed)” — and then continues:
I should like to insist here that [the excerpted passage’s] underlying theme of the inaccuracy of perception furnishes us with the first clue to the significance of all this optical imagery. The science of optics forever shows the errors of our vision, the distortion from accuracy, deviations from the straight line. Photographic precision is only an accepted version of deformation. …
Marcel’s comedy of errors continues until he and we begin to understand that art itself is an optic, but a superior optic which will finally transform error into truth for our mortal eyes. (13, 17–19)
At the same moment technological claims for logical truth are being promulgated by Russell, Proust is already reminding readers of its artifice and unsuitability, at least as a single captured “instantané (still or snapshot).”
Though it remains the basic unit of observation and memory, the single instantané turns out to be an orphan, a meaningless fragment snatched out of the flux. The still camera must yield to other optical devices to provide metaphors for our pursuit of reality: the magic lantern, the kaleidoscope, the cinematograph. All three depend upon a succession of images and describe the fly by reproducing it in a schematic form. They reflect time by partially submitting to its ceaseless modifications. (Shattuck 19, 23)
Unmoored information “snatched out of the flux” is meaningless. But if we take a different approach and use time instead of just annihilating it, we can approach reality. We can find depth.
“Depth, or what is called in optics penetration effect, cannot be found in a single image, a single instantané,” Shattuck tells us — instead it must be produced stereoscopically, by layering images so our binocular vision can remake them in their full three dimensions. Taking up memory,
Proust undertakes a transposition of spatial vision into a new faculty. The accumulation of optical figures in A la recherche gradually removes our depth perception from space and re-erects it time. (42–43)
Proust is constructing a topology of time. In the final volume of A la recherche, when Marcel confronts the figures from his past once more, now aged and changed from their previous selves, it is clear that “Proust set about to make us see time” (46) through his inaccurate optical memories — and succeeds when they are gathered.
Memory, in its alternate form of recognition, progressively sets one image beside other chronologically separated images and sees in them not change …, not trompe l’oeil, but revelation of true identity, the “optical view.” … Multiplicity now brings not confusion but dimensionality and depth. … Merely to remember something is meaningless unless the remembered image is combined with a moment in the present affording a view of the same object or objects. (47)
Instants are arranged by kind, like the oral narratives Gleick was so happy to escape, only now in their context they provide the depth of understanding required to see life truthfully.
Shattuck outlines three principles for generating deep from insantanés: the cinematographic, the montage, and the stereoscopic.
The first “employs a sequence of separately insignificant differences to produce the effect of motion or animation…. This is the simplest and most familiar of the the three principles for it appears to conform to the continuity of normal experience.”
The second “rejects the accumulation of small differences [that is characteristic of the first] for the exploitation of larger associative or dissociative leaps that suggest the meaning of a scene or situation by contrast.”
The third, the stereoscopic,
abandons the portrayal of motion in order to establish a form of arrest which resists time. It selects a few images or impressions sufficiently different from one another not to give the effect of continuous motion, and sufficiently related to be linked in a discernible pattern. This stereoscopic principle allows our binocular (or multi-ocular) vision of mind to hold contradictory aspects of things in the steady perspective of recognition, of relief in time. (Shattuck 49–51)
What a perfect version of memory to invest into our object. It is autopoeitic, arraying the network of memories that constitute and give rise to our consciousness in the topology of time. This construction is suited to the discrete data technology was built for but only works in terms of private juxtaposition. In my invested object, I can gather what I would call moments of inflection.
The Moment of Inflection
I chose the term inflection over instantanés, moments bienhereux or even moments of recognition, because these moments, while somewhat more than an instanté in their readiness for layering are still rather less than a moment bienhereux.
For, “If an image or sensation out of the past,”
is to be truly recognized, in the Proustian sense and not merely recollected, it must be summoned back by a related experience in the present and after a period of absence. (63)
This recognition itself differs from the moments bienhereux, Shattuck writes, in “that the recognitions contain an active participation of the mind, a conscious resolve ….” In this way, they last and give insight. By contrast, “The perpetually elusive pleasure of the moments bienhereux” may give rise to semi-permanent realizations, but are ultimately raw and ungraspable (Shattuck 38).
The moments of inflection ought to be moments bienhereux saved and filed away for active intellectual engagement at a later time. They are less incomplete, because we put them away, but do not have the force alone of a moment of recognition. Given their close relation to moments bienhereux, it is perhaps worthwhile to consider the structure of the latter before moving on to §6 and the genesis of the Oublié/trouvé project.
Each of these moments bienhereux [that Shattuck has identified and described in a set of tables] follows or partially follows a uniform pattern. … First, Marcel is always in a dispirited state of mind — bored, usually tired, alone (or if not, annoyed by the presence of others interfering with his solitude), and deeply entangled in the train-train of habitual living. Second, he experiences a physical sensation, which comes unexpectedly and by chance through any one of his senses or a complex of them. Third the sensation is accompanied by a clear feeling of pleasure and happiness …. These three components, which occur together in the present, combine in the fourth step to lift Marcel steeply out of the present and raise him high enough to see what he has lost sight of: an analogous and forgotten event in the past. (Shattuck 69)
(This is followed sometimes by a fifth step that can only exist in literature: a premonition of the future.)
Looking at this pattern, we begin to have an idea of the situations the object would appear in as well as the information it might hold.
While I already had noticed in myself a predilection for something similar to moments bienhereux, moments of intense recollection and the sensation of tying to other moments in rapid succession, I decided to test my project idea by spending a few weeks collecting the moments in a paper notebook — before any tech was added. It was a great success.
Like Marcel, these moments tended to come when I was alone, often when I was on the subway or walking around — the times when I was not reading my phone but instead let my mind wander. While the experiment proved I was in fact in a good place to use an invested object of the sort I imagined, I was mostly surprised by the pleasure of the work. Looking closely, finding the threads and knowing they weren’t doomed to partial or total loss but available for sustained engagement felt good. The world seemed brighter; colors were more intense — all of those glorious cliches of pleasure.
I also discovered that the distinction of public and private, so relevant in terms Proust v. Perec is irrelevant in New York, today — at least for me. Most moments occurred when I was on the subway or walking, in a mentally private space but a physically public one. I am surrounded by the impersonal and public but immersed in the private, each playing together reflexively. This may be a feature of New York and New Yorkers, of our facility for making a private space in a crowd or in the distinction between the 19th, 20th, and 21st centuries. With a system of annotations living around the object, as with the notes of the back of photographs, this is finally moot: one can simply annotate the public or private values of the situation.
This was a good idea.
6. The Idea
And so the goal became: create a real, private, reflexive, transportable thing-that-one-can-think-with, and do so in a way one can fabricate alone, given a consumer-level fabrication lab. Embed this investable object in a system where it can become a container or interface for memory. The memory should consist of inflected moments ripe for stereoscopic juxtaposition. Make use of the computing options we have, but let it feel a little different from eternal screens. Let it be something we can have a relationship with. Let it be something personal; something I can have relationship with.
Oublié/trouvé is a hardware and software system for ingesting and juxtaposing memories. In its final form, it makes use of a BLE-enabled organic object to communicate with an iPhone app. In addition to being an interface, the object is a talisman and worry stone. The app collects and organizes saved moments, offering different groupings for review at any time. It also seeks moments that match the grouping conditions and notifies its owner (or human partner) via the object when these conditions are present.
In her paper on phenomenology, Martha Ladly discusses the “embodied experience of ‘Savouring’ and ‘Fulfilling’” in Heidegger:
Heidegger devised the idea of formal notification in communication, as a demand that the other person (the one with whom we are communicating), when shown a thing, must look at it themselves. They must see for themselves what is notified (or shown) in order to fulfill that thing, with their own experience of it. With fulfillment comes the opportunity to savour the thing. We can only do this if we are present, and in the flow of temporal experience. (145)
Combining some conventions of data visualization with obscured data, Oublié/trouvé wants to make it possible for its human partner to experience the notification-fulfillment-savoring phenomenon and in so doing re-encounter herself. Oublié/trouvé should also provide a way to investigate a more contextual sense of information, something that rubs against the abstracted, riven concept with which we are usually presented.
This section reviews the initial design documents involved in the genesis of the system and a few objections to the project.
In an early phase of the work, I investigated three related themes that were present in my desire to push back against watcher objects and contextual information. These were: design for techno-human symbiosis, the nature of humanized objects, and sense-induced experience. In each case, I undertook a basic review of the theme’s presence in extant research and then curated a set of icons to visually explore my conception of each theme.
Figure 6.1, the icon set for design for techno-human symbiosis already makes visible the assumption that melding the technical and human comprises a melding of art and chips, and perhaps a royal user. I also managed to choose three different memory technologies without realizing: a chip, a floppy disk, and a server. This may point to the icons available in a set designed for web development but also highlights how much memory is in fact at the core of computing.
Figure 6.2, created for the nature of humanized objects focuses on messages and representations of the physical world, which is to say, the other important elements in computing. For good measure I also included some burning cash: what would we make if we weren’t concerned with profit?
Sense-induced experience, shown in 6.3, is concerned with mystery and intimation. The figure is no longer a king nor a detective but is instead barely there: he is his trappings, his metadata, a costume. Experience is like a flag on a buoy, crystals, or a basic collection of shapes. It is distilled and harsh. The outside world here is seen as desert, instead of the mountain representations from the nature of humanized objects. These icons seek intensity.
The icon sets then formed the ground for my first collection of storyboards, shown in figures 6.4 and 6.5.
These already show two currents at work in this project: a sense of history and a sense of future history — or projecting ourselves through time. They also include a sense of unease with today; the presumption that all is not right. The second iteration moved from situating the idea in history to considering a sense of experience, which is the same trajectory the explorations took. I was clear on why I wanted to investigate this space and the largest problem I was attempting to solve. (Though this has morphed through research into doubt about the problem-solving uses of design and interest in the option-exploring uses, at this point I had yet to question this basic design assumption.)
The next step was to explore ways to express memory experience.
Reading Response Film & Classifications
In response to an assignment asking us to create an artistic reflection on our reading, I chose to focus on the sense-memory experience stimulated by the books. To do so, I created a watercolor illustration (see figure 6.6) of a scent-palette for scents I recalled while reading. (The full film is available on the project site.) These include direct scents, like fixative, the smell of photography, or ozone, the smell of some computers; also represented are more abstract scents: your parent’s house at night (see figure 6.7 for annotated stills). I then drew in the webs of connections between each, as they exist in the world and in my mind.
While if I were to remake the film as a film experience I would slow it down and allow more time to concentrate on each scent indicator, I took great pleasure in playing with memory webs and exploring moments backwards and forwards. Would it be possible to focus on a meditative moment encounter?
To think through this a little more, I created a series of classifications cards to illustrate possible ways to consider moments (see figures 6.8–6.11). Drawing these out instead of making them in Sketch gave me the chance to think about each a little longer, and I went from notes to thumbnail to watercolor. Altogether, these four cards illustrate the breadth of information qualities and mark-making approaches the project might take. In fact, nearly all the ideas in “Ways to Make a Mark” found their way into materials testing for the object.
The simplest ideas from “Ways to Describe a Place” — distance and elevation — made it into the current prototype. Volume of people and volume of water are on the list for the next iteration. The former develops greater resonance when we recall from the previous section that one of the key requirements for a moment bienhereux in Proust is that the narrator be alone or seeking solitude.
Finally, while it focuses on the two most dispersed densities of feeling, the project allows for at least three methods of bridging ambiguity — projection, imagination, and interpolation. Specification and rejection are repudiated as they are tools that presume a classic sense of information. Both methods seek to eliminate ambiguity rather than acknowledge or engage with it.
System Overview & Character Questions
With a set of concepts and a rough sense of the tools at hand, it was time to get more specific, which in this case meant sketching the system (figure 6.12) and asking specific questions about each of the three nodes: the object, the application, and the user (figure 6.13).
If we consider the system of culture and communication shown in figure 3.1, this work focuses on the two rightmost spheres: the physical world and the sphere of communication.
I described it in the initial blog post:
The system comprises five stages, which take place both in the physical world and in the minds of two types of observers: the user-creator-operator (or user for short) and external observers, who may or may share a temporal space with the user.
The object will be instantiated in the creation phase. Here we must determine the desirability of the object’s being fabricated entirely by the user with reference only to set a plans, from a kit that includes hard-to-fabricate objects, or being made most-accessible but least-special by being manufactured. Another path to consider is the modification of extant objects.
Once created, the object becomes a part of the investment-playback-reflection loop. The interaction is the core of my action to pollute the possible by exposing an interaction that is digital but not context-less. An application and the object will be able to play off one another in order to create a reflexive experience: characterized by intersection, layering, juxtaposition and possibly even surprise. The capabilities and functionalities required to enable this experience — and to keep it from being a job in itself — must be investigated and refined, and this action will be the task of the thesis.Finally, the system must consider the question of memorialization. The action of freezing and elaboration seems necessary for a complete work of memory and for the social function of talismans. However, it may be a bit of a stretch for this work to fully approach. There is a lot of uncertainty here. (Groff Hennigh-Palermo, thesis blog, spring section, “009: System 1: Overview & Aspect 2: Materials”)
Here I still use standard design terms, like user, even if I make a gesture towards expanding it in “user-creator-operator.” It is only now, having investigated the history of information theory (§2) and polluting the possible via critical design (§3) that using design terms feels discordant to the project goals. The user carries implications of a system determining possible interactions and goals, rather than being a malleable tool or location of continued encounter. Perhaps rather than the user, we should design for the investigator, the explorer, the human partner?
The investment-playback-reflection loop forms the heart of the diagram and stayed intact throughout the process. Testing has shown that finding the balance between feeling as though one is working for the machine versus with the machine remains elusive quarry. How to do so might be the hardest question of the entire project.
The general system summary made it possible to begin to ask questions about the three constituents, as shown in figure 6.15. This led directly to materials testing, which is covered in §8, and to the final design system documents: a second set of storyboards.
These storyboards (figure 6.14) are more specific than the first round. They use some elements of the original icon set, and many additions, to outline the interaction that has been brought to life in the prototype. The explorer experiences a moment of inflection and saves it. Moments are made available both through notification and to curiosity at any time. In retrospect, while the lighthouse has made it into the visual design of the application, the flagged buoy is perhaps a better representation of the moments saved. They surface and we mark them; then off they bob, floating on the currents.
Pitfalls and Objections
Now that the impetus and justification for the system have been indicated; the reasons for a physical object and memory as content bullet-pointed and described; the workings of the system sketched, it is worth taking a moment to consider the pitfalls and objections that meet all projects of ambition.
Lanier, for instance, might argue that creating an object we hope to have a relationship with, one that can contain “the residue of the uniquely human” (Hayles 67) would be expanding the “empathy circle” too far, undermining that which is special about humanity (Lanier 40). He might also argue that the nostalgic — and when we talk about using memory we must be clear we are talking explicitly about captured and induced nostalgia — is fundamentally non-authentic and a “waste” of our new technological capabilities (Lanier 130). In looking at related works and Oublié/trouvé, I hope I’ve made the case for the depth of memory and nostalgia that obviates novelty seeking as the primary goal for digital art.
A more complex pitfall arises in the preface to The Inner History of Devices. This is introduced via the story of Gordon Bell, computer pioneer (this is beginning to sound familiar) and creator of MyLifeBits. Working with a team from Microsoft, Bell is animated by the “idea of a complete, digitally accessible life.” This means scanning all of his books, recording all of his lectures, capturing everything from notes to logos in pursuit of a total archive for their descendants (Turkle, Devices 24). Here again is Pierre Nora’s archival nightmare.
The text contrasts this approach with two others. The first is described with a quote from Lillian Hellman, who writes that meaning “comes in pentimento, in the painter’s layering of paint, in his ‘repentance’ as he finds what he wants in the process of repainting” (Hellman quoted in Turkle, Devices 24). Turkle then asks, “What will become of this kind of reworking, when, in digital culture, people’s fantasies shift from telling the story of a life to having a complete record of it?” (24) Implied in this contrast is the necessity of forgetting and imperfection — incompleteness — in the making of meaning and understanding.
Turkle illustrates this further by recalling her mother’s favored way of engaging the past: saving photographs with poems on the back in a large drawer, pulling them out to savor again, sometimes in joy, sometimes in mourning. This is approach that will be corroded by the digital:
Of course, the digital archive is only a resource; it remains for us to take its materials as the basis for a deeply felt enterprise of recollection. But one wonders if the mere fact of the archive will not make us feel that the job is already done. (Turkle, Devices 24, 26–27)
In both circumstances the idea is that the totalization of digital memory, its inhuman perfection, will drive out reflection, curation, accretionism. And that this is immanent in technology itself.
In response to this, I want to push back on the idea that there is an unchangeable digital character as opposed to a contingent digital deployment. Performance artist Susan Kozel, in her reflections on her piece Telematic Dreaming, argues similarly:
My “performance” in Telematic Dreaming (which felt more like a dwelling) took place in 1994. This piece is significant for revealing, in an accessible way, that basic human qualities such as touch, trust, vulnerability, pain, and embodiment are not lost when people engage with each other through technologies: we just need an appropriate methodological framework in order to see and validate this. (Kozel 88)
In the performance, Kozel was in a separate room from her digital self, which was projected on a bed in a gallery. Over the course of her residency, she was able to have a range of improvised interactions with visitors, many intimate and surprising. She reports physically feeling certain interactions, despite the fact that visitors were only touch her projection. This is itself “an indication of the strong physicality of the piece, of the powerful link between the body on the screen and the bundle of emotions, thoughts, and movement that make up my material body” (Kozel 94).
She then explicitly pushes away the idea that digital works must be a certain way.
The mechanization or computerization of human experience is generally thought to diminish the physical and emotional sides of life, yet in the virtual world of Telematic Dreaming questions of privacy, intimacy, and identity were central. This was not just my experience as a performer: many members of the public were overwhelmed by their experiences on the other bed. Some felt protective toward me, or stayed on the bed because they didn’t want me to be alone in my virtual world. Others claimed to have been “changed” by the experience. The installation was paradoxical not only for using technology to provide a forum for experiencing the basics of human intimacy, but also for situating this private interaction within a public domain. (Kozel 95)
That is, the experience of technology and space can be different than the norm if it is part of a different undertaking. Supporting imperfect, incomplete, and partial approaches is worthwhile challenge for Oublié/trouvé. Can it be made to allow for small, pleasant engagements, for augmentations and returns?
Finally, the last pitfall for this project is its duration. The vector for change, via polluting the possible, takes a long time. Creating possibilities for others to elaborate and replicate until they become technological dreams is a process that works on the scale of centuries, as we can see in the history of information’s development in §2.
Likewise testing memory-based interactions is challenging. As we have seen both in §5 and above, memory makes meaning through forgetting and long-term juxtaposition. Though these types of durational goals are not well-suited to standard results-oriented user testing, I believe they are even more important than incremental improvements. Fortunately, alternate methods for testing the success of the project are available. We will take a look at these in §7.
Before considering ways to evaluate Oublié/trouvé, however, it is worth considering related projects. While Oublié/trouvé is unique, it is not without precedent. I considered other memory collection systems, like MyLifeBits, Reporter, and Moodnotes, as well as at alternative approaches in Memory Device, Datacatcher, and Dear Data.
MyLifeBits, mentioned earlier, is one of the earliest totalizing life-data projects. Creator Bell, together with a team from Microsoft, created a database and tools to save and link ephemera from his past. Once this was underway, the project expanded to location tracking via GPS and data-collection via a light-activated camera that would take a snap on a timer or when a new person came into the frame (Gemmel 2).
The project is unsurprisingly predicated on the notion that it is human memory that is fallible and the all-remembering computer that is the preferable option. The project is introduced in Scientific American:
Human memory can be maddeningly elusive. We stumble upon its limitations every day, when we forget a friend’s telephone number, the name of a business contact or the title of a favorite book. People have developed a variety of strategies for combating forgetfulness — messages scribbled on Post-it notes, for example, or electronic address books carried in handheld devices—but important information continues to slip through the cracks. Recently, however, our team at Microsoft Research has begun a quest to digitally chronicle every aspect of a person’s life, starting with one of our own lives (Bell’s). (Bell 58)
The rest of the article is full of the totalizing promises that underlay watcher objects: prediction, vast archives for future historians, and the ability to remember exactly what color shirt we wore fifteen years ago. The project explicitly positions itself as the logical culmination of computing trends. This is the life Oublié/trouvé prefers to counter.
Nicholas Felton’s Reporter app is similar to MyLifeBits in that it looks to collect details multiple times each day. Created in 2014, about fifteen years after MyLifeBits, Reporter is able to take advantage of the integrated sensors in phones in a way Bell could only anticipate. Unlike MyLifeBits, Reporter was created to collect data that would be reported and sorted manually.
Felton is known for his personal annual reports — beautiful folios that visualize personal data from the previous year. An early version of Reporter was created as a tool for Felton to use in the creation of his 2012 report. A refined version was then released to the public. At random moments throughout the day, the app alerts you it has a survey ready and prompts you to answer a set of questions about what you are doing, where you are, who you are with. It comes with preset questions and users are able to add their own (see figure 6.15d–f). Answers are then displayed on in-app graphs as well as made available for export. The premise here is that we can find ourselves in the accretion of small samples, a very mainstream data proposition.
I was a huge fan of Felton’s work and was very excited to download the app when it first came out. And then I hated it.
From seeing Felton’s reports, I thought knowing the same information about myself would be fascinating and enlightening. But it turns out I dislike a phone interrupting me and asking me questions constantly — even if constantly is only a few times each day. I often forgot to tell the app I was awake, and I never wanted to put in the effort to structure my data, despite the app offering different question types to facilitate this work. I was never able to discover if the information was interesting in the aggregate; the interruption and up-front effort was too much.
Like MyLifeBits, Moodnotes is built around the watcher-object proposition: it will surveil us and it will help us be better. Though the premise here is less appealing that the insight promise of Reporter, Moodnotes seems to have solved the data structuring issue by presenting users with mostly structured questions and some convenient ambiguity.
For the primary question — “How are you feeling?” — Moodnotes presents a clever slider along with the request that users enter a general feeling (shown in figure 6.15a–c). The ambiguity in the input, avoiding words and instead relying on a basic face and color, allows users to engage the the application and their feeling simultaneously, without worrying about a mismatch in terms. Users can also enter a text description and more refined emotions from a list.
Like Reporter, Moodnotes also includes screens visualizes entered data, so users can review and seek insight. Both apps, like Oublié/trouvé only store data on the user’s phone.
The three foregoing apps each engage memory and journaling from the traditional machine-oriented point of view: the app works to augment our fallible minds; the pure mind of the machine is available to able to make explicit the trends we may miss. Ishaac Bertran’s Memory Device is a true deviant.
The device comprises a small display with a single button (see figure 6.16). On the project site, Bertan explains its descent, writing,
We rely on technology to collect and make sense of our own data, but very often technology doesn’t give us the right to filter what we want to remember, or what we want to forget.
The Memory device reminisces back to a time when people used to tie a string around their fingers, or pin a piece of paper on their clothes, to help them remember something.
In this case, when the user presses the button, a line is added to the screen. Previous days may be accessed by turning the knob, but no other information is available. The device also honors forgetting in a gesture against the totalizing conception of memory we see in the previous three projects. This also means, however, that it targets a different sort of memory that Oublié/trouvé — what Shattuck would recognize as the cinematic. Like Reporter, it accumulates small changes, even thought it visualizes these differently.
Using a single line to represent a moment, Memory Device shows how expressive even the barest of marks can be.
Expressive mark-making is also the salient feature of the Dear Data project from Giorgia Lupi and Stefanie Posavec. Both women have a long history of making aesthetically expressive data visualization pieces outside standard charting approaches. (In fact, both create data visualizations without coding. They literally stand slightly removed from computer-defined information.)
Posavec’s First Chapters, which charts sentence length in first chapters of classic books by generating a line whose density varies based on sentence length, was one of the first forms of alternate visualization I ever encountered and, as such, is one of the first cells in this project’s genesis. Lupi’s work with Accurat, particularly visualizations of painters’ and writers’ lives, is the second.
Recently, the two collaborated on a project, Dear Data, in which they created personal visualization postcards and sent them to one another. The project has blown up, resulting in a book, a program to match personal-data penpals, contests, and even Dear Data–related museum activities at MoMA.
The project itself presents a number of strong options for visualization using shape and color encoding and evidence for the value of aesthetic data communication. Even more, the appetite for the project and the public’s interest in manually collecting data as a source for art suggests a project like Oublié/trouvé will have resonance.
Datacatcher, a project from Gaver and the other researchers at Goldsmiths, echoes the push-and-twist interface of Memory Device and the location-awareness of Oublié/trouvé. Unlike both of these projects, however, Datacatcher is concerned with public, social data.
The handheld machine presents users with a variety of facts about the neighborhood they are in. These facts are drawn from official sources, social media, and reflections from users that are fed into the system again. The presentation centers more on comparative facts than cold statistics, seeking to capture a texture of the location.
In interviews, users reported enjoying dipping in and out of the system as they went about their days. Some were skeptical of the purpose of a specific device over a phone app and many remarked on the size, wishing for something easier to carry. In the end, though, the strangeness of the object seemed to engage people further (Boucher 133–48). Altogether the Datacatcher, like Dear Data, gives evidence of the appeal of specific data, even if is more location-related than personal. It also presents further evidence that users will engage with a non-phone object regularly.
Altogether, then, Oublié/trouvé, as a combined system dedicated to capturing and reflecting memories in a private, allusive way fits within the series of memory collection apps and art projects. I would make more use of computers than art works like Dear Data and Memory Device but maintain their more expressive approach to data display.
It was time to dive in.
With the idea in place, the next step is to consider how to evaluate it. However, it seems inappropriate to evaluate a non-traditional project in a traditional way. With its complex and durational goals — recreating a relationship with personal information over years — Oublié/trouvé is not suited to user centered testing. The style focuses on short testing sessions and quantitative evaluation, where the best design is the one most people prefer or are quickest to use; Oublié/trouvé is a single exploration focused on generating a new relationship. What are the ergonomics of that? What does it mean for an object of contemplation to be usable and is that even the right word?
Furthermore, to step away from the implicit positivism is to honor the core principles of the project itself. If the project is pulling away from the underlying assumptions of positivist theories — the notion that truth is a quantity in the world we can discover through the application of general logical principles — so should the research. This is all about ambiguity, the personal, and the contextual.
Luckily research models that take the ambiguous, contextual, personal, and durational into account already exist. Bill Gaver and the team at the Goldsmiths Interaction Research Studio undertake durational and personal design. The phenomenological approach, which considers research via reflection and experience, originated with Merleau-Ponty and has been given new life by Susan Kozel, Kate Sicchio, and other researchers working with the performing arts and technology. In the visual arts, Graeme Sullivan’s art-practice-as-research outlines the workings of research through doing, including a robust, multivalent framework. Finally, autoethnography, a personal approach to social studies research, provides us with an array of written research products that have been useful to communicate social experiences within a field that still runs on quantitative work.
Goldsmiths Interaction Research Studio
As they explain on their homepage, the Goldsmiths Interaction Research Studio undertakes “practice-based research … to produce prototype products embodying new concepts for interaction.” They continue, “Because our prototypes are evocative and open ended, a crucial part of our process involves asking volunteers to live with our designs to see how their experiences evolve.”
One of the ways they do so is through the distribution of cultural probes, which studio head Bill Gaver first deployed for a project with our critical design friend from §3, Anthony Dunne.
They describe their approach:
Unlike much research, we don’t emphasize precise analyses or carefully controlled methodologies; instead, we concentrate on aesthetic control, the cultural implications of our designs, and ways to open new spaces for design. Scientific theories may be one source of inspiration for us, but so are more informal analyses, chance observations, the popular press, and other such “unscientific” sources.
Unlike most design, we don’t focus on commercial products, but on new understandings of technology. This allows us — even requires us — to be speculative in our designs, as trying to extend the boundaries of current technologies demands that we explore functions, experiences, and cultural placements quite outside the norm.
Instead of designing solutions for user needs, then, we work to provide opportunities to discover new pleasures, new forms of sociability, and new cultural forms. (Gaver et. al, “Cultural Probes,” 24–25)
In this approach, designers seek to be neither doctors nor servants (Gaver et. al, “Cultural Probes,” 25): the goal is not to diagnose a problem nor to create a system that works exactly as a user might demand. Instead, though friendliness, lack of goals, and a willingness to explore, insight and new paths may come to light. These paths will be contextual, they note, because the questions and explorations are as well.
The cultural probes were successful for us in trying to familiarize ourselves with the sites in a way that would be appropriate for our approach as artist–designers. They provided us with a rich and varied set of materials that both inspired our designs and let us ground them in the detailed textures of the local cultures. …
The real strength of the method was that we had designed and produced the materials specifically for this project, for those people, and for their environments. (Gaver et. al, “Cultural Probes,” 29)
They call this process “user-centered inspiration” and highlight how it is used to influence ideas without dictating them. The process matches the uncommercial and exploratory desires.
That this method continues to be used at Goldsmiths and continues to produce many projects, papers, and talks, attests to its validity as an insight-seeking methodology. With the Datacatcher project, which is reviewed with other related works in §5, this method is put to use to transmute data into a lived moment.
From the Interaction Research Studio approach then, I take the value of asking people about their lives without the data being used literally and the value of durational works. And, more importantly, I take the value of a long test. Though we differ is in terms of timelines and resources, the primary point of departure is in terms of who we are studying. The Interaction Design studio seeks to research and provoke social moments. In this case, I am investigating and provoking individual moments.
The Goldsmiths model, though innovative for design, shares a flaw with critical design: it still expects to operate within current social structures. Personal, experiential research requires one step further — into phenomenology. Closely associated with Maurice Merleau-Ponty, phenomenology asserts the power of knowledge through individual perception, through interaction with the world and through personal reflection.
In Closer, Susan Kozel makes the case for rehabilitating this slightly old-fashioned theory and for the ability to draw truth from our relationship with ourselves, through an erasure of the subject-object divide:
Phenomenology appeared exhausted in the 1990s, seeming to offer less rigor, insight, and inspiration than other philosophical lenses for examining the world. The critique of it as fundamentally a male, subject-centered approach to transcendent meaning overshadowed the validity of its basic tenets for a while — for a short while. Still, the core of phenomenology survived: that it calls for a return, again and again, to lived experience; that it takes as its starting point a position prior to, or beyond, the subject-object divide; that it shapes a reflective process that opens itself onto the richness of pre-reflective experience; that it is inclusive of a variety of experiences and not bound to a narrow and abstracted notion of truth; that it provides scope for the many dimensions of what we are as human beings to contribute to the expansion of knowledge and creation of cultural artifacts. Bodies, thought, imagination, memories, material conditions of life, and affect find a voice through phenomenology. (4–5)
Centered as it is around “thought, imagination, memories, material conditions of life,” the Oublié/trouvé appears well-suited for a phenomenological investigation.
Kozel also explicitly locates phenomenology in opposition to the positivist currents in contemporary tech culture:
To counterbalance this anachronistic, and simply weird, positivist turn occurring at the beginning of a new century, it became clear that a compelling approach to the validity of the subjective position in research was needed: a respect for the lived experience of the scientist, researcher, artist, designer, and writer, and an acknowledgment of the sometimes anarchic results borne from the marriage of perception and imagination in the process of thinking and doing. (12)
This finds its echo in Kozel’s use of Francisco Varela’s cognitive work to buttress the scientific truth of the phenomenological approach — the same Varela who was a student of Humberto Maturana and part of the reflexiveness-focused cybernetic second wave (10–11).
But what does “doing a phenomenology” (Kozel 52) look like exactly? At first, Kozel’s focus the body seemed difficult to reconcile with the invested object; this work that is only about the physical body in the sense that it attempts to pull computing out of a screen slightly and more into an object which with we can have a material relationship. Kozel, though, suggests “[t]he experience of using technologies in close proximity with bodies … can go in many directions: corporeal, spiritual, immanent, external, through memory, across dissolution, morphing, and transubstantiation, backward and forward in time. This particular proximity needs a range of under-formulated and highly poetic concepts to help explain it and to make links with the wider bodies of human knowledge and experience” (46).
With these thoughts as a guide, the invested object can be seen as an embodied work through the use of memory, reaching to make links with “wider bodies” of knowledge. It also buttresses the durational experience to insight approach from Goldsmiths, locating truth in the poetic and physical experience of living with the work.
The suitability of phenomenology for this project can also be considered in terms of telematics. As Martha Ladly writes,
Telematic performance describes the process of engagement with the long-distance transmission of digital, visual or kinetic information, and the interaction of the mind, the body and the senses, with the information received. As such, telematic performance can act as a catalyst for understanding the wider social and cultural implications of digital technology.” (139)
Oublié/trouvé is of course concerned with the transmission of contextual digital information over the long distance of time, making it a telematic project indeed. As Kozel demonstrates in great detail, various types of phenomenologies can be done around telematic projects, from the straightforward but electrically evocative reflections on Telematic Dreaming through the dialogic investigations of Ghosts and Astronauts and the pedagogy of Liftlink. In all cases, attention is paid to the physical and mental experience of the works, in particular to the space where two categories or experiences meet, to the space where bodies and the personal meet the world. This is the space Merleau-Ponty calls flesh, and which Kozel reworks in her new phenomenology.
This reconsideration of bodies and the spaces between them is also key to the reworking of phenomenology offered by this book. According to this reworking, bodies and the space between people are not simply matter or substance: they are connective tissue, electric and transforming, they are channels of communication. They are flesh. (30–31)
Truth emerges from flesh, through sensation and reflection. This is not without pitfalls, however. Though experience and perception — the pre-reflective — hold a truth, we must attempt to access it through reflection itself.
Kozel sketches the problem:
Obviously there is a logical problem in trying to use reflective practices to obtain access to the pre-reflective, but this is what phenomenology is all about. The tensions are clear and need to be stated up front: Can a process of thought reveal the pre-thought? Can a reflective practice bracket itself in order to reach a pre-reflective state without violating this state and itself in the process? … The impulse toward phenomenology as a method is based on the realization that we can loosen our rationalist structures of meaning sufficiently to permit qualities that are associated with the pre-rational, such as ambiguity of meaning, fluidity of existential and conceptual structures, scope for entirely new thought, perceptions, including contradictions, reversals of meaning, or paradoxes. (18)
and draws out the phenomenological solution:
It is impossible to prevent thought from attempting to pin down experience just as it is impossible to prevent experience from blowing holes in structures that attempt to codify it. Merleau-Ponty knew this, and attempted to respect the being of thought and the being of sensation, hence the circle he described in which reflection and the pre-reflective enter into relationship, and his dynamic notion of hyper-reflection.
Hyper-reflection is a process of thought that takes into account its own functioning. Like the recognition in contemporary physics of the impact of the observer on the observed, another operation beyond the “conversion of sense experience into reflection” is necessary: “a sort of hyper-reflection (sur-réflexion) that would also take itself and the changes it introduces into the spectacle into account” (Merleau-Ponty 1968, 38). The circle of hyper-reflection can be so finely grained, in such tiny loops that it can be done in the midst of experience. (Kozel 22)
What does this look like as a methodology? How does it communicate its truth beyond the hyper-reflection in the moment?
As a first-person methodology, a phenomenological description is received subjectively. As a purveyor of lived experience with the potential for new knowledge contained within it, one person’s phenomenological account can be received by others within circles of shared truth. Truth according to this model may be objective and verifiable through repeated experiments, but it also may be entirely unrepeatable and subjective. The truth offered through phenomenology is better expressed as relevance, and the way it functions is described effectively by Gaston Bachelard in terms of reverberation, resonance, and repercussion. …
The phenomenological impulse is to move from subjectivity to transsubjectivity. The phenomenological experience of another person unfolds across physical description with latent conceptual elements extrapolated and can be relevant to me based on my having experienced the same thing, or simply because I have the ability to construct meaning empathically, perhaps through imagination or previous experience. Quite simply, I can resonate with another’s experience. According to Bachelard, first there is a reverberation, followed by the experience in oneself of resonances, and then there are repercussions in the way we see or feel the world. The poetic image “takes root in us” even though it originates from another, and “we begin to have the impression that we could have created it, that we should have created it.” (Kozel 25)
A reflexive methodology for a reflexive object. I knew I was on the right track here. This feeling deepened when I discovered this approach also matches the way in which Shattuck discusses Proust’s goals for his own work, that readers read themselves into him (117–121). It is the same mechanism: to vibrate and continue that vibration.
Also on the same wavelength is Kate Sicchio’s image of she who undertakes a phenomenology, the immanent researcher. “In order for research to consider the interrelations and interconnectedness of ideas within a practical exploration, a continuum of multiple aspects are considered, as well as a continuum of the positions for the researcher in the work,” Sicchio writes (157). That is, depth of meaning can be produced by overlaying the aspects of the researcher as developer on those of the researcher as writer and researcher as user or test subject. The interactions and reflections that result work to produce truth in the same way as narrator’s stereoscopic views in A la recherche.
Seeking other practical methods to implement this phenomenological methodology, I looked to visual arts.
Graeme Sullivan’s practice-as-research theories and framework are grounded in questions around the uses of visual art in higher education and focus on understanding art-making as a research practice within the university.
Echoing the phenomenologists, Sullivan sets practice-based inquiry in opposition to a more positivist approach. “Rather than seeing inquiry as a linear procedure or an enclosing process, research acts can also be interactive and reflexive whereby imaginative insight is constructed from a creative and critical practice” (“Research Acts,” 19–20). He also points to the generation of knowledge as a result of the interaction of different aspects, and the way this can undo empirical divides between maker and observer:
Conceiving art practice as a theoretical act within a framework of inquiry sets in place the prospect of doing research in artmaking. When used as a site for research, art practice brings into play the seamless relationship between the “researcher” (artist) and the “researched” (art practice) and this builds on all the discursive arguments that disrupt untenable dichotomies such as the fictive subjective-objective divide. (Sullivan, “Research Acts,” 31)
Or, as John Baldacchino puts it,
Art practice is, in and of itself, a specific and special form of research. In the arts, the very idea of a qualitative-quantitative divide becomes irrelevant because by its distinct nature arts research calls for a different set of categories where the arts do not search for facts or stuff but they generate it. (quoted in Sullivan, Art As Research, 57, emphasis in the original)
Generating facts, in this project, may be known by another name: polluting the possible. The invested object project is practice-as-research.
Building up the phenomenological case in a research environment, Sullivan also highlights reflexive processes as critical to the undertaking. “The transformative nature of practice-as-research is best seen in its reflexivity and postdiscipline structure, and these are best represented in structures that are described as braided and self-similar.” (Art As Research, 110, emphasis in the original)
The latter “describes the way visual arts research takes place within and beyond existing discipline boundaries” (Sullivan, Art As Research, 111); it is clear this describes this research and program.
Sullivan expands the former, listing four types of reflexiveness that occur in the process. The first, the self-reflexive is the phenomenological reflexivity “that is directed by personal interest and creative insight, yet informed by disciple knowledge and research expertise.” The second, the meta-analytic, comes from the artist-researcher “responding to empirical understandings … so as to review conceptual strategies used and to consider other approaches.” The third and fourth focus on engagement with widening circles — the dialogic reflection of research discussion within the field and the questioning of social context and problems unearthed by the work (Sullivan, Art As Research, 111).
Again, it should be clear that the invested object project undertakes each reflexive practice. The second and fourth are to be found in the genesis of the project as a reaction to the social and empirical constructions of information. This paper is part of the academic engagement with research, to be reviewed, questioned, and responded to. And the first, the self-reflexive, is to be found in the reflections that constitute the primary data of this research.
He then moves on to present his deep, braided framework, a series of levels that begin with visual arts research and ascend through knowing and contexts into the practices that “describe creative and critical habits of mind that are at the core of thinking and making processes involved in art as research” (Sullivan, Art As Research, 193). The invested object project spans each of the four groupings (see figure 7.1), undertaking interpretive discourse and empiricist inquiry through the visualization of moments and their surrounding metadata, as well as through the construction of a memory narrative through stereoscopic juxtaposition. The entire Oublié/trouvé system can be read as the visualized idea of a new relationship with data, and it stands on the critical process, born from a critique of the current situation.
As a work of visual arts research, the invested object inhabits the thesis-exegesis structure. Sullivan describes this as “another public face for exploring and presenting art practice as research.” In this case, a physical work, like Oublié/trouvé is accompanied by a work of explanation and theory, like this paper. Though some find the structure redundant, Sullivan acknowledges, others find the exegesis valuable because it “acknowledges that visual arts theorizing is a diverse practice that can be articulated in many visual and verbal forms.” And yet this “active documentation” (as Nancy Freitas has named it) is not the research outcome it might be in a qualitative empirical work, but instead is another way to think through the ideas and theories resident in the visual or physical work. In the end, Sullivan argues, “an exegesis is not merely a form of documentation that serves preliminary purposes, that records in-process activity, or displays outcomes: it is all of these” (Art As Research, 221, emphasis original).
Though Sullivan finishes Art As Research reviewing a few projects, a deeper look at varying documentation in pursuit of insight comes from sociology — and the autoethnographers. In The Ethnographic I, Carolyn Ellis situates autoethnography beside more traditional ethnographies, sketching how it extends the ethnographic approach “writing about or describing people and culture using firsthand observation and participation in a setting or situation” into writing more impressionistic works, usually taking the researcher’s self as the core of a narrative work (Ellis 26–32). In fact, The Ethnographic I is itself a work of autoethnography — a history and explanation of the methodology in the form of a novel.
Ellis summarizes the approach:
Back and forth autoethnographers gaze: First they look through an ethnographic wide lens, focusing on social and cultural aspects of their personal experience; then, they look inward, exposing a vulnerable self that is moved by and may move through, refract, and resist cultural interpretations. (37)
Like all the foregoing methodologies, Ellis sees autoethnography in contrast to more quantitative, positivist approaches:
Rather than believing in the presence of an external, unconstructed truth, researchers on this end of the continuum embrace narrative truth, which means that the experiences they depict become believable, lifelike, and possible. Through narrative we learn to understand the meanings and significance of the past as incomplete, tentative and revisable according to the contingencies of present life circumstances. In interpretive, impressionistic, narrative work, authors are concerned about issues of validity, reliability and generalization but these terms take of different meanings in narrative research.(30)
Yet here the cracks between autoethnographic and phenomenological or practice-based research begin to appear. Ellis makes a number of claims for the value of personal experience within the social sciences, which means she still looks at the production of works as part of a tradition of divining research instead of generating knowledge.
Autoethnography, then, has less to say about how to use documentation reflection produces; where it excels is in offering up a panoply of works as valuable research methods. Ellis recounts how essays, novels, one-act plays, interviews, reconstructed conversations, and even paintings can work as effective objects and documents of inquiry, each communicating a truth in a resonant format.
She also presents an approach to field notes that has been useful in this project. I use notes more as reference material than sources of truthful representation and keep in mind the distinction Ellis offers when I choose what to record: “If you viewed your project as closer to art than science, then your goal would be not so much to portray the facts of what happened to you accurately … but instead to convey the meanings you attached to the experience” (116).
In addition to presenting options and suggestions, Ellis’s work also offers candidates for validity evaluation that can be important in buttressing these choices in the face of objectors who insist on a validity beyond the philosophical. (It can be argued that even this demand tilts the table towards their own assumptions, but when we can, meeting objections where they begin cannot hurt.)
She cites Yvonna Lincoln and Egon Guba’s evaluative methods: Fairness looks at whether all views are represented. “Ontological and educative authenticity” look to see if participants minds are changed and “catalytic authenticity” finds validity in the actions participants undertake. She offers her definition of validity: “To me, validity means that our work seeks verisimilitude; it evokes in readers a feeling that the experience described is lifelike, believable, and possible.” Ellis goes on to consider Laurel Richardson’s use of “a metaphor of a crystal to deconstruct traditional validity,” wherein distinctions in interpretation are not read as undermining validity, but as different aspects on a structure, before coming to rest on Patti Lather’s “counter-practices of authority that rupture validity as a ‘regime of truth’ and lead to a critical political agenda.” These include “four subtypes”:
ironic validity, concerning the problems of representation; paralogical validity, which honors differences and uncertainties; rhizomatic validity, which seeks out multiplicity; and voluptuous validity which seeks out ethics through practices of engagement and self-reflexivity. (Ellis 124–125)
These last two are the invested object’s claims to validity.(Unsurprisingly they are the counter-practices. Sorry, empiricists!) In this practice, the research methods are worthwhile and knowledge produced true because it rhymes and resonates with other works in art and literature on the same topics. Self-engagement and reflection, through a variety of media, if done rigorously and well will communicate an experience that is true.
Output & documentation
Reviewing these approaches has allowed me to come up with a list of research deliverables that are true to the spirit of the project and rigorous within its demands.
System Test and Observation
The primary basis for self-reflection is engagement with the Oublié/trouvé system itself. For this, I spent a month testing the system and its position in my daily life. I then undertook a phenomenology using Richard Lanigan’s phenomenological process as recounted by Ladly:
The first step in the method is to formulate a phenomenological description using phenomenological intuition, dealing with the capta, or conscious experience of the phenomena. The next step is to make a phenomenological reduction, whereby the observer determines which parts of the description are essential. The goal is to isolate the object of consciousness, the thing, situation, emotion or person that constitutes the experience. The description then becomes a reduction or a depicting definition, based directly on the experience, rather than on a conception of what the experience may be like. The final step is to produce a phenomenological interpretation, an attempt to signify meaning.(142)
The un-distilled reflection are in the field notes I took. The resulting essential experience is documented in §8, alongside the presentation of Oublié/trouvé. In this way I attempt to signify meaning.
Reflections on the Development and Design Process
In addition to reflections on being she who used the system, I took field notes on the process of being the hardware manufacturer, the software developer, and the designer and information artist. These reflections are also documented in §§ 8 and 9.
This approach also echoes elements suggested by Estelle Barrett in her appendix to Practice as Research: Approaches to Creative Arts Enquiry, particularly the discussion of the studio practice and the ensuing “re-versioning or retelling of the process as well as the discussion work itself” (Barrett & Bolt 199).
I interviewed myself about my project goals and approach at the beginning of the project and again at the end. These are reproduced in full in the appendix. While I don’t think there are any deep insights in the works, the format helped me to think through the types of questions I normally neglect and focus on self-reflection.
Seeding the app with prior memories, including some collected on paper in the moment and others generated by recollection, provided a space to observe what I chose to include as a moment of inflection. I used short vignettes to prepare for testing, in order to think through my expectations and identify the functions a test system needed to fulfill. In a different attempt, I created a zine and theoretical talisman kit to investigate polluting the possible of makerspaces.
These can all be found in the appendix, with the exception of the zine, which is shown in figure 8.6.
The two primary objections to this approach are a perceived lack of rigor and lack of external input.
I hope this section and the reflective output have put to rest the notion that an experience-based research project has less claim to validity than the more traditional positivist approach. In the performing and media arts as well as in the social sciences, knowledge is being produced through practice and reflection. Validity can be measured rhizhomatically and voluptuously.
The question of external input is a bit trickier. A good phenomenology should resonate further than the creator, even if a personal experience is a valid one. At the same time, limitations of time and scope prevent what I would consider to be a full test — giving systems out for multi-year tests.
I compromised by running two surveys, discussed §§4 and 8. The former considers the cultural resonance of my ideas of objects that are carried and that we remember with; the latter, the idea of Oublié/trouvé itself. Because these ideas will be received without context, the low-context setting of a survey is not the negative it usually is. Likewise, the intended pool of resonators is expected to have high capacity for self-reflection and thus not need much hand-holding to produce thoughtful responses.
So let’s see what happened.
8. The Project
Creating and Testing the Object
Development for the project began with manufacturing a placebo object. The current vogue for at-home fabrication — captured in Gershenfeld’s Fab, articles with titles like “Desktop Milling Machines on the Rise,” and pages of Kickstarter projects — meant a number of desktop digital manufacturing tools were available, often with beginner-friendly interfaces and tutorials.
I chose to use the MAGNET Fab Lab’s Othermill, a desktop CNC milling machine. Subtractive technologies allow us to work with a wider variety of materials than additive technology and to avoid plastic materials, since they just have to carve it, not melt and print it. Because I wanted the tactile interface to feel more human and less manufactured, I wanted the object to be made of anything other than plastic. I adapted a chocolate mold-making kit from Othermill’s project section to create talisman molds. This process combined some of our newest computer-aided technology with some of our oldest art-making processes — single mold casting.
I used the mill to carve positives of the talisman, created silicone negatives from these, and then used the silicone molds to cast final objects. The materials testing took place in my kitchen, completing the movement from factory to lab to the home. Echoing other kitchen experiences I have had, I found the baking-like tasks of mixing compounds, pouring and waiting more suited to my temperament than finer, heat driven work, like soldering.
The first mold followed the Othermill’s chocolate medallion sizing and produced four talismans, each with a diameter of 2 centimeters. I cast these in plaster and resin (shown in figure 8.1). Though I had wanted to avoid plastic, clear resin presented the possibility of being literally invested with other materials: glitter, soil, grass. Plaster and resin also presented different ways of taking and showing age and different hand-feels; the resin was smoother and less susceptible to outside forces. It also picked up finer detail than the plaster. Unfortunately, it aged worse, taking on a yellow tint rapidly and coming to seem dirty (see figure 8.2a).
Each set of talismans included four different designs on the top. Two were literal representations pulled from the icon sets I used in early design documents (reviewed in §6) and two were different shapes I thought might be pleasing to touch, one corrugated and one smooth and blobby. The icons were drawn so finely that the plaster did not pick them up, as we can see in figure 8.1b. I thought the corrugation, styled to look like water, might give more tactile interested to touching the object and the blob not enough. In fact, even the corrugation was too sharp, too like the spiky frustration objects in Isbister et al.’s SEI study (320–21; also discussed in §4).In terms of the “sculptural dimensions of interest” outlined in the same study:
- rounded vs. spiky (positive to negative valence),
- smooth vs. bubbly or protruding surface (low vs. high arousal),
- symmetrical vs. asymmetrical (calmer or more directed/resolved vs. confused/chaotic) (Isbister, et al. 327)
the sharp objects, though symmetrical and perhaps calmer for being so, were high-arousal objects. Their narrow, discrete protrusions added a sense of an interrupted surface and spikiness. By contrast, only the blob, with its seemingly lower surface protrusion and its relative smoothness would be associated with lower arousal. Its asymmetric shape reflected a bit of the confusion one might feel when reaching out to the object, and thereby kept a sense of emotional reflection while also being calming. This shape was also reminiscent of the test object Isbister et al. called a barbapapa, which American participants associated with humor.
And so I chose plaster and the blob to move forward with into a test of a larger object. I knew eventually I would need to put electronics into the object and the small item would not be suitable. Also, the size of the initial batch made the objects easy to lose. Later, too, I found larger objects worked better as worry stones / thinking objects. The broader top gave the fingers space to play, space to draw my mind along. The blob was the only form left standing from the first test and though I had intended to explore other related shapes, I fell in love with its look and feel; it was just right for me. It was also the one design I had drawn freehand, without adapting. As with test talismans, which I cannot throw away, even when they are broken, the more effort I put into making the thing, the more it means to me. The handmade object becomes invested with the context of its production thereby becomes a treasure.
I was not able, however, to escape fully the confines of the industrial. The sizes of the next test pieces were predicated on the sizes of precut wax available. I tested both thin and thicker options: 4 centimeter–diameter objects with a depth of 0.5 and 1 centimeter. The hefty 1 cm object became my favorite test object and I created a silicon case for it as well (see figure 8.2c). This object travelled with me all summer, to California and Colorado, and it shows its trip. Plaster, like leather, can take age and make it beautiful. It also has great “durability in the ground” as Gaimster puts it (59). A plaster object endeavors to be around in the future and once there can communicate its intimate relationship with its previous owner.
To test the balance between creating a patina and strengthening the material for even greater durability over time, I experimented with two further materials: wood glue–plaster mixes and concrete. The former mixes were based on advice I found online for creating strong magnets (Bechtel). These wound up too plasticky, like the resin, and were even more finicky, being liable to separate (see figure 8.3). They also did not take on any patina.
Concrete is beautiful and does take on some signs of wear. It can be cured rough or smooth (shown in figure 8.4) and, as I discovered in my visit to the Material Connexion library, takes glass and metal objects into its mix, opening up some of the ideas I had around resin (see figure 8.5). Sadly, in testing it did not stand up to wear and tear in my pockets or bag (aftermath shown in 8.2b, and cracks are visible already in 8.4). The talisman shape did not play to the material’s strengths.
The last failed test was mixing scents into plaster, to activate the olfactory recall that memory is associated with. I mixed nontraditional perfumes that I had used in an earlier project into plaster mix and let it set. The scents began as overwhelming and quickly dissipated.
A Zine of One’s Own
At this point, the pleasure of engaging with maker culture on art terms became apparent — as did the lack of obvious venues for that type of engagement. Without the theoretical structure that had gotten me into the fabrication room with a weird plan already up my sleeve, I would have been at the mercy of traditional introductory projects: robots, chocolate or jewelry. And I wouldn’t have enjoyed it.
The joy in manufacturing was, as Gershenfeld discovered, to be found in fulfilling idiosyncratic desires. But first I had to get into the lab. And so, as a small research project, I created a zine as the first element in a speculative design for a kit that could be used to encourage some other art nerds to get in front of a desktop mill. (The full un-folded zine is shown in figure 8.6) The work uses the vernacular of feminist punk zines, which are associated most prominently with 1990s riot grrl culture, though they remained popular with indie rockers and DIY enthusiasts through the first decade of the 21st century — including a younger version of the Sarah typing this.
Though this path remains undeveloped in terms of the current project, the reception of the zine among my peers validates for me its approachability and ability to generate excitement. In informal presentations and show and tells, everyone who has seen the zine has read and loved it. Most asked when I was making the kit. An invested object that came with a DIY talisman kit would get excited weirdoes into makerspaces at least once and add a tiny pollutant to the stream.
Taking the Object Electric
Once the external form of the object was set, it was time to take it electric. As shown in the second storyboards (see figure 6.14), the object works as a non-screen interface to the application, which means it needs to do two things: send a signal when asked to by the explorer and receive a signal from the application. It also needed to be able to be attached to a bag or pocketless skirt or dress.
I began by testing basic components — vibrating motors and LEDs — in plaster to be sure they survived. I also embedded magnets in the test objects. These were all successful (see figure 8.7).
To connect the components to the app, of course, I also needed a microcontroller. After researching various options, I chose the Light Blue Bean from Punchthrough. The Bean has an integrated BLE chip and an Arduino, plus a Software Development Kit for creating apps on the iPhone. It was the smallest option I could find: 4 x 2 cm with the full prototyping board, 3 x 2 cm with the extra board removed. Its biggest drawback is its price, so I tested embedding a cheaper Red Bear BLE microcontroller in plaster while developing with the Bean (see figure 8.8). This too was successful: even when embedded in plaster the unit could connect to my phone and transmit proximity data.
The final hurdle in embedding the microcontroller unit in the talisman, though, was power. Since I needed to test a version of the system for at least a month leading up to this paper, I chose to create an object that embedded the motor and button in plaster while keeping the microcontroller free. For this, I used the Othermill to create a circuit board to hold the elements and embedded this in plaster (see figures 8.9). This is the object that was tested with the system.
Reflections on Being a Domestic Manufacturer
The most notable results of my experience as a domestic manufacturer has been an increased interest in the materiality of the objects that populate my life, a curiosity into the circumstances of their production, and a demystification of everyday electronics.
One evening I was in the lab milling a new wax positive. This process takes a while, as the machine cuts the facing, the blob, and the pedestal in separate passes. I had also made a mistake on the maximum cutting depth setting — leaving it at the incredibly shallow default — which means I was sitting there for quite a while.
I was reading Dunne & Raby’s Speculative Everything to pass time, flipping though the book and studying photographs of speculative industrial design projects. As I listened to the machine grind its way through the wax, I was struck by the incredibly obvious fact that everything in this book was made. These school projects, these speculative chairs and Herzian boxes, what did their fabrication rooms look like? Were they a kitchen, a studio, a workshop? Where did the materials come from? How were they paid for? Did they, like the Goldsmith’s team, have shops they could hire to take this block of expertise on for them? Where did these materials come from and who had the expertise to fold them like this? What did they feel like? Did they feel like this table, this pen, this machine?
I knew, of course, that someone made everything we have. Someone is inside those factories. But in my domestic manufacture it became real.
My visit to the Material Connexion library, unfortunately, underscored the gatekeeping that still exists one step beyond the open culture of desktop machines and fabrication labs. The space is called a library, and makes samples available to be browsed and inspected. The breadth of materials available was invigorating. We could be filling our homes and recording studios with tiles of sound-dampening moss right now. Or having bacteria secrete plastic replacement from fruit juice.
But then, unlike at other libraries, the knowledge ends. The database is limited to where these materials can be purchased. The gate closes. And it is maddening.
I felt the sting of exclusion here all the more keenly for its proximity to the liberatory demystification of DIY electronics. It is easy these days for machines to feel like magic — comprehended only by engineers after many years of sadistic lab classes. But the availability of electronics self-education, made possible and popular through Brand-influenced notions of a “hacker ethic,” pulls back the curtain and makes them intelligible. The motor in my phone works like these little motors I am soldering in. Circuits are circuits, and they have a history. The other elements are small and complex but I am capable of knowing them and their principles. To make is to find out.
By instantiating counterculture–cybernetic ideas of personal liberation through machines, desktop manufacturing machines have given me the tools to instantiate a different idea — one that works to counter the toxic notions bound up in contemporary techno-liberation. Taking on the role of small manufacturer in conjunction with that of artist highlights the contingency of today’s technological narratives. Because I can see myself in a inhabiting a position counter to the narrative, I gain an aspect on revolt from within.
The application is object’s antipode in the investment-playback-reflection loop. (see figure 6.12) The goal of the application is to facilitate steroscopic memory by creating various concordances of moments and presenting a platform for reflecting on these.
Roger Shattuck summarizes the working of depth in pursuit of understanding in A la recherche,
Within the limits of the novel Proust creates a form of double consciousness, which I have examined at length as stereologic or binocular vision in time. As our two pupils, when properly functioning, form one three-dimensional image in the mind, so the experience of two related events separated and connected by the proper interval of oubli [that is, forgetting] forms one four-dimensional image in the consciousness — a moment bienhereux when it occurs fleetingly and without lasting effect on our life pattern, a self-recognition and piercing of the veil of illusion when we are able to sustain our consciousness at this level. (131)
Through repeated engagement, each of us can encounter our own topology of time and thereby our own autopoeitic self. Creating an application to facilitate this serves as an experiment into engaging with more contextual information, information that holds traces of its origin and requires its context to be fully understood.
To imagine what this would look like in terms of screens, I put together four mockups that covered the essential screens as I saw them (see figure 8.10). These remained mostly unchanged through implementation, although the augmented view screen fell victim to scope reductions.
Rather than spend more time on design documents, I moved into coding the application and working out design details as I went. Though the development process was integrated and iterative, it is perhaps easiest to review the application screen-by-screen. The full architecture can be seen in figure 8.11.
Home Screen, Figure 8.12a
Using a gentle seafoam background, the home screen establishes the feminine aesthetic that pervades the work. The combination of a geometric sans serif and serif letter in the Oublié/trouvé logo honors the modern and Victorian ideas that underlie the project. In inner screens, the monospace-inspired typeface brings in an aestheticized-tech feel.
This screen is essentially as it was in the first screen designs; however, tapping the main logo does not bring up the save moment form — only the main form button does so. This is to deemphasize the application as a quick moment-saving interface; that is the role of the object. At the same time, the Record Form button remains available on every page, should I be inspired while using the app.
Record Moment Form, Figure 8.12b
The form auto-populates the title with the date and time. In testing, I rarely used the app to save a moment. This is a victory for the value of a tactile interface, though it did cause problems with annotating moments.
Bean Screen, Figure 8.12c
Accessed from the button to the left of the Record Form button on the home screen, the Bean screen provides administrative functions for the object, including the ability to test the motor buzz.
All Moments List, Figure 8.12d
Moment Details, Figure 8.13
The list of all saved moments allows me to access each moment without going through a concordance group. They are arranged chronologically.
The moment screen itself presents the smallest information view available in the system. It contains the moment’s title, its data representation, and any notes I have added.
Figure 8.14 shows two moments captured via the object and left unannotated. I have forgotten what some of these moments represent, and therefore why they were saved. In an informal critique, one viewer suggested using voice notes in the object to send data to the application without typing.
It is also possible to interpret forgotten moments not as a problem but as an indication. We can consider it evidence of a moment that just didn’t make it over the threshold; it is a type of false positive that did not rise to the requirement for sustained intellectual effort that Shattuck identified as necessary for veil-piercing recognition.
It may be the detritus of an effort to develop a habit, to make the system into a friend. When testing, especially early on, I struggled to remember to take the object with me; I was afraid of breaking it; I forgot to charge its batteries. Then time turned work into care: the object was a pet. The same may happen with moments themselves over a longer testing period. Work may cease to be a problem naturally or I may have to iterate explicitly on the interaction.Figure 8.13 shows the moments that were originally captured in the paper test. The group’s data representations give an idea of what kinds of similarities and variations are produced by moments collected in one location at one time of year. These are as I hoped.
However, the interaction with them is still lacking. I sometimes found myself wanting to — in a calm moment — browse the series of moments. In this mode, toggling back and forth from the moment to the list becomes disruptive and interferes with the ability to consider moments in relation to one another. One of the first post-presentation upgrades will be to make it possible to swipe from moment-to-moment directly. As a designer and developer, I made the mistake of allowing the hierarchy of the data to interfere with the rhizomatic experience I sought.
Concordance List, Figure 8.15a–c
Concordance Image, Figure 8.15d–f
Concordance Moments, Figure 8.12f
Of course, it is the concordance creation and view that gives the application its idiosyncratic character. Concordances are formed by matching a complex of metadata among moments. This may include the weather, the distance from home, or the elevation associated with the moment.
The images are meant to be objects of meditation, a focal point when considering the grouped moments, which are accessed from a list similar to all moments list. The list-detail relationship in this section suffers from the same flaw as above.
The list makes available all concordances, without naming them. In the first round of tests, the concordances were simple groups, based on a single type of data, like temperature. Knowing the grouping principle made it difficult to engage with the grouping as a topography deserving exploration and consideration, so for the second half of testing, when the complex concordances were generated, I removed the group names.
When a concordance is detected and the object is notified, the application opens to the related concordance image. When this occurred during testing and the timing was right — which to say, I wasn’t distracted or cranky — it was exciting to discover the associated image and representative moments. One unseasonably warm October day gave me my best set of memories, a set of nostalgic intensities around dry warmth that I have experienced before but still am working to understand. It begins in the type of childhood memories that are only identified by a location and fleeting images — the YMCA, dry leaves, a wild rabbit — and then layers on days until I have an image of myself as a child of the West, just as surely as I have when I read Joan Didion or encounter other more explicit memory instigators. While I cannot claim a complete recognition, the system is capable of delivering concordances that are worth considering repeatedly, each time clarifying the reality at the core of this selection.
Developing the Data Views
The data views are not just objects of meditation or spurs to investigation. They exist to give depth to the text and the moments as a whole, providing context aesthetically instead of explicitly.
In developing the approach to information, I drew on Gaver et al.’s concept of ambiguity of information. He argues for ambiguity as a tool in human–computer interaction that is counter to traditional, cybernetic-descended interactional design, but valuable for its expressive, rich qualities.
Ambiguity can be frustrating, to be sure. But it can also be intriguing, mysterious, and delightful. By impelling people to interpret situations for themselves, it encourages them to start grappling conceptually with systems and their contexts, and thus to establish deeper and more personal relations with the meanings offered by those systems.
The ability for ambiguity to evoke personal relationships with technologies is particularly relevant as digital technologies are designed to support activities outside of work. Traditional concerns for clarity and precision are superseded in such systems by the need to provide rich resources for experience that can be appropriated by users (“Ambiguity as a Resource” 233).
They identify three types of ambiguity: ambiguity of information, ambiguity of context, and ambiguity of relationship. The first, which we are concerned with, pertains to ambiguities that arise based on how information is presented. The simplest example given is the Mona Lisa’s smile: what does it mean? There is no way to actually know. Or a GPS system may not have resolution to the level that the screen can display it. The information is ambiguous. This is the type of ambiguity that data visualization normally works against.
Gaver et al. give suggestions for enhancing ambiguity of information. “These focus on creating or reflecting uncertainties about information that are in some way significant,” they write. “The purpose may be merely to make the system seem mysterious and thus attractive, but more importantly it can also compel people to join in the work of making sense of a system and its context.”
Use imprecise representations to emphasise uncertainty. …
Over-interpret data to encourage speculation. …
Expose inconsistencies to create a space of interpretation. …
Cast doubt on sources to provoke independent assessment. … (“Ambiguity as a Resource” 237–38)
For the data view in this application, I chose the first tactic.
Imprecise displays such as these are often described as ‘ambient’, but they are ambiguous as well. This ambiguity, we believe, is crucial for understanding their appeal: they may be perceptually undemanding, but they require users to fill in the gaps in information that is purposefully imprecise. (“Ambiguity as a Resource” 238)
Each moment of inflection saved to the application can appear in any number of concordances, which is to say, a grouping in which some portion of the metadata is shared among each element. A concordance is therefore a list of moments in which a particular element appears, much like a traditional concordance lists passages in book (often the Bible), where particular words or ideas appear. In a concordance here, the word or idea becomes some element of the context in which the moment occurred. This might be as simple as the temperature range or as complex as elevation + distance + humidity.
I decided to create visual representations of the concordances that were driven by the metadata on which the concordance was based but which were unable to be read directly. This way different groupings would feel different but would not lose their magic. For moments, the goal would be the same, and it was important they be recognizable as a group.
Various treatment explorations can be found in figures 8.16 and 8.17. Based on feedback from a number of design friends that the organic bean moments were more legible as data, I chose the big texture treatment for moments, since my goal was to obscure. For the concordances, the hardest choice was between the wash effects and the big texture effects. The washes are far more ambient feeling and undemanding, but I chose the big texture to keep a tighter relationship between the moments and their groupings.
Code & Reflections on the Developer Role
To evaluate the success of this approach to personal information and to maximize possibility polluting potential, it was important Oublié/trouvé be a working application and not simply a design prototype.
The code for the project is available on Github. Figure 8.18 shows a diagram of the system.
Although most of the development straightforward, trying to work ambiguity into a system predicated on a total lack of it definitely felt as though I was working against the grain. In order to keep from generating too many concordances or from generating them too consistently, for instance, the code uses two different randomizers: one to select whether to look at all and another that generates possibly concordances from raw categories without filtering for populated categories first. While the code for this is not difficult, I did wish for an ambiguous data type or event.
More generally, as I was a kitchen-table hardware manufacturer, so I was for software. In both cases, the experience of small-scale making was one of integration and pride. I made the thing and I made it exactly how I wanted. I was powerful and brought change to bear on the world.
The biggest disjunction between the roles was the experience of time and iteration. In hardware development, time refuses to be malleable; there is no collapse, no annihilation, no bricolage or juxtaposition, just long, grinding unified time. You wait the time it takes to grind away at a block of wax and when you make a mistake, you must wait to start again. Software can be tweaked quickly and the change is effected instantly.
Despite the improvisatory freedom, though, the role of developer is a site of anxiety. The process of writing code and testing it can can feel like supplication to a compiler, even if you are equipped with unending offerings, a new one every second. A culture of presumed expertise and unfriendly documentation often left me feeling fretful when working. And then again, like most developers, I did sometimes lose track of time when coding. The creation experience of software — worry and absorption — brackets the intense but open self creation experience that the application provides for.
Inhabiting the role of developer meant inhabiting the core of information theory while working oneself out of it — alternately oppressive and hopeful.
A Website for the Project
My experiences through various roles were personally compelling and meaningful. However, when explaining the application to my cohort and other interested parties, it was not always clear that the project was comprehensible as a whole — and it was challenging to describe the system and its goals succinctly.
To address these concerns, and to investigate resonances of the app absent the reflections that this paper provides, I created a faux-marketing page at lost-time.club. The page plays off standard startup marketing tropes to explain Oublié/trouvé. I then set up a short survey which I posted to Twitter and Slack, as I did with the Lovable Objects survey detailed in §4. I received 19 responses, which are reproduced in the appendix.
Other than one respondent who found the exercise boring and thus stopped reading the site partway through (but still decided to complete the survey!), most readers did understand the system to a large degree; some even had cleverer descriptions than I came up with, like “Quasiquantified self-reflection” and “NaaS (nostalgia as a service).” I attribute the clarity to the system illustration primarily (figure 8.18).
The system appears to be a decent gambit for polluting the possible through use. Only 16% of respondents were definitively uninterested in having the system in their own lives; the rest split equally between yes and not sure. Some were interested in the object as an art piece; others as a tool. One response made it clear that the satire of the site had been read into the piece as a whole.In terms of polluting the possible through inspiring others to create their own idiosyncratic tools, the project also appears to be a success, with 21% giving a strong yes to the question “Does this make you want to invent your own kind of object?” and another 30% saying maybe.
Reflections on the Role of the Designer
If knowledge is built phenomenologically, in the role of tester and researcher, it is still easiest to disseminate as designer. Having been a interaction designer professionally for a number of years, this role was the most familiar and the tools were within reach.
At first I wanted to avoid traditional design documents, seeing them as tools of the product design system the invested object is meant to work against. Besides, if I was both manufacturer and designer, did I need illustrations? I only had myself to communicate with. In fact, being able to communicate the goal of the object before years of memory were embedded in it was important. Pollution is a long-term project and design documents are useful to compress time somewhat.
Comparing the project to critical design also made it clear that what needed to be changed in order to make a wider future available through design work is not form but the status of authority. As we saw in §3, critical design begins its work with the forecasts of experts, who are by their nature committed to current plausible futures — and often their authors.
In the introduction to Writing Material Culture History, Gerritsen and Riello relate the rise of material culture history is to the democratization of history as field and the subsequent focus on everyday lives.
In the 1960s, historians first became interested in understanding the everyday lives of ordinary people who had lived in the past. This was a shift away from the idea of history as determined by the few at the top, to what has come to be known as ‘history from below in which agency is given to the lives of common people. Yet the people who replaced kings and queens, prime ministers and generals as the subjects of history did not necessarily leave substantial written records. One of the ways in which we can trace their lives is through the material goods they left behind. (4)
Though we may not have the sway to leave interviews or shelves on manufactured goods behind us as we attempt to pollute the possible, even leaving design documents preserves a trace of our yearnings. Using design tools to spread idiosyncratic projects — as long as we avoid appeal to authority — does make an effective route to polluting the possible with our own, small-people ideas. It also provides the written materials some material culture scholars, like Styles, look for. Our information can be made plain along with our experiences.
Life with Oublié/trouvé
I have reflected so far on each role except the most important — using Oublié/trouvé as the human portion of the symbiotic system, as the explorer. For this test, I carried a version of the object and app with me, beginning October 17 and extending through today. The app was pre-seeded with memories collected via paper testing and reminiscence. (These are reproduced in the appendix.) Visualizations were not added to the test until November 1. Concordances were changed from simple, single-dimension lists to complex groupings at the same time.
Although I was excited to test my system, the first few days were difficult. I was afraid to break the object and the bright LED drew attention to the electronics. (I didn’t want to turn it off because it was how I knew the object was connected to the application and the battery was charged.) The battery was constantly running out. Then, in a process of acclimation similar to moving in with someone for the first time, the moments of finickiness turned into opportunities for care. I became less afraid to hurt the object. A light in the early morning quiet was a sign the system was working, an external part of my memory humming next to my coffee cup.
Concordance notifications varied with concordance complexity. I only received one after the more complex checks were introduced; it was much anticipated and much appreciated, although in that case it was more of a distraction than moment for reflection. Overall, I received notifications in convenient and inconvenient moments. As I might have anticipated after my experiences with Reporter, there were times I was annoyed to be interrupted; these were mostly when I was working. In a broader test, this would be less of a problem because I would not be spending most of my days working on the object or system that was also notifying me. There were also times I was happy and receptive but was unable to engage because other people were around. (Just like Proust’s poor narrator. It appears the system successfully targeted moments bienhereux.)
The effect of a notification could continue long beyond the moment of engagement, too. The best receptive moment, in fact, started badly — on my least favorite kind of gray, swampy day. I can be a bit of a plant, and days without sunshine are trials to be survived; worse, I don’t even find the rain nourishing. So when the object buzzed and I saw the Humidity: Swamp concordance had triggered it I was not happy. This was why I needed to get onto those complex groupings, dammit! I have zero nostalgic memories of rain! I grew up in Southern California under a drought! What a stupid mistake.
But then, an hour or so later, the irritation mellowed slightly, the right song came on my headphones at the right train stop and it all came back. It worked. There were gray days that had this flavor, this depressed thoughtfulness. And how many times on those days had I listened to the Psychedelic Furs? I loved the graveyard here at the Wilson L train stop in the sunlight, but also again, vague in the soft rain. Views stacked up; juxtapositions. I can recall the freezing publishing office cum bookstore I worked in when was 22 and consider how much and how little I am that same narrator.
Overall, the subway was a highly inflected place. A few times, it was the feeling of hominess on an evening G train — the loud yellow and orange seats in bright counterpoint to the hushed conversation. Another, it was the sight of station mural I’ve walked past for over a decade and the thoughts of all the different moods it saw: happy to be home, running to pee, drunk, in love. In fact, I had to be sure to design the application to handle a moment when GPS was not available, leading to the underground elevation category.
I don’t know that in any case I can point to a recognition that became clear here, but I am still much younger that Proust’s narrator is when he attends his ultimate illusion-piercing party, so I have hope.
As for moment saving — I have hope, too. I did not collect every moment and I failed to annotate many. Towards the end of the test, though, I did forget the object and regret bitterly being unable to add a moment. I was on the go, I did not have pockets, I had that inflected feeling and I was consumed with regret. It was working. But it was not consuming. I willfully let a moment pass when noting it would be disturbing. I did not become a monster.
My interaction with the system settled into a frequency befitting a project of long duration. The object is usually nearby and mostly comes out with me, but notifications no longer appear daily. While this is not ideal for a month-long test, it is ideal for enjoyment.
Often when thinking, I played with the object. A few times, I opened the app to peruse the moments and concordance lists, but this has been the least explored aspect in day-to-day use. As I document the piece, though, I have come to enjoy looking at the individual moment expressions; their images that are just different enough to be interesting. I return to sets of moments far more often than I ever checked the graphs in Reporter. My goal isn’t to decode them even though doing so would be trivial; instead I enjoy seeing them together as variations on an impossible core experience that is itself inaccessible except through slight glimpses. It is a different, interesting approach to information, and I am slowly building a meaning with them.
Overall, then, the experiment has been successful, if too short. I was able to form a friendship with my object and a rapport with moments and their context. Many of the scenarios I generated to test around occurred — particularly wanting to save moments both with and without having the object and sometimes being annoyed at the buzz. Others, particularly around widely disparate moments, have yet to occur.
Through Other Lenses
Though the truth of my own experience was the primary goal of this research project, it is also worth investigating how the system fares when viewed through other lenses introduced in this paper: phenomenologies, material culture, the characteristics of an invested object, and Turkle’s “digital character.”
Phenomenologies and Alternate Research Methods
The project hewed closely to the phenomenological methods outlined in §7. The reflections in this section follow the more straightforward style Kozel models in her Telematic Dreaming phenomenology, as do the reflections on other roles — manufacturer, developer, designer. Taken together, they are the work in the style of Sicchio’s immanent researcher, examining the personal changes experienced when a single researcher lives each role. The reflections were produced using the three-step process described in Ladly: field notes as capta, this paper as the reflected experience and pursuit of meaning-making.
The self-interviews, imaginative goal setting and marketing site satire expanded the project into some of the more creative document types suggested both in Kozel and Ellis.
In §7, we note Sullivan’s assertion that “an exegesis is not merely a form of documentation that serves preliminary purposes, that records in-process activity, or displays outcomes: it is all of these” (Art As Research, 221, emphasis original). Altogether, this paper and related writings; the Oublié/trouvé system; and the process blog make up a project in the thesis-exegesis style.
It is therefore not surprising to see this project engaging in all four types of reflexiveness Sullivan identified as characteristic of practice-as-research works: the self-reflexive is present in the reflections; future plans outlined below represent both a meta-analytic response to empirical surveys and a response to problems unearthed within this research; and the paper and its defense are themselves an engagement with the dialogic academic practice.
Based on the feedback to the marketing site outlined in §8, particularly the 21% of respondents who said the project description made them want to make something, the project can be considered moderately successful in meeting the phenomenological goal of resonance or vibration within the community.
Part of the purpose of an exegesis is also to reflect on the creative process that generated the entire project. That is, to also ask what there is to say in the role of artist. This question could also be restated as What would someone else learn if they decided to make something for themselves? or What will those 21% find when they seek to create prototypes to pollute the possible with their dreams? For the desire to make something for myself is what drove the phenomenological evaluation and every subsequent decision.
Having a background in user-centered design work, in particular as part of design and advertising agencies, as well as within startups, I chose to use a standard process but to turn it on myself: I would create exploratory documents to define my vision, map out the system, create mockups, and then code the work, as we saw in the deliverables in §6. In some ways, designing for oneself is easier than designing for a separate client because there is no communication gap — I generally know what I mean. I know the need I am attempting to meet.
In fact, anyone designing for themselves has likely had the dream long before the tools. We saw this in Gershenfeld’s story about How to Make (Almost) Everything, his initial class at MIT that drove the fab labs as a movement. Students already wanted a scream container or runaway alarm clock (Gershenfeld 19–21, 23–24); they just needed to know how to make it. By separating the what from the how, at least at first, the user-centered process put me in the position of seeking skills to make my dream object. I was able to do so by take a digital fabrication class in the spring of 2016; other inspired makers may have to go looking at fab labs or other types of makerspaces. I would expect them to discover the same sense of revelation and liberation I found in learning how to become a kitchen-table manufacturer (see “Reflection on Being a Domestic Manufacturer” in §8).
More difficult may be defending the value of creating a personal object as an artwork instead of as a product. Much more so than creating a work that sits on a screen or a wall, the idea of creating a product as art project can be difficult for people to interpret, especially when it can first appear similar to contemporary tracking products. When explaining the project to friends and friends-of-friends, the assumption was always that I wanted to sell the system. Even Dunne & Raby grounded their critical design approaches in salability (see §3) and the potential for change that exists in consumers’ wallets. I would hope anyone else undertaking a project similar to Oublié/trouvé, possibly even making their own sort of invested object, would be able to experience the same surprise at the lack of a profit-motive and find illumination there.
More broadly, seeking illumination through a repurposing of the user-centered design process requires the ability to inhabit multiple roles and to alternately set down requirements and to fulfill them. You have to be on top of interrogating yourself and then able to turn that off to produce a work that meets the uncovered needs. Traditional design documentation came in handy here, proceeding from icon selection to system definition allowed me to document my requirements upfront and then check the system progress against them. When trying to decide, for instance, whether missed concordance notifications should accumulate or simply expire, I was able to reach back to idealized scenarios and storyboards and realize that notifications were meant to be contextual and contingent, so they did not need to stack up.
The self-interviews, by forcing me to explicitly inhabit separate roles, also helped me keep track of the goals of the work and to evaluate its success on completion. Beyond that, there was the pleasure of being paid attention to by an interviewer, even if the interviewer is oneself, and the fun of attempting to explain how one sees the world. This pleasure expands into the joy of being able to manifest a very particular product — via one-user-centered-design.
In the end, by lavishing the care usually reserved for clients on our own wants, the process of designing for oneself echoes the refusal in Oublié/trouvé itself to aggregate the metadata around moments of inflection and look for high-level trends. In both cases, the value of the particular is defended. This may be the ultimate insight generated in turning a standard process into something else: the particular is valuable and we can repurpose our tools to support it. In doing so, we may even find a little liberation.
The Oublié/trouvé research project therefore lives up to the goals set for it in §7. But what might the object communicate to future material culture historians? In “Material Culture and the History of Artefacts,” Viccy Coltman presents a series of questions that can be asked of a work of art or other artefact to “pierce the mute carapace of objecthood to let the work speak” (Jules Prown, quoted in Coltman 20). These include questions about the object itself, its production and consumption, and its afterlife (see figure 9.1).
Answering these questions, we can sketch what a material culture historian might see:
The object is made of plaster and electronics. It is either broken or incomplete, as it has a button and motor but does these are not attached to one another. It is reasonably small and light. It appears to be handmade, and may be unique. This is perhaps the most distinguishing aspect of the object, as it can be dated to the time of ubiquitous small electronics mass-production. It was mobile and carried a lot, as we can see from the wear. The owner did not take special care of it.
Other questions are unanswerable, still. If the software is no longer available, which is plausible, given issues with conserving computer art already, future historians are unlikely to be able to understand what it was for.
The object’s intimacy may remain the most durable meaning possible to take from the artifact. It also can communicate the desire to make electronics belong to us; the personal in personal computing is usually the expression produced with an impersonal machine, but in this case the personal is in the development of the machine itself. Perhaps if the theory behind the work is lost with this exegesis, the raw desire to make the impersonal personal will be communicated to the future.
Characteristics of an Invested Object
Back here, with the exegesis at hand, we can consider Oublié/trouvé in terms of the goals set out in §4 (see figure 9.2). Overall, the object succeeds at being an invested object.
The object portion of the Oublié/trouvé system is a real thing: it is physical; the surface decoration communicates its use as a tactile object and the material takes on a patina over time. Each scratch makes each iteration unique. The durability of the plaster and the internal electronics is able to communicate its use. (See the various test objects in figures 8.2 and 8.3.)
It is a thing I can think with, and have thought with. The object has worked as worry stone and become the repository for my feelings about the project. It lives on my desk and in my hand as I work; I rub it when I am having a hard time. In accordance with the interpretations in the SEI study, the asymmetric, blobby top shape is calming while reflecting some of the incomplete thoughts that may bring it out. Under my thumb, the wobbly line mirrors the unease of an unresolved bug.Practically, the object is transportable. It is small — pocket-sized. It has magnets embedded to allow it to be attached to bags for users — mostly women — who often do not have pockets. It is also something that users can make themselves. Manufacturing requires desktop machines that are available in community fabrication labs.
The system is private: the structure is open and the surface is malleable. The data visualization is personal and nontransparent, though the color and arrangement were chosen to echo the emotions I associate with dimensions of metadata. For instance, in general the more preferred intensity (clear days, warm temperatures) results in a bolder pattern. Because data lives on the phone, it cannot be communicated to others without permission; the unstructured irreducibility of text also keeps meaning private and away from the surface.
As we’ve noted previously, the system expresses its reflexivity by allowing the user to chart and investigate a topology of personal time. It also attempts to maintain context for information through the metadata is saves with each moment. We do not try to avoid feedback or neuroticism, or conserve stasis in the system. Over time it can amplify repeated metadata and the related patterns will become characteristic of the app’s aesthetic.
In this way, the project reaches strongly towards being a different sort of information experience.
Digital Character / Object as Work
This is where we run into the most difficult interactional hurdle. Turkle expressed this in terms of digital character — the idea that the possibility for completeness in a digital archive would drive completionism in users, overwhelming slower methods of meaning making. Likewise, it was possible that automated memory-gathering gives us the impression that we do not need to think through recollection, to build up meaning bit-by-bit. This project attempted to create a digital archive where this was not the case, but instead was more like browsing through a drawer of photos.
This is provided for by, on the one hand, creating supports for annotations and browsing and, on the other, by divorcing concordance notifications from direct linkages and over-completeness. Because saving a moment requires direct action and is not a background process, the human is prevented from the sense that the machine can do everything.
At times during testing, the project did feel like work, particularly at the beginning. Once I reminded myself that I was engaging in a longer term endeavor, beyond this report, the worry dispersed. Engaging with the app as a self-practice and focal point of reflection was engaging without being overwhelming, and but for being part of an academic work with a deadline, it felt like a recreation.
While imperfect, the project was, in the end, a digital work with the characteristics of a more analog set of remembrances. Through the metadata chosen to create concordances, the app grounded memories in the circumstances in which they arose, working against cybernated abstractions. Likewise, the presentation of data through expressive, interpretive visualizations kept the data private.
It was a different experience than I’ve had with other tracking apps and as such, a success.
Revisiting the Watcher Object
In working to counter the watcher object, I did end up creating a system that also grabs contextual clues and checks in on its location at least a few times a day, which means checking on its human counterpart’s location as well. In so doing, did I merely recapitulate the watcher object — keeping tabs, reporting back?
The sharpest distinction between the watcher and invested versions of objects are in their context and narrative, of course. Oublié/trouvé is explicitly not for personal improvement and it does not promise to explain me to myself or make me safer by being used on others. It does not try to find trends in numbers — rather it looks to express connections via images; it does not aggregate data; it does not promise to find deeper human truths by aggregating one person’s data with another’s.
The distinction, too, can be found in the sensors and physical components used in Oublié/trouvé. Rather than use particle sensors like the Point or gather visual or personal physical data, like Lively, Sense, and Pavlok, Oublié/trouvé works only with location data and generalizable categories. It cannot tell you who was there, like MyLifeBits, or how loud it was, like Reporter. It can only give the public physical context of the moment and it does this only expressively.
The invested object is fundamentally private, whereas a watcher object is not. And the commitment to privacy goes deep: into the code itself. Rather than grasp specific numbers and retain them infinitely, the Oublié/trouvé application converts the quantitative into qualitative. That is, readings are sorted into labelled, vague categories — humidity, for instance, can be swamp, normal, or desert — and the numeric value is thrown away, thereby increasing the privacy value of the system. Likewise, the moments are only stored on the phone, keeping them localized to the user’s world.
The most interesting conclusion I draw from these differences is that the technology we have today can support a different type of interaction and exist in a distinct narrative without much modification. That means the ends to which we choose to deploy it are determined culturally — which means we can start making different choices now, if we want to. This is another way in which the work was a success. It demonstrates that our current path is not part of a technical teleology and we have so much room to make new dreams.
In what may be the final evidence of success, there are a number of routes that might be taken to further the project and better investigate the questions it has raised.
Making it possible for the battery to be charged inductively and creating custom PCBs for the object hardware that integrates the microcontroller and components into a single round board would make it possible to put together a home-manufacturing kit. This kit could be used as the basis of wider tests and research similar to the Goldmith’s Datacatcher trials. While I maintain the truth value of phenomenological research, wider tests would be a vector to spread the idea further.
Within the application itself, I would like to continue to work on data collection and visualization. Most practically, getting vector patterns working would allow me to test and create finer and more complex visualizations. Implementing different interactions to provide for easier visualization browsing would also boost the visualization experience. I would like to add in other types of metadata mentioned in the classifications exploration such as the volumes of people and water associated with a moment; I would also like to explore other methods of giving a sense of location, such as building height and green space. Being able to ask for the current concordance would also provide another approach for interacting with my data.
I considered keyword search within memories when I first designed the app, and it can be seen in the initial design images. Originally, I included it on the list of features to add, but after considering the experience of privacy inherent in irreducible text, I do not believe it would be a beneficial addition. Rather, the ability to forget, to hide and retrieve moments would further complicate the information available in the app and allow for a different, possibly more challenging experience.
Finally, while the project engaged with nearly the entire system diagrammed in Figure 6.12, the last phase, memorialization went unaddressed. This is part of the reason it was hard to project how the work might be interpreted by future historians, but it also limits the project’s experience today to owning it oneself. Memorialization could focus on self-expression and support for sharing various concordances or topologies; it could also focus on disposal for the object once it is retired. For instance, moment images could be etched into metal and buried along with a broken object as a personal time-capsule.
We can do so much more to unravel and re-situate information. I look forward to it.
Appendix A: Surveys
General Population Makeup
For both surveys I sent the links out via my Twitter account. I wanted to reach further than MAGNET, which is traditionally the testing ground for thesis projects. Because one goal for the project was to have it resonate with art-savvy, technical peers, an audience of people who had chosen to follow me was a very good filter.
Figure A.1 shows the general demographic breakdown of my 972 followers, while A.2 highlights their interests. This data was current October 29, 2016.
The Lovable Objects survey sought to find out what types of objects people loved and how often they carried these objects with them.
Figure A.3 shows the breakdown of responses to the two structured questions and A.4 shows a graph I created from the unstructured responses.
In this survey, respondents read through the mock marketing site at http://lost-time.club and then answered questions about the site and the Oublié/trouvé system as a product. Figure A.5 shows the responses to the basic structured questions.
Appendix B: Other Writings
Self Interview 01
The first self-interview was created as a alternate user research method in April 2016.
Ready to jump into this interview thing?
Yes, let’s go!
Tell me a little about yourself.
I’m a woman. I’m [age redacted]. I’m married; I live in Brooklyn; I really like the subway and books. I have an overactive imagination.
An overactive imagination. What are the signs of that?
[Laughs] Well, you know, I get very absorbed in things. When I was a kid, I didn’t even play with toys really. I just read books and acted out stories in my head. Now that I’m older, it just seems like what’s out there [motions to room] is only the tiniest bit of what … well, what the world is made of. And then the imagination part, too. There was an article recently about a guy who has aphantasia, he can’t see pictures in his head. And then I was reading a thread about it on this website, and people were talking about all their mental imaginations. And even though its been two days, I can’t stop imagining how a plastic milk carton feels, because someone mentioned picking up milk. I don’t even like milk! But the sensation is so real and so pleasant. Even though its not real. I dunno. Does that make sense?
So you find the sensation of imagination pleasant. Does it matter if it is real?
No, I don’t think so. As long as I know the difference between real and imagined and I am not going to end up in a sanitarium. Well, maybe it is more fun to remember things that were real, that happened, than conjure up out of whole cloth. I think that is because it adds another dimension on top of everything, like a search for patterns of truth on top of the pleasure of remember, which is on top of the fun of there being more than just sits before us. It can get a little overwhelming though, dimension on dimension. I’d like to organize it more, but I don’t really have time.
You don’t usually have time. Talk to me about a normal day.
Oh god. I get up at 7, so I have some quiet time before the storm begins. Then lots of school, work, commuting back and forth. I always feel like I am looking for moments that I can take and think a little — but then I always shove my nose in a book, so I don’t know how hard I am really trying to find those moments.
What are you trying to find time to think about?
How everything ties up together, I think. I like to build systems, and so I want to make a system out of the things I read and experience, like a unified philosophy of everything, you know? But, like I know that’s not actually possible; I just like working on it forever. Like the Watts Towers or something. I also feel like I never notice things enough, never look into them deeply enough. I know the very topmost layer, but there is so much more to review and understand and link up. Like there’s this really great book, The Brothers K, by David James Duncan, and I love it because I can read it every year and there is something else buried in there. Or you go back to things you loved when you were in school, and it turns out they hold up or they don’t, or you know so much more now … I just keep wanting to find those things and capture them because they make me happy. Because I think they are beautiful.
You want to capture things. Are you afraid they will get away then?
Yes, definitely, so afraid. Because some day I am going to die. And then all these lovely things and notices will die with me. It would be nice to save them. Although, it is a little indulgent and most people probably won’t care. But maybe there is like one person who will find that garden and be into it. And that would be super cool. But I have to find a way to get it all out first! And then even if it fails, I made a thing that made me happy.
Okay, so now I want to pivot and ask you a little more meta questions. Why are you doing this? It seems kind of bullshit.
Well, I do a lot of design work which is for other people. I am interested in a thing for me. Also, if we think of cybernetics’ second wave concept of an autopoeitic system, I am looking at the level of self-consciousness, which is worth discovering. In the final work, having others test the work may add another layer, but it does not seem appropriate right now.
So are you just not going to talk to anyone?
No! I plan to chat with plenty of people for research, including Katherine Isbister when we are both in SF this summer and Christopher Brosious. But the user-centered design approach just doesn’t feel appropriate for this topic.
Well, look at Proust. He made his best work by gathering up a lifetime of impressions, then holing up in a cork-lined room to put them down in the way that made the most sense to him. Its powerful because it is particular.You just can’t get through anything without mentioning that guy, can you?
[Laughs.] No! Because he is so cool and so good at capturing a world that is super familiar to me, despite the face that in general the Belle Epoque is not that interesting. Because in being particular he hit on something, well not unusual but recognizable. I think that comes from not sanding all the edges off. Broad usability is important when it is like industrial design or something everyone can be expected to use, but we gotta keep making weird stuff, too.
Hm, interesting. Well thanks for letting me interview you today, Sarah.
Yeah, thanks for listening!
These were created to help me think through the goals of the object test and the functionality necessary to get there. By imagining how I wanted the test to go, I gave myself something to live up to.
I am on the G train, going home. Its been a long, gray day. The train isn’t too crowded yet; mostly everyone has a seat. There are some cute kids and across from me is a guy who looks like a teacher, reading a creased novel, and a young dude also gazing into space. We’ve just pulled out of Clinton-Washington. As my eyes, blur a little and I relax into the feeling of Metropolitan camaraderie — just then it happens. Time loosens a little and I can just about make out the traces of other moments hovering about this one. The object is in my hand. I press the button.
I’m walking through the park to brunch. It is a blustery day. I’ve been here before, on this blustery day! Omg, that was so long ago. Back then, I lived in Greenpoint and … . I reach into my bag, find the object, hit the button.
I’m walking through the park to brunch. It is a blustery day. I’ve been here before, on this blustery day! Omg, that was so long ago. Back then, I lived in Greenpoint and … . I reach into my bag — oh fuck I left it at home. I grab my phone and open the app. At least I can save it here.
I’m walking through the park to brunch. It is a blustery day. I’ve been here before, on this blustery day! Omg, that was so long ago. Back then, I lived in Greenpoint and — oh! I wonder if my app has resonances. I reach into my bag and pull out the phone. Clicking the lefthand gem button, I see the moments the app has pulled up to match.
It is Sunday afternoon. This morning I went swimming and now I am laying on the couch reading. It is sunny and drowsy feeling. I have been playing with the object but now it rests on my chest. It buzzes! What could be there?
It is Tuesday afternoon. This morning I got in my 5,000 words and now I am laying on the couch, reading for media law. I’ve been playing with the object, rubbing the surface to help me focus, but I’ve put it down on my chest to concentrate. Fuck, why is it buzzing???
A ferry on Lake Champlain. I’ve never been here before but it’s gorgeous. The leaves are changing and the sky is bright. By bag buzzes; I check. Oh all these other adventures, this same mind.
I am in New York but this is what I call earthquake weather. Save, save, save, save. Phone, object, whatever. California, I’ve never missed you so badly.
Los Angeles. Earthquake weather. Bougainvillea. Nothing.
Happy hour ended one drink ago and I’m waiting for the bus. Sometimes when you are tipsy, you are also nostalgic. I switch from Twitter to the full list of moments and scroll. Moments come back one-by-one. I add a few notes, top up some titles. I wonder what this random time is. Well, I can’t delete it. Maybe I’ll remember later. Scroll, scroll, scroll.
These are the moments I seeded the app with for the test. The set labelled No.1 through No. 6 are from the initial paper moment tests. The rest were recalled at seeding time. The dates are approximate but the weather is historically accurate. It was interesting to note that I wrote these not directly into the database but as literature first.
Old In the 80s
Duvet with flowers, 70s botanical style
Reminds me of blankets that were old when we were kids, especially Heather’s house and her old sagging bed
Near Queens Plaza
/* use date */
Empty and vaguely industrial like Williamsburg used to be. I think BBQs at Mike Legrand’s and the first Bike Kill out by the river. We had to sneak through the fence but there are condos there now.
Henry Street, Brooklyn, Cafe
Reading passages in Swann’s Way where the narrator talks about reading books as a child and the sensation of being swept away on a long summer afternoon. Very soothing on an afternoon where we don’t really live anywhere and I am convinced I am a failure.
Sunset District, SF
/* use date */
Little wood houses near the beach with buoys and stucco and weathered wood. Going to the beach at Huntington.
Santa Ana, CA
/* use date */
Ugly low building fronts remind me of preschool ramps and allium out front.
/* use date */
First crisp fall day. Pavement comes on. My first autumn in Providence. Leaves were yellow!
Sunset District, SF
/* use date */
Can see the Farallones. I think of Catalina, perfect days. Wet sand and yearning. Proust on Balbec.
Sunset District, SF
/* use date */
Foggy mornings. Trying to get lost on the school field at Eader. My eternal sorrow that it was never foggy enough to actually be lost.
Wilson L stop. Brooklyn.
Wilson Ave graveyard. I would like to be buried here.
Lorimer L stop, underground
The lettering on the tiles. All the different reasons I have come to this stop. When I lived with Mark. Eternal transfers. Our wedding day.
The trees are in bloom again. Providence. Kissed spring afternoons. Taking drugs and climbing trees after the last paper is turned in. Gloria in excelsis.
Hoyt Street, Downtown Brooklyn
Raw wet morning, like the blustery day in McCarren Park back when I still lived in Greenpoint. Feeling as though one were on the cusp of something.
Warm feelings of orange and all in it togetherness.
East Village, NYC
Flower smell outside the bodega. Summers of being the only one in town, hot Fridays.
/* use date */
Snow Mountain Ranch, Granby, Colorado
/* use date */
Sledding on a tube. My first winter camp when I was 10. The first snow I saw. Getting the wind knocked out of me after launching off a jump.
Bernal Hill, SF
/* use date */
Dusty paths. Low scrub. The fun of catching lizards and hiking around the park at the North County Y.
Mission District, SF
A whiff of chlorine passing CCSF on Valencia Street. I think of Baruch, leaving Wednesday nights in the dark. The sisterhood at John Jay. A place I was and can never return.
Palladium Pool, Manhattan
My God Is a Chlorine God
Sundays at the pool and the feeling of holiness about it. Everyone is gone and it is time for quiet magic. Doing things I never thought I could. Long pull sets or kick with fins. Sliding through the water.
Close to home
/* use date */
Lights coming up. Cold air. Christmas decorations. Carrying the tree home and all the other narcissistic celebrations of a December.
Emeryville Marina, East Bay, Calif.
/* use date */
I see the boats arrayed at Emeryville and think of swimming out to the last row of yachts and back when I was 9, 10 maybe. We’d stop by and talk to people sitting on their boats. Just kids. I never wore a bikini because I had to swim. I had a one piece with a hole in the middle cut out instead. We were undersupervised, certainly.
Self Interview 02
The second self-interview was created as a spur to reflection in November 2016.
Hey there, Sarah.
Here we are back at this self-interview thing again.
Yeah. It’s pretty silly but also pretty fun. Also a good way to work on making talking to yourself cool again.
(Laughs.) It sure is. Okay, let’s get started. It’s been a little since we last talked. In that time, you’ve developed this project and tested it out for the past five weeks or so now. How did the test go? Was carrying the object around weird?
Not really. Maybe if I were someone else and hadn’t been talking about this project forever my friends would have thought it was weird to pull out this mishmash of plaster and wires while I was talking. But I’ve put a lot of effort into establishing I am a weirdo, so I don’t think anyone was surprised.
What about strangers?
Well, they didn’t seem to notice much. To be fair, when I was out — on the train for instance — I would just hold the plaster and leave the rest inside my bag, so I don’t think it was super obvious. Partly that was because I didn’t want to be too obvious but mostly it was because the embedded LED is so bright. I know I could have changed the code to make it dimmer, but I’m lazy.
Speaking of laziness, did that effect the project at all?
Mmm … yea, I think so. There were definitely times I didn’t bring the object with me when I went out. Or when I left it in my bag. But I think that is part of how it is supposed to work. It’s not meant to be this constant thing. Otherwise I could just use Reporter and make my own visualizations with the exported data.
Although Reporter is different. It doesn’t really talk back to you.
What did you think of Oublié/trouvé’s notifications?
They were pretty chill. Well, at first they were annoying and then they tapered off and I missed them a little. I am not sure how important the notification is. Well, no, maybe I am wrong. There were times that it bugged me but then I was glad it did. So I guess it is important to have that uncertainty, to wonder when it is going to come alive. That way it is a little friend. If you knew it was never going to talk to you, it would be just a really weird knob for your phone, wouldn’t it?
That’s one way to think about it. Overall, did you like the test? Do you plan to keep using Oublié/trouvé?
I do think I’ll keep using it. I want to see what kinds of things I can discover over a longer time period. I mean, the last book of A la recherche happens like twenty years after the rest of the action. And the whole Swann in Love flashback is twenty years before the narrator is even born. The insight there takes a really long time. If I could use this my whole life it would be amazing, but maybe even just a few years.
That good, huh?
Well, not good, but not like bad either? It is hard to tell! I like it enough to want to see where it goes and not throw it across the room. And the pictures are cool. I want to see how many different kinds there are to be made.
Do you think this experiment has changed you?
Oh, hm, that’s hard to say. Like, I already thought a lot about the past and nostalgia and how you become who you are because of what happened to you before. But I used to worry a lot, like go over what sort of things I would have done in the past if I knew what I know now and start wondering about what I was missing today. I like this project because it lets me make nice things out of the nostalgia impulse, instead of just feeling like it was a reminder of what I can’t live up to.
And it does make me pay attention better to what things are like. Where I am, the weather, all that sort of stuff. It makes me wish I could sometimes open the app and just see what the current concordance is. It would be cool to add that.
Noted. One more question: How do you think the project went as a whole? Did it have an effect on your life beyond making you very busy?
Oh, I dunno. I think I am just very tired. And I wish it were ten thousand times better than it is, even if it is ok. More beautiful, I think.
Appendix C: Resources
Westley Hennigh-Palermo provided endless debugging help, code architecture advice, and fabrication help.
Materials and Vendors
The icons used throughout are from Streamline Icons.
All milling was done on an Othermill. Silicone and some wax blocks and silicone were sourced from Othermill; the remainder of the wax is Ferriss machining wax sourced from Rio Grande. PCB blanks are from Tinkersphere.
All plaster is Amaco casting compound. Resin is Pebeo Gede crystal resin. Concrete was Quikcrete mix.
The microcontroller is the Light Blue Bean from Punchthrough. It runs on 3.6V rechargeable LIR2032 batteries sourced from Battery Junction. The button and motor components are PCB mount momentary two-pin SPST tactile switches (6 x 3 x 4.3mm) and 3V DC coin vibration motors (10mm x 3.6mm) from Uxcell.
The iPhone app was created for iOS 9, using React Native 0.37. The design was optimized for the iPhone SE. Weather information is retrieved using the Open Weather Map API.
The data model for moments is shown in figure C.1; for concordances, C.2. A diagram of the code system is shown in C.3.
Ada, Michael. Dominance by Design. Cambridge: Harvard University Press, 2009. Print.
Barrett, Estelle, and Barbara Bolt, eds. Practice as research: Approaches to creative arts enquiry. Ib Tauris, 2014.
Bechtel, Martha. “Perfecting the Plaster of Paris and Wood Glue Mix.” Martha.net 18 December 2015, www.martha.net/2015/12/perfecting-the-plaster-of-paris-and-wood-glue-mix.
Bell, Gordon, and Jim Gemmell. “A digital life.” Scientific American 296.3 (2007): 58–65.
Benjamin, Walter. Illuminations. New York: Pantheon Books, 1968. Print.
Bertran, Ishac. “Memory Device.” Ishac Bertan. 2014, ishback.com/memory/index.html.
Blauvelt, Andrew, ed. Hippie Modernism: The Struggle for Utopia. Minneapolis: Walker Art Center, 2015. Print.
Boucher, Andy, et al., eds. Datacatcher. London: Goldsmiths, University of London/Interaction Research Studio, 2015. Print.
Brand, Stewart. “Spacewar.” Rolling Stone: 7 December 1972. Web.
Castillo, Greg. “Counterculture Terrior: California’s Hippie Enterprise Zone.” Blauvelt 87–101.
Charpy, Manuel. “How Things Shape Us.” Gerritsen & Riello 199–221.
Coltman, Viccy. “Material Culture and the of Art(efacts).” Gerritsen & Riello 17–31.
Dubberly, Hugh and Paul Pangaro. “How Cybernetics Connects Computing, Counterculture, and Design.” Blauvelt: 126–141.
Dunne, Anthony, and Fiona Raby. Speculative Everything: Design, Fiction, and Social Dreaming. Cambridge: The MIT Pess, 2013. Print.
Ellis, Carolyn. The Ethnographic I. Walnut Creek: AltaMira Press, 2004. Print.
Farr, Ian, ed. Memory. Cambridge: MIT Press, 2012. Print.
Fer, Briony. “The Infinite Line.” Farr 71–75.
Gaimster, David. “Material Culture, Archaeology, and Defining Modernity: Case Studies in Ceramic Research.” Gerritsen & Riello 59–66.
Gaver, William W., Jacob Beaver, and Steve Benford. “Ambiguity as a resource for design.” Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2003. Print.
Gaver, Bill, Tony Dunne, and Elena Pacenti. “Design: cultural probes.” interactions 6.1 (1999): 21–29.
Gemmell, Jim, Aleks Aris, and Roger Lueder. “Telling Stories with Mylifebits.” ICME. Vol. 8. No. 2005. 2005.
Gershenfeld, Neil. Fab. New York: Basic Books, 2008. Print.
Gerritsen, Anne and Giorgio Riello, eds. Writing Material Culture History. New York: Bloomsbury Academic Press, 2015. Print.
Gertner, Jon. The Idea Factory: Bell Labs and the Great Age of American Innovation. New York: Penguin Books, 2013.
Gleick, James. The Information: A History, A Theory, A Flood. New York: Vintage Books, 2012. Print.
Glen, Joshua and Carol Hayes. Taking Things Seriously: 75 Objects With Unexpected Significance. New York: Princeton Architectural Press, 2007. Print.
Goldsmiths Interaction Research Studio. www.gold.ac.uk/interaction. Accessed 27 November 2016.
Halbwachs, Maurice. “Space and the Collective Memory.” Farr 47–49.
Hayles, N. Katherine. How We Became Posthuman. Chicago: University of Chicago Press, 1999. Print.
Herz, Rachel. “A Naturalistic Analysis of Autobiographical Memories Triggered by Olfactory Visual and Auditory Stimuli.” Chem. Senses 29: 217–224, 2004. Web.
Higgins, Hannah. Fluxus Experience. Berkeley: University of California Press, 2002. Print.
Hodgson, Francis. “Idris Khan.” Farr 94–96.
Höök, Kristina, Anna Ståhl, Petra Sundström, Jarmo Laaksolahti “Interactional Empowerment.” CHI 2008 Proceedings: Dignity in Design: 647–656, 2008. Web.
Hendren, Sara. “A ‘Technology of Distance’.” Ablerism: May 19, 2011. Web.
Isbister, Katherine, Kia Höök, Jarmo Laaksolahti, Michael Sharp. “The sensual evaluation instrument: Developing a trans-cultural self-report measure of affect,” International Journal of Human-Computer Studies 65: 315–328, 2007. Web.
Kozel, Susan. Closer: Performance, Technologies, Phenomenology. MIT Press Scholarship Online: August 2013. DOI: 10.7551/mitpress/9780262113106.001.0001
Kracauer, Siegfried. “Memory Images.” Farr 45–47.
Laaksolahti,Jarmo, Katherine Isbister and Kristina Höök. “Using the Sensual Evaluation Instrument.” Digital Creativity 20:3, 165–175, 2009. Web.
Ladly, Martha. “Being there: Heidegger and the phenomenon of presence in telematic performance.” International Journal of Performance Arts and Digital Media 3.2–3 (2007): 139–150. Print.
Lehrer, Jonah. Proust Was a Neuroscientist. New York: Mariner Books, 2008. Print.
Malpass, Matt. Contextualising Critical Design: Towards a Taxonomy of Critical Practice in Product Design. PhD thesis, Nottingham Trent University.
Maturana, Humberto R., and Francisco J. Varela. Autopoiesis and cognition: The realization of the living. Vol. 42. Springer Science & Business Media, 1991.
Meireles, Cildo and Charles Merewether. “Memory of the Senses.” Grand Street 64: 216–223, Spring 1998. Web.
Morris, Robert. “In Conversation with W.J.T. Morris.” Farr 92–94.
Murray, Janet H. “Inventing the Medium.” The New Media Reader. Noah Wardrip-Fruin and Nick Montfort, eds. Cambrige: MIT Press, 2003. 3–11. Print.
Nora, Pierre. “Realms of Memory.” Farr 61–66.
Petersen, Anne Helen, “Big Mother Is Watching You,” Buzzfeed: January 1, 2015. Web.
Proust, Marcel. Swann’s Way. Trans. Lydia Davis. New York: Viking, 2003. Print.
—————. In the Shadow of Young Girls in Flower. Trans. James Grieve. New York: Penguin, 2004. Print.
—————. The Guermantes Way. Trans. Mark Treharne. New York: Penguin, 2005.
Revell, Tobias. “Critical Design / Design Fiction lecture finally written up. (loooooong)” Occasional blog of Tobias Revell. December 2013, blog.tobiasrevell.com/2013/12/critical-design-design-fiction-lecture.html.
Sadler, Simon. “Mandalas or Raised Fists? Hippie Holism, Panther Totality, and Another Modernism.” Blauvelt 114–125.
Scott, Felicity D. “Networks and Apparatuses circa 1971: Or, Hippies Meet Computers. Blauvelt 102–113.
Shattuck, Roger. Proust’s Binoculars: A Study of Memory, Time, and Recognition in A la Recherche du temps perdu. Princeton, New Jersey: Princeton University Press, 1983. Print.
Sheringham, Michael. “Perec: Uncovering the Infra-Ordinary.” Farr 86–92.
Sicchio, Kate. “Immanent Researcher.” Unpublished chapter. September 2016. PDF.
Simon, Stephanie. “Biosensors to monitor students’ attentiveness.” Chicago Tribune (2012).
Styles, John. “Objects of Emotion: The London Foundling Hospital Tokens: 1741–60.” Gerritsen & Riello 165–171.
Sullivan, Graeme. Art Practice As Research: Inquiry in Visual Arts, 2nd ed. Los Angeles: Sage Publications, 2010. Print.
—————. “Research acts in art practice.” Studies in Art Education 48.1 (2006): 19–35.
Turkle, Sherry. Inner History of Devices. Cambridge: MIT Press, 2008. Print.
—————. Evocative Objects. Cambridge: MIT Press, 2007. Print.
Turner, Fred. From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism. Chicago: The University of Chicago Press, 2006. Print.
Ullman, Ellen. Close to the Machine. New York: Picador, 2012. Print.
Voros, Joseph. “A primer on futures studies, foresight and the use of scenarios.” Prospect: The Foresight Bulletin 6.1 (2001). Print.
de Waal, Edmund. The Hare with Amber Eyes. New York: Picador, 2010. Print.
Walter-Hermann, Julia and Corinne Büching. Fablab: Of Makers, Machines, and Inventors. Bielefed: Transcript, 2013. Print.
Weiser, Mark. “The Computer for the 21st Century.” Draft of article for Scientific American. Web. Accessed October 2, 2016.