Intentional and Responsible: Incorporating Social Values Into the Design Process

Unintended consequences of technology are ubiquitous and often negative. Certain communities, the Amish for example, have minimized these consequences by understanding and agreeing upon the social values they seek to preserve, and by making technology choices based on these values. Although similar strategies are not widely implemented in Western society, technology developers should be intentional and responsible about the impact of their work. Existing design strategies that deal with values (e.g., contextual design) focus on adapting to business culture rather than preserving social values. This article proposes a new process model that focuses on articulating social values during the design phase as a method by which to evaluate the attributes of the system being developed, and discusses a planned experimental design intended to measure the effectiveness of this approach. By considering the relationship between design decisions and social values, this model may result in a design strategy that is both intentional and responsible.


INTRoDUCTIoN
In contrast with Western society, the Amish exercise high control over their communities' pace and direction of change (Wetmore, 2007).Amish elders carefully consider the potential impact of new technology systems on their way of life.They are particularly concerned with how these systems could threaten the social values they seek to protect.Based on their evaluation, the elders decide whether to accept, reject, or modify the technology systems with which they are presented (Kraybill et al., 2013).
The values-based decisions reached by elders in an Amish community are declared in a set of rules and guidelines by which the community's citizens live (Kraybill et al., 2013).Using this process, the Amish have minimized the negative unintended consequences of technology on their society.
A technology system can succeed by achieving its stated goals but still produce negative and unintended consequences.Negative unintended consequences of technology systems are undesired outcomes separate from the systems' stated goals.This type of consequence suggests a lack of deliberateness by the technology system's designers (Ash et al., 2007).
Negative unintended consequences of technology systems are extensive and widespread (Alberts, 1996).In extreme cases, these consequences have included the loss of human life, as in the case of the 2018 and early 2019 Boeing 737 MAX crashes in Indonesia and Ethiopia, when the flight control software system to prevent stalls malfunctioned (Herkert et al., 2020).Researchers have identified negative unintended consequences in medical informatics (Ash et al., 2007) and the use of instant messaging in business settings (Rennecker & Godwin, 2003).There are reports of negative unintended consequences of technology systems in other fields, including cyber-crime (Reyns et al., 2013), military and defense (Lin et al., 2012), computer-mediated communication (Debatin et al., 2009), and forestry (Thomson & Schmoldt, 2001).An extensive interdisciplinary list of incidents demonstrating the risks of computer and information technology to the public has been compiled by P.G.Neumann (2015).Several categories of the risks included in Neumann's list fall under the umbrella of "negative unintended consequences" (Neumann, 2015).
Many negative unintended consequences of technology systems are subtle and do not receive the same prioritization as those that implicate survival (Jonas, 1979).However, even subtle unintended consequences can contribute to significant societal changes in the long term.For example, a subtle negative unintended consequence of smartphones is the way these devices entice users with virtual activities that replace interpersonal interaction.A familiar observation is the phenomenon of families and friends out together in public where each person is transfixed on their smartphones (Al-Saggaf & O'Donnell, 2019).

DeFINITIoNS
Defining technology as a general concept includes nearly all human endeavors (Beniger, 1986), but MacKenzie and Wajcman (1985) offer further clarification.They identify three layers of meaning in "technology": • physical objects (such as the personal computer); • activities and processes (such as computer programming); and • what people know and do (such as navigating a website) (MacKenzie & Wajcman, 1985).This paper's use of the terms technology and system emphasizes the second and third layers of meaning: activities, processes, and what people know and do.The limited definition facilitates a focus on the impact of a given technology system upon the values of the society that adopts it and on technology systems developed by organizations using design strategies.
Values are trans-situational goals or principles that motivate the behavior and beliefs of individuals and groups (Schwartz et al., 2012).In this paper, the term social values denotes the motivating principles of a society to distinguish it from the motivating principles of other groups such as businesses or political parties.
Technology systems designers who strive to be intentional and responsible have an implicit goal similar to that of the Amish-minimization of a technology's negative unintended consequences on the social values of their society-but may lack a corresponding decisionmaking process or design strategy.
A design strategy is how an organization manages and utilizes its design resources (Garcia, 2012).Included in these resources are the designers themselves: the humans within an organization responsible for making design decisions during the development of a system.Sometimes, a design strategy is emulated by multiple organizations, as in the example of agile software development (State of Agile Report, 2022).

DeSIGN STRATeGIeS
There is no exhaustive list of design strategies, which conventionally have focused only on the technical qualities of the system being developed (Wood-Harper et al., 1996).User-centered design strategies depart from this tradition by focusing on the system's users.One user-centered design strategy involving some consideration of values is Contextual Design.Contextual Design attempts to tailor a system to the values of its users (Beyer & Holtzblatt, 1998).However, Contextual Design does not explicitly consider the system's impact on social values.
Another design strategy, Value Sensitive Design (VSD), uses conceptual, empirical, and technical investigations to account for social values in developing a technology system.These investigations are performed iteratively across the life cycle of the system being developed.The conceptual investigations determine which and whose values could be implicated by interactions with the technology system.Competing values of primary and secondary users of the system are considered.The empirical investigations observe how the conceptual investigations play out in practice, emphasizing humans and social structures.The technical investigations prescribe design decisions that account for specific values.Alternatively, technical investigations can take a form similar to the empirical ones but emphasizing technology.VSD accounts for some social values, but the model focuses primarily on values with moral implications (Friedman et al., 2008).
Technology-related organizations such as ACM, CCSR, and IEEE have published codes of ethics for developers that might capture some consideration of a system's impact on social values.Unfortunately, the codes are not enforced and hold individuals (typically, "members" or "professionals") rather than organizations responsible for outcomes (Thomson & Schmoldt, 2001) and, therefore, lack the motivational aspects necessary for successful internalization (Stead et al., 1990).
Though technology system development is centralized-it occurs via design strategies within organizations-technology's impact on society is dispersed (Collins et al., 1994) and often transformative through second or higher-order effects.Technology systems designers might not understand the ethical implications of the systems they develop and believe their work to be ethically neutral when, in fact, each design decision is an ethical one, and designers, in effect, act as moral agents (Wood-Harper et al., 1996).There is a need for a design strategy that considers the negative unintended consequences of technology systems on social values and treats protecting those values as its central goal.

PRoPoSeD PRoCeSS
To address the negative unintended consequences of technology systems on social values, this paper proposes a process model based on the vetting method of the Amish and a method of risk analysis: a design strategy for technology systems designers to evaluate the attributes of the system they are developing in terms of each attribute's protection of or threat to social values.This strategy is termed the Social Values Protection (SVPro) process model.
SVPro has three essential steps: 1.The designers work with their organization to determine which social values they will attempt to protect through their design decisions.These values are articulated in what this paper terms prime and conditional values, together termed value bundles.Prime and conditional values are specified to avoid protecting one social value at the expense of other equally or more important values.Prime and conditional values are discussed further in the next section.2. The designers evaluate the planned attributes of the technology system in development based on whether each attribute protects or threatens each value bundle.An attribute is a component, feature, subsystem, or functionality that aims to benefit the user.One example of an attribute is the ballpoint on an ink pen: it provides the benefit of a continuous line of consistent width to the person using the pen.3. The designers decide whether to keep, reject, or modify the planned attribute based on the threat to value bundles.
An important fourth step, outside the scope of this paper, is for the designers to invite other project stakeholders and the public in general into their conversation so that the broadest possible scope of insights and concerns may be captured and a vocabulary and conceptual framework of potential risks can be built up.

Choosing Prime Values
First, designers work with their organizations to choose the social values that they intend to protect from a list of 19 values common to every society, as cataloged by Schwartz et al. (2012) and shown in Table 1.The list of 19 social values is established and universal, and designers, with the participation of their organizations, should decide which values best reflect the balance of values in their society and choose the ones that they consider most important.This way, designers and their organizations invest in their values-based design decisions and possible outcomes.One benefit of Schwartz et al.'s values is that they are defined in terms of motivational goals, implying a behavior to support each value.Unintended consequences are often the result of behavior, whether it is a behavior of the technology system or, more often, a behavior of its users.These values are related to each other, forming a continuum.So long as their continuity is maintained, it is possible to choose 10, four, or two social values for particular research purposes (Schwartz et al., 2012).Designers may, therefore, limit the number of chosen values to one of these lesser quantities based on project needs.Each chosen value is termed a prime value.

Conditional Values and Creating Value Bundles
A team of technology system designers is unlikely to share a rich and precise understanding of the social values they hold in common.A simple list of values might need to be revised to evaluate the planned attributes of the technology system they are developing.A design decision intended to protect a single value could result in other essential values being unintentionally threatened.Therefore, articulating a balance of values is necessary to provide a more robust shared understanding from which technology system designers can evaluate a system's attributes.
Two or more social values might be equally important, but a technology system attribute that protects one value might threaten the other.Designers brainstorm scenarios in which a technology system protects a prime value to uncover these potential conflicts.For example, consider a dronemounted camera.Designers might envision a scenario in which the social value of self-direction-action (SDA)-the freedom to determine one's actions-is protected via the drone-mounted camera, which affords the user a significant amount of freedom to determine heights and angles from which to take aerial photographs.Then, the scenario is examined based on whether other values on the continuum are threatened with an unacceptable result.Since the drone-mounted camera also grants the user unauthorized access to private areas, safety and stability in the broader society (SES) is threatened.A value deemed threatened by the protection of the prime value is considered a conditional value of the prime value, and these values are grouped in what is termed a value bundle.In this model, a prime value may have more than one conditional value.Within a value bundle, prime and conditional values conform to the following formula: Protection of the prime value should not threaten the conditional value(s).
Each prime value chosen by the designers and their organization should be weighed against conditional values to create a list of value bundles.Value bundles may be prioritized by perceived importance to provide a starting point for Step Two.
Step Two: evaluation In this step, designers evaluate each planned attribute of the technology system in development for its potential threats to value bundles.The model uses a modified failure mode and effects analysis (FMEA), a risk analysis method, to complete this step.
FMEA examines the plausible ways a system's components' malfunctions (or failure modes) could result in an undesired outcome.For example, the desired outcome of an ink pen is to allow the user to write.Failure modes for an ink pen's components include the ink cartridge becoming empty, the ballpoint becoming dry or clogged, or the clicker breaking, preventing the user from writing-an undesired outcome.Each failure mode is then categorized according to its likelihood and severity.FMEA is comprised of the following stages: 1. Determine the separate components of the system and their sequential relationship.2. Identify each component's failure modes.3. Estimate the likelihood and severity of each failure mode and ease of detection or mitigation.
For SVPro, where planned system attributes are evaluated according to their threat to value bundles, FMEA is modified so that each plausible interaction between a system attribute and a human is treated as a component.Any threat to value bundles through an interaction's first-order or (1+n)th-order effects is treated as a failure mode.Thus modified, the method is comprised of the following stages: 1. Determine the interactions affected by or necessary to use each planned system attribute and how they are sequenced.(See Figure 2) 2. Estimate each interaction's plausible first-order and (1+n)th-order effects as fully as possible.
(See Figure 2) 3. Determine whether each effect protects or threatens each value bundle.
During Step 2, designers may access existing literature about the effects of technology systems on aspects of society that might map to prime or conditional values.For instance, Turkle (2012) and Lucey and Noctor (2022) have written about the implications of information technology on psychological well-being and interpersonal relationships (which might map to the Universalism or Benevolence values).Abby Smith Rumsey ( 2016) has written about the implications of data storage on human memory and culture (which might map to the Tradition or Self-direction values).In Stage 1 of the modified FMEA, designers might benefit from prioritizing the planned system attributes so that attributes most likely to be implemented are addressed first.In Stage 2, designers might consider humans' tendency to form mental and behavioral habits with repeated actions and what might be lost or given up when adopting a new way of doing things.In Stage 3, the priority of planned system attributes and value bundles may help determine the most important design decisions in minimizing unintended negative consequences.

Step 3: Design Decisions
The Amish respond to new technology systems in five ways: acceptance, rejection, acceptance only after requisite modification, invention of new systems to provide similar benefits but protect Amish social values, or restriction on access or ownership (Kraybill et al., 2012).Because the latter three responses imply modification to derive benefits from some part of the system while rejecting others, the five Amish responses to technology systems can be reduced to three essential types for the SVPro model: acceptance, rejection, or modification.These three types form a template for designers to make more intentional and responsible decisions about each planned system attribute.

eXPeRIMeNTAL DeSIGN AND ReSULTS
Because the Social Value Protection process model aims to protect social values, verifying SVPro's effectiveness necessitates studies focusing on different parts of the process and its underlying assumptions.We conducted a pilot study to explore how designers' consideration of value bundles affected their design decisions.

Location and Participants
Our goal was to gather data from at least 120 people.We guest-hosted multiple user-experience professional meetup group sessions in Ohio and one in Michigan.These meetup groups were chosen because their attendees typically work in technology in some capacity, are involved in design decisions, and are familiar with software development practices.From these groups, we had 40 participants.We organized our sessions as workshops, each one lasting approximately 90 minutes.Our participants were told that they were part of a study about how technology development teams make design decisions; participants were kept naive to both SVPro and VSD.Participants were randomly assigned to teams of four.Each team was randomly assigned to one of two instruction types, A or B.

Scenario
All teams were presented with a scenario in which they were designing a smart refrigerator for a cutting-edge home appliance company.A list of smart refrigerator attributes was given to participants (Table 2).The refrigerator would be built according to the team's design decisions and released to a general mass-market audience.

Method
After being presented with the scenario, all teams were provided with materials and instructions.The materials were: • a description of the smart refrigerator they will be designing-a new type of refrigerator meant to revolutionize the way perishable food is stored and accessed; • high-level information about the "client" organization and its industry-a bold start-up company intent on disrupting decades-old conventions of domestic cold food storage and retrieval; and • a randomized list of planned attributes of the technology system.
An external dispenser provides through-the-door access to any combination of eight beverage choices, plus either crushed or cubed ice, based on presets programmed by the user on a smartphone and recognizable by wireless close-proximity signal 6 Sensors, an online service, and a smartphone app alert the user when the refrigerator door is left ajar for over 60 seconds 7 A cool-down compartment is used for isolating hot items so that they do not warm up the interior of the refrigerator as they cool 8 A color-adjustable LED interior lighting system lasts the lifetime of the refrigerator 9 An on-board backup battery keeps the cooling unit operational for up to 24 hours in the event of interior temperature rising above a threshold when power is off 10 A distinctive spherical shape gives the refrigerator a unique look In addition to the small sample size, we are reevaluating the pilot study methodology to explore different team sizes (other than four to a team) and specific constructs and methods that might lead to clearer outcomes.

CoNCLUSIoN
Technology systems touch nearly every corner of human activity, and negative unintended consequences follow their adoption.Many of these consequences are subtle; they threaten the quality rather than the existence of life and, therefore, do not attract the attention of the designers, their organizations, or the public.
Having maintained their way of life for over a century, the Amish demonstrate that it is possible to mitigate this situation with a technology decision-making process foregrounding the protection of social values.This paper has borrowed from the Amish formula and a risk analysis method to develop a process model to help systems designers be more intentional and responsible.
In the Social Value Protection process model, designers and their organizations choose a set of social values defined in terms of motivational goals.This set of values is articulated based on how the protection of a prime value might threaten conditional values in various scenarios.Prime and conditional values are considered together as value bundles.The model then guides designers in evaluating planned technology system attributes according to whether they protect or threaten value bundles.If the model is a good fit, designers can make more informed decisions about keeping, omitting, or modifying a planned attribute.
Because this is an early-stage process model, more research is planned to determine its effectiveness, practicability, and limitations.In particular, follow-up research initiatives may explore alternative practical methodologies for the subprocesses comprising the Social Value Protection process model's three steps, and additional input and feedback may provide rigor to its theoretical foundations.Research may also identify where to assign responsibility for design decisions informed by SVPro since the model incorporates choices made at the organizational and designer levels and potentially at the societal level by including Step 4 (discussed in Proposed Processes).

Instructions for Team B
You are designing a "smart refrigerator" for a cutting-edge home appliance company.The refrigerator will be built according to the design decisions of the team and released to a general mass-market audience.
The product is a new type of refrigerator meant to revolutionize the way perishable food is stored and accessed.
Your client is a bold start-up company intent on disrupting decades-old conventions of domestic cold food storage and retrieval.
Below is a list of proposed features for the product.Give each feature a grade of pass (P) or fail (F).
Additional comments or caveats about your design decisions (feel free to use the back of this paper if necessary).

Figure 1 .Figure 2 .
Figure 1.Stages one and two of the modified FMEA used in step two

Robert
Keefer is computer scientist and human factors engineer with a Ph.D. in Computer Science from Wright State University in Dayton, OH.His research interests include natural language process, document image processing, and user experience design.His work has been published in peer reviewed journals, conference proceedings, and online magazines.Nadav Zohar has been working as a UX Researcher since 2013 and studying technology ethics since 2014, with a particular focus on organizational technology adoption and change.Lisa Douglas is a research psychologist with a Ph.D. in Human Factors Psychology from Wright State University in Dayton, OH.Her areas of expertise include human factors, cognitive systems engineering, and research and user experience in software development.She has created several custom metrics, including the "Data Confidence Score," a metric that improves corporate stakeholders' ability to leverage data from user research.Her work has been published in the International Journal of Human-Computer Interaction, UX Magazine, and numerous conference proceedings.

Table 1 . Schwartz et al.'s 19 universal values Value Conceptual definitions in terms of motivational goal
Note.The values highlighted in yellow are referenced in Conditional Values and Creating Value Bundles.