Social work thinking for UX and AI design

• Read original article here

What if, instead of solely looking for the best AI talent, tech companies and startups created a Chief Social Work Officer? This is the question Greg Epstein posed to me during our conversation about why AI needs more social workers [1]. My response: The social work difference is that from inception, when considering why or if AI should be created or integrated in society, we would ask: Who should be in the room making that decision? Social work thinking underscores the importance of anticipating how technological solutions operate and activate in diverse communities. Social workers have long been the trained "blind spotters" that UX and AI design needs. In fact, social work thinking might have been useful in Google's latest challenges with facial-recognition systems.

Recently, Google contractors attempted to create a racially diverse dataset [2] for their facial-recognition system by hiring temps to collect facial scans; this involved paying individuals experiencing homelessness. The desire to reflect diversity in UX is just fine, but diversifying the corpus for algorithmic accuracy without informed consent, protection of confidentiality, or a concern for the general well-being of the participants should raise concerns about how "AI for good" is pursued. Here, instead of leveraging AI to help those experiencing homelessness access supports and services [3], efforts to diversify training data were actually used for capitalistic gain.

Over the past few years, it has become popular to pursue AI for good or human-centered AI, ideas that suggest solving longstanding social problems, with diverse community members, that were once too hard to approach with traditional research methods. Academic institutions and technology companies have invested vast resources in using AI to deepen our understanding of how society works, with goals like improving well-being and creating more technologies that work for all. If you take a look at the websites of such organizations, you'll probably have a hard time finding individuals representing diverse and complicated lived experiences. According to AINOW [4], 18 percent of authors at leading AI conferences are women and more than 80 percent of AI professors are men. The numbers are more dire with respect to race. At Google, only 2.5 percent of the workforce is Black, while Facebook and Microsoft both hold steady at 4 percent. We should also be concerned about the lack of voices representing Black, Latinx, or Native American people within the U.S. context, from all socioeconomic classes and individuals with less than a college degree or past justice involvement, or practitioners and first-line workers (e.g., social workers, counselors, nurses, outreach workers). Taking a step back, how might AI impact society in more positive ways if these communities were consulted often, paid, and recognized as integral to the development and integration of these technologies that aim to improve life for all?

I am an African American, male-identified, gay associate professor at the Columbia School of Social Work and am the founding director of the SAFElab (https://safelab.socialwork.columbia.edu/). The SAFElab is a research group composed of social workers, community members, and data scientists that aims to use qualitative insights, community participatory methodologies, and data science to understand and intervene in gun violence.

Over the past five years, I have engaged in a unique interdisciplinary collaboration between natural language processing (NLP) and social work researchers to study the role of social media in gun violence, leading to the development of a system that predicts aggression and loss [5,6,7]. Our approach is distinct: How we define the problem, the identification of the training dataset, and the creation of labels is led by the social work team and community experts who live in communities with high rates of gun violence. We use an inductive, close-read qualitative strategy to determine what social media expressions relate to expressions of loss following a shooting in Chicago. In our case, we hire and train social work annotators to carry out this task. We selected annotators who are current students in a Master of Social Work program. Annotators selected have work experience in youth development, criminal justice, and social work practice with youth of color.

The annotation training includes: 1) a general overview of the domain informed by our domain experts, outlining their role as annotators (e.g., the tasks, purposes, and goals of the analyses), 2) an in-depth annotation-system tutorial, 3) a weeklong deep immersion in the specific social media domain, and 4) annotation practice and feedback. The qualitative analysis influenced the development of our computational approach for analyzing social media content and automatically identifying relevant posts. We then developed a set of algorithmic systems that automatically identify text and images associated with aggression and grief [8] with a relatively high level of accuracy (Table 1).

The development of our algorithmic systems occurred through close interaction between the computer scientists and social work researchers and was informed by experience with the real-world tensions inherent in these situations. In our collaborations with computer scientists, the social work team underscores the importance of vacillating between the real-world consequences of AI, concerns that keep me up at night. On the one hand, in our qualitative work, we hear from Black families with children who are vulnerable to gun violence and simply want their children to be safe and protected. They are willing to listen and contend with possible computational tools that may support those efforts. I also hear from youth in Brownsville, New York, who are OK with social media surveillance if it keeps sexual predators away. On the other hand, as a social work researcher, I know that these same tools can be—and are—used against children in another form of violence: state violence, enacted by surveilling the hyperlocal language and pictures on social media that can be used as evidence or negative character testimony within the criminal justice system [9]. As such, I haven't identified an ethical AI framework that wrestles with the complexities and realities of safety and security within an inherently unequal society, and as a social worker, that means we cannot deploy or integrate AI technologies in community settings without a more reflective framework that involves privileging community input. As such, we constantly wrestle with a set of questions that drive our analysis and perspective on deploying AI technologies: Can AI keep kids safe, or will it send them to prison? Can both be true at once?

While our ultimate goal for the AI systems mentioned is to develop computational tools to aid social workers and violence outreach workers in their intervention and prevention efforts, I am constantly worried about whether this is the most ethical thing to do. I'm also not sure this is the right approach given the problem we're trying to solve. On one end, AI technologies have advanced in ways that allow more direct application, or in this case, a tool for potential gun violence prevention. But without reflection on how data generated by the technologies will be used and for whom, the negative consequences might outweigh the possibilities. But I believe there is hope.

One strategy is to radically shift who is deemed a domain expert. While AI offers an unprecedented opportunity to analyze billions of data points that reflect the complexity of the human experience—once too difficult to reach through traditional methods—society's most marginalized individuals may not be reflected in how we conceptualize, develop, analyze, and integrate AI technologies in society. In the SAFElab, we recognize our lack of knowledge regarding the hyperlocal language and broader context surrounding our analysis of tweets from Black youth living in communities with high rates of gun violence in Chicago. In this context, misinterpreting a tweet or overidentifying innocuous language as threatening can have detrimental consequences for our population of young Black Twitter users. Our solution? Develop an ethical annotation process and create research-assistant positions that would enable us to hire youth who live in the same communities from which our Twitter corpus was derived.

Working with Black youth as domain experts [10] created a more robust understanding of context, including hyperlocal language; their backstories helped us understand the context surrounding a post and the triggering events that shifted posting behavior. These insights changed how our data-science colleagues approached the development of algorithms that could affect the types of services and experiences to which the Twitter users in our study have access (e.g., Table 1).

This was not to say that the process was easy. Our domain experts have lives outside of academic research, and when life happened, they attended to personal needs. We should have been more invested in developing trust and rapport with the domain experts, as opposed to being solely focused on products and outcomes. We overcame this challenge through a mentoring system. Our community partners identified mentors who could participate in a broader support system, aiming to work with our domain experts under conditions that worked better for their lives. For example, collaborations with domain experts produced a new methodology for centering context and community voices in the preprocessing of training data. Together, we created the Contextual Analysis of Social Media (CASM) approach. CASM is a contextually driven methodological process used to annotate and qualitatively analyze social media posts for the training of an NLP computational system. CASM involves several steps. Phase 1 involves getting a baseline interpretation of a social media post from an annotator, identifying various forms and types of context (Figure 1), interpretation and contextual analysis assessment, and labeling of the post.

In Phase 2, community experts and social work researchers debrief, evaluate, and reconcile disagreements with the labeled social media posts. The social work researchers rely on the expertise of the domain experts to inform their coding process and will recode their initial annotations based on what they learn in reconciliation meetings. Once codes are thoroughly and iteratively evaluated by domain experts and social work researchers, a final label will be applied to each social media post (e.g., labeling the social media post "public safety" or "community trust"). Finally, the labeled social media dataset is handed to the data-science team to use in further training of the NLP computational system that automatically detects public safety-related content.

CASM provides the context necessary to eliminate the use of NLP processes in a cultural vacuum; similar efforts must continue at every step along the research process to produce equitable, applicable, and well-rounded research. As a social worker working in AI, I am guided by the National Association of Social Workers' Code of Ethics (https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English), which dictates how I leverage AI and the extent to which it is actually "for good." For example, a core principle is for social workers to help people in need and address social problems. In the case of my lab's research, gun violence and mass incarceration are two social problems that share roots in inequality, racism, and poverty. Thus, any approach that leverages AI must anticipate how these social problems intersect, and the impact AI may have on preventing or extending these problems. Another core value is understanding the importance of human relationships. Before I ask if AI is the right tool for gun violence prevention, I must engage in extensive and critical conversations with communities in which gun violence is an issue, and work with them to better understand how the problem is defined by the community.

I believe UX and AI design should value individuals beyond academia as domain experts. When we respect and value diverse lived experiences, we create an opportunity to anticipate needs rather than simply respond to them after a crisis. Centering all lived experiences offers a holistic way to move beyond classifying human behavior into neat bins of data, forcing us to consider who overall gets to participate in UX and AI design, how data is collected and analyzed, the ways in which those systems are deployed, and, most important, for whose benefit. Hopefully, we might also contend with the idea that the answer may require devaluing the power and role of technology in order to center humanity.

1. Epstein, G. Why AI needs more social workers, with Columbia University's Desmond Patton. TechCrunch. Aug. 9, 2019; https://techcrunch.com/2019/08/09/why-ai-needs-more-social-workers-with-columbia-universitys-desmond-patton/

2. Adams, G. and Dillon, N. Google using dubious tactics to target people with 'darker skin' in facial recognition project: sources. New York Daily News. Oct. 2, 2019; https://www.nydailynews.com/news/national/ny-google-darker-skintones-facial-recognition-pixel-20191002-5vxpgowknffnvbmy5eg7epsf34-story.html

3. Stern, C. How to solve homelessness with artificial intelligence. OZY. Apr. 21, 2019; https://www.ozy.com/rising-stars/how-to-solve-homelessness-with-artificial-intelligence/92033/

4. West, S.M., Whittaker, M. and Crawford, K. Discriminating systems: Gender, race and power in AI. AI Now Institute, 2019; https://ainowinstitute.org/discriminatingsystems.html

5. Blevins, T., Kwitakowski, R., Macbeth, J., McKeown, K., Patton, D., and Rambow, O. Automatically processing tweets from gang-involved youth: Towards detecting loss and aggression. Proc. of the 26th International Conference on Computational Linguistics: Technical Papers. COLING, 2016, 2196–2206.

6. Chang, S., Zhong, R., Adams, E., Lee, F-Z., Varia, S., Patton, D., Frey, W., Kedzie, C., and McKeown, K. Detecting gang-involved escalation on social media using context. Proc. of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2018, 46–56.

7. Zhong, R., Chen, Y., Patton, D., Sealous, C., and McKeown, K. Detecting and reducing bias in a high stakes domain. Proc. of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2019, 4765–4775.

8. Blandfort, P., Patton, D.U., Frey, W.R., Karaman, S., Bhargava, S., Lee, F-T., Varia, S., Kedzie, C., Gaskell, M.B., Schifanella, R., McKeown, K., and Chang, S-F. Multimodal social media analysis for gang violence prevention. Proc. of the International AAAI Conference on Web and Social Media 13, 01(2019), 114–124; https://arxiv.org/pdf/1807.08465.pdf

9. Patton, D., Brunton, D., Dixon, A., Hackman, R., and Miller, R. Stop and frisk online: Theorizing everyday racism in digital policing in the use of social media for the identification of criminal conduct and associations. Social Media + Society 3 (2017). DOI: 10.1177/2056305117733344

10. Frey, W., Patton, D., Gaskell, M., and McGregor, K. Artificial intelligence and inclusion: Formerly gang-involved youth as domain experts for analyzing unstructured Twitter data. Social Science Computer Review 38, 1 (2018), 42–56.

Desmond Upton Patton is an associate professor of social work, sociology, and data science at Columbia University; associate dean for curriculum innovation and academic affairs at Columbia University; and director of SAFElab. His work leverages social media, qualitative methods, and artificial intelligence for gun violence prevention. He holds a Ph.D. from the University of Chicago. dp2787@columbia.edu

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2020 ACM, Inc.