🔬 Research Summary by Devansh Saxena, a doctoral candidate in the Dept. of Computer Science at Marquette University. Their research focuses on algorithmic systems used in the public sector, especially the child-welfare system. Their current work examines collaborative child-welfare practice where decisions are mediated by policies, practice, and algorithms.
[Original paper by Devansh Saxena, Erina Seh-Young Moon, Dahlia Shehata, Shion Guha]
Overview: Artificial Intelligence (AI) systems are being developed in the child-welfare system using quantitative administrative data that captures citizens’ interactions with government services. However, there are significant gaps in this data and concerns that algorithms amplify racial and systemic biases embedded in this historical data. In this paper, we turn towards a source of data that has been hard to study computationally so far but carries more contextual information – caseworkers’ casenotes. Casenotes contain more critical details about caseworkers’ interactions with families, circumstances surrounding a case, uncertainties, and the impact of systemic factors. We conducted the first computational inspection of these casenotes where we highlight invisible work practices, systemic constraints, and power asymmetries that impact street-level decisions in child welfare.
The U.S. Child-Welfare System (CWS) faces significant challenges. CWS has limited resources, burdensome workloads, and high staff turnover and faces intense public scrutiny on harm caused to children who are removed from their parents but also when child abuse tragedies occur. These challenges have mounted pressure on CWS to employ AI systems and prove that they follow consistent and objective decision-making processes. Researchers have made significant contributions in developing algorithms that aid frontline caseworkers in deciding which calls (i.e., allegations of abuse) should be screened in for an investigation. Researchers have also used crowdsourcing platforms such as Amazon Mechanical Turk (MTurk) to study people’s perceptions of algorithmic decisions and their impact on human judgment. However, there are drawbacks in these studies that need redressing: 1) algorithms built from quantitative administrative data in CWS only account for a narrow set of predictors, have significant gaps, and only offer a deficit-based framing of families, and 2) experiments conducted on crowdsourcing platforms do not account for organizational constraints or day-to-day bureaucratic protocols that impact caseworker decision-making. Here, collaboratively curated caseworker documentation (i.e., casenotes) may offer a more holistic picture of street-level interactions and bureaucratic complexities. Unlike administrative quantitative data, casenotes offer a more credible source of information by revealing workers’ interactions with families, uncertainties in a case, and the impact of bureaucratic constraints on decision-making. These narratives offer much of the desiderata necessary for computational narrative analysis. Case notes about families are highly contextual but also share core similarities because they describe similar pathways that most families follow in CWS.
Work Practices are evolving through technology but going unnoticed by policymakers
We analyzed two years of casenotes written by the family preservation team, a specialized child-welfare team that works with parents in their efforts to achieve reunification with their children. This team engages with children, birth parents, and foster parents on a regular basis and understands the risks, needs, and protective factors present in every case. We conducted computational text analysis using LDA topic models to highlight work practices that caseworkers were undertaking while working with families. Our results highlight much of the hereto hidden, street-level discretionary work that caseworkers do while helping families (e.g., managing medication schedules, conducting quantitative assessments, establishing caregiving roles, navigating court proceedings etc.). These casenotes are collectively curated by CW staff involved at the front-end of case planning and offer a holistic picture for collaborative decision-making. What makes our results really important is that they revealed patterns of work that were not even uncovered during an extensive ethnography at the same agency consisting of observations of meetings and interviews with caseworkers to understand their daily work practices (Saxena et al. 2021). For instance, caseworkers help manage medical consent, medication administration, as well as accompany clients to medical appointments and court hearings. These topics were not highlighted during the ethnography even though they are collaboratively discussed in the casenotes. This suggests that qualitative deconstruction of work practices may not reveal all the nuances of invisible labor and in fact, demand complementary methodological lenses. By extension, this advocates for a need for both a qualitative and quantitative critique of sociotechnical systems.
Systemic factors that constrain discretion
Different systemic constraints arise for families with different needs.
Families in child welfare have varying needs and interact with caseworkers in different capacities. Based on prior literature, we used the number of interactions that families have with caseworkers as a proxy for their level of need and divided the families into three groups – low (G1), medium (G2), and high needs (G3). Next, we assessed how different patterns (topics) of work are highlighted at different times through the life of a case and illustrate different interventions for these groups. Our results highlight how constraints affect the work that caseworkers need to do in order to provide better outcomes for children. We find that all children in CWS are not treated the same as some have higher needs than others (hence, our groups – G1, G2, G3) and this differential need is affected by constraints (e.g., resource, bureaucratic, temporal etc.). For instance, CW staff help secure essential resources for families. However, for G1 (low need), this generally takes the form of economic resources such as employment, food, clothing, and preventive services such as parenting classes. This requires CW staff to reach out to local parent support groups and family resource centers to connect clients to such services. Similarly, G2 (medium need) requires CW staff to find court-ordered services for their clients such as domestic violence classes, AODA (alcohol and other drug abuse) classes, therapy, etc. This requires CW staff to reach out to each of these service providers and find room for their clients. Much of this disparately available information can be curated into a system and made more accessible to CW staff. Here, an important implication arises for algorithms in CWS. Much of the current work has focused on (a) developing more sophisticated machine learning-based risk assessment algorithms to improve the status-quo or (b) understanding breakpoints, biases, and ways in which caseworkers make decisions from currently implemented AI systems. What is left unexplored at the current moment is whether (a) we need to be developing AI applications in CWS in the first place as well as (b) if simpler, non-algorithmic applications can help in removing some existing constraints that caseworkers work around. That is, there is a need to design sociotechnical systems centered in the nature of practice that support labor, help mitigate organizational constraints, and streamline bureaucratic processes.
Caseworkers must balance power asymmetries that arise for different cases.
Our results find some evidence to support that CW staff exercised a more collaborative, power-with role where they played a supporting role for groups G1 and G3 and only assumed more power-over relationships (in the case of group G2) when the primary goal was to expedite reunification such that cases did not transition into long-term foster care (i.e., group G3). This also provides some evidence for the efforts made within CWS from both a policy and practice standpoint to transition towards a “Families as Partners” model where parents are supposed to act as equal partners in the case planning process and have agency in the decision-making process. Different power relationships also help uncover the differences in different families involved in child-welfare and highlight the need to support both the families and CW staff in different capacities. For instance, CW staff is involved in a supporting, power-to relationship in both G1 and G3 groups, where they help secure resources for families. However, for G1, this translates into finding material resources (e.g., adequate food/clothing, childcare). Whereas, for G3, CW staff must find ongoing professional services (e.g., therapy domestic violence). On the other hand, G2 cases require that CW staff have a more power-over role in managing the needs of multiple foster placements. Moreover, different power relationships also directly impact how data is collected about children, how their needs are assessed, and have serious implications for algorithmic decision-making. For instance, our prior ethnographic study conducted at this agency revealed that foster parents exercised significant control over how children’s risks and needs were quantitatively scored, which impacted their compensation rates and the services offered to children. This in turn leads to the manipulation of data and the algorithm such that foster parents receive higher compensations. In sum, our analysis unpacks different kinds of work power relationships (e.g., power-with, power-over, power-to etc.) between CWS stakeholders depending on the context and align well with prior social work literature on power relationships in CWS. These results imply that algorithm design in child welfare needs to understand and consider these power relationships to support the primary objective of providing positive outcomes for foster children.
Between the lines
This study offers the first computational inspection of child welfare casenotes and introduces them as a critical data source for studying complex sociotechnical systems. Computational methods can help us uncover critical systemic issues and disparities that were even hidden from an extensive ethnography conducted at the same agency. This paper highlights the complexities within child-welfare (i.e., invisible labor, systemic constraints, power asymmetries) that are often overlooked in computational research and advocates for a need for both qualitative and quantitative critique of sociotechnical systems.