Keeping people meaningfully involved in algorithmic systems.

We study threshold tests for detecting discrimination under environmental shifts, focusing on the Veil-of-Darkness (VoD) setting where visibility changes between daylight and darkness. We show that standard threshold tests, when applied separately to daylight and darkness data, violate key assumptions: risk distributions drift across contexts and thresholds fluctuate arbitrarily. We propose a cross-context threshold test that enforces distributional invariance and monotonic threshold decay. Using New York City stop-and-frisk data and synthetic experiments, we demonstrate that this model yields more reliable thresholds, improves bias detection, and aligns with the counterfactual logic of the VoD test. Our framework generalizes to fairness auditing whenever environmental context influences decisions.

Ranking schemes drive many real-world decisions, like, where to study, whom to hire, what to buy, etc. Many of these decisions often come with high consequences. For example, a university can be deemed less prestigious if not featured in a top-k list, and consumers might not even explore products that do not get recommended to buyers. At the heart of most of these decisions are opaque ranking schemes, which dictate the ordering of data entities, but their internal logic is inaccessible or proprietary. Drawing inferences about the ranking differences is like a guessing game to the stakeholders, like, the rankees (i.e., the entities who are ranked, like product companies) and the decision-makers (i.e., who use the rankings, like buyers). In this paper, we aim to enable transparency in ranking interpretation by using algorithmic rankers that learn from available data and by enabling human reasoning about the learned ranking differences using explainable AI (XAI) methods. To realize this aim, we leverage the exploration–explanation paradigm of human–data interaction to let human stakeholders explore subsets and groupings of complex multi-attribute ranking data using visual explanations of model fit and attribute influence on rankings. We realize this explanation paradigm for transparent ranking interpretation in TRIVEA, a visual analytic system that is fueled by: (i) visualizations of model fit derived from algorithmic rankers that learn the associations between attributes and rankings from available data and (ii) visual explanations derived from XAI methods that help abstract important patterns, like, the relative influence of attributes in different ranking ranges. Using TRIVEA, end users not trained in data science have the agency to transparently reason about the global and local behavior of the rankings without the need to open black-box ranking models and develop confidence in the resulting attribute-based inferences. We demonstrate the efficacy of TRIVEA using multiple usage scenarios and subjective feedback from researchers with diverse domain expertise.

Explainable AI (XAI) methods such as SHAP can help discover unfairness in black-box models. If the XAI method reveals a significant impact from a "protected attribute" (e.g., gender, race) on the model output, the model is considered unfair. However, adversarial models can subvert the detection of XAI methods. Previous approaches to constructing such an adversarial model require access to underlying data distribution. We propose a simple rule that does not require access to the underlying data or data distribution. It can adapt any scoring function to fool XAI methods, such as SHAP. Our work calls for more attention to scoring functions besides classifiers in XAI research and reveals the limitations of XAI methods for explaining behaviors of scoring functions.

As automated decision systems (ADS) get more deeply embedded into business processes worldwide, there is a growing need for practical ways to establish meaningful transparency. Here we argue that universally perfect transparency is impossible to achieve. We introduce the concept of contextual transparency as an approach that integrates social science, engineering and information design to help improve ADS transparency for specific professions, business processes and stakeholder groups. We demonstrate the applicability of the contextual transparency approach by using it for a well-established ADS transparency tool: nutritional labels that display specific information about an ADS. Empirically, it focuses on the profession of recruiting. Presenting data from an ongoing study about ADS use in recruiting alongside a typology of ADS nutritional labels, we suggest a nutritional label prototype for ADS-driven rankers such as LinkedIn Recruiter before closing with directions for future work.