April 29, 2026
12:00 PM - 1:00 PM
Abstract:
Foundation models have achieved remarkable performance across a wide range of tasks, yet their internal representations remain opaque, entangled, and difficult to control, limiting safety, alignment, and adaptability. In this talk, I present a unified framework for controlling multimodal foundation models based on representing activations as sparse linear combinations of concept vectors from a curated dictionary. First, I introduce Parsimonious Concept Engineering (PaCE), which constructs concept dictionaries in LLMs and enables targeted intervention via suppression of undesirable activation components, mitigating unaligned responses without degrading core capabilities. Next, I present Concept Lancet, which extends this framework to diffusion models, enabling fine-grained editing of images through decomposition and recombination of visual concepts. Finally, I introduce Dictionary-Aligned Concept Control (DACO), which extends this approach to multimodal settings by learning shared concept dictionaries across modalities, enabling scalable, granular control of model behavior for improved safety and alignment. Together, these approaches establish a paradigm for controlling foundation models through concept-based sparse representations, enabling interpretable intervention and reliable alignment across domains.
Biography:
René Vidal, a global pioneer of data science, is the Rachleff University Professor, with joint appointments in the Department of Radiology in the Perelman School of Medicine and the Department of Electrical and Systems Engineering in the School of Engineering and Applied Science. Dr. Vidal has been named a Penn Integrates Knowledge University Professor at the University of Pennsylvania.
René Vidal received his B.S. degree in Electrical Engineering (highest honors) from the Pontificia Universidad Catolica de Chile in 1997 and his M.S. and Ph.D. degrees in Electrical Engineering and Computer Sciences from the University of California at Berkeley in 2000 and 2003, respectively. He was a research fellow at the National ICT Australia in 2003 and joined The Johns Hopkins University in 2004 as a faculty member in the Department of Biomedical Engineering and the Center for Imaging Science.