All Projects
Active Projects
Automated Ontology and Knowledge Graph Construction
We are designing an automated methodology to construct ontologies and knowledge graphs from text, supporting applications such as legal statute interpretation, improved case retrieval, and recommendations.
Formalism Switching for the Selection of Best Formal Languages
We systematically survey the literature and conduct expert studies to create a mechanistic, two-tier rubric for selecting the most appropriate formal language for a given problem, based on must-have properties and good-to-have rankings.Episodic Memory
Exploring neuro-symbolic approaches to understanding and answering questions about long-form videos.
Braille and Place Labels for 3D-Printed Tactile Maps
Tactile Map Annotation: On a tactile map, a building under the finger is an unnamed shape. Our tool gives it a name in Braille. Ten blind and low-vision readers compared labeled and unlabeled prints. By linking the 3D model with a real-world map, annotation becomes easy and sharable across map modalities and places.
SLuG - Schema Logical Guidance
SLuG - Schema Logical Guidance is a project that tries to understand current LLMs/VLMs’ ability to provide reliable guidance when visual input is insufficient to answer an image-based question.
Real-Time Multimodal Agent Grounded in Touch
TouchingSpace: Blind users explore a map by touch and ask a voice agent about the place under their finger. We built it with a blind co-designer and tested it in person with 14 blind and low-vision adults.Faithful NeSy Reasoning
This project aims to apply neuro-symbolic approach to instruct the COT process of LLMs for better verifiability and faithfulness/trustworthinessLayerSkip
Understanding residual bias in LLMs.
Propagators
Propagators are a constraint-based programming model which we use to diagnose, understand, and monitor complex systems
Neuron Explanations
We explain what neurons learn using logical combinations of visual concepts, developing richer explanations across activation levels, open-vocabulary concepts, and algorithms with guaranteed optimality.
Causality in Autonomous Vehicles
Understanding how autonomous vehicles reason about causality.
Explanatory Grading Guidance (EGG)
A project to make automated assessment tools or autograders more explainable.
Abductive Learning
Reducing LLM hallucinations through neuro-symbolic reasoning and abductive learning for natural-language-to-Prolog translation.AV Ethics
Exploring ethical decision-making in autonomous vehicles, including dilemmas such as the trolley problem.Challenge team
Competing in open autonomous vehicle research challenges using open datasets, spanning Waymo, LiDAR, synthetic data, and vision-language-action models.Dynamic Path Planning
Exploring autonomous vehicle path planning with algorithms such as D* and D* Lite.
Frame-based monitoring
Translate autonomous system logs into a common, symbolic frame-based representationHybrid Note
Turning handwritten notes into searchable text.Olelo
Building a custom, open-source notetaker for our lab.
Self-Explainable Deep Neural Networks
Inherently interpretable neural networks whose explanations influence predictions and reveal model reasoning, with a focus on computer vision, novel settings, architectures, and training methods.Transfer Learning
Developing interpretable reinforcement learning algorithms that transfer knowledge to new and unfamiliar environments.
WISPER
A collective advancing ethical AI through interdisciplinary collaboration, undergraduate mentorship, and justice-oriented co-creation of knowledge.
Completed Projects
Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning
We evaluated the faithfulness of LLMs as autoformalizers and solvers in NDA entailment classification and identified their failure modes.
Learning to Draw ASCII Improves Spatial Reasoning in Language Models
People sketch to reason about space. We ask whether language models can do the same, and find that LLMs read ASCII layouts better than they draw them, and that training a model only to draw these layouts improves its spatial reasoning from text alone.
Knowledge-augmented LLMs for Logical Fallacy Classification
We experimented with several approaches to augment LLMs with knowledge and evaluated whether, and by how much, they could improve performance in logical fallacy classification.
Interpretable Scene Understanding for Autonomous Driving
Frame-Based Scene Understanding: Self-driving systems detect objects, but they rarely say what a scene means. We organize raw driving data into sample, object, and scene frames that people can read.
Camera Guidance for Vision-Language Models
Right This Way: When a photo can’t answer the question, our models tell blind users how to move the camera. Training on our label-free synthetic data raised open vision-language models by up to 16 points, and the best model beat GPT-4o.