In everything from autonomous vehicle testing to improving driver safety, video recordings of drivers provide important data, but to be useable, the data must first be extracted. Compared to human coders, automated algorithms have the potential to reduce expense and increase speed, while increasing the accessibility of the wealth of information captured about drivers’ behavior. Our project will focus on developing strategies for automating video data extraction to record vehicle occupant behavior, as well as objects and actions in the vehicle’s forward view. Moreover, techniques could be extended to other research areas that collect data using video (e.g., education, policing). However, in all of these areas, there is a potential to introduce an unintended bias in the algorithms that could have negative societal implications. Measuring and reducing this bias will be a key goal of our work.

To address both bias and efficiency, we will evaluate the process used by human coders, eventually including recording eye gaze as they code videos. We will identify strategies for selecting video training sets that prevent the algorithms from developing bias, and we will develop ways to evaluate algorithms for the presence of bias. The project will also investigate how context provided by other datasets could be used to increase accuracy and speed of algorithms.

The process of developing and implementing computer vision algorithms contains many component steps and each step raises a host of interesting technological challenges and research questions. Our program is organized around these steps:

At every step, we some basic questions: 1) What roles do the human and the computer play in this? 2) How can we find and address potential bias here? 3) Where are inefficiencies in this step and how can they be addressed?

Meeting time and location:

For academic credit, our MDP course is classified as a hybrid course but will mainly meet remotely, following university public health informed guidelines. Our MDP team meets Mondays at 4:45 pm – 6:00 pm ET) using video conferencing. Our lab is located at the UMTRI Building, 2901 Baxter Road; Online meeting options are available as needed; team has been successfully working remotely since March 2020. Each subteam arranges a convenient time to meet and work together following university guidelines. A two-term commitment will begin January 2021.

Team organization:

This team has flexible sub-teams that allow students to deepen their learning: Technology Development, Research Projects, and Process sub-teams. The teams are flexibly structured to enhance creativity and opportunity for student growth. Each of three subteams has a student team leader that reports to the faculty PIs.

Open Lab Meeting:

More information

First-year undergraduates through masters graduate students are welcome to apply, and all will be encouraged to stay on the team for more than the two-semester minimum. Leadership roles are available in the lab, and experienced students will be a natural fit for these positions as their knowledge grows over time.

Data Analyst (3 Students)

Specific Tasks: Statistic analyses, data management

Required Skills: Completed at least 3 college-level stats courses, experience with R programming

Preferred Skills: Python, some exposure to Bayesian Statistics

Likely Majors: Stats, Biostats, Data Science

Human Cognition & Process Specialist (4 Students)

Specific Tasks: Define instructions for labelers; compare human and computer labeling performance; apply principles of human cognition to algorithm development, monitor and evaluate efficiency of entire process, identify roadblocks and pinch-points, design and implement technology solutions

Preferred Skills: Some programming experience , organization, experimental design

Likely Majors: Cognitive Science, Psychology, Data Science, Industrial and Operations Engineering (IOE), information science

Applied Software Engineer (3 Students)

Specific Tasks: Labeling software development, coding implementation for speed

Required Skills: Completed EECS 281 and 2 other introductory EECS courses (or equivalent), experience with C++ or Python

Preferred Skills: Experience with data structures

Likely Majors: EE, CSE/CS-LSA, Information

Research Software Engineer (2 Students)

Specific Tasks: Determine how to leverage existing work for our projects, develop strategies for accomplishing computer vision tasks

Required Skills: Completed 3 introductory EECS courses or equivalent (e.g., 183, 280, 281) plus some course exposure to machine learning, experience with C++ or Python

Preferred Skills: Experience with TensorFlow, PyTorch, CuDNN or other deep learning libraries, previous experience with computer vision problems

Likely Majors: CSE/CS-LSA, Data Science

Apprentice Researcher/Labeler (4 Students)

Requirements: Interest in project material, willingness to develop skills. Open to first-and second-year undergraduate students ONLY.

Likely Majors: Any

Faculty Sponsor

Kathleen Klinich
Associate Research Scientist, Transportation Research Institute (UMTRI) and Associate Director, UMTRI Dr. Klinich is an associate research scientist in UMTRI’s Biosciences Group, and is currently serving as an associate director of UMTRI. She has 27 years of experience performing research on how to protect occupants in motor-vehicle crashes, particularly child passengers. She has experience in the analysis of motor-vehicle crashes and crash databases, crash dummy design, laboratory reconstruction of real-world loading events, injury criteria development, occupant-anthropometry and posture evaluation, engineering analysis of Federal Motor Vehicle Safety Standards (FMVSS), child-passenger-safety issues evaluation, and finite element modeling. Her most recent research efforts have focused on improving compatibility between child restraints and vehicles, examining belt fit of child occupants, assessing the role of legislation on restraint use, and analysis of crash data to guide future safety priorities.


Jared Karlow
Jared Karlow has more than 7 years of experience as a professional software developer. At UMTRI, he works on systems that enable the querying, modeling, and visualization of state and national crash datasets. He develops systems that model the impact of various safety systems and the introduction of autonomous vehicles on crash and injury populations.

Students: 12-16

Likely Majors: Biostats, Data Science, Cognitive Science, CE, Computer Science (CSE/CS-LSA), Data Science, EE, Psychology, Statistics, IOE, School of Information (SI), Any

Summer Opportunity: Summer research fellowships may be available for qualifying students.

Citizenship Requirements: This project is open to all students on campus.

IP: Students who successfully match to this project team will be required to sign an Intellectual Property (IP) Agreement prior to participation in January 2021.

Course Substitutions: Honors