In everything from autonomous vehicle testing to improving driver safety, video recordings of drivers provide important data, but to be useable, the data must first be extracted. Compared to human coders, automated algorithms have the potential to reduce expense and increase speed, while increasing the accessibility of the wealth of information captured about drivers’ behavior. Our project will focus on developing strategies for automating video data extraction to record vehicle occupant behavior, as well as objects and actions in the vehicle’s forward view. Moreover, techniques could be extended to other research areas that collect data using video (e.g., education, policing). However, in all of these areas, there is a potential to introduce an unintended bias in the algorithms that could have negative societal implications. Measuring and reducing this bias will be a key goal of our work.
To address both bias and efficiency, we will evaluate the process used by human coders, eventually including recording eye gaze as they code videos. We will identify strategies for selecting video training sets that prevent the algorithms from developing bias, and we will develop ways to evaluate algorithms for the presence of bias. The project will also investigate how context provided by other datasets could be used to increase accuracy and speed of algorithms.
The process of developing and implementing computer vision algorithms contains many component steps and each step raises a host of interesting technological challenges and research questions. Our program is organized around these steps:
Meeting time and location:
For academic credit, our MDP course is classified as a hybrid course but will mainly meet remotely, following university public health informed guidelines. Our MDP team meets Mondays at 4:45 pm – 6:00 pm ET) using video conferencing. Our lab is located at the UMTRI Building, 2901 Baxter Road; Online meeting options are available as needed; team has been successfully working remotely since March 2020. Each subteam arranges a convenient time to meet and work together following university guidelines. A two-term commitment will begin January 2021.
This team has flexible sub-teams that allow students to deepen their learning: Technology Development, Research Projects, and Process sub-teams. The teams are flexibly structured to enhance creativity and opportunity for student growth. Each of three subteams has a student team leader that reports to the faculty PIs.
Open Lab Meeting:
First-year undergraduates through masters graduate students are welcome to apply, and all will be encouraged to stay on the team for more than the two-semester minimum. Leadership roles are available in the lab, and experienced students will be a natural fit for these positions as their knowledge grows over time.
Data Analyst (3 Students)
Specific Tasks: Statistic analyses, data management
Required Skills: Completed at least 3 college-level stats courses, experience with R programming
Preferred Skills: Python, some exposure to Bayesian Statistics
Likely Majors: Stats, Biostats, Data Science
Human Cognition & Process Specialist (4 Students)
Specific Tasks: Define instructions for labelers; compare human and computer labeling performance; apply principles of human cognition to algorithm development, monitor and evaluate efficiency of entire process, identify roadblocks and pinch-points, design and implement technology solutions
Preferred Skills: Some programming experience
Likely Majors: Cognitive Science, Psychology, Data Science, Industrial and Operations Engineering (IOE), information science
Applied Software Engineer (3 Students)
Specific Tasks: Labeling software development, coding implementation for speed
Required Skills: Completed EECS 281 and 2 other introductory EECS courses (or equivalent), experience with C++ or Python
Preferred Skills: Experience with data structures
Likely Majors: EE, CSE/CS-LSA, Information
Research Software Engineer (2 Students)
Specific Tasks: Determine how to leverage existing work for our projects, develop strategies for accomplishing computer vision tasks
Required Skills: Completed 3 introductory EECS courses or equivalent (e.g., 183, 280, 281) plus some course exposure to machine learning, experience with C++ or Python
Preferred Skills: Experience with TensorFlow, PyTorch, CuDNN or other deep learning libraries, previous experience with computer vision problems
Likely Majors: CSE/CS-LSA, Data Science
Apprentice Researcher/Labeler (4 Students)
Requirements: Interest in project material, willingness to develop skills. Open to first-and second-year undergraduate students ONLY.
Likely Majors: Any
Carol Flannagan, Ph.D.
Research Associate Professor, Transportation Research Institute (UMTRI) and Director, Management of Information for Safe and Sustainable Transportaion (CMISST)
Dr. Carol Flannagan is a Research Professor and Director of the data center at the University of Michigan Transportation Research Institute (UMTRI). She earned an M.A. in Statistics and a Ph.D. in Mathematical Psychology from the University of Michigan. Dr. Flannagan develops analytical methods for and analyzes a wide variety of transportation-related data, including travel, driving, crash, and injury outcome data. Her work has been implemented both in state planning processes and in in-vehicle algorithms for automatic crash and injury notification. In recent years, she has been thinking about applications of Bayesian statistics and cognitive psychology to improve computer image processing.
Associate Research Scientist, Transportation Research Institute (UMTRI) and Associate Director, UMTRI Dr. Klinich is an associate research scientist in UMTRI’s Biosciences Group, and is currently serving as an associate director of UMTRI. She has 27 years of experience performing research on how to protect occupants in motor-vehicle crashes, particularly child passengers. She has experience in the analysis of motor-vehicle crashes and crash databases, crash dummy design, laboratory reconstruction of real-world loading events, injury criteria development, occupant-anthropometry and posture evaluation, engineering analysis of Federal Motor Vehicle Safety Standards (FMVSS), child-passenger-safety issues evaluation, and finite element modeling. Her most recent research efforts have focused on improving compatibility between child restraints and vehicles, examining belt fit of child occupants, assessing the role of legislation on restraint use, and analysis of crash data to guide future safety priorities.
Jared Karlow has more than 7 years of experience as a professional software developer. At UMTRI, he works on systems that enable the querying, modeling, and visualization of state and national crash datasets. He develops systems that model the impact of various safety systems and the introduction of autonomous vehicles on crash and injury populations.
Likely Majors: Biostats, Data Science, Cognitive Science, CE, Computer Science (CSE/CS-LSA), Data Science, EE, Psychology, Statistics, IOE, School of Information (SI), Any
Summer Opportunity: Summer research fellowships may be available for qualifying students.
Citizenship Requirements: This project is open to all students on campus.
IP: Students who successfully match to this project team will be required to sign an Intellectual Property (IP) Agreement prior to participation in January 2021.
Course Substitutions: Honors