Data Elements from Video using Impartial Algorithm Tools for Extraction
In everything from autonomous vehicle testing to improving driver safety, video recordings of drivers provide important data, but to be usable, the data must first be extracted. Compared to human coders, automated algorithms have the potential to reduce expense and increase speed, while increasing the accessibility of the wealth of information captured about drivers’ behavior. Our project will focus on developing strategies for automating video data extraction to record vehicle occupant behavior, as well as objects and actions in the vehicle’s forward view. Moreover, techniques could be extended to other research areas that collect data using video (e.g., education, policing). In all of these areas, there is a potential to introduce an unintended bias in the algorithms that could have negative societal implications. Measuring and reducing this bias will be a key goal of our work.
To address both bias and efficiency, we will evaluate the process used by human coders, eventually including recording eye gaze as they code videos. We will identify strategies for selecting video training sets that prevent the algorithms from developing bias, and we will develop ways to evaluate algorithms for the presence of bias. The project will also investigate how context provided by other datasets could be used to increase accuracy and speed of algorithms.
The process of developing and implementing computer vision algorithms contains many component steps and each step raises a host of interesting technological challenges and research questions. Our program is organized around these steps:
At every step, we ask two questions: 1) What roles do the human and the computer play in this? 2) How can we find and address potential bias here?
Meeting Date, Time and Location: A best time will be finalized once students are identified. Each subteam arranges a convenient time to meet and work together. We will meet at UMTRI.
Team organization: This newly formed start-up team will develop sub-teams with student team leaders that report to the PIs. The teams are flexibly structured to enhance creativity and opportunity for student growth.