The CLUES Project uses artificial intelligence methods to index both what was said and visually presented in class sessions. Through this innovation an unprecedented data set is being created that identifies the knowledge being transmitted in college courses and allows students the ability to search for specific moments in their class sessions. It is a goal of the CLUES Project to automatically generate study guides for courses based on what was presented in class including contextual linkages to additional resources in a student’s learning ecosystem.

The CLUES Project is an outgrowth of a research program funded, in part, by the University of Michigan College of Engineering and the National Science Foundation. It is expected that the tools and methods developed in this project will be made available to a wide range of colleges, universities, and industry.

The CLUES research project is seeking students with a broad range of backgrounds and interests.  The research team will:

  1. Build upon an existing toolset of AWS cloud-based functions that ingest and analyze transcripts and videos from class sessions to identify key concepts being taught.
  2. Design, implement and test search algorithms and machine learning classifiers to identify key moments in class sessions.
  3. Design, implement and test methods for identifying additional resources related to what is being discussed in class.
  4. Develop machine learning methods that identify which resources students find more useful in a course.
  5. Assess and revise existing application designs and functionality.
  6. Plan, conduct and analyze usability studies.
  7. Conduct customer discovery studies to identify potential use cases for the technology beyond higher education.

Team Details 

Meeting time and location:  Students will enroll in ENGR 255, 355, 455, or 599 for 2 credits per term.  Annual, two-term enrollment committments begin in January.

The MDP-CLUES team holds all team meetings twice each month on Friday afternoon via Zoom.  The PI/Faculty will lead discussions on topics such as technical developments in the field, group management and strategy planning as well as hold working sessions (e.g., addressing a particular problem).  In addition, MDP-CLUES will hold an in-person all-team research review meeting at least once per term (subject to University guidelines).

Sub-teams and additional small project groups meet weekly with their sub-team leader in person at mutually convenient times.  These are work together sessions to make progress on the semester’s goals and address individual or sub-team specific questions.  The PI and other faculty may attend sub-team meetings as required.  Team members are expected to meet and work together in person.

The sub-team leaders will meet weekly with the faculty PI to regularly report the progress of each sub-team.  These meetings will be in-person or remote, as needed, but not hybrid.

MDP-CLUES maintains a Slack channel to asynchronously coordinate progress, seek support from others, document challenges, and generally keep each other informed.

Team organization:  Each CLUES sub-team is coached by an experienced student leader that reports to the CLUES faculty and the PI.   Students will work closely with their sub-team leader.

The sub-teams are flexibly structured to promote creativity, provide opportunity for student growth, and nurture team-science.  Students may move between sub-teams as workload and individual interest dictates.  As students develop skills and build confidence, they should expect increasing responsibility on assignments with multiple parts of the CLUES Project.

We currently have the following project sub-teams: Back-End Programming, Front-End Programming, Artificial Intelligence (NLP & ML), Usability/UI, Product Development/Entrepreneurship, and Apprentice Researcher. 

See more information

First-year undergraduates through master’s students are welcome to apply, and all will be encouraged to stay on the team for more than the two-semester minimum. Leadership roles are available in the lab, and experienced students will be a natural fit for these positions as their knowledge grows over time.

Back-End Programming (3 Students)

Preferred Skills: Python, React.js, HTML5, API development, AWS tools, Database design


Front-End Programming (3 Students)

Preferred Skills: React.js, HTML5, JavaScript, AWS tools, Intuitive UI/UX design

Likely Majors: SI/MSI (HCI/UX), DATA, CSE/CS-LSA, ARTDES (with programming skills)

Artificial Intelligence (NLP & ML) (4 Students)

Preferred Skills: Python, Natural Language Processing, image/video recognition or related machine learning domains

Likely Majors: CSE/CS-LSA, SI/MSI, DATA, STATS, MIDAS, LING (must have NLP skills)

Usability (3 Students)

Preferred Skills: Usability study design, plan surveys/focus groups, collect and analyze user feedback to guide design decisions (quantitative analyses)

Likely Majors: SI/MSI-HCI, CSE/CS-LSA

Product Development/Entrepreneurship (2 Students)

Preferred Skills: Customer discovery, customer segments and channels

Likely Majors: SI, BBA, ENG, any major with practical experience

Apprentice Researcher (4 Students)

Requirements: Interest in project material, willingness to develop skills. Open to first and second-year undergraduate students ONLY.

Likely Majors: CSE/CS-LSA, SI, BBA, ENTR, Any

Faculty Sponsor

Kevyn Collins-Thompson
Associate Professor in the School of Information and College of Engineering, Dept. of Electrical Engineering and Computer Science (affiliate, CSE Division)
Kevyn Collins-Thompson is an affiliate faculty member of the Artificial Intelligence Lab and the Michigan Institute for Data Science (MIDAS), as well as an academic director of the Masters of Applied Data Science Program. Research interests include information retrieval, machine learning, natural language processing, and large-scale data mining to optimally connect people with information, especially to help them learn and discover.


Perry Samson
Arthur Thurnau Professor of Climate and Space Sciences and Engineering,  and in the School of Information
Perry Samson was honored as a Professor of the Year in the state of Michigan and was the recipient of the 2021 University of Michigan “Distinguished Innovator Award,” as co-founder of The Weather Underground and educational technologies LectureTools and LearningClues. A proponent of experiential learning, he organizes summer expeditions for undergraduates to chase supercell thunderstorms in the Great Plains and expeditions to Greenland to make climate-related measurements and observations.


Number of Students: 12 – 15


Summer Opportunity: Some summer research internships may be available for qualifying students.

Citizenship Requirements: This project is open to all students on campus.

IP/NDA: Students who successfully match to this project team will be required to sign an Intellectual Property (IP) Agreement prior to participation in January 2023.