The CLUES Project uses artificial intelligence methods to index both what was said and visually presented in class sessions. Through this innovation an unprecedented data set is being created that identifies the knowledge being transmitted in college courses and allows students the ability to search for specific moments in their class sessions. It is a goal of the CLUES Project to automatically generate study guides for courses based on what was presented in class including contextual linkages to additional resources in a student’s learning ecosystem.
The CLUES Project is an outgrowth of a research program funded, in part, by the University of Michigan College of Engineering and the National Science Foundation. It is expected that the tools and methods developed in this project will be made available to a wide range of colleges, universities, and industry.
The CLUES research project is seeking students with a broad range of backgrounds and interests. The research team will:
- Build upon an existing toolset of AWS cloud-based functions that ingest and analyze transcripts and videos from class sessions to identify key concepts being taught.
- Design, implement and test search algorithms and machine learning classifiers to identify key moments in class sessions.
- Design, implement and test methods for identifying additional resources related to what is being discussed in class.
- Develop machine learning methods that identify which resources students find more useful in a course.
- Assess and revise existing application designs and functionality.
- Plan, conduct and analyze usability studies.
- Conduct customer discovery studies to identify potential use cases for the technology beyond higher education.
Team Organization
Each CLUES sub-team is coached by an experienced student leader that reports to the CLUES faculty and the PI. Students will work closely with their sub-team leader.
The sub-teams are flexibly structured to promote creativity, provide opportunity for student growth, and nurture team-science. Students may move between sub-teams as workload and individual interest dictates. As students develop skills and build confidence, they should expect increasing responsibility on assignments with multiple parts of the CLUES Project.
We currently have the following project sub-teams: Back-End Programming, Front-End Programming, Artificial Intelligence (NLP & ML), Usability/UI, and Product Development/Entrepreneurship.
First-year undergraduates through master’s students are welcome to apply and are encouraged to participate for more than two-semester minimum to develop a deep understanding of the research process. Leadership roles are available within the group and experienced students are a natural fit for these positions as their knowledge grows over time.
Below are the skills needed for this project. Students with the following relevant skills and interest in the project are encouraged to apply! Although the team consists of subteams, students apply to the project as a whole, rather than individual roles on the team.
Back-End Programming (3 Students)
Preferred skills: Python, React.js, HTML5, API development, AWS tools, Database design
Likely majors: CS, SI/MSI, MIDAS, DATA
Front-End Programming (2 Students)
Preferred skills: React.js, HTML5, JavaScript, AWS tools, Intuitive UI/UX design
Likely majors: CS, SI/MSI (HCI/UX), DATA, ARTDES (with programming skills)
Artificial Intelligence (NLP & ML) (2 Students)
Preferred skills: Python, Natural Language Processing, image/video recognition or related machine learning domains
Likely majors: CS, SI/MSI, DATA, STATS, MIDAS, LING (must have NLP skills)
Usability/UI (3 Students)
Preferred skills: Usability study design, plan surveys/focus groups, collect and analyze user feedback to guide design decisions (quantitative analysis)
Likely majors: SI/MSI-HCI, CS
Product Development/Entrepreneurship (2 Students)
Preferred skills: Customer discovery, customer segments and channels
Likely majors: BBA, SI, ENGR, any major with practical experience
Faculty Sponsor
Kevyn Collins-Thompson
Associate Professor in the School of Information and College of Engineering, Dept. of Electrical Engineering and Computer Science (affiliate, CSE Division). Also, an affiliate faculty member of the Artificial Intelligence Lab and the Michigan Institute for Data Science (MIDAS). Academic director of the Masters of Applied Data Science Program. Research interests include information retrieval, machine learning, natural language processing, and large-scale data mining to optimally connect people with information, especially to help them learn and discover.
Perry Samson
Arthur Thurnau Professor, Climate and Space Sciences and Engineering and School of Information. Honored as a Professor of the Year in the State of Michigan and, was the recipient of the 2021 University of Michigan “Distinguished Innovator Award” as co-founder of The Weather Underground, and educational technologies, LectureTools and LearningClues. A proponent of experiential learning, he organizes summer expeditions for undergraduates to chase supercell thunderstorms in the Great Plains and expeditions to Greenland to make climate-related measurements and observations.
Weekly Meeting Time and Location: Students will enroll in ENGR 255,355,455, or 599 for 2 credits per term. Annual, two-term enrollment commitments begin each January.
The MDP-CLUES team holds all-team meetings twice each month on Friday afternoon via zoom. The PI/Faculty will lead discussions on topics such as technical developments in the field, group management and strategy planning as well as hold working sessions (e.g., addressing a particular problem). In addition, MDP-CLUES will hold an in-person all-team research review meeting at least once per term.
Sub-teams and additional small project groups meet weekly with their sub-team leader in person at mutually convenient times. These are work together sessions to make progress on the semester’s goals and address individual or sub-team specific questions. The PI and other faculty may attend sub-team meetings as required. Team members are expected to meet and work together in person.
The sub-team leaders will meet weekly with the faculty PI to regularly report the progress of each sub-team. These meetings will be in-person or remote, as needed, but not hybrid.
We maintain a Slack channel to asynchronously coordinate progress, seek support from others, document challenges, and generally keep each other informed.
Course Substitutions: Honors
Citizenship Requirements: This project is open to all students on campus
IP/NDA: Students who successfully match to this project team will be required to sign an Intellectual Property (IP) Agreement prior to starting participation in January.
Summer Opportunity: Some Summer research internships may be available for qualifying students
Learn more about the expectations for this type of MDP project