Audio localization using ad-hoc microphone arrays for education

The goal of this summer research project is to develop algorithms that can analyze audio recordings to be able to locate people (a teacher) in a noisy environment (classroom).

Imagine a high school teacher in a classroom who is wearing a microphone around their neck.  There are additional microphones placed at various fixed locations around the room, at tables where students are working in small groups during class. The fundamental question we're seeking to answer is this:  If the locations of the microphones at the student tables is known, can we use all of the audio recordings (which we assume to be synchronized somehow) to determine the location of the teacher (who is moving around the room during the recording)?

There is a large literature around audio localization (using audio signals to locate people/objects) going back decades. But, this application of audio localization seems to be unique given that we are trying to do the location in a reverberant, noisy environment; that we are not using carefully arranged microphones but a more ad-hoc locations for microphones; that we have an audio recording corresponding to the person/object that we are trying to geolocate. There are other technical challenges to have to overcome as well.

This summer research project builds off of the work done by Athena Li and Darryl Yong during 2021-22, and will hopefully lead to the implementation of simple tools for automatically producing location data given a set of audio recordings. This research project itself relies on some knowledge of mathematics (signal processing) and programming. There is a larger reason why we're trying to do this work, which is described below if you're interested.

This research project would take place remotely during the summer of 2022. If you prefer to live on HMC campus, there is funding for housing included as well.

 

Name of research group, project, or lab
Project TAU
Why join this research group or lab?

Project TAU (Teaching Amidst Uncertainty) is a multi-institution research group formed by Ilana Horn (Vanderbilt University), Brette Garner (University of Denver), Ben Rydal Shapiro (Georgia State University), and Darryl Yong (HMC). Participation in this summer research project doesn't require you to know much about the education research background behind the project, but in case you are interested, here are more details.

Decades of research have shown that students learn best and instruction is more inclusive when students have opportunities to talk about mathematics. For this reason, many conceptually-oriented mathematics instructional approaches emphasize peer-to-peer discussion and group work in small teams. Yet research diverges around questions of how teachers should foster and manage such discussions and activities.

This research team is focusing specifically on the phase of instruction we're calling "groupwork monitoring"--this is the period of time after an instructor has given instructions for a task, when teachers are actively monitoring students as they work on the task in small groups or answering questions. We're interested in this phase of instruction because it's really interesting and relatively under-theorized. Groupwork monitoring is an inherently improvisatory kind of teaching practice; it is more challenging to describe and plan for. Furthermore, this phase of instruction is often where instructors and students might marginalize each other because of implicit bias and other prejudices embedded in their words and actions.

The way that this research team studies groupwork monitoring is through audiovisual recordings of real classrooms with real teachers and students. We have recorded many hours of these classroom videos and are building new tools for analyzing these recordings. One of these tools is the ability to geolocate a teacher in the classroom during the lesson: https://www.benrydal.com/software/igs. The only drawback of these analyses is that one has to watch and listen to recording and manually locate persons of interest. The goal of this summer research project is to try to automate that process!

If successful, this research would enable our team to be able to much more easily analyze these classroom recordings and also to enable other teachers the ability to analyze their own classrooms in new ways. Ultimately, the goal is to help teachers improve their classroom instruction and help more students learn in more inclusive environments.

Logistics Information:
Project categories
Mathematics
Student ranks applicable
First-year
Sophomore
Junior
Senior
Student qualifications
  • Programming skills: Python, Mathematica (not necessary, but helpful)
  • Familiarity with signal processing tools like the Fast Fourier Transform (You might have sees a bit of Fourier Analysis in E79. Students who have taken Prof. Tsai's signal processing course are especially invited to apply. This background knowledge will make it less likely that a first-year student will have the necessary background, but first-year students with such background are invited to apply as there is nothing else limiting who can participate.)
  • An interest in educational research is a bonus, but not necessary at all.
  • Openness to learning new things! :)
Time commitment
Summer - Full Time
Compensation
Paid Research
Number of openings
1
Techniques learned

Participants will become familiar with a large body of literature in audio localization, will deepen their scientific computing and signal processing skills, and likely also develop their Python skills (particularly with using Numpy and working with audio data). 

Contact Information:
Mentor name
Darryl Yong
Mentor email
dyong@hmc.edu
Mentor position
Professor
Name of project director or principal investigator
Darryl Yong
Email address of project director or principal investigator
dyong@hmc.edu
1 sp. | 16 appl.
Hours per week
Summer - Full Time
Project categories
Mathematics