Music Information Retrieval Research

The Music Information Retrieval (MIR) Research Lab explores the intersection of music, signal processing, and machine learning.  Our goal is to train machine learning and signal processing ninjas using music as a playground.

Projects are designed around students' interests and expertise (both technical and musical) or are continuations of existing work.  Some recent projects have included:

• Composer Recognition.  The goal of this project is to predict the composer of a piece of music based on its compositional style.  Students converted the music into a sequence of discrete tokens, pretrained a Transformer-based language model on unlabeled data, and fine-tuned the model on a small amount of labeled data.  We are interested in considering multimodal approaches to this problem.
• Virtual Concerto Accompaniment.  The goal of this project is to enable a soloist to play a concerto with a virtual orchestral accompaniment that adapts to their playing.  In collaboration with a research lab in Germany, we have collected and annotated a set of concerto data and studied offline approaches to the problem.  We would like to expand this to online and real-time approaches.
• Extending Dynamic Time Warping.  Dynamic Time Warping (DTW) is a widely used tool for calculating similarity between time series data in an offline manner.  We are currently exploring extensions of DTW that allow for flexible boundary conditions, have reduced memory or runtime costs, or are suitable for online applications.

You can find short video summaries of other projects here.

We are currently soliciting applications for 4-6 positions across three different categories:

• Signal Processing.  Strong applicants will have a solid foundation in signal processing (e.g. performance in E101) and programming, and must be able to enroll in E207 (music signal processing) in spring semester.  These positions will extend to summer internships.
• Machine Learning.  Strong applicants will have demonstrated experience and proficiency with machine learning and data processing.  Ideally, applicants will enroll in independent research for 2-3 units during spring semester and continue their work through the summer internship.
• Human Centered Design.  Strong applicants will have taken 1 or more human centered design (HCD) courses and have demonstrated proficiency on past HCD projects.  Ideally, applicants will enroll in independent research for 3 units during spring semester and conduct a study on how to make a selected MIR technology (like automated accompaniment or transcription) more human-centered.  This position has the possibility of being extended to a summer internship.

Expertise and experience with music is highly valued, but not required.  To apply, please follow the instructions on our lab website.