Machine Learning Pipeline Optimization when Querying Video Data

In real-world deployments of machine learning (ML) models, many models and computer vision (CV) operations are logically stitched together to return the final result. For example, to detect abandoned luggage at an airport, we would need to know that an object is not a person, is part of the foreground, has been static for some time, and is not within some distance of a person. Currently, if such a query is to be run on  video data, someone must manually stitch these pieces together to form a pipeline. Further, this person would need an in-depth understanding of CV and ML because they must decide how to balance the user's requirements (accuracy, speed, etc.) with optimal use of computational resources (CV and ML tasks can vary widely with regards to the computational resources required). Ideally however, this person would not need such a background and could simply issue an SQL-like query to get the final result.

In this project, a collaboration with UCLA researchers, we aim to automate the formation of a pipeline with two steps: 1) build a language that is simple to use but expressive enough to convey concepts such as time and space 2) use the language to build an optimized machine learning pipeline. Students will work on building a library of "primitives" with regards to video queries and building a ranking system for pipelines that considers their resource usage and performance. 

Essay prompt: In 1-2 paragraphs, please describe why you are interested in this area, what you hope to get out of this summer, and any prior research experience. Please also include any questions about this project. 

Name of research group, project, or lab
SysML Lab
Why join this research group or lab?

While machine learning is seeing explosive growth, it's often extremely difficult and expensive to use in real-world scenarios. For example, while computer vision models are getting increasingly robust, they're also getting larger and slower, making them less suitable for tasks such as self-driving cars. The SysML lab applies topics from computer systems to solve such problems in machine learning usability. We look at monitoring resource usage and ways to optimize the end-to-end machine learning pipeline to make deployment more feasible.

Logistics Information:
Project categories
Computer Science
Computer Vision
Machine Learning
Student ranks applicable
Student qualifications

Experience using an ML framework (PyTorch or TensorFlow) is preferred. 

Time commitment
Summer - Full Time
Paid Research
Number of openings
Techniques learned

Applying concepts from computer systems to ML problems, optimization methods

Contact Information:
Arthi Padmanabhan
Computer Science Professor
Name of project director or principal investigator
Arthi Padmanabhan
Email address of project director or principal investigator
1 sp. | 10 appl.
Hours per week
Summer - Full Time
Project categories
Machine Learning (+2)
Computer ScienceComputer VisionMachine Learning