In real-world deployments of machine learning (ML) models, many models and computer vision (CV) operations are logically stitched together to return the final result. For example, to detect abandoned luggage at an airport, we would need to know that an object is not a person, is part of the foreground, has been static for some time, and is not within some distance of a person. Currently, if such a query is to be run on video data, someone must manually stitch these pieces together to form a pipeline. Further, this person would need an in-depth understanding of CV and ML because they must decide how to balance the user's requirements (accuracy, speed, etc.) with optimal use of computational resources (CV and ML tasks can vary widely with regards to the computational resources required). Ideally however, this person would not need such a background and could simply issue an SQL-like query to get the final result.
In this project, a collaboration with UCLA researchers, we aim to automate the formation of a pipeline with two steps: 1) build a language that is simple to use but expressive enough to convey concepts such as time and space 2) use the language to build an optimized machine learning pipeline. Students will work on building a library of "primitives" with regards to video queries and building a ranking system for pipelines that considers their resource usage and performance.
Essay prompt: In 1-2 paragraphs, please describe why you are interested in this area, what you hope to get out of this summer, and any prior research experience. Please also include any questions about this project.
While machine learning is seeing explosive growth, it's often extremely difficult and expensive to use in real-world scenarios. For example, while computer vision models are getting increasingly robust, they're also getting larger and slower, making them less suitable for tasks such as self-driving cars. The SysML lab applies topics from computer systems to solve such problems in machine learning usability. We look at monitoring resource usage and ways to optimize the end-to-end machine learning pipeline to make deployment more feasible.