Quantifying Space and Time Dependencies in Deep Networks

In this work we will explore how deep convolutional networks focus on and represent information in both spatial and temporal domains. Deep convolutional networks are widely used in the field of computer vision, representing the state of the art approach to many problem domains. There are nevertheless a number of aspects to their behaviour which are unexplored or poorly understood, and we frequently find that the way that these models solve visual problems is quite different from (and more brittle than) human visual representations. By uncovering these differences, we hope to both better understand how deep networks function and identify possible avenues for improvement, as well as deepen our understanding of human visual cognition. In this project we will:

  • Review existing literature in deep convolutional network behaviour, particularly in the domains of object detection, semantic segmentation, and action recognition.
  • Work on adapting visualization techniques such as Grad-CAM to target networks of interest.
  • Compare the behaviour of deep networks to human visual behaviour (with a focus on eye tracking)
  • Stretch goal: Incorporate our findings into novel training or architecture designs to improve model behaviour.

Essay prompts (address the following):

1. What interests you in the project?
2. Do you have any prior experience with computer vision and/or deep learning?
3. What do you hope to get out of this research experience?

Name of research group, project, or lab
Lab for Cognition and Attention in Time and Space (CATS)
Why join this research group or lab?

The Lab for CATS seeks to understand visual cognition, and help build more robust and unbiased artificial visual agents. There is a lot of hype in the world of computer vision and machine learning, and we seek to keep a grounded focus on fair and realistic evaluations of model behaviour with the goal of identifying when common benchmarking and evaluation practices might result in unanticipated deficits in novel or unconstrained environments.

We are a new organization; you will get to play an active part in laying the groundwork for a new research group!

Representative publication
Logistics Information:
Project categories
Computer Science
Artificial Intelligence
Computer Vision
Student ranks applicable
Sophomore
Junior
Senior
Student qualifications

Have an active interest in learning/relevant experience in the following:

  • computer vision
  • deep learning (experience with PyTorch is a bonus)
  • experience with general processing on the GPU (i.e. using a system with CUDA)
  • cognitive science and/or psychophysics
Time commitment
Summer - Full Time
Compensation
Paid Research
Number of openings
2
Techniques learned

Students should expect to gain experience with:

  • reading and analyzing computer vision literature
  • working with neural networks
  • understanding ways in which human and computer vision differ
Contact Information:
Mentor name
Calden Wloka
Mentor email
cwloka@hmc.edu
Mentor position
Principal Investigator
Name of project director or principal investigator
Calden Wloka
Email address of project director or principal investigator
cwloka@hmc.edu
2 sp. | 12 appl.
Hours per week
Summer - Full Time
Project categories
Computer Vision (+2)
Computer ScienceArtificial IntelligenceComputer Vision