Martian Terrain Explorer for Advanced Terrain Segmentation

This project aims to revolutionize Martian exploration through a two-part initiative. First, we will harness the power of large language models (LLMs) and contextual understanding to curate an extensive synthetic dataset, auto-captioning Martian terrain features to ensure high-quality data for machine learning. The second phase focuses on developing a cutting-edge self-supervised learning framework that leverages this synthetic data to pre-train models, followed by fine-tuning on limited real Martian datasets. An example Martian dataset can be found at here. The resulting system will accurately segment and classify Martian terrain types, providing critical support for navigation and scientific exploration on the Red Planet.

Name of research group, project, or lab
Prof. Mohanty's Research Group
Website
https://sites.google.com/view/adyasha-mohanty/projects
Why join this research group or lab?

Research in this lab directly addresses critical challenges in robotics, including improving navigation in uncharted environments, enhancing sensor fusion techniques, and developing robust autonomous systems. These applications extend beyond space exploration to fields like autonomous vehicles, drones, and smartphones. Key student outcomes are:

  • Students in the lab will emerge with a broad and valuable skill set, having collaborated across robotics, aerospace engineering, computer vision, and AI. 

  • Students will gain hands-on experience with cutting-edge robotics platforms, advanced sensors, and powerful computational tools, and exposure to advanced programming, machine learning, deep learning, sensor fusion, and hardware integration. 

  • Students will be equipped to develop novel solutions to complex robotics challenges by using creative thinking and engineering prototyping. 

  • Students will gain the opportunity to collaborate with either industry or government organizations on impactful projects, leading to real-world applications.

Logistics Information:
Project categories
Computer Science
Engineering
Computer Vision
Robotics
Student ranks applicable
Junior
Senior
Student qualifications

Strong skills in programming languages such as Python, C++, or MATLAB are essential. Experience with libraries and frameworks such as TensorFlow, PyTorch, or scikit-learn is required.  A solid understanding of machine learning principles, including supervised and unsupervised learning, neural networks, and deep learning. Familiarity with self-supervised learning techniques is a plus. Experience working on projects, particularly those involving AI, machine learning, or robotics, is highly desirable. This could include independent study, internships, or project-based coursework.

Interested students can directly email Prof. Mohanty with their CV/Resume and their responses to these questions:

  1. Given a dataset of images that include various terrain features, describe how you would segment the different types of terrain (e.g., rocks, dust, craters) using a combination of synthetic data and real images. Outline your approach, including any preprocessing steps, algorithms, and potential challenges you might face.
  2. After training a model to segment Martian terrain, you notice that it performs well on some types of terrain but poorly on others. How would you diagnose and address this issue?
  3. Can you share an example of a situation where you faced unexpected challenges in a project? How did you adapt your approach, and what did you learn from the experience?
Time commitment
Fall - Part Time
Spring - Part Time
Summer - Part Time
Compensation
Academic Credit
Number of openings
2
Techniques learned
  • Applying advanced computer vision techniques for image segmentation and classification
  • Utilizing LLMs for data curation and auto-captioning, involving natural language processing (NLP) and contextual data generation
  • Evaluating and validating machine learning models using several metrics
  • Creating high-quality synthetic datasets, including data augmentation and the use of generative models
Project start
Fall 2024
Contact Information:
Mentor
Adyasha Mohanty
admohanty@hmc.edu
Principal investigator
Name of project director or principal investigator
Dr. Adyasha Mohanty
Email address of project director or principal investigator
admohanty@g.hmc.edu
2 sp. | 4 appl.
Hours per week
Fall - Part Time (+2)
Fall - Part TimeSpring - Part TimeSummer - Part Time
Project categories
Engineering (+3)
Computer ScienceEngineeringComputer VisionRobotics