Embodied AI Computer Vision Agent for Tracking and Detecting Objects

This project aims to utilize a robotic AI agent in order to operate a camera mounted on a 2-axis swivel which could be used to track unmanned aerial vehicles or for identifying humans in rescue operations. The goal is to be able to locate and track objects around an area using a using a voice or text-based interface and a computer vision and object detection AI. The poster will detail the development process of both hardware and software to enable this capability.

Intern: Aanya Palety

Mentor: Matt Abernathy (AMDS/A5B)

Aerospace and Space-Related Engineering

Aerospace and Space-Related Engineering

Aerial Swarming in Search and Rescue Over Water Using Small Target Motion Reactions: Multi-Agent Systems in the Commerical and Defense Sector

Aerial Swarming in Search and Rescue Over Water Using Small Target Motion Reactions: Multi-Agent Systems in the Commerical and Defense Sector

Comparative Validation of Mid-Fidelity Vortex-Based Simulations for Dragonfly Rotorcraft Aerodynamics.

Comparative Validation of Mid-Fidelity Vortex-Based Simulations for Dragonfly Rotorcraft Aerodynamics.

Cosmic Contamination: Investigating Outgassing as a Result of Heat Sealing

Cosmic Contamination: Investigating Outgassing as a Result of Heat Sealing

Creating a Design Reference Mission for a Refueling Spacecraft

Creating a Design Reference Mission for a Refueling Spacecraft

Dragonfly: Constructing Rotor-Protection Covers on Flight Lander

Dragonfly: Constructing Rotor-Protection Covers on Flight Lander

Flying Back to Fly Forward: Reinventing the SR-71 Blackbird

Flying Back to Fly Forward: Reinventing the SR-71 Blackbird

Improved Physics-Based Rail Modeling: Approaching Tip-Off Forces in Rail Launched Missiles

Improved Physics-Based Rail Modeling: Approaching Tip-Off Forces in Rail Launched Missiles

Lasereyes: Full Loop Laser Detection and Tracking System

Lasereyes: Full Loop Laser Detection and Tracking System

LiDAR-IMU SLAM: Autonomous UAV Localization in GPS-Denied Environments

LiDAR-IMU SLAM: Autonomous UAV Localization in GPS-Denied Environments

Monitoring Psyche's GRNS Instrument: Creating and Plotting Calibrated Neutron Spectrometer Data

Monitoring Psyche's GRNS Instrument: Creating and Plotting Calibrated Neutron Spectrometer Data

The DBL: Cost Efficent High Voltage Device Encapsulation

The DBL: Cost Efficent High Voltage Device Encapsulation

XRSSITE: Using Extended Reality to Streamline Spacecraft Integration and Testing

XRSSITE: Using Extended Reality to Streamline Spacecraft Integration and Testing

AI and Machine Learning

AI and Machine Learning

ABLATE: Architecture-Based Layered Ablation Testing & Evaluation of Synthetic Patients

ABLATE: Architecture-Based Layered Ablation Testing & Evaluation of Synthetic Patients

Accelerating Home Recovery: AI-Driven Accuracy Scoring of Physical-Therapy Exercises Using RGB Video from the KiMoRe Dataset

Accelerating Home Recovery: AI-Driven Accuracy Scoring of Physical-Therapy Exercises Using RGB Video from the KiMoRe Dataset

AImong Us IRL: Cooperative and Adversarial Gameplay Between Robots

AImong Us IRL: Cooperative and Adversarial Gameplay Between Robots

AI-Powered Exercise Tracker: Automatic Rep-Counting and Camera Centering

AI-Powered Exercise Tracker: Automatic Rep-Counting and Camera Centering

Autonomous Safe-Landing Drone with Computer Vision-Based Terrain Detection

Autonomous Safe-Landing Drone with Computer Vision-Based Terrain Detection

Below-the-Canopy Computer Vision for Drone-Based Search and Rescue

Below-the-Canopy Computer Vision for Drone-Based Search and Rescue

Creating a Custom Reinforcement Learning Environment for Coding Agents

Creating a Custom Reinforcement Learning Environment for Coding Agents

Developing Agentic Behaviors for Collaborative Crazyflie Teams

Developing Agentic Behaviors for Collaborative Crazyflie Teams

Embodied AI Computer Vision Agent for Tracking and Detecting Objects

Embodied AI Computer Vision Agent for Tracking and Detecting Objects

Evaluation on Hyperparameter Optimizers for Improving Neural Network Prediction Accuracy

Evaluation on Hyperparameter Optimizers for Improving Neural Network Prediction Accuracy

Eyes for the Blind: A Multi-Agent Assistant for the Visually Impaired

Eyes for the Blind: A Multi-Agent Assistant for the Visually Impaired

Fighting Fire with AI: Using Reinforcement Learning to Train AI in a Variable Multi-Actor Strategy Game

Fighting Fire with AI: Using Reinforcement Learning to Train AI in a Variable Multi-Actor Strategy Game

Let's Play Disc Golf: Exploring the Effects of AI Verbosity on Motor-Skills Training in VR

Let's Play Disc Golf: Exploring the Effects of AI Verbosity on Motor-Skills Training in VR

Leveraging Agentic AI for Cyber Attack Execution and Threat Analysis

Leveraging Agentic AI for Cyber Attack Execution and Threat Analysis

Machine Learning for Visual Understanding: An Exploration of Embedding Spaces

Machine Learning for Visual Understanding: An Exploration of Embedding Spaces

Machine Learning In Alzheimer's Disease Screening

Machine Learning In Alzheimer's Disease Screening

Multimodal Pipeline Framework for Generative AI Disinformation

Multimodal Pipeline Framework for Generative AI Disinformation

Multi-Sensor Electric Field Sensing Experiments

Multi-Sensor Electric Field Sensing Experiments

PICKLE: Probabilistic Intelligent Computational Knowledge-Based Language Engine

PICKLE: Probabilistic Intelligent Computational Knowledge-Based Language Engine

PULSE: Physical Unit for Language, Sight, and Execution

PULSE: Physical Unit for Language, Sight, and Execution

Small Scale Security: AI in Tracking and Triangulation

Small Scale Security: AI in Tracking and Triangulation

Sound Classification Using Whisper Embeddings

Sound Classification Using Whisper Embeddings

The Predictably Unpredictable Patient: Exploring the Impact of AI-driven Personality Profiles on Medical Encounters

The Predictably Unpredictable Patient: Exploring the Impact of AI-driven Personality Profiles on Medical Encounters

VULCAN: VULnerability Cascade Analysis for Networked Infrastructure tool

VULCAN: VULnerability Cascade Analysis for Networked Infrastructure tool

Engineering

Attempting Invisibility

Attempting Invisibility

Biomanufacturing of Aligned Cellulose Growth

Biomanufacturing of Aligned Cellulose Growth

Custom Quadcopter Testbed for Evaluating Fractional Control Algorithms

Custom Quadcopter Testbed for Evaluating Fractional Control Algorithms

Data Center Water Usage: USA Data Center Water Usage Analytics Dashboard

Data Center Water Usage: USA Data Center Water Usage Analytics Dashboard

Design of a Switch for Inrush Testing

Design of a Switch for Inrush Testing

Miniaturizing A High Pressure Pump

Miniaturizing A High Pressure Pump

Modeling and Simulation of Multiple Agents in a Dynamic Environment

Modeling and Simulation of Multiple Agents in a Dynamic Environment

Modeling Operational Workflows with Python Based Discrete Event Simulation

Modeling Operational Workflows with Python Based Discrete Event Simulation

Optical Test Target

Optical Test Target

Psychoacoustic Listening Curves: An Interactive App

Psychoacoustic Listening Curves: An Interactive App

SHOOT: Super High-tech Object Obliterating Technology

SHOOT: Super High-tech Object Obliterating Technology

Stabilizing and Drilling for Ice Cores: Mars' Polar Ice Caps

Stabilizing and Drilling for Ice Cores: Mars' Polar Ice Caps

Testing the Performance Impact of Electronically Controlled Baluns: Measuring Insertion Loss of a Differential Balun

Testing the Performance Impact of Electronically Controlled Baluns: Measuring Insertion Loss of a Differential Balun

Web Based Control Interface for Modular Emulator Platform: A Browser-Based Solution for Remote Configuration on an Embedded Processor

Web Based Control Interface for Modular Emulator Platform: A Browser-Based Solution for Remote Configuration on an Embedded Processor

Environmental, Earth and Life Sciences

Environmental, Earth and Life Sciences

Analyzing Volumetric Neuroimaging: Using Datasets to Recognize Synaptic Patterns

Analyzing Volumetric Neuroimaging: Using Datasets to Recognize Synaptic Patterns

Is the underwater world 2D or 3D?: An acoustical modeling exploration

Is the underwater world 2D or 3D?: An acoustical modeling exploration

Propionic Acid Production in Pseudomonas Putida: Metabolic Engineering to Optimize Sustainability

Propionic Acid Production in Pseudomonas Putida: Metabolic Engineering to Optimize Sustainability

Sponge-Bacteria Coupled Biosensors for Marine Environments

Sponge-Bacteria Coupled Biosensors for Marine Environments

The Bioproduction of Epinephrine: Turning Amino Acids Into Life-Saving Medicine in Yeast

The Bioproduction of Epinephrine: Turning Amino Acids Into Life-Saving Medicine in Yeast

The Impact of Climate Change on the Spread of Diseases Carried by the Black-Legged Tick

The Impact of Climate Change on the Spread of Diseases Carried by the Black-Legged Tick

Math and Data Analysis

Math and Data Analysis

County-Level Risk Analysis for Opioid Overdose and Resource Allocation in Maryland

County-Level Risk Analysis for Opioid Overdose and Resource Allocation in Maryland

Injury-Specific Casualty Morbidity Modeling: Resolving Patient Progression by Body Region in Medical Mission Simulations

Injury-Specific Casualty Morbidity Modeling: Resolving Patient Progression by Body Region in Medical Mission Simulations

ISR Posture and Early Campaign Dynamics: Phase Attainment, Tempo, and Escalation Pathways; Project PACES -- Posture Analysis for Campaign Escalation and Success

ISR Posture and Early Campaign Dynamics: Phase Attainment, Tempo, and Escalation Pathways; Project PACES -- Posture Analysis for Campaign Escalation and Success

Lots-a-Lassa: A Mathematical Model for Lassa Fever Induced Sensorineural Hearing Loss

Lots-a-Lassa: A Mathematical Model for Lassa Fever Induced Sensorineural Hearing Loss

Modeling COA Uncertainty for Value Focused Thinking

Modeling COA Uncertainty for Value Focused Thinking

Predicting ICU Demand for Healthcare Operational Readiness: Analyzing Historic COVID ICU Data

Predicting ICU Demand for Healthcare Operational Readiness: Analyzing Historic COVID ICU Data

Probabilistic Undersea Search and Detection Modeling

Probabilistic Undersea Search and Detection Modeling

Physical Science

Physical Science

Analysis of Ocean Currents for Stochastic Modeling

Analysis of Ocean Currents for Stochastic Modeling

Detecting Atmospheric Gravity Waves the Not So EZIE Way: A Novel Method to Detect Atmospheric Gravity Waves by Proxy of the Zeeman Effect

Detecting Atmospheric Gravity Waves the Not So EZIE Way: A Novel Method to Detect Atmospheric Gravity Waves by Proxy of the Zeeman Effect

Fueling the Future: Investigating Space Contested Logistics by Modeling Orbital Transfers

Fueling the Future: Investigating Space Contested Logistics by Modeling Orbital Transfers

Neutron and Gamma Ray Spectroscopy for Homeland Protection and Materials Science

Neutron and Gamma Ray Spectroscopy for Homeland Protection and Materials Science

Tuning Hydrogels for Peripheral Nerve Tissue Engineering

Tuning Hydrogels for Peripheral Nerve Tissue Engineering

Quantum Plumbing: Fixing Leakage in Transmon Systems using Optimal Control

Quantum Plumbing: Fixing Leakage in Transmon Systems using Optimal Control

vFOX: Measuring Oxygen Concentration on Venus

vFOX: Measuring Oxygen Concentration on Venus

Programming and Computer Science

Programming and Computer Science

Building a Sim-to-Real Robotics Pipeline: Imitation Learning, Digital Twins, and Synthetic Training for the SO-101

Building a Sim-to-Real Robotics Pipeline: Imitation Learning, Digital Twins, and Synthetic Training for the SO-101

ChronoSync: Using Control Theory to Correct Real-Time Clock Error

ChronoSync: Using Control Theory to Correct Real-Time Clock Error

ELEET: Automating and Streamlining Data Entry

ELEET: Automating and Streamlining Data Entry

Exploring Recommender Systems through Agent Driven Text-Based Social Media Simulation

Exploring Recommender Systems through Agent Driven Text-Based Social Media Simulation

From Files to the APL Wikipedia: Populating the APL Wikipedia Using Confluence

From Files to the APL Wikipedia: Populating the APL Wikipedia Using Confluence

IMPRINT

Integrating Medical Devices and Robotics to Provide Medical Care

Integrating Medical Devices and Robotics to Provide Medical Care

Leaf Queuer: Interface System Test Scheduler

Leaf Queuer: Interface System Test Scheduler

OODA Sense: A Geography-Informed Multi-Agent Search and Rescue Drone Algorithm

OODA Sense: A Geography-Informed Multi-Agent Search and Rescue Drone Algorithm

Recording Mass Properties: Streamlining Data Management

Recording Mass Properties: Streamlining Data Management

Splat Sorcery: An APL Local Splatting Pipeline

Splat Sorcery: An APL Local Splatting Pipeline

IT and Cybersecurity

IT and Cybersecurity

Guided Exploration for One-way Function Families (GEOFF): Tokenization-Enhanced Reinforcement Learning for Red-Team Password Cracking

Guided Exploration for One-way Function Families (GEOFF): Tokenization-Enhanced Reinforcement Learning for Red-Team Password Cracking

Interactive Dashboard For System Management: Using Power BI To Improve Information System Tracking

Interactive Dashboard For System Management: Using Power BI To Improve Information System Tracking

Project SENTINEL: Cybersecurity Vulnerability Assessment & Risk Analysis

Project SENTINEL: Cybersecurity Vulnerability Assessment & Risk Analysis

STEM Communications and Education

STEM Communications and Education

The Code-Off: A Comparative Analysis of AI-Generated and Student-Written Programs

The Code-Off: A Comparative Analysis of AI-Generated and Student-Written Programs

The Cognitive Standard: A Rubric-Based Analysis of Critical Thinking in High School Mathematics

The Cognitive Standard: A Rubric-Based Analysis of Critical Thinking in High School Mathematics