Publications

Publications are listed according to their type and in reverse chronological order. For an updated list please check on Google Scholar. * denotes equal contribution

Journal Publications

2024

1. AISY

   Motion Enhanced Multi-Level Tracker (MEMTrack): A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

   Medha Sawhney, Bhas Karmarkar, Eric J. Leaman, Arka Daw, Anuj Karpatne, and Bahareh Behkam

   Advanced Intelligent Systems, 2024

   Abs Bib Website

   Tracking microrobots is challenging due to their minute size and high speed. In biomedical applications, this challenge is exacerbated by the dense surrounding environments with feature sizes and shapes comparable to microrobots. Herein, Motion Enhanced Multi-level Tracker (MEMTrack) is introduced for detecting and tracking microrobots in dense and low-contrast environments. Informed by the physics of microrobot motion, synthetic motion features for deep learning-based object detection and a modified Simple Online and Real-time Tracking (SORT)algorithm with interpolation are used for tracking. MEMTrack is trained and tested using bacterial micromotors in collagen (tissue phantom), achieving precision and recall of 76% and 51%, respectively. Compared to the state-of-the-art baseline models, MEMTrack provides a minimum of 2.6-fold higher precision with a reasonably high recall. MEMTrack’s generalizability to unseen (aqueous) media and its versatility in tracking microrobots of different shapes, sizes, and motion characteristics are shown. Finally, it is shown that MEMTrack localizes objects with a root-mean-square error of less than 1.84 μm and quantifies the average speed of all tested systems with no statistically significant difference from the laboriously produced manual tracking data. MEMTrack significantly advances microrobot localization and tracking in dense and low-contrast settings and can impact fundamental and translational microrobotic research.

   @article{msawhney2024memtrack,
     author = {Sawhney, Medha and Karmarkar, Bhas and Leaman, Eric J. and Daw, Arka and Karpatne, Anuj and Behkam, Bahareh},
     title = {Motion Enhanced Multi-Level Tracker (MEMTrack): A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments},
     journal = {Advanced Intelligent Systems},
     pages = {2300590},
     year = {2024},
     keywords = {bacteria, biohybrid microrobotics, collagen, computer vision, machine learning, multiobject tracking, object detection},
     doi = {https://doi.org/10.1002/aisy.202300590},
     url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/aisy.202300590},
     category = {Journal Publications},
     dimension = {true},
     publisher = {Wiley Online Library},
   }

Workshop Papers

2024

1. AAAI

   Are Pre-trained Vision Language Models (VLMs) Decent Zero-shot Predictors in Scientific Contexts?

   M. Maruf, Arka Daw, Medha Sawhney, Kazi Sajeed Mehrab, Mridul Khurana, Harish Babu, and Anuj Karpatne

   AAAI, 2024

   Oral Presentation in Imageomics Workshop at AAAI 2024

   Website

2023

1. 

   Detecting and Tracking Hard-to-Detect Bacteria in Dense Porous Backgrounds

   Medha Sawhney*, Bhas Karmarkar*, Eric Leaman, Arka Daw, Anuj Karpatne, and Bahareh Behkam

   CVPR, 2023

   Oral + Poster Presentation in CV4Animals Workshop at CVPR 2023

   Abs Poster Slides Website

   Studying bacteria motility is crucial to understanding and controlling biomedical and ecological phenomena involving bacteria. Tracking bacteria in complex environments such as polysaccharides (agar) or protein (collagen) hydrogels is a challenging task due to the lack of visually distinguishable features between bacteria and surrounding environment, making state-of-the-art methods for tracking easily recognizable objects such as pedestrians and cars unsuitable for this application. We propose a novel pipeline for detecting and tracking bacteria in bright-field microscopy videos involving bacteria in complex backgrounds. Our pipeline uses motion-based features and combines multiple models for detecting bacteria of varying difficulty levels. We apply multiple filters to prune false positive detections, and then use the SORT tracking algorithm with interpolation in case of missing detections. Our results demonstrate that our pipeline can accurately track hard-to-detect bacteria, achieving a high precision and recall.

Preprints

2023

1. 

   MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

   Medha Sawhney, Bhas Karmarkar, Eric J. Leaman, Arka Daw, Anuj Karpatne, and Bahareh Behkam

   arXiv, 2023

   Abs arXiv Bib Code

   Tracking microrobots is challenging, considering their minute size and high speed. As the field progresses towards developing microrobots for biomedical applications and conducting mechanistic studies in physiologically relevant media (e.g., collagen), this challenge is exacerbated by the dense surrounding environments with feature size and shape comparable to microrobots. Herein, we report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for detecting and tracking microrobots using synthetic motion features, deep learning-based object detection, and a modified Simple Online and Real-time Tracking (SORT) algorithm with interpolation for tracking. Our object detection approach combines different models based on the object’s motion pattern. We trained and validated our model using bacterial micro-motors in collagen (tissue phantom) and tested it in collagen and aqueous media. We demonstrate that MEMTrack accurately tracks even the most challenging bacteria missed by skilled human annotators, achieving precision and recall of 77% and 48% in collagen and 94% and 35% in liquid media, respectively. Moreover, we show that MEMTrack can quantify average bacteria speed with no statistically significant difference from the laboriously-produced manual tracking data. MEMTrack represents a significant contribution to microrobot localization and tracking, and opens the potential for vision-based deep learning approaches to microrobot control in dense and low-contrast settings. All source code for training and testing MEMTrack and reproducing the results of the paper have been made publicly available this https URL.

   @article{sawhney2023memtrack,
     title = {MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments},
     author = {Sawhney, Medha and Karmarkar, Bhas and Leaman, Eric J. and Daw, Arka and Karpatne, Anuj and Behkam, Bahareh},
     year = {2023},
     eprint = {2310.09441},
     archiveprefix = {arXiv},
     primaryclass = {cs.CV},
     category = {Preprints},
     dimension = {true},
     journal = {arXiv},
   }

2022

1. 

   Deep Learning Enabled Label-free Cell Force Computation in Deformable Fibrous Environments

   Abinash Padhi, Arka Daw, Medha Sawhney, Maahi M. Talukder, Atharva Agashe, Sohan Kale, Anuj Karpatne, and Amrinder S. Nain

   bioRxiv, 2022

   Abs bioRxiv Bib

   Through force exertion, cells actively engage with their immediate fibrous extracellular matrix (ECM) environment, causing dynamic remodeling of the environment and influencing cellular shape and contractility changes in a feedforward loop. Controlling cell shapes and quantifying the force-driven dynamic reciprocal interactions in a label-free setting is vital to understand cell behavior in fibrous environments but currently unavailable. Here, we introduce a force measurement platform termed crosshatch nanonet force microscopy (cNFM) that reveals new insights into cell shape-force coupling. Using a suspended crosshatch network of fibers capable of recovering in vivo cell shapes, we utilize deep learning methods to circumvent the fiduciary fluorescent markers required to recognize fiber intersections. Our method provides high fidelity computer reconstruction of different fiber architectures by automatically translating phase-contrast time-lapse images into synthetic fluorescent images. An inverse problem based on the nonlinear mechanics of fiber networks is formulated to match the network deformation and deformed fiber shapes to estimate the forces. We reveal an order-of-magnitude force changes associated with cell shape changes during migration, forces during cell-cell interactions and force changes as single mesenchymal stem cells undergo differentiation. Overall, deep learning methods are employed in detecting and tracking highly compliant backgrounds to develop an automatic and label-free force measurement platform to describe cell shape-force coupling in fibrous environments that cells would likely interact with in vivo.Competing Interest StatementThe authors have declared no competing interest.

   @article{Padhi2022.10.24.513423,
     author = {Padhi, Abinash and Daw, Arka and Sawhney, Medha and Talukder, Maahi M. and Agashe, Atharva and Kale, Sohan and Karpatne, Anuj and Nain, Amrinder S.},
     title = {Deep Learning Enabled Label-free Cell Force Computation in Deformable Fibrous Environments},
     elocation-id = {2022.10.24.513423},
     year = {2022},
     doi = {10.1101/2022.10.24.513423},
     publisher = {Cold Spring Harbor Laboratory},
     url = {https://www.biorxiv.org/content/early/2022/10/25/2022.10.24.513423},
     journal = {bioRxiv},
     category = {Preprints},
     biorxiv = {10.1101/2022.10.24.513423v1},
     dimension = {true},
   }