Multimodal retrieval and ranking for ads · Ph.D. in Computer Science, The City University of New York
I build vision-language systems that decide which images belong with which ads across the Microsoft Audience Network — retrieval, ranking and evaluation at the scale of billions of ads, shipped and measured in production.
Before Microsoft, I did a Ph.D. in computer vision at The City University of New York, working on action understanding in video, cross-modal retrieval and sign language recognition, with papers at CVPR and IEEE TPAMI.
Launched a multimodal pipeline that replaces generic stock images with the advertiser's own photos of each property, across billion-scale travel text ads. Ads are matched to properties through destination-URL entity resolution, and a fine-tuned scene classifier filters out unsuitable photos so that only representative hotel scenes are shown. Validated in an online A/B test.
Redesigned how images are chosen for text ads: landing-page content is embedded with a vision-language model and matched with DiskANN vector search, with a tuned retrieval-distance gate that suppresses poor matches. Built with LLM-as-judge evaluation in the loop; shipped through a backfill plus a daily pipeline and evaluated online.
Replaced random fallback images with imagery of what each merchant sells, for logo-less cashback ads at hundred-million scale. An offline pipeline classifies each ad into a fine-grained taxonomy with thousands of categories, retrieves candidate images per category via embedding retrieval, and shortlists them with a fine-tuned re-ranker. Validated in an online A/B test.
Built an embedding-based text-to-image and image-to-image retrieval system over the full ad corpus for incident triage, with automated reporting and mitigation workflows that cut investigation time from hours to minutes.
Earlier: Research Science Intern at Dataminr (fall 2021) and Data Science Intern at Expedia Group (summer 2021).
A pseudo-label-oriented transformer for weakly supervised action localization that requires only a single labeled point per action instance during training, achieving a 5% average mAP improvement over weakly supervised methods.
A comprehensive survey of deep learning methods for temporal action detection in untrimmed videos across supervision levels — fully, weakly, unsupervised and semi-supervised.
A cross-modal framework that maps images, meshes, and point clouds into a shared embedding space, trained end-to-end with a cross-modal center loss, improving cross-modal retrieval by 8% mAP on ModelNet40.
An educational tool that processes sign language students' video assignments and provides immediate feedback by detecting grammatically important nonmanual elements in continuous signing, recognizing five types of ASL grammatical errors with 60% recall.
Our solutions for the image-based vehicle re-identification and multi-camera vehicle tracking tracks of the AI City Challenge 2019; the re-identification framework improved on the prior state of the art on the VeRi dataset by 16.3%.
A multi-modal, multi-channel framework for real-time recognition of American Sign Language signs from RGB-D video.
A new dataset of color and depth videos of fluent ASL signers performing 100 signs, recorded with a Kinect v2 sensor.
An O(log n)-approximation algorithm for the NP-hard problem of scheduling deadline-constrained jobs on the minimum number of channels in a sensor network.
* Equal contribution.
Ph.D. in Computer Science, The City University of New York — defended December 2023.
Dissertation: Deep Learning-Based Human Action Understanding in Videos. Advised by Professor Yingli Tian at the Media Lab of The City University of New York; research spanning action detection in untrimmed videos, cross-modal retrieval, sign language understanding, time-series analysis and vehicle re-identification.
Reviewer for CVPR, AAAI, TMM, CVIU, MVAP, and JVCI.
Hiking, yoga, strength training, and reading.
The fastest way to reach me is email: ellie.vahdani@gmail.com. I'm also on LinkedIn, GitHub and Google Scholar.