Research Experience

Supporting production ML workflows for Falcon-H — TII's large-scale multi-modal model. My work spans internet-scale data acquisition, cleaning, and model alignment dataset generation.

• Architected ETL pipelines to crawl, deduplicate, and normalize internet-scale multi-modal datasets.
• Processed 3M+ PDFs via OCR, layout parsing, and CV-based structured text extraction.
• Synthesized large-scale SFT datasets using GPT-4, Gemini, Claude, and Qwen for Falcon-H alignment.
• Unified data from 10+ agent platforms into training-ready multi-modal corpora with quality-scoring pipelines.
• Built STEM-VQA evaluation pipelines covering charts, equations, and scientific plots.
• Trained VLMs on GCP and AWS across distributed Slurm clusters, optimizing GPU utilization.

Designed and contributed to VisRes Bench, a benchmark for systematically evaluating the visual reasoning capabilities of vision-language models across diverse and challenging tasks.

• Developed evaluation protocols and dataset curation pipelines targeting multi-step visual reasoning.
• Benchmarked a wide range of state-of-the-art VLMs, revealing systematic failure modes.
• Accepted at CVPR 2026 (Brigitta T., Dahou Y., Huynh N. D., et al.).

Investigated fundamental failure modes of VLMs on visually obvious reasoning tasks, contributing to a co-authored paper accepted at ICCV 2025.

• Designed probing tasks that expose systematic blind spots in leading VLMs.
• Developed and integrated ASR, VQA, OCR, and LLM inference components into unified end-to-end evaluation pipelines.

Designed and built SVLA, a unified assistant capable of jointly processing speech, images, and text in a single end-to-end framework.

• Integrated ASR, VQA, OCR, and LLM inference components into a cohesive multi-modal pipeline.
• Enabled the model to take audio and visual inputs simultaneously and generate coherent language responses.
• Published as a preprint on arXiv (arXiv:2503.24164, 2025).