AI Language Model Analyst

AI Language Model Analyst — Synthetix Labs

Seattle, WA | January 2024 – Present

• Refined prompt engineering architectures and system instructions for 6 production conversational models, decreasing user-reported model hallucinations by 28% and boosting overall sentiment score by 14%.
• Curated and cleaned training datasets containing 800,000+ instruction-tuning pairs, improving model task accuracy on complex reasoning evaluations by 18% based on MMLU benchmarks.
• Analyzed model output logs using custom automated evaluation frameworks, identifying and mitigating structural bias to reduce toxic output occurrences by 42% over a 6-month period.
• Coordinated with 8 cross-functional engineers to implement API routing algorithms, reducing average payload latency by 150ms and lowering overall operational API expenses by 31%.

NLP Data Analyst — Kortex Solutions

Bellevue, WA | July 2021 – December 2023

• Managed data annotation pipelines with 15 external annotators, verifying label consistency to achieve a 95% inter-annotator agreement rate across 200,000 labeled text samples.
• Built Python web scraping scripts to extract and preprocess 40GB of domain-specific textual data, accelerating pre-training database expansion speed by 35%.
• Conducted fine-tuning diagnostics on 3 BERT-based classification models, improving categorical precision from 81% to 93% on client-facing support tickets.
• Generated weekly data quality reports on model alignment metrics for engineering leadership, reducing dataset curation bottlenecks by 22% over a 12-month period.

Prompt engineering, Model evaluation, Benchmark design, Dataset curation, Sentiment analysis, Fine-tuning, RAG system evaluation, Python, SQL, PyTorch, Hugging Face, LangChain, LlamaIndex, Weights & Biases, Git

Automated Evaluation Framework

Role: Lead Analyst

Tools: Python, Hugging Face, Git, Weights & Biases

Created a custom automated benchmarker for internal LLMs, reducing manual model review time by 65% and tracking drift on 4 custom task domains.

RAG Pipeline Optimization

Role: AI Language Model Analyst

Tools: LangChain, LlamaIndex, OpenAI API, Qdrant

Evaluated and restructured retriever indexing pipelines, increasing retrieval accuracy by 24% and reducing context tokens by 18%.

• Google Cloud Professional Machine Learning Engineer (2024)
• AWS Certified AI Practitioner (2025)

• Languages: English (Native), French (Conversational)
• Volunteer Work: Technical advisor for non-profit open-source AI projects (2023-present)
• Availability: 2 weeks notice