AI Agent Data Pipeline Intern

Executive Summary

Build pipelines to ingest and organize experiment-related data from team communications, meeting notes, experiment plans, analysis documents, metrics, and evaluation results. Use LLM-based methods to clean noisy unstructured data, extract experiment-relevant information, and convert fragmented discussions into structured records. Design data schemas, metadata, and quality checks that make experiment context easier to search, trace, and use in downstream agent workflows. Support retrieval and indexing workflows, including semantic search or RAG-style pipelines, so the agent can access relevant experiment context. Prepare curated datasets for agent evaluation and, where applicable, LLM fine-tuning or instruction-tuning. Work with MLEs and platform engineers to understand experiment workflows, data gaps, and the types of insights most useful for planning and analysis. Evaluate whether the agent uses curated experiment data correctly to generate summaries, comparisons, recommendations, and analysis insights. Contribute to internal tools, dashboards, or reports that help teams monitor experiment status, outcomes, and trends. Strong skills in Python, SQL, and data processing. Experience working with structured and unstructured data, including text-heavy sources such as documents, notes, messages, or logs. Familiarity with data pipelines, ETL workflows, or large-scale data processing. Interest in LLM development, LLM evaluation, agentic AI systems, RAG pipelines, semantic retrieval, prompt engineering, or LLM-assisted data processing. Familiarity with machine learning workflows, model training, evaluation metrics, or MLOps concepts. Strong analytical thinking and attention to data quality, consistency, and reliability. Comfort working with ambiguous data sources and collaborating with ML and platform engineers to clarify requirements. Previous experience building internal tools, automation scripts, or data quality checks. A fun, supportive and engaging envi