MEANINGFUL WORK AND PERSONAL IMPACT
As a Senior Principal AI Data Engineer, you will turn raw, multi-source data into reliable, high-performance inputs that directly power AI models and advanced analytics. Your work will make it faster and easier for teams across engineering, analytics, and products to develop, deploy, and improve AI capabilities by ensuring datasets are ready and accessible.
JOB DUTIES (INCLUDE BUT ARE NOT LIMITED TO)
Design, develop, and implement scalable data pipelines and ETL processes using Apache Airflow, with a focus on data for AI
Build and tune search and retrieval capabilities using ElasticSearch/OpenSearch, including indexing strategies, schema mappings, and relevance/performance optimization.
Enable low-latency retrieval for AI inference and RAG applications by optimizing data access patterns, caching approaches, and index refresh strategies.
Collaborate with analytic teams to define requirements, schemas, and interfaces for downstream consumption.
Use Git for version control, peer code reviews, CI/CD workflows, and reproducible pipeline deployments across environments.
Operate within Linux environments and perform performance tuning across pipeline components, storage layers, and compute resources.
REQUIRED SKILLS
Experience with Apache Airflow for workflow orchestration.
Strong programming skills in Python.
Experience with ElasticSearch/OpenSearch for data indexing and search functionalities.
Understanding of vector databases, embedding models, and vector search for AI applications.
Expertise in event-driven architecture and microservices development.
Hands-on experience with cloud services (e.g. MinIO), including data storage and compute resources.
Strong understanding of data pipeline orchestration and workflow automation.
Working knowledge of Linux environments and database optimization techniques.
Strong understanding of version control with Git.
WHAT YOU’LL NEED TO SUCCEED
Clearance: Current TS/SCI Clearance with current or willingness to obtain CI polygraph