AI Enablement: We help developers at ZipRecruiter leverage AI as a part of their tech stack by building and maintaining an agentic AI data exploration and analysis tool to democratize access to data, collaborate on insights, and accelerate data driven decision making. Starting from a natural language chat interface, the AI assisted tool assists with every aspect of using data, from finding the right data sources and writing queries to using them to visualize and analyze the results. We also manage an LLM Gateway which powers all of ZipRecruiter’s LLM usage in our customer facing products.
Key Focuses
As a Big Data Engineer in Tech Enablement, you will
Data Ingestion & Datalake Management: Design, build, deploy, and maintain the data pipelines and systems that make web events available in our datalake and via event streams for various downstream applications to process.
Big Data Processing: You'll tackle challenges with large, evolving datasets, using a mix of batch and streaming techniques to make critical trade-offs between speed, efficiency, and reliability.
MLOps: Design, build, deploy, and maintain frameworks and systems that accelerate ML development across teams and products at ZipRecruiter, with the goals of improving the velocity, quality, observability, and reliability of ML development and ML powered products.
Data Documentation Framework: Design, build, deploy, and maintain frameworks and systems to improve the comprehensiveness and quality of data documentation at ZipRecruiter through processes and tooling that allows both humans and automated processes to easily add, maintain, and discover documentation.
Data Analysis Agent: Design, build, deploy, and maintain an agentic data exploration and analysis tool, Sift, to democratize access to data, collaborate on insights, and accelerate data driven decision making.
Minimum Qualifications
4+ years of professional software development experience, including building and operating data pipelines that feed into a customer facing product
Hands-on Apache Spark experience: writing, tuning and debugging batch and/or streaming jobs at scale
Experience deploying and operating containerized services and batch jobs
Solid computer science fundamentals: data structures, algorithms, distributed systems, and testing
Professional software development experience, including building and operating data pipelines and infrastructure that feed into a customer facing product
Preferred Qualifications
4+ years of professional software development experience with a focus on data-intensive systems
Working knowledge of AWS (S3, IAM, EC2, MSK or equivalents) and of instrumenting jobs with metrics, logs and alerts
Experience with a data lake stack: parquet on object storage, a data catalog such as AWS Glue, and an open table format such as Apache Iceberg or Delta Lake
Streaming experience: Kafka producers and consumers, Apache Flink, and/or Spark Structured Streaming
Experience with a workflow orchestrator such as Apache Airflow: authoring DAGs, backfills, dependency management and on-call for scheduled pipelines
Experience with open table format internals: schema evolution, partitioning, and compaction
Experience with deploying services on Kubernetes with Docker
Experience building and maintaining a data platform that powers a user-facing product including infrastructure and tools for streaming and batch data ingestion, building ETL pipelines, data permissions management, and data discovery.
BS/MS/PhD in Computer Science, Mathematics, Physics or a related field, or equivalent practical experience
As part of our team you’ll enjoy
Competitive compensation
Exceptional benefits package
Flexible Vacation & Paid Time Off
Employer-matched 401(k) plan
The US base salary range for this full-time position is $190,000 - $250,000. Our salary ranges are determined by role, level, and location, and the range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location, role-related knowledge and skills, depth of experience, relevant education or training, and additional role-related considerations.
Depending on the position offered, equity, bonuses, commission, or other forms of compensation may also be provided as part of a total compensation package, in addition to a full range of medical, financial, and other benefits.
#LI-Remote
ZipRecruiter is proud to be an equal opportunity employer and provides equal employment opportunities (EEO) to all employees and applicants without regard to race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity or genetics.
Privacy Notice: For information about ZipRecruiter's collection and processing of job applicant personal data for this job, please see our Privacy Notice at: https://www.ziprecruiter.com/careers/job-applicant-privacy-notice
