Quest Global
From Quest Global's careers page
Status: listed on the employer's own careers page when we last checked (daily). Confirm on their site before applying.
Job Requirements
Design and develop robust data ingestion and ETL pipelines for batch and workflow-based processing requirements.
Build, optimize, and maintain data processing solutions using Apache Spark, PySpark, and Apache NiFi.
Develop, schedule, monitor, and manage workflows in Apache Airflow to ensure reliable orchestration.
Work on the Cloudera Data Engineering platform and the broader Hadoop ecosystem, including HDFS and Hive.
Write efficient Hive SQL and Python code for data transformation, validation, and operational automation.
Support, troubleshoot, and optimize production data pipelines for performance, reliability, and scalability.
Collaborate with data architects, analysts, platform teams, and business stakeholders to understand requirements and deliver data solutions.
Follow engineering best practices for code quality, version control, documentation, deployment, and monitoring.
Work Experience
8+ years of relevant experience in data engineering, big data platforms, and production pipeline delivery.
Strong hands-on experience in designing data pipelines using NiFi, Spark/PySpark, Hive SQL, and Python.
Good understanding of Hadoop concepts, HDFS storage patterns, data partitioning, and distributed processing.
Experience managing Airflow DAGs, dependencies, retries, monitoring, and failure handling.
Ability to identify pipeline bottlenecks and perform performance tuning across ingestion, transformation, and orchestration layers.
Strong problem-solving mindset with the ability to support critical production workloads.
A free account lets you apply, save this job, see how well you match and check your resume against it.
Scores your resume against this job the way resume scanners do: the job's hard and soft skills, searchability, recruiter tips and formatting.
Create a free account to see your score. Your resume stays in this browser until then, and we never store it.
Already a member? Sign in