Develop in-flight transformation logic leveraging Apache Flink, Kafka Streams, KSQLDB, and Kafka Single Message Transforms (SMTs).
Build and maintain scalable mapping frameworks for transforming source system data into harmonized enterprise data models.
Implement complex event processing, enrichment, filtering, aggregation, and windowing logic for streaming workloads.
Optimize streaming pipeline performance, ensuring low latency, high throughput, and efficient resource utilization.
Design solutions to handle late-arriving and out-of-order events using event-time processing, watermarks, checkpointing, and state management.
Integrate streaming applications with Kafka, Schema Registry, APIs, and downstream analytical platforms.
Implement robust error handling, retry mechanisms, dead-letter queues (DLQ), and monitoring for streaming applications.
Collaborate with Solution Architects, Data Architects, API teams, and business stakeholders to define transformation rules and streaming integration patterns.
Develop automated testing frameworks and CI/CD pipelines for streaming applications.
Monitor production streaming jobs and troubleshoot performance, scalability, and reliability issues.
Required Skills & Experience
6–10+ years of experience in Data Engineering with strong expertise in real-time streaming solutions.
Hands-on experience with Apache Flink for enterprise-scale stream processing.
Strong knowledge of Apache Kafka, Kafka Streams, KSQLDB, Kafka Connect, and Single Message Transforms (SMTs).
Experience designing event-driven data architectures and real-time data integration pipelines.
Strong understanding of event-time processing, watermarks, windowing, checkpointing, stateful stream processing, and fault tolerance.
Experience developing data transformation and harmonization frameworks for enterprise data platforms.
Proficiency in Java and/or Scala; Python knowledge is desirable.
Experience with Avro, Protobuf, JSON, Schema Registry, and serialization techniques.
Strong SQL skills and experience working with large-scale structured and semi-structured datasets.
Experience with cloud platforms such as Azure, AWS, or GCP is preferred.
Familiarity with CI/CD, Git, Docker, Kubernetes, and monitoring tools is an advantage.
Preferred Qualifications
Experience with Confluent Platform and enterprise Kafka deployments.
Exposure to modern data lakehouse platforms such as Databricks, Snowflake, or Delta Lake.
Knowledge of CDC patterns, API integration, and enterprise integration architectures.
Experience working in Agile/Scrum delivery environments.
Excellent analytical, problem-solving, and stakeholder communication skills.
Qualifications
Bachelors - Computer Science, Bachelors - Data Science, Bachelors - Statistics, Masters - Data Science
- Certifications
- Certified Data Management Professional - Riversand, Databricks Certified Associate Developer for Apache Spark 3.0 - Databricks Academy
Required Skills Apache Kafka, Apache Spark, API Integration, AWS Data Pipeline, Cloud Applications, Databricks Platform, Data ETL, Data Governance, Data Modeling, Data Security, Document Management, Metadata Management, Performance Optimizations, Python (Programming Language), Quality Assurance (QA), Snowflake (Platform), SQL Databases, Terraform (Software)
- Language Proficiency -
- English (OVERALL - Proficient - C2)
Additional Job Location -
- Master Skill List -
- Data Engineering
- Work Shift -
- Flex Time (India)
Why join Genpact?
Lead AI-powered transformation – Drive innovation and solve real-world business challenges that matter
Make an impact – Help global enterprises solve business challenges that matter
Accelerate your career – Gain hands-on experience, mentorship, and world-class learning opportunities to stay ahead
Work with the best – Join 140,000+ bold thinkers and problem-solvers who push boundaries every day
Thrive in a values-driven culture – Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress
Come join the 140,000+ coders, tech shapers, and growth makers at Genpact and take your career in the only direction that matters: Up.
Let’s build tomorrow together.
Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation.
Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a 'starter kit,' paying to apply, or purchasing equipment or training.