Build and maintain ETL/ELT pipelines using GCP services: BigQuery, Cloud Functions, Cloud Run, Cloud Composer, and DataForm.
Integrate with external APIs to extract and load data into BigQuery.
Develop event-driven architectures using Pub/Sub and Eventarc to trigger workflows and manage real-time data streams.
Automate deployment and testing workflows using Cloud Build and CI/CD pipelines.
Optimize performance and cost of data processing and storage across GCP services.
Collaborate with cross-functional teams to translate business requirements into scalable data solutions.
Ensure data security, governance, and compliance across all pipelines and storage layers.
Must have skills
1. 10 years of experience in data engineering, with at least 4+ years on Google Cloud Platform.
2. Strong proficiency in Python, including OOP principles, modular design, and clean coding standards.
3. Hands-on with GCP services: BigQuery, Cloud Functions, Cloud Run, Cloud Build, Dataform, Pub/Sub, Eventarc, Cloud Storage, and Cloud Composer.
4. Solid understanding of SQL, data modeling, and data warehousing concepts.
5. Familiarity with CI/CD pipelines, Git-based workflows etc.
6. Strong problem-solving skills and ability to work independently in a fast-paced environment.
Good to have skills
1. UNIX shell scripting, SnowFlake, Reshift, Familiar with NoSQL such as MongoDB, etc
2. ETL tool
3. Experience in Cloud platforms - AWS / Azure
4. Client-facing skills & Technical Team management
5.Experience with CI/CD pipelines
6. Requirement gathering
Location
DGS India - Pune - Indiqube Orchid
Brand
Merkle
Time Type
Full time
Contract Type
Permanent
