Customer Technical Backlog Management: Own and actively manage the customer technical backlog—translating business epics into technical user stories, prioritizing technical debt, defining sprint milestones, and aligning backlog items with business ROI.
Modern Ingestion & Streaming: Design and implement data pipelines supporting both real-time and batch ingestion using modern orchestration (Cloud Composer, Azure Data Factory, Airflow) and streaming/messaging frameworks (Kafka, GCP Pub/Sub, Azure Event Hubs).
Governance & Lineage: Establish and enforce best practices for unified data cataloging, metadata management, automated lineage, and data quality across multi-cloud and on-prem systems using Dataplex, Microsoft Purview, or open-source governance tools.
Security & Compliance: Define and implement robust data security, identity management (IAM / Azure Entra ID), encryption, row/column-level access control, and regulatory compliance standards across clouds.
Team Technical Leadership & Mentorship: Steer the overarching technical direction of engineering teams; establish architectural standards, mentor senior and mid-level engineers, conduct architecture review, and ensure engineering excellence.
Hands-on Delivery & Prototyping: Balance strategic architecture discussions with hands-on solutioning, architectural spikes, Proof-of-Concepts (POCs), and performance tuning of complex multi-cloud distributed workloads.
- Cross-Functional Partnership: Partner with enterprise architects, delivery leads, DevOps engineers, and customer stakeholders to drive solution adoption, operational scalability, and long-term platform maintainability
- Technical Experience & Tooling
- Candidates should demonstrate hands-on architecture and design experience across the following domains:
Cloud Platforms: Deep expertise in both Google Cloud Platform (GCP) and Microsoft Azure enterprise data ecosystems.
Processing Engines: Apache Spark, PySpark, Databricks, Azure Synapse / Fabric Spark, GCP Dataproc.
Modern Migration & AI Automation: Building automated, agentic, or metadata-driven data migration pipelines, automated schema conversion, and data validation agents for on-prem to cloud lift-and-shift/re-platforming.
Messaging & Streaming: Apache Kafka, GCP Pub/Sub, Azure Event Hubs.
Orchestration: Apache Airflow, Google Cloud Composer, Azure Data Factory (ADF).
Analytical Storage & Warehouses: GCP BigQuery, Azure Synapse, Microsoft Fabric, Delta Lake/Parquet.
NoSQL / Cache Storage: MongoDB, Redis, Bigtable, Azure Cosmos DB.
Containerization & Deployment: Kubernetes (GKE and AKS), Cloud Run, Azure Container Apps.
API Frameworks: Python (FastAPI, Flask) for data service endpoints.
Databases & Querying: Advanced SQL validation and extraction from legacy/RDBMS systems (SQL Server, Oracle, PostgreSQL).
- CI/CD & Version Control: Git, GitHub, GitLab CI/CD, Azure DevOps, and Infrastructure as Code (Terraform).
- Required Qualifications & Skills
- 15+ years of progressive experience in Software Engineering, Data Engineering, and Enterprise Data Platform architecture.
5+ years architecting, migrating, and deploying production data platforms across GCP and/or Azure, with proven multi-cloud integration patterns.
Demonstrated track record leading on-premises/cloud to cloud modernization/migration initiatives, including data ingestion, transformation, automated data reconciliation, and cutover strategies.
Experience designing or integrating autonomous, agent-driven, or LLM-augmented data workflows (e.g., automated schema mapping, code conversion, data quality validation agents).
Proven hands-on experience designing and deploying Lakehouse platforms incorporating Medallion Architecture (Bronze/Silver/Gold) and Data Mesh / Data Product principles.
Deep knowledge of multi-cloud data governance, access management, and cataloging (Dataplex, Microsoft Purview, Unity Catalog).
Strong leadership capability in technical backlog management, agile scoping, sprint planning, and setting technical strategy for distributed teams.
- Exceptional stakeholder communication skills: capability to articulate architectural trade-offs to C-level executives as smoothly as reviewing code with engineers.
- Preferred Qualifications (Optional / Good to Have)
- Multi-cloud certifications: Google Cloud Professional Data Engineer / Cloud Architect and/or Microsoft Certified: Azure Data Engineer Associate / Azure Solutions Architect Expert.
Hands-on knowledge of Snowflake and Databricks
Familiarity with AWS data services (Amazon S3, Redshift, Glue, EMR).
- Experience with LangChain, LlamaIndex, or agentic framework integrations in data operations.
- Education
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or equivalent practical experience.
