Optimize Snowflake performance through clustering, caching, micro-partitioning, and resource monitoring to ensure Data Products are delivered with reliability and speed.
Data Marketplace & Data Governance
Support the design and population of the Data Marketplace, ensuring Data Products are cataloged, discoverable, and compliant with enterprise data governance standards.
Define and enforce data contracts, SLAs, and quality standards for each Data Product published to the Marketplace.
Define and automate data validation, profiling, and quality checks embedded within ETL pipelines to guarantee Data Product integrity.
Collaborate with data governance teams to implement naming standards, lineage tracking, metadata management, and access policies for Data Products.
Ensure data lineage is traceable end-to-end from source systems through bronze/silver/gold layers to the final Data Product.
Collaboration & Delivery
Work closely with data analysts, data scientists, AI/ML teams, product teams, and business stakeholders to understand requirements and translate them into well-defined Data Products.
Partner with the Data Marketplace product owner to define the Data Product lifecycle (creation, certification, publication, deprecation).
Lead code reviews, provide mentorship, and support best practices across the engineering team.
Contribute to architectural decisions and participate in strategic data roadmap planning.
Automation & DevOps
Build CI/CD pipelines for Matillion job deployments and Snowflake object management.
Develop reusable ETL frameworks, templates, and patterns to accelerate the creation of new Data Products.
Implement monitoring, alerting, and operational excellence practices for data pipelines and Data Product health.
Automate Data Product certification workflows, including quality gates and approval processes.
Required Qualifications
5+ years of experience as a Data Engineer or similar role.
Strong expertise with Snowflake (SQL, performance tuning, data modeling, warehouse configuration, streams, tasks).
Hands-on experience with Matillion on AWS (or comparable ETL/ELT tools such as Talend, Informatica, dbt, Azure Data Factory).
Solid understanding of Data Product and Data Marketplace concepts (data contracts, data-as-a-product, self-service data consumption).
Advanced SQL skills; ability to write complex queries and optimize them.
Strong understanding of data modeling, dimensional models, star schemas, and data warehouse/data lake concepts (bronze/silver/gold architecture).
Experience with cloud platforms, specifically AWS.
Experience with Oracle Business Applications (ERP/EBS) as source systems is a plus.
Proficiency in Python, Airflow, or other orchestration tools.
Familiarity with Git, CI/CD, and general DevOps practices.
Knowledge of data governance principles, metadata management, and data cataloging tools.
Preferred Qualifications
Experience building or contributing to an enterprise Data Marketplace or Data Catalog.
Familiarity with data mesh or data product thinking methodologies.
Experience with near real-time data ingestion tools (e.g., OpenFlow, Kafka, CDC).
Exposure to AI/ML data preparation and delivering AI-enabled datasets.
Experience with data lineage and impact analysis tooling.
