Build templates, parameterized job definitions (ingestion/batch/streaming/SQL) with standardized logging, notifications, and governance.
Enforce run‑as service principals, job versioning, change auditability, and artifact immutability.
Automate provisioning via Terraform (Databricks provider), Databricks SDK (Python/Go), and REST API 2.1 for Jobs, DLT, Repos, SQL Warehouses, and Unity Catalog.
Unity Catalog – Automation, Governance & Migration
Catalog & Schema Provisioning (Automation‑first): Idempotent creation of catalogs, schemas, default locations, owners, grants, volumes, storage credentials, and external locations using Terraform/SDK/REST.
Grant‑as‑Code: Standardize RBAC with templates (OWNERSHIP, USE CATALOG/SCHEMA, SELECT, MODIFY, CREATE, READ/WRITE FILES); integrate SCIM groups and service principals.
Security & Audit: Dynamic views for row/column‑level security, data classification tags, audit logging, token policies, periodic permissions drift detection & remediation.
Delta Sharing: Automate providers/recipients, share objects, and scheduled access reviews/expiry.
Unity Catalog Migration (HMS → UC or multi‑workspace to UC): discovery, wave planning, external/managed table registration, permission model migration, dual‑run validation (row counts, schema parity, query replay), cutover & rollback.
Reliability & Data Quality
Define SLOs for pipeline success, latency, and data freshness; drive ≥99% T1 success and low MTTR.
Implement data quality checks (DLT expectations, dbt‑expectations/GX), idempotency, schema evolution strategy, checkpointing & replay.
Build incident response playbooks; lead RCAs and preventative actions.
Must Have Skills
5–8 years of experience in data engineering, platform engineering, or cloud infrastructure roles.
Strong hands-on experience with Databricks administration (workspace management, cluster policies, access controls, cost optimization).
Working knowledge of Databricks workflow development (job orchestration, scheduling, pipeline monitoring).
Proficiency in Terraform for infrastructure as code and automated provisioning.
Solid experience with AWS cloud services and architecture.
Good Python scripting skills for automation and tooling.
Good understanding of CI/CD practices and version control (Git).
Exposure to or experience implementing Agentic AI solutions, Genie spaces and apps primarily or anything similar.
Professional Skills
Strong problem-solving skills and ability to work in a collaborative, fast-paced environment.
Excellent English communication skills
Bridgenext is an Equal Opportunity Employer
#LI-AP1
#LI-REMOTE
