Data Platform Architecture & Lakehouse Engineering
Design and implement scalable data warehouse and Lakehouse architectures utilizing Microsoft Fabric and Azure.
Architect and organize data structures within OneLake, Fabric Warehouse, and Lakehouse using a medallion (bronze, silver, gold) approach.
Develop and maintain ingestion and transformation pipelines using Data Factory in Fabric (pipelines, Copy jobs) and Spark/Python notebooks.
Ingest data from REST APIs and relational databases (MySQL/MariaDB, PostgreSQL/Redshift, SQL Server), including sources operating behind on-premises data gateways.
Design dimensional models and star schemas with SCD Type 2 history and conformed dimensions to support cross-functional reporting.
Centralize business logic in curated gold-layer tables to ensure semantic models remain consistent and feed near-real-time operational dashboards.
Performance Tuning & Capacity Optimization
Profile and tune Spark and SQL workloads to accelerate transformation run times via partitioning, file sizing, join strategies, and caching.
Redesign legacy full-reload processes into incremental and change-data-capture (CDC) pipelines to minimize compute overhead.
Monitor and manage Fabric capacity (CU) consumption with the Fabric Capacity Metrics app to prevent compute throttling during peak workloads.
Maintain Delta tables using OPTIMIZE, VACUUM, and V-Order to ensure fast read performance in Direct Lake and warehouse queries.
Audit existing pipelines, notebooks, and dataflows to eliminate redundant processing and consolidate duplicate logic.
Data Reliability, Observability & Stakeholder Collaboration
Implement automated freshness, volume, and completeness monitoring alongside proactive alerting on all production pipelines.
Reconcile loaded data against source systems to resolve ingestion gaps and anomalies at the root cause.
Execute safe historical data corrections and backfills with dry runs and validation prior to deploying to production.
Collaborate closely with vendor partners, internal business users, BI, and analytics teams to deliver cost-efficient, scalable data solutions.
Who You Are
We value adaptability, a solutions-oriented mindset, and deep technical curiosity:
Bachelor's degree in Computer Science, Data Science, Information Systems, or a related field (or equivalent professional experience).
7+ years of hands-on experience in Data Engineering or Data Platform Development, including direct production experience with Microsoft Fabric and Azure data technologies.
Proven ability to identify, decompose, and solve complex data challenges independently and as part of a team. You adapt smoothly to shifting priorities, take initiative in fast-paced environments, and maintain resilience when troubleshooting intricate system issues.
Strong proficiency in Python, PySpark, and advanced SQL. Proven track record of reducing run times and compute costs through workload optimization, incremental loading, and dimensional modeling (star schema, SCD Type 2).
Practical experience with Git-based development and deployment promotion workflows using Azure DevOps or GitHub.
Excellent communicator with the ability to bridge conversations across executive leadership (CFO, COO), vendor partners, and non-technical stakeholders alike.
Bonus Points (Great to Have)
Microsoft Fabric Data Engineer Associate (DP-700) certification.
Familiarity with Azure AI Foundry or Azure Machine Learning.
Experience leveraging AI-assisted coding tools to accelerate pipeline development.
Familiarity with secondary cloud ecosystems such as Google Cloud Platform (GCP).
Prior database administration (DBA) experience managing data security, redundancy, and high availability.
What We Offer
Comprehensive medical, dental, and vision insurance coverage options.
Paid vacation and paid company holidays throughout the year.
Lunch is on us every Friday!
A vibrant team culture featuring regular quarterly social events, team outings, and annual holiday parties.
401(k) retirement savings plan options to support your long-term financial health.
Direct visibility and reporting to executive leadership (CFO & COO), giving you the autonomy to architect and scale our enterprise data footprint.
We are an Equal Opportunity Employer and consider all qualified applicants without regard to race, color, religion (including religious dress and grooming practices), sex (including pregnancy, childbirth, breastfeeding, and related medical conditions), gender, gender identity, gender expression, sexual orientation, marital status, national origin, ancestry, age, physical or mental disability, medical condition, genetic information, military or veteran status, or any other status protected by applicable federal, state, or local laws. We are committed to providing reasonable accommodations and to maintaining a workplace free from discrimination and retaliation. #StaffAccountant #GeneralLedger #NetSuite #AccountingCareers #Big4
