The role does not require deep specialization in a single data technology or ownership of enterprise architecture. Success requires solid engineering fundamentals, technical versatility, attention to data quality, and the ability to execute effectively within an architecture established by others.
Primary Responsibilities
Build and maintain enterprise data products according to architecture, data models, transformation rules, and technical specifications established by the Data Architect
Develop datasets and data pipelines across bronze, silver, and gold layers using modern medallion architecture principles
Develop complex SQL and PySpark transformations that integrate data from multiple source systems
Implement source-to-target mappings, transformation rules, business logic, joins, aggregations, and data-quality controls defined by the architecture and data-product design
Build scalable ETL/ELT pipelines that ingest, transform, validate, and publish enterprise data
Develop and maintain data pipelines, notebooks, jobs, and workflows using Databricks and Microsoft Azure technologies
Translate logical and physical data models into working datasets and production data structures
Implement incremental processing, historical data handling, reconciliation, deduplication, error handling, and restart/recovery capabilities
Optimize SQL, Spark jobs, pipelines, and storage structures for reliability, scalability, and performance
Perform unit testing, integration testing, data reconciliation, and data-quality validation before promoting data products into production
Troubleshoot data pipeline failures, incorrect transformations, data-quality issues, and performance problems
Participate in code reviews and technical design reviews and incorporate feedback from the Data Architect and senior engineering team
Follow established architecture standards, engineering patterns, naming conventions, security requirements, and development practices
Document data pipelines, transformations, dependencies, data mappings, and operational support procedures
Collaborate with Data Architects, analysts, Product Owners, Finance stakeholders, and other engineers throughout the data-product development lifecycle
Support deployment, monitoring, troubleshooting, and ongoing maintenance of production data products
Contribute reusable engineering patterns and components that improve development efficiency and consistency across data products
Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements
- Data Quality & Governance
- The Data Engineer should understand and implement data-quality and governance requirements as part of engineering delivery
Implementing data-quality rules defined within the data-product design
Validating completeness, accuracy, uniqueness, consistency, and referential integrity
Building reconciliation checks between source systems and target data products
Implementing appropriate logging and exception handling
Maintaining appropriate technical metadata
Supporting data lineage and traceability
Following established data security and access-control standards
Working within enterprise data-retention, privacy, and governance requirements
Required Qualifications
Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or a related discipline, or equivalent relevant professional experience
5+ years of experience in data engineering, data warehousing, software engineering, analytics engineering, or related data disciplines
3+ years of solid hands-on experience developing production data pipelines and datasets using modern cloud data technologies
Hands-on experience with Databricks or a comparable modern cloud data platform
Solid hands-on experience with SQL
Solid hands-on development experience with PySpark / Apache Spark
Experience implementing ETL/ELT pipelines
Experience integrating data from multiple source systems
Experience building and maintaining production-grade datasets
Experience with Microsoft Azure or comparable cloud infrastructure
Experience with Git-based source control and modern engineering-development practices
Understanding of medallion architecture and bronze, silver, and gold data layers
Demonstrated ability to work effectively from architecture specifications and established technical designs
Demonstrated solid debugging, analytical, and problem-solving skills
Software & Technical Tools
Data Engineering & Processing
Databricks: Hands-on experience developing notebooks, jobs, workflows, Delta tables, and production data pipelines
PySpark: Solid hands-on experience developing distributed data-transformation and processing workloads
Delta Lake: Experience building and managing reliable lakehouse datasets and understanding concepts such as ACID transactions, schema evolution, and incremental processing
Apache Spark: Good understanding of distributed data processing, partitioning, transformations, performance optimization, and troubleshooting
SQL: Solid proficiency developing complex queries, joins, CTEs, window functions, aggregations, transformations, reconciliation logic, and data-quality validations
Cloud & Data Platforms
Experience building cloud-based data solutions on Microsoft Azure
Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2)
Experience with Azure Data Factory or comparable orchestration technologies
Familiarity with Databricks Unity Catalog and modern data-governance capabilities
Data Architecture & Modeling
The Data Engineer is not expected to define enterprise architecture but should have sufficient architecture knowledge to correctly implement an established design
Knowledge of the following
Bronze, silver, and gold medallion architecture
Data Lake and Lakehouse concepts
Relational data modeling
Dimensional modeling concepts
Primary and foreign keys
Data grain and relationships
Fact and dimension structures
Historical data management
Slowly changing dimensions
Source-to-target mappings
Schema evolution
Batch and incremental processing
Multi-source integration
Data-quality and reconciliation patterns
The Data Engineer should be able to review data model or architecture specification and translate the design into working datasets, transformation logic, and production pipelines
ETL / ELT & Pipeline Engineering
Solid understanding and practical experience with
ETL and ELT development patterns
Batch data processing
Incremental and change-based data processing
Multi-source ingestion
Data transformation
Complex dataset joins
Data standardization and cleansing
Data reconciliation
Error handling and exception management
Data pipeline monitoring
Pipeline restart and recovery
Performance optimization
Production support and troubleshooting
Engineering & DevOps Tools
Experience with automated build, testing, and deployment practices
Experience following branching, peer-review, coding-standard, and release-management processes
Knowledge of Git for source control and collaborative development
Knowledge of Azure DevOps and/or GitHub for code repositories, pull requests, work management, and CI/CD
Familiarity with DataOps and modern software-engineering practices for data products
Demonstrated ability to diagnose production issues using platform logging, monitoring, and observability capabilities
Experience with Databricks Unity Catalog, Microsoft Purview, or comparable governance/catalog technologies is preferred.
Domain Knowledge
Domain expertise is secondary to solid data engineering capability for this role.
Experience working with Finance or healthcare datasets is preferred but not required.
Useful Finance experience includes exposure to
General Ledger and accounting data
Billing and Accounts Receivable
Accounts Payable
Revenue
Financial reporting
Customer and vendor data
Product and contract data
Financial hierarchies and Chart of Accounts
Transactional and financial master/reference data
Exposure to financial data from SAP S/4HANA, SAP ECC, Oracle ERP/eBusiness Suite, PeopleSoft, or comparable enterprise ERP platforms is beneficial
Preferred Qualifications
Experience with Databricks Lakehouse and Delta Lake
Experience with Microsoft Azure, ADLS Gen2, and Azure Data Factory
Experience with Databricks Unity Catalog
Experience working with high-volume enterprise datasets
Experience integrating data from enterprise ERP or financial applications
Experience operating in large, complex, or regulated enterprise environments
Snowflake experience
Experience with CDC and streaming/event-driven ingestion is a plus but is not a core requirement for this role unless needed by the assigned data product
Healthcare experience involving claims, billing, provider, member, clinical, or operational datasets
Finance data experience
Healthcare data experience
Understanding of CI/CD and DataOps practices
Leadership & Communication Skills
- This is primarily an execution-focused engineering role rather than an architecture or technical-leadership position.
- The Data Engineer will:
Work effectively from architecture, data models, and technical designs established by the Data Architect
Ask appropriate technical questions when requirements or designs require clarification
Communicate implementation challenges, dependencies, data-quality issues, and technical risks early
Collaborate closely with Data Architects and senior engineers during implementation
Participate constructively in design and code reviews
Provide realistic engineering estimates and communicate progress and delivery risks
Produce clear technical documentation
Work collaboratively across distributed and cross-functional teams
Take ownership of assigned engineering deliverables from development through production deployment
Escalate material deviations from the architecture rather than independently changing the established design
Contribute engineering observations and recommendations while respecting architecture-level design ownership
The Data Engineer should demonstrate solid technical ownership without requiring architecture ownership or formal people-management
At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.