Security architecture: Design the security and compliance architecture with InfoSec: Unity Catalog access model, PII/NPI protection, encryption, audit logging and lineage.
Migration planning: Plan and sequence the migration of legacy Hadoop/Hive/EMR and warehouse workloads, including cutover strategy and risk mitigation.
Cost and business case: Build the cost and capacity model (DBU forecasting, FinOps guardrails) and the business case for platform investments.
Governance: Chair the architecture review board and maintain standards, reference patterns and decision records.
Vendors: Manage the relationship with the Databricks account team and cloud provider, and lead vendor evaluations and proofs of concept.
Must-Have Skills (candidate must meet all)
Experience: 12+ years in data engineering or architecture, with 5+ years on Databricks lakehouse platform.
Data platform fundamentals: Strong Spark, Delta Lake and SQL, plus data modeling (medallion, dimensional).
AWS architecture: Networking, IAM, encryption and multi-account design on AWS.
Secure design: Platform design for PII/NPI, including Unity Catalog access models and lineage.
Migration: Migration planning from legacy Hadoop/Hive/warehouse systems.
Streaming and batch design: Ability to choose between batch and streaming (Kafka, Structured Streaming) and design for both.
Resilience: Disaster recovery and high-availability design.
Cost modeling: FinOps and Databricks cost and capacity modeling, including a business case.
Communication: Stakeholder management, vendor management and clear architecture documentation.
Good-to-Have Skills (preferred, not required)
Banking, lending or other regulated financial services experience
Marketing data architecture (customer 360, consent, campaign measurement)
Collections data architecture (delinquency, agency and dialer feeds)
ML/MLOps architecture on Databricks (feature store, MLflow)
Databricks and AWS Solution Architect certifications
Screening Guide
Quick checks to shortlist candidates before the technical round
Has the candidate designed a full Databricks or lakehouse architecture (not only one component)?
Can they walk through a migration plan, including cutover and risk mitigation?
Have they designed the AWS network, IAM and multi-account setup for a data platform?
Can they show a cost model or business case they produced?
Have they chaired architecture reviews or managed vendor relationships?
Search keywords: Solution Architect, Databricks, lakehouse, Unity Catalog, AWS, multi-account, IAM, PII, NPI, medallion, Hadoop migration, FinOps, disaster recovery, Kafka, banking, architecture review board
How this role differs from its partner role
The Solution Architect owns the target-state architecture, cross-domain decisions, vendor relationships and the cost model. The Principal Engineer is the hands-on builder who delivers the detailed designs, pipelines and engineering standards within that architecture.
Oversee organizational data engineering solutions. Lead highimpact initiatives. Ensure alignment with organizational goals. Provide executive guidance.
Graduate in Computer Science, Data Science, or related field. 6-8 years of experience in data engineering or related field.
