Company
Qualcomm India Private Limited
Job Area
Information Technology Group, Information Technology Group > IT Data Engineer
General Summary
The Databricks Architect will be responsible for technical decision-making, code quality, and the successful delivery of complex data projects using Delta Lake, Unity Catalog, and Mosaic AI. The Databricks Architect/Lead owns the end-to-end strategy for data ingestion, storage, and analytics and serve as a liaison between business stakeholders and engineering teams, ensuring that the platform is secure, cost-optimized, and capable of supporting advanced AI/ML workloads.
Key Responsibilities
- Lakehouse Design: Architect end-to-end data solutions using the Medallion Architecture (Bronze/Silver) and Delta Lake.
- Platform Governance: Implement and manage Unity Catalog for centralized data governance, fine-grained access control, and lineage tracking.
- Technical Leadership: Lead and mentor cross-functional teams of data engineers; conduct code reviews and define engineering best practices in Python, PySpark, and SQL.
- Optimization & Performance: Drive cost-efficiency by tuning Spark jobs, managing cluster policies, and leveraging serverless compute and auto-scaling.
- DevOps & DataOps Integration: Integrate Databricks workflows into CI/CD pipelines using tools like Databricks Asset Bundles, Terraform, and Azure DevOps or GitHub.
- AI/ML Support: Provide architectural guidance for integrating machine learning models, including feature engineering and model deployment via MLflow.
- Deep hands-on technical understanding in building complex data models for supporting various analytics needs
- Comfortable in working with cross-functional teams such as Data Engineering, Analytics, SMEs and Business stakeholders
- Passion to understand complex existing footprint of data solutions and build out the strategy for migrating to Databricks medallion architecture
- Accountability and Ownership of the delivery in driving the projects to completion
Show the full description
- 14+ years of experience in data management, data engineering, or data science roles, with at least 7 years in building enterprise level data models as part of data modernization and DW migration efforts.
- 8+ years of work experience with Data Structures and algorithms
- 8+ years of work experience with programming (e.g., Java, Python)
- 8+ years of work experience with SQL or NoSQL Databases
- 3+ years of work experience in a role requiring interaction with senior leadership, leading modernization of Data platforms.
- Bachelor's or Master's degree in Computer Science, Data Science, Information Technology, or a related field.
- Strong understanding and hands-on experience with lakehouse medallion data architecture and govenance
- Proven implementation experience with migrating legacy DW data models to 3-layer medallion architecture.
- Strong passion in exploring GenAI based solutions for documenting the existing DW footprint and build out strategic data model for Databricks platform
- Experience with machine learning, artificial intelligence, and predictive analytics is a plus.
- Excellent leadership, communication, and collaboration skills in a global hybrid delivery model