Responsibilities
Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Databricks, and Azure storage services.
Ingest data from multiple sources including relational databases, APIs, and file-based systems into Azure data platforms.
Implement data transformations on HPCC using ECL and on Azure platform using SparkSQL and PySpark in Azure Databricks.
Develop and optimize notebooks and jobs using Azure Databricks.
Write efficient Spark jobs for large-scale data processing.
Design and implement data models on HPCC using ECL and also using Delta Lake, Parquet, or other columnar storage formats.
Optimize ECL code, SQL, and Spark workloads for performance and scalability.
Work closely with business analysts, data scientists, and application teams to understand data requirements and promote data engineering best practices.
Requirements
Strong proficiency in SQL and data transformation logic with Solid understanding of data warehousing concepts and ETL/ELT patterns.
Hands-on experience with Azure Databricks, Spark SQL, and PySpark.
Experience with Azure Data Factory, Azure Data Lake Storage (ADLS Gen2), and Azure SQL / Synapse.
Experience working with large-scale, structured and semi-structured datasets.
Experience with Agile development methodologies.
Must have a positive attitude and desire to learn new processes and technologies.
Knowledge of SDLC process like requirement gathering, design, implementation (Coding), testing, deployment, and maintenance. Solid understanding of software development principles, design patterns, and best practices. Proficient in requirement gathering and documentation.
Excellent communication skills – verbal and written, Strong problem-solving skills with good knowledge of Insurance terminology.
Our Commitment to Inclusion
We welcome applications from individuals of all backgrounds, including underrepresented groups, people with disabilities, career changers, and candidates with non-traditional education or experience. If you require accommodations at any stage of the recruitment process, please let us know. We are committed to supporting your success.
Learn more about the LexisNexis Risk Solutions team and how we work
- 






- We know your well-being and happiness are key to a long and successful career. We are delighted to offer country specific benefits. Click here to access benefits specific to your location.
We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our Applicant Request Support Form or please contact 1-855-833-5120.
Criminals may pose as recruiters asking for money or personal information. We never request money or banking details from job applicants. Learn more about spotting and avoiding scams here.
Please read our Candidate Privacy Policy.
We are an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law.
USA Job Seekers
EEO Know Your Rights.
