Exploring and implementing new tools and technologies to enhance ETL platform performance.
Functional Skills
Must-Have Skills
Proficient in SQL for extracting, transforming, and analyzing complex datasets from both relational and columnar data stores. Proven ability to optimize query performance on big data platforms.
Proficient in leveraging Python, PySpark, and Airflow to build scalable and efficient data ingestion, transformation, and loading processes.
Ability to learn new technologies quickly. Strong problem-solving and analytical skills. Excellent communication and teamwork skills.
Good-to-Have Skills
Experienced with SQL/NOSQL database, vector database for large language models
Experienced with data modeling and performance tuning for both OLAP and OLTP databases
Experienced with Apache Spark, Apache Airflow
Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops
Experienced with AWS, GCP or Azure cloud services
Education and Professional Certifications
5 to 9 years of experience and Bachelor’s degree in computer science and engineering preferred, other Engineering field is considered
AWS Certified Data Engineer preferred
Databricks Certificate preferred
Soft Skills
Excellent analytical and troubleshooting skills.
Strong verbal and written communication skills
Ability to work effectively with global, virtual teams
High degree of initiative and self-motivation.
Ability to manage multiple priorities successfully.
Team-oriented, with a focus on achieving team goals
Strong presentation and public speaking skills
.