What You Will do
Participate in architecture design and implementation of high-performance, scalable, and optimized data solutions.
Good Sql understanding with ability to create the data model from scratch.
Help write and optimize in-application SQL statements.
Ensure performance, security, and availability of databases.
Prepare documentations and specifications
Handle common database procedures such as upgrade, backup, recovery, migration, etc.
Profile server resource usage, and optimize and tweak as necessary
Design, build and automate the deployment of data pipelines and applications to support data scientists and researchers with their reporting and data requirements.
Integrate data from a wide variety of sources, including on premise databases and external data sources with rest APIs and harvesting tools.
Collaborate with internal business units and data science teams on business requirements, data access, processing/transformation and reporting needs and leverage existing and new tools to provide solutions.
Effectively support and partner with businesses on implementation, technical issues, and training on the datalake ecosystem.
Work with team on managing AWS resources (EMR, ECS clusters, etc.) and continuously improve deployment process of our applications
Work with administrative resources and support provisioning, monitoring, configuration, and maintenance of AWS tools.
Promote the integration of new cloud technologies and continuously evaluate new tools that will improve the organization’s capabilities while leading to lower total cost of operation.
Show the full description
Support automation efforts across the data analytics team utilizing Infrastructure as Code (IaC) using Terraform, Configuration Management, and Continuous Integration (CI) / Continuous Delivery (CD) tools such as Jenkins.
Work with the team to implement data governance, access control and identify and reduce security risks.
What you will need
7-9 years of experience in Solution, Design and Development of Cloud based data models, ETL Pipelines and infrastructure for reporting, analytics, and data science.
Experience working with both structured and unstructured data.
Strong proficiency with SQL and its variation among popular databases
Experience with some of the modern relational databases
Skilled at optimizing large complicated SQL statements
Knowledge of best practices when dealing with relational databases
Capable of configuring popular database engines and orchestrating clusters as necessary
Ability to plan resource requirements from high level specifications
Capable of troubleshooting common database issues.
Experience working with Spark, Glue, EMR, Apache Kafka/AWS Kinesis
Experience with version control tools (Git, Subversion)
Experience using automated build systems (CI/CD)
Experience working in different programming languages (Java, python, scala)
Experience of Data Structures and algorithms
Knowledge of different databases technologies (Relational, NoSQL, Graph, Document, Key-Value, Time Series, etc…). This should include building and managing scalable data models.
Knowledge of ML model deployment
Knowledge of Cloud based platforms (AWS)
Knowledge of TDD/BDD
Strong desire to improve upon their skills in software development, frameworks, and technologies