Data Modelling: Expertise in data modelling techniques and tools, such as ER modelling, dimensional modelling, and UML (Unified Modelling Language). Design conceptual and logical data models and flowcharts, including source-to-target mappings, transforms, and data lineage.
Partner with Application Architects to design data-intensive applications with APIs and streaming data pipelines.
Design solutions to prepare and transform data into a usable state for multi-platform consumption.
Create data architecture, engineering and migration patterns for MSSQL, MYSQL, Oracle, IBM DB2 assisting data tier migrations with multiple technologies including No SQL/SQL, Relational, Analytical, Big Data, Data Stores, Data Warehousing, supporting Solera Technology Standards and Strategies (TSS) and Cloud Technology Mapping for software versions, licensing, cost savings and implementation in AWS
Key Responsibilities
Lead the design, development, and maintenance of our data engineering infrastructure
Architect and implement scalable data pipelines for batch and real-time processing
Establish best practices, coding standards, and technical documentation
Mentor and provide technical guidance to junior data engineers
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements
Optimize data models and database designs for performance and reliability
Implement data quality controls and monitoring systems
Evaluate and recommend new data technologies and tools
Deploy and manage cloud-based data solutions (AWS/Azure/GCP)
Ensure data security, compliance, and governance standards are met
What You’ll Bring
Bachelor's degree in Computer Science, Engineering, or related technical field (Master's preferred)
10+ years of experience in data engineering roles
2+ years of experience in a technical leadership position
Utilizing Real-Time, Batch, and NoSQL/SQL.
Proficiency in data warehousing, data modelling, ETL design, and optimization
Deep understanding of data enrichment, transformation, security, movement, data architecture, golden record, and data integrity.
Experience with big data technologies (e.g., Hadoop, Spark, Kafka)
Experience with data orchestration tools (Airflow, Luigi, etc.).
Familiarity with AWS migration services like Database Migration Service (DMS) and Server Migration Service (SMS).
Technical Skills
Programming Languages: Python, SQL, Java, Scala
Big Data Technologies: Apache Spark, Hadoop, Kafka, Hive
Any Cloud Platforms
AWS (S3, EC2, RDS, Lambda, DynamoDB, Glue)
Azure (Data Factory, Synapse, Databricks)
GCP (Big Query, Dataflow, Dataproc)
Data Warehousing: Snowflake, Redshift, Big Query, Fabric Data Warehouse
ETL/ELT Tools: Airflow, dbt, Informatica, Talend, Glue
Databases
Relational: PostgreSQL, MySQL, SQL Server, Oracle
NoSQL: MongoDB, Cassandra, DynamoDB
Time-series: Influx DB, Timescale DB
Container & Orchestration: Docker, Kubernetes
Infrastructure as Code: Terraform, CloudFormation
Version Control: Git, GitLab/GitHub workflows
CI/CD: Jenkins, GitHub Actions, GitLab CI
Monitoring & Observability: Datadog, Grafana
Data Architecture Skills
Data modeling (dimensional, relational, NoSQL)
Database design and optimization
Batch and real-time pipeline architecture
Data mesh/data fabric architectures
Data lake and Lakehouse implementation
API design and development
Microservices architecture
Leadership Skills
Experience leading data/analytics engineering teams
Ability to translate business requirements into technical solutions
Strong communication skills to present insights to stakeholders
Track record of improving data quality and analytics processes
Required Qualifications
Bachelor's degree in Computer Science, Engineering, or related technical field (Master's preferred)
Hands-on technical involvement
Experience with agile development practices
Strong problem-solving and debugging skills
Product Based Company Experience Mandatory