4-7 years of experience delivering enterprise software solutions
Proficient with Java, Spark, Scala, Python, AWS Cloud technologies
2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala
Flair for data, schema, data model, how to bring efficiency in big data related life cycle.
Experience with Agile Development methodologies.
Experience with data ingestion and transformation
Have understanding for secure application development methodologies.
Experience in with Airflow and Python will be preferred.
Understanding of automated QA needs related to Big data technology.
Strong object-oriented design and analysis skills
Excellent written and verbal communication skills
Responsibilities
Utilize your software engineering skills including Spark, Python, Scala to Analyze disparate, complex systems and collaboratively design new products and services
Integrate new data sources and tools
Implement scalable and reliable distributed data replication strategies
Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases
Perform analysis of large data sets using components in AWS Cloud
Own product features from the development, testing through to production deployment
Evaluate big data technologies and prototype solutions to improve our data processing architecture
Automate different pipelines
- Knowledge and hands-on experience of using AI technologies for development, Testing and automation.
- Knowledge of Cloud Technologies is a must such as AWS and its services