Data Engineer (Hadoop) - 6 months Contract | Banking

About the Role

As a Senior Data Engineer, you will focus on designing, building, and maintaining scalable data solutions that support business needs and AI applications. You will play a key role in:

1. Data Pipeline & Architecture Development
   • Build and optimize automated data pipelines for ingesting, transforming, and processing large datasets.
   • Design efficient data architectures that support analytics, machine learning, and real-time applications.
2. Cloud Migration & AI Enablement
   • Support cloud migration efforts, transitioning on-premises data workflows to cloud-based platforms like Databricks.
   • Collaborate with data scientists to improve feature selection, feature engineering, and enable end-to-end AI workflows from model training to deployment and monitoring.
3. CI/CD & Automation
   • Develop CI/CD pipelines to streamline data pipeline deployments and ensure stable, automated workflows.
   • Improve monitoring and observability to maintain system reliability.
4. Collaboration & Business Impact
   • Work with data scientists, product teams, and platform engineers to align data solutions with business objectives.
   • Ensure data quality, security, and compliance with industry standards.
   • Contribute to best practices in data governance, documentation, and automation.

Qualifications & Skills

• Degree holder of Information Technology, Mathematics or Statistics with at least 3 years of experience in data engineering.
• Expert in Python, Java, SQL, Linux Shell.
• Experience in UNIX environment, Git Flow, CI/CD automation, Jenkins, Bitbucket.
• Hands-on experience with Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), Spring Boot, etc. to build big data products & platforms.
• Proficiency with a modern cloud or hybrid-cloud stack (AWS, Databricks, Cloudera, etc).
• Experience in building and deploying production-level data-driven applications and data processing workflows or pipelines.