Software Development Engineer

Mastek; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Mastek is seeking an experienced data engineering professional to design and maintain end-to-end data pipelines, leveraging Databricks and AWS for scalable analytics and BI use cases.

You will develop backend services with Python and Flask, implement containerization with Docker and orchestration via Kubernetes, and contribute to automated CI/CD pipelines for data solutions.

Qualifications

• Experience in data engineering and analytics ecosystems.
• Proficiency in Python and Flask for backend APIs.
• Hands-on with cloud-native architectures on AWS and data platforms.

Responsibilities

• Design, develop and maintain end-to-end data pipelines and ETL/ELT processes.
• Build cloud-native analytics solutions on AWS (S3, EKS) for scalable workloads.
• Develop backend services and APIs with Python and Flask for data access.

Tools

Python

Flask

Docker

Kubernetes

AWS

Databricks

CI/CD

Job description

Role & responsibilities

Key Responsibilities

• Design, develop, and maintain end-to-end data pipelines and ETL/ELT processes using Databricks and other data engineering tools to support analytics and BI use cases.
• Build and deploy cloud-native data and analytics solutions on AWS, leveraging services such as EKS, S3, and other relevant components for scalable and resilient architectures.
• Develop backend services and APIs using Python and Flask to enable data access, automation workflows, and integration with downstream applications and dashboards.
• Implement containerization and orchestration using Docker and Kubernetes (EKS preferred) to ensure reliable, portable, and scalable deployment of data and analytics workloads.
• Set up and maintain CI/CD pipelines using Jenkins/Jenkins Core to automate build, test, and deployment processes for data engineering and analytics solutions.
• Collaborate with data scientists, analysts, and BI developers to enable AI, machine learning, and advanced analytics use cases through well-structured, high-quality data.
• Optimize data models, queries, and pipelines for performance, cost efficiency, and reliability across large-scale datasets and complex analytics workloads.
• Implement data quality checks, monitoring, logging, and alerting to ensure data reliability, integrity, and timely availability for business stakeholders.
• Contribute to data automation initiatives by identifying opportunities to streamline manual processes and implementing robust automation solutions.
• Ensure adherence to best practices in cloud security, data governance, and compliance for all data engineering and analytics solutions.
• Support troubleshooting, root cause analysis, and performance tuning for data pipelines, cloud infrastructure, and analytics platforms.
• Mentor junior team members and contribute to knowledge sharing, coding standards, and best practices within the data engineering and analytics team.