Data Engineer

ShyftLabs; Coimbatore District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

ShyftLabs is seeking a Data Engineer in India to support production and development tasks for client engagements. The role requires 2–4 years of hands-on data engineering experience, with strong Python, SQL, and Spark skills.

You’ll balance pipeline support with building scalable data processes in cloud environments. Ideal candidates will have prior client-facing project exposure and the ability to optimize data workflows for performance and cost efficiency.

Qualifications

• 2-4 years of experience in a Data Engineering role.
• Proficiency in Python and SQL.
• Hands-on experience with Apache Spark.
• Experience working on client-based projects (mandatory).
• Ability to balance support and development responsibilities.
• Strong problem-solving and communication for client-facing work.

Responsibilities

• Provide day-to-day support (50%) for existing data pipelines, jobs, and platforms—monitoring, troubleshooting, and resolving issues to ensure smooth operations.
• Design, build, and maintain scalable data pipelines and ETL/ELT workflows using Python, SQL, and Spark.
• Collaborate with cross-functional and client teams to understand data requirements and translate them into technical solutions.
• Perform root-cause analysis on data/pipeline issues and implement fixes with minimal downtime.
• Optimize existing data workflows for performance, reliability, and cost-efficiency.
• Document processes, pipeline architecture, and runbooks for knowledge continuity.
• Participate in on-call/support rotations as needed for the client engagement.
• Work with Databricks and/or AWS cloud environments to build or support data solutions.

Skills

Python

SQL

Spark

Tools

Databricks

Airflow

CI/CD

AWS

Job description

Position Overview:

We are seeking a Data Engineer to join a client-focused engagement, with responsibilities split evenly between production support and technical/development work. This role requires 2-4 years of hands-on experience in data engineering, strong proficiency in Python, SQL, and Spark, and prior exposure to client-based project environments. The ideal candidate will be comfortable balancing operational support duties with building and optimizing data pipelines.

Job Responsibilities

• Provide day-to-day support (50%) for existing data pipelines, jobs, and platforms - monitoring, troubleshooting, and resolving issues to ensure smooth operations
• Design, build, and maintain (50%) scalable data pipelines and ETL/ELT workflowsusing Python, SQL, and Spark
• Collaborate with cross-functional and client teams to understand data requirementsand translate them into technical solutions
• Perform root-cause analysis on data/pipeline issues and implement fixes withminimal downtime
• Optimize existing data workflows for performance, reliability, and cost-efficiency
• Document processes, pipeline architecture, and support runbooks for knowledgecontinuity
• Participate in on-call/support rotations as needed for the client engagement
• Work with Databricks and/or AWS cloud environments where applicable to build orsupport data solutions

Basic Qualifications

• 2-4 years of experience in a Data Engineering role
• Strong proficiency in Python and SQL
• Hands-on experience with Apache Spark
• Prior experience working on client-based projects (mandatory)
• Ability to work across both support and development responsibilities
• Strong problem-solving and communication skills for client-facing situations

Preferred Skills

• Experience working with Databricks
• Familiarity with AWS Cloud services (e.g., S3, Glue, EMR, Lambda, Redshift)
• Exposure to CI/CD pipelines for data engineering workflows
• Experience with workflow orchestration tools (e.g., Airflow)

We are proud to offer a competitive salary alongside a strong insurance package. We pride ourselves on the growth of our employees, offering extensive learning and development resources.