Senior Data Engineer

ESP Engineered; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

ESP Engineered is seeking a skilled Data Engineer based in Hyderabad, India. The role involves leading the design and optimization of large-scale cloud-based data pipelines and mentoring junior engineers. Candidates should have over 4 years of experience, proficiency in Python and SQL, and familiarity with cloud platforms like Azure and AWS. This position offers an exciting opportunity to collaborate closely with data teams and contribute to innovative data solutions.

Qualifications

• 4+ years’ experience as a data engineer or relevant role.
• Deep understanding of data modeling, ETL/ELT processes, and data warehousing.
• Experience with cloud data platforms.

Responsibilities

• Lead design and optimization of large-scale cloud-based data pipelines.
• Mentor junior engineers and establish best practices.
• Collaborate with data architects, analysts, and stakeholders.

Skills

Python

SQL

Apache Spark

Big Data technologies

ETL/ELT processes

DataLake

DeltaLake

Data warehouse concepts

JSON

Avro

Parquet

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Tools

Azure Data Factory

AWS Glue

GCP Dataflow

Snowflake

Databricks

Apache Airflow

Terraform

Job description

Lead the design and optimization of large-scale, cloud-based data pipelines and systems.

Develop infrastructure to collect, transform, combine, and publish/distribute customer data

Mentor junior engineers and contribute to best practices for data engineering standards.

Collaborate closely with data architects, analysts, and stakeholders to deliver reliable data solutions.

Implement and optimize ETL/ELT processes for performance and scalability.

Ensure data quality, integrity, and security across all environments.

Research and implement new tools, frameworks, and automation strategies to enhance productivity.

Support DevOps initiatives through CI/CD pipeline management and infrastructure-as-code practices.

Qualifications

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.

4+ years’ experience as a data engineer or relevant role

Understanding of Big Data technologies, DataLake, DeltaLake, Lakehouse architectures

Advanced proficiency in Python, SQL, and Apache Spark.

Deep understanding of data modeling, ETL/ELT processes, and data warehousing concepts.

Experience with cloud data platforms such as Azure Data Factory, AWS Glue, or GCP Dataflow.

Strong background in performance tuning and handling large-scale datasets.

Familiarity with version control, CI/CD pipelines, and agile development practices.

Proficient with handling different file formats: JSON, Avro, and Parquet

Knowledge of one or more database technologies (e.g. PostgreSQL, Redshift, Snowflake, NoSQL databases).

Preferred Qualifications

Cloud certification from a major provider (AWS, Azure, or GCP).

Hands‑on experience with Snowflake, Databricks, Apache Airflow, or Terraform.

Exposure to data governance, observability, and cost optimization frameworks.