Data Engineer

Talent Tribe Nexus; Chennai District; On-site; INR 1,200,000 - 1,700,000; Full time

Job summary

Talent Tribe Nexus is hiring a Data Engineer to build scalable data pipelines on AWS using PySpark, SQL and Python for analytics in the insurance domain.

You will develop ETL/ELT pipelines, implement real-time ingestion, and ensure data quality while collaborating with business and analytics teams across Chennai/Bangalore campuses.

Qualifications

• Hands-on experience with PySpark/Spark, Python and SQL in production environments.
• AWS services including S3, Glue, EMR, Redshift and Athena.
• Experience with Airflow and building scalable data pipelines.
• Knowledge of Parquet/ORC formats and partitioning strategies.
• Strong problem-solving and collaboration skills.
• Nice to have: Delta Lake/Iceberg/Hudi, Kafka/Kinesis streaming, data modeling, and insurance domain exposure.

Responsibilities

• Build and maintain ETL/ELT pipelines using PySpark, SQL and Python.
• Develop batch and near real-time ingestion processes.
• Implement incremental loads (CDC) and data quality checks.
• Work with AWS: S3, Glue, EMR, Redshift, Athena.
• Develop Airflow DAGs for orchestration and monitoring.
• Support CI/CD pipelines (CodeCommit/Bitbucket).
• Ensure data quality, validation and governance.
• Optimize Spark jobs and SQL queries for performance.
• Follow data security and governance practices.
• Collaborate with business and analytics teams to deliver insights.

Skills

PySpark

Spark

Python

SQL

AWS

S3

Glue

EMR

Redshift

Athena

Lambda

EC2

RDS/DynamoDB

Airflow

Parquet/ORC

partitioning

Data pipelines

Insurance domain exposure

Delta Lake

Kafka/Kinesis streaming

Data modeling

Stable career progression

Job description

Role: Data Engineer

Location: Chennai/Bangalore

Experience: 47 years

Role Summary:

Data Engineer to build and maintain scalable data pipelines on AWS using PySpark, SQL, and Python, supporting analytics use cases in the insurance domain.

Key Responsibilities:

• Build and maintain ETL/ELT pipelines (PySpark, SQL, Python)
• Develop batch & near real-time ingestion pipelines
• Implement incremental loads (CDC)
• Work with AWS: S3, Glue, EMR, Redshift, Athena
• Develop Airflow DAGs for orchestration
• Support CI/CD pipelines (CodeCommit/Bitbucket)
• Ensure data quality, validation, and monitoring
• Optimize Spark jobs and SQL queries
• Follow data governance and security practices
• Collaborate with business and analytics teams

Required Skills:

• Hands-on: PySpark/Spark, Python, SQL (intermediate–advanced)
• AWS (working knowledge):
• S3, Glue, EMR, Redshift, Athena
• Lambda, EC2, RDS/DynamoDB
• Experience with Airflow & data pipelines
• Understanding of Parquet/ORC, partitioning
• Strong problem-solving skills

Nice to Have:

• Exposure to Delta Lake / Iceberg / Hudi
• Basics of Kafka/Kinesis streaming
• Knowledge of data modeling
• Insurance domain exposure
• Stable career progression