Data Engineer

Synechron; Chennai District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Synechron is seeking a Data Engineer – PySpark & Cloudera CDP to join our data engineering team. The ideal candidate will design, develop, and maintain scalable data pipelines ensuring high data quality, reliability, and availability across the organization.

The candidate should have strong experience with PySpark, Apache Spark, CDP, Python, SQL and distributed data processing technologies. Cloud-native tools and modern data engineering practices are a plus.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Proven experience as a Data Engineer or Big Data Engineer.
• Strong hands-on experience with PySpark and Apache Spark.
• Strong programming experience in Python and SQL.
• Experience working with Cloudera Data Platform (CDP) or the Cloudera Hadoop ecosystem.
• Experience building and supporting scalable ETL/ELT data pipelines.

Responsibilities

• Design, develop, test, and maintain scalable data pipelines using PySpark, Apache Spark, Python, and SQL.
• Build and optimize ETL/ELT processes for structured and unstructured data.
• Develop data processing solutions using the Cloudera Data Platform and related big data technologies.
• Work with distributed storage and processing technologies such as HDFS, Hive, Impala, Kafka, and Spark.
• Implement data quality checks, validation rules, reconciliation processes, and monitoring capabilities.
• Collaborate with data architects, analysts, application teams, DevOps engineers, and business stakeholders.

Skills

PySpark

Apache Spark

Python

SQL

Git

CI/CD

Cloud platforms

Education

Bachelor's degree in CS/IT/Engineering

Tools

Cloudera CDP

HDFS

Hive

Impala

Kafka

Airflow

Job description

Synechron is a leading digital consulting firm with 17,000+ collaborative employees in 55+ global offices across 17+ countries.

From our solid financial services industry foundation, we have become a prominent global digital consulting firm for large financial services and technology firms.

With a key focus on trust, and in partnership with our clients, we’re leading modernization and digital optimization journeys with expertise that spans Consulting, Data, Design, Cloud and Engineering across various industries.

We can provide you with customized end-to-end solutions that drive business value.

We are looking for a highly skilled and motivated Data Engineer – PySpark & Cloudera CDP to join our data engineering team. The successful candidate will be responsible for designing, developing, and maintaining scalable data pipelines that ensure high data quality, reliability, and availability across the organization.

The ideal candidate will have strong experience with PySpark, Apache Spark, the Cloudera Data Platform (CDP), Python, SQL, and distributed data processing technologies. Experience with cloud-native tools and modern data engineering practices will be an added advantage.

Job Description:

Responsibilities:

• Design, develop, test, and maintain scalable data pipelines using PySpark, Apache Spark, Python, and SQL.
• Build and optimize ETL/ELT processes for structured and unstructured data.
• Develop data processing solutions using the Cloudera Data Platform and related big data technologies.
• Work with distributed storage and processing technologies such as HDFS, Hive, Impala, Kafka, and Spark.
• Implement data quality checks, validation rules, reconciliation processes, and monitoring capabilities.
• Optimize Spark jobs and data pipelines through efficient partitioning, joins, caching, resource utilization, and query optimization.
• Troubleshoot data pipeline failures, performance issues, data inconsistencies, and production incidents.
• Collaborate with data architects, analysts, application teams, DevOps engineers, and business stakeholders.
• Support data migrations, platform enhancements, and cloud-native data engineering initiatives.
• Implement version control, CI/CD, automated testing, and deployment practices for data engineering solutions.
• Maintain technical documentation, data lineage, operational runbooks, and support procedures.
• Follow data security, access-control, governance, and compliance standards.

Required Skills & Experience:

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Proven experience as a Data Engineer or Big Data Engineer.
• Strong hands‑on experience with PySpark and Apache Spark.
• Strong programming experience in Python and SQL.
• Experience working with Cloudera Data Platform (CDP) or the Cloudera Hadoop ecosystem.
• Experience building and supporting scalable ETL/ELT data pipelines.
• Good knowledge of distributed data processing, data storage, and big data architecture.
• Experience with technologies such as HDFS, Hive, Impala, Kafka, HBase, Airflow, or similar tools.
• Experience implementing data quality, validation, monitoring, and reconciliation processes.
• Strong understanding of Spark performance tuning and troubleshooting.
• Experience working with Git, CI/CD pipelines, and Agile delivery practices.
• Strong analytical, problem-solving, communication, and collaboration skills.

Preferred Skills:

• Experience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud.
• Knowledge of Spark SQL, Scala, Docker, Kubernetes, Jenkins, or Terraform.
• Experience with data governance, data lineage, metadata management, and data security.
• Exposure to modern data lake, data warehouse, or lakehouse architectures.
• Experience working in financial services, banking, fintech, or another regulated industry.

Awards:

2022: Voted in Top 25 Best Companies to Work for by the Business Intelligence Group

2021: Winner of the Gold Globee Award for Employer Excellence in Career Growth, Development & Training

2021: Synechron recognized with a Silver Level Award for COVID-19 Support Strategy

2020: Winner of the Team of the Year Award at the US FinTech Awards

2020: Synechron FinLabs won a Gold Stevie Award at the Asia-Pacific Stevie Awards

We are proud to be an equal opportunity employer. Our Diversity, Equity, and Inclusion initiative, Same Difference, is committed to fostering an inclusive culture that promotes equality, diversity, and respect.

We encourage applicants from diverse backgrounds, races, ethnicities, religions, ages, marital statuses, genders, sexual orientations, and abilities to apply.

We offer flexible workplace arrangements, mentoring, internal mobility, and learning and development programs to support our global workforce. Empowerment and collaboration are at the core of how we operate.