Pyspark Data Engineer

Synechron; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Synechron is seeking an experienced Data Engineer (PySpark) to join our Data Engineering team. You will design, develop, and maintain large-scale data pipelines, data marts, and analytical solutions using Python, PySpark, SQL and data warehousing technologies.

You will participate in the complete SDLC, ensure data quality, and collaborate with cross-functional teams to deliver timely results in a dynamic environment.

Qualifications

• Experience designing, developing and maintaining large-scale data pipelines.
• Strong expertise in Python, PySpark, SQL and data warehousing.
• Hands-on SDLC experience and ability to work with SDLC processes.

Responsibilities

• Design, develop, and maintain scalable ETL pipelines and data processing frameworks.
• Build and support Data Mart solutions to meet business and analytical requirements.
• Develop high-quality, maintainable code using Python and PySpark.
• Participate in the full SDLC including requirement analysis, development, testing, UAT, and production deployment.
• Debug and optimize PySpark apps for performance and reliability.
• Write and optimize complex Oracle SQL queries.
• Work with structured, semi-structured, and unstructured data.
• Implement data quality, validation, and monitoring processes.
• Collaborate with cross-functional teams and adhere to CI/CD practices.

Skills

PySpark

Hadoop Ecosystem

MapReduce

Hive

ETL Development

Data Mart Development

Data Warehousing

Python

Tools

Jupyter Notebook

Oracle SQL

SQL

NoSQL Databases

Query Optimization

Data Analysis

Jenkins

Git

Job description

We are seeking an experienced Data Engineer (PySpark) to join our Data Engineering team. The ideal candidate will be responsible for designing, developing, and maintaining large-scale data pipelines, data marts, and analytical solutions. The role requires strong expertise in Python, PySpark, SQL, Data Warehousing, and Big Data technologies, along with hands‑on experience across the end‑to‑end Software Development Life Cycle (SDLC).

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines and data processing frameworks.
• Build and support Data Mart solutions to meet business and analytical requirements.
• Develop high-quality, maintainable, and efficient code using Python and PySpark.
• Participate in the complete Software Development Life Cycle (SDLC), including:
• Requirement Analysis
• Development
• Unit Testing
• UAT Support
• Production Deployment
• Perform complex data analysis and troubleshoot data-related issues.
• Debug and optimize PySpark applications for performance and reliability.
• Write and optimize complex Oracle SQL queries.
• Work with structured, semi-structured, and unstructured datasets.
• Implement data quality, validation, and monitoring processes.
• Collaborate with cross-functional teams to resolve dependencies and ensure timely project delivery.
• Follow software engineering best practices, coding standards, testing methodologies, and CI/CD processes.

Required Skills & Experience

• PySpark
• Hadoop Ecosystem
• MapReduce
• Hive
• ETL Development
• Data Mart Development
• Data Warehousing

Programming

• Python
• Jupyter Notebook

Database Technologies

• Oracle SQL
• SQL
• NoSQL Databases
• Query Optimization
• Data Analysis

Workflow & Automation

• Jenkins
• Git
• Data Cleansing
• Data Linking
• Data Validation and Quality Checks

Preferred Experience

• Exposure to production support and deployment activities.
• Experience working in Agile environments.

Desired Competencies

• Strong analytical and problem-solving skills.
• Excellent debugging and troubleshooting capabilities.
• Ability to lead technical initiatives with ownership and accountability.
• Strong collaboration and stakeholder management skills.
• Excellent verbal and written communication skills.
• Ability to communicate effectively with both technical and non-technical stakeholders.
• Ability to prioritize tasks and perform effectively in a fast-paced environment.