Data Engineer II

PDI Technologies, Inc.; Chennai District; On-site; INR 1,200,000 - 2,100,000; Full time

Benefits offered by this job

Hybrid work model

Quarterly perks program

Continuous learning opportunities

Job summary

PDI Technologies is hiring a Data Engineer II to design and maintain scalable cloud-based data pipelines in AWS. You will enable analytics, reporting, and BI across the organization by integrating enterprise apps and ensuring data quality and performance.

You will work with stakeholders to translate requirements into scalable data solutions, supporting governance and automated CI/CD practices. Hybrid working arrangements and a competitive salary are offered.

Qualifications

• Bachelor's degree or equivalent experience in a related field.
• 2-4 years of data engineering or cloud data platform experience.
• Experience with AWS data services (S3, Glue, Redshift, AppFlow, Step Functions).
• Proficient in SQL, Python, and PySpark.

Responsibilities

• Design, develop, and maintain cloud-based data pipelines for analytics.
• Build and optimize ETL/ELT processes using AWS data services.
• Integrate Salesforce, NetSuite, and other SaaS apps into data platforms.
• Develop scalable data models to support BI initiatives.
• Monitor pipelines, troubleshoot data issues, and improve data quality.
• Collaborate with stakeholders to translate requirements into solutions.
• Support data governance with quality, metadata, and docs.
• Contribute to automation, CI/CD, and IaC practices.
• Participate in code reviews and promote best practices.

Skills

SQL

Python

PySpark

AWS

ETL/ELT

Data modeling

Git

CI/CD

Terraform

AI/ML for data eng

Education

Bachelor's degree in Computer Science or related

Tools

S3

Glue

Redshift

AppFlow

Step Functions

Job description

At PDI Technologies, we empower some of the world's leading convenience retail and petroleum brands with cutting-edge technology solutions that drive growth and operational efficiency.

By “Connecting Convenience” across the globe, we empower businesses to increase productivity, make more informed decisions, and engage faster with customers through loyalty programs, shopper insights, and unmatched real-time market intelligence via mobile applications, such as GasBuddy. We’re a global team committed to excellence, collaboration, and driving real impact. Explore our opportunities and become part of a company that values diversity, integrity, and growth.

Role Overview:

As a Data Engineer II, you will design, develop, and support scalable cloud-based data pipelines that enable analytics, reporting, and business intelligence across the organization. Working within AWS-native technologies, you will integrate enterprise applications into centralized data platforms while ensuring data quality, reliability, and performance. You will collaborate with business stakeholders and technical teams to translate reporting requirements into scalable data solutions that support informed business decisions.

Key Responsibilities

• Design, develop, and maintain cloud-based data pipelines supporting enterprise reporting and analytics.

• Build and optimize ETL/ELT processes using AWS-native data services and modern data engineering practices.

• Integrate enterprise source systems, including Salesforce, NetSuite, and other business applications, into centralized data platforms.

• Develop and maintain scalable data models supporting reporting and business intelligence initiatives.

• Monitor pipeline performance, troubleshoot data issues, and implement improvements to ensure data accuracy and reliability.

• Partner with business stakeholders to understand reporting requirements and translate them into technical solutions.

• Support data governance initiatives by maintaining data quality, metadata, and documentation.

• Contribute to automation, continuous integration, and infrastructure-as-code practices to improve development efficiency.

• Participate in code reviews and promote engineering best practices across the team.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field, or equivalent experience.

• 2-4 years of experience in data engineering or cloud data platform development.

• Experience developing data pipelines using AWS services such as S3, Glue, Redshift, AppFlow, and Step Functions.

• Experience working with SQL, Python, and PySpark.

• Understanding of dimensional modeling, data warehousing concepts, and modern data lake technologies.

• Experience integrating enterprise applications such as Salesforce, NetSuite, or similar SaaS platforms.

• Familiarity with Git, CI/CD pipelines, and Infrastructure as Code (Terraform preferred).

• Practical experience with applying AI/ML techniques to data engineering problems, such as automated data quality checks, anomaly detection, or intelligent pipeline monitoring

• Strong analytical and problem-solving skills.

• Excellent collaboration and communication skills.

Behavioral Competencies

• Ensures Accountability

• Manages Complexity

• Communicates Effectively

• Balances Stakeholders

• Collaborates Effectively

PDI is committed to offering a well-rounded benefits program, designed to support and care for you, and your family throughout your life and career. This includes a competitive salary, market-competitive benefits, and a quarterly perks program. We encourage a good work-life balance with ample time away [time away] and, where appropriate, hybrid working arrangements. Employees have access to continuous learning, professional certifications, and leadership development opportunities. Our global culture fosters diversity, inclusion, and values authenticity, trust, curiosity, and diversity of thought, ensuring a supportive environment for all.