Tech Lead - Data Engineer

Opus Technologies Inc.; Pune District; On-site; INR 2,400,000 - 4,200,000; Full time

Job summary

Opus Technologies Inc. in Pune is seeking a seasoned Data Engineer to design and own scalable data pipelines in the payments domain, leveraging Spark, Hadoop, and modern orchestration tools.

You will collaborate with product and security teams to ensure data quality, optimize BI performance, and help implement PCI-DSS aligned data practices. 7-9 years of experience and strong Scala, Python, SQL skills are required.

Qualifications

• 7–9 years of hands-on data engineering experience.
• Experience with ETL/ELT frameworks and data pipelines.
• Proficiency in Scala, Python, SQL; Java a plus; shell scripting mandatory.
• Strong SQL and database optimization; NoSQL a plus.
• Familiarity with Airflow, Prefect, Luigi.
• Big data techs (Hadoop/Spark) preferred.
• Knowledge of data modelling and BI performance optimization.
• Understanding of data quality, metrics, test automation, and monitoring.

Responsibilities

• Design and maintain scalable data pipelines for payments data.
• Ensure data quality and monitoring across platforms.
• Collaborate with cross-functional teams and stakeholders.
• Optimize BI reporting performance.
• Adhere to PCI-DSS and security standards in data handling.

Skills

Data engineering

ETL/ELT

Scala

Python

SQL

Java

Shell scripting

NoSQL

Airflow

Prefect

Luigi

Spark

Hadoop

Data modelling

BI optimization

Test automation

Monitoring

Education

Bachelor's or Master's in CS/Engineering

Tools

Apache Spark

Hadoop

Job description

• 7-9 years of hands-on experience in data engineering, preferably in the payments/financial services domain.
• Extensive experience with ETL/ELT frameworks and data pipeline tools.
• Strong programming skills in Scala, Python, SQL; experience with Java is a plus & shell scripting is mandatory.
• Strong SQL and database optimization skills; experience with NoSQL is a plus.
• Familiarity with orchestration tools such as Airflow, Prefect, Luigi, or similar.
• Experience working with big data technologies (Hadoop/Spark) preferred.
• Knowledge of data modelling (star/snowflake schemas), dimensional design, and BI performance optimization.
• Understanding of data quality frameworks, metrics, test automation tools, and monitoring.

Domain Knowledge

• Strong understanding of payments processing, transaction lifecycle, settlements, authorizations, disputes/refunds, and reconciliation flows.
• Experience with payment gateways, merchant acquiring, issuing, and banking interfaces is a plus.
• Familiarity with compliance and security standards relevant to payments (e.g., PCI-DSS).

Qualifications

• A bachelor's or master's degree in computer science, Engineering, or a related discipline.
• Ability to communicate effectively across multiple audiences, including firm-wide business units, senior leaders, associates and clients.
• Exceptional interpersonal skills, including teamwork, facilitation, and negotiation.
• Strong planning and organizational skills

Required Skills

• Apache Spark, Hadoop , shell scripting Python

Job Info

• Posting Date 01/19/2026, 12:31 PM
• Locations Maharashtra, Pune, Maharashtra, 411006, IN