Tech Lead - Data Engineer

Opus Technologies Inc.; India; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Opus Technologies Inc. in Pune is seeking an experienced Data Engineer to help build and optimize data pipelines for payments and financial services. You will work with Spark, Hadoop, and SQL to deliver scalable solutions and ensure data quality across platforms.

Ideal candidates have 7–9 years of hands-on data engineering experience, strong programming in Scala, Python, and SQL, and familiarity with Airflow, Prefect, or Luigi. Experience with NoSQL and data modelling is a plus.

Qualifications

• 7-9 years of hands-on data engineering experience.
• Experience with ETL/ELT frameworks and data pipelines.
• Strong programming in Scala, Python, SQL; Java is a plus and shell scripting is mandatory.
• Strong SQL and database optimization; NoSQL is a plus.
• Familiarity with orchestration tools such as Airflow, Prefect, Luigi, or similar.
• Experience with big data technologies (Hadoop/Spark) preferred.
• Knowledge of data modelling (star/snowflake schemas), dimensional design, and BI performance optimization.
• Understanding of data quality frameworks, metrics, test automation tools, and monitoring.

Skills

Scala

Python

SQL

Shell scripting

Java

Education

Bachelor's or Master's degree in CS/Engineering

Tools

Airflow

Prefect

Luigi

Apache Spark

Hadoop

Job description

• 7-9 years of hands-on experience in data engineering, preferably in the payments/financial services domain.
• Extensive experience with ETL/ELT frameworks and data pipeline tools.
• Strong programming skills in Scala, Python, SQL; experience with Java is a plus & shell scripting is mandatory.
• Strong SQL and database optimization skills; experience with NoSQL is a plus.
• Familiarity with orchestration tools such as Airflow, Prefect, Luigi, or similar.
• Experience working with big data technologies (Hadoop/Spark) preferred.
• Knowledge of data modelling (star/snowflake schemas), dimensional design, and BI performance optimization.
• Understanding of data quality frameworks, metrics, test automation tools, and monitoring.

Domain Knowledge

• Strong understanding of payments processing, transaction lifecycle, settlements, authorizations, disputes/refunds, and reconciliation flows.
• Experience with payment gateways, merchant acquiring, issuing, and banking interfaces is a plus.
• Familiarity with compliance and security standards relevant to payments (e.g., PCI-DSS).

Qualifications

• A bachelor's or master's degree in computer science, Engineering, or a related discipline.
• Ability to communicate effectively across multiple audiences, including firm-wide business units, senior leaders, associates and clients.
• Exceptional interpersonal skills, including teamwork, facilitation, and negotiation.
• Strong planning and organizational skills

Required Skills

• Apache Spark, Hadoop , shell scripting Python

Job Info

• Posting Date 01/19/2026, 12:31 PM
• Locations Maharashtra, Pune, Maharashtra, 411006, IN