Senior Data Engineer

o9 Solutions, Inc.; Bengaluru; Hybrid; INR 900,000 - 1,300,000; Full time

Job summary

o9 Solutions, Inc. in Bengaluru (Hybrid) is seeking a Senior Technical Consultant – RDAF/MFG to design and implement data ingestion, transformation, and integration pipelines. You will work with cross-functional teams to enable scalable data platforms and ensure data quality.

You will lead data engineering tasks, support production deployments, and mentor junior consultants while adhering to best practices and industry standards. Strong Python/PySpark/SQL and Airflow experience are required.

Qualifications

• 3+ years of experience in Data Architecture, Data Engineering, or related fields.
• Strong experience in data modelling, ETL processes, and cloud-based data platforms.
• Hands-on experience with Python, PySpark, SQL.
• Experience with workflow management tools such as Airflow and SSIS.
• Experience with Parquet, JSON, RESTful APIs, HDFS, Delta Lake, Hive, and Presto.
• Strong SQL expertise, including query writing and relational databases.
• Experience with GitHub or Azure DevOps.
• Familiarity with Agile development methodologies.
• Strong problem-solving and proactive learning mindset.
• Excellent verbal and written communication skills.

Responsibilities

• Manage and execute complex project tasks related to data ingestion, transformation, validation, and publishing.
• Review and analyze data provided by customers, understanding its technical and functional intent along with dependencies.
• Work closely with functional teams to understand end-to-end data flows and integration requirements.
• Build and maintain data ingestion and egress pipelines capable of handling large volumes of data.
• Develop data transformation functions using Python, PySpark, SQL, and SSIS.
• Integrate data from various sources including Teradata, SAP ERP, SQL Server, Oracle, Sybase, ODBC connectors, and flat files using API or batch integrations.
• Support production deployment activities.
• Assist in issue triaging, testing, and root cause analysis.
• Resolve batch automation disruptions to ensure customer SLA compliance and accurate data delivery.
• Provide technical guidance to junior consultants and review their code for best practices and quality.

Skills

Data Architecture

Data Engineering

ETL Processes

Cloud Data Platforms

Python

PySpark

SQL

Airflow

SSIS

Parquet

JSON

RESTful APIs

HDFS

Delta Lake

Hive

Presto

GitHub/Azure DevOps

Agile Methodologies

Problem-Solving

Communication

Tools

Airflow

SSIS

GitHub

Azure DevOps

Teradata

SAP ERP

SQL Server

Oracle

Job description

Role: Senior Technical Consultant (Data Engineer) - RDAF/MFG

Location: Bangalore, Hybrid

About the role...

We are looking for a Senior Technical Consultant – RDAF/MFG with strong experience in data engineering, ETL pipelines, and cloud-based data platforms. The role involves designing and implementing data ingestion and transformation pipelines, collaborating with functional teams, and ensuring seamless data integration across multiple systems.

What you will do in this role:

Advanced Execution & Data Management

• Manage and execute complex project tasks related to data ingestion, transformation, validation, and publishing.
• Review and analyze data provided by customers, understanding its technical and functional intent along with dependencies.
• Work closely with functional teams to understand end-to-end data flows and integration requirements.

Data Integration & Pipeline Development

• Build and maintain data ingestion and egress pipelines capable of handling large volumes of data.
• Develop data transformation functions using technologies such as Python, PySpark, SQL, and SSIS.
• Integrate data from various sources including Teradata, SAP ERP, SQL Server, Oracle, Sybase, ODBC connectors, and flat files using API or batch integrations.

Production Deployment & Hypercare

• Support production deployment activities.
• Assist in issue triaging, testing, and root cause analysis.
• Resolve batch automation disruptions in a timely manner to ensure customer SLA compliance and accurate data delivery.

Technical Leadership

• Provide technical guidance to junior consultants and review their code to ensure alignment with best practices and quality standards.
• Follow and promote o9 ways of working and industry standards for efficient project execution.

What you’ll have...

• 3+ years of experience in Data Architecture, Data Engineering, or related fields.
• Strong experience in data modelling, ETL processes, and cloud-based data platforms.
• Hands-on experience with: Python, PySpark, SQL
• Experience with workflow management tools such as Airflow and SSIS.
• Experience working with Parquet, JSON, RESTful APIs, HDFS, Delta Lake, and query frameworks such as Hive and Presto.
• Strong SQL expertise, including query writing and working with relational databases.
• Experience with version control platforms such as GitHub or Azure DevOps.
• Familiarity with Agile development methodologies.
• Strong problem-solving skills and a proactive learning mindset.
• Excellent verbal and written communication skills.

Good to Have

• Hands-on experience with Delta Lake.
• Experience working with Supply Chain Planning applications.
• Exposure to cloud platforms such as AWS, Azure, or Google Cloud.

What we’ll do for you:

• Flat organization: With a very strong entrepreneurial culture (and no corporate politics).
• Great people and unlimited fun at work.
• Possibility to really make a difference in a scale-up environment.
• Support network: Work with a team you can learn from every day.
• Diversity: We pride ourselves on our international working environment.
• DavOs Insight (Turning VUCA into value with Neuro-Symbolic AI): ngQoe
• Work-Life Balance: IHSZeUPATBA?feature=shared
• Feel part of A team: watch?v=O6B2lEJl2oI

Good luck!