Senior Data Engineer

o9 Solutions, Inc.; Bengaluru; On-site; INR 3,200,000 - 5,200,000; Full time

Benefits offered by this job

Flat organization

International team

Job summary

o9 Solutions, Inc. is seeking a Senior Technical Consultant (Data Engineer) to design and implement robust data ingestion, transformation, and integration pipelines. You will work with Teradata, SAP ERP, SQL Server, Oracle, and other data sources, using Python, PySpark, SQL, and SSIS to enable scalable data platforms.

You will guide junior consultants, ensure best practices, and collaborate across teams to deliver reliable data solutions in a fast-paced environment.

Qualifications

• 3+ years in Data Architecture or Data Engineering.
• Strong SQL and data modelling skills.
• Experience with cloud-based data platforms and ETL pipelines.

Responsibilities

• Design and implement data ingestion and transformation pipelines.
• Review customer data and understand technical/functional intent and dependencies.
• Collaborate with functional teams on end-to-end data flows and integrations.
• Develop data ingestion and egress pipelines capable of handling large volumes of data.
• Build data transformation functions using Python, PySpark, SQL, and SSIS.
• Integrate data from Teradata, SAP ERP, SQL Server, Oracle, Sybase, and flat files via API or batch.
• Assist in issue triaging, testing, and root-cause analysis.
• Resolve batch automation disruptions to ensure SLA and accurate data delivery.

Skills

Data Architecture

Data Engineering

ETL processes

SQL

Python

PySpark

Airflow

SSIS

Data modelling

Problem-solving

Verbal & written communication

Tools

Airflow

SSIS

GitHub/Azure DevOps

Hive

Presto

Delta Lake

Job description

Role: Senior Technical Consultant (Data Engineer) - RDAF/MFG

About the role...

We are looking for a Senior Technical Consultant – RDAF/MFG with strong experience in data engineering, ETL pipelines, and cloud-based data platforms. The role involves designing and implementing data ingestion and transformation pipelines, collaborating with functional teams, and ensuring seamless data integration across multiple systems.

What you will do in this role:

Advanced Execution & Data Management

• Manage and execute complex project tasks related to data ingestion, transformation, validation, and publishing.
• Review and analyze data provided by customers, understanding its technical and functional intent along with dependencies.
• Work closely with functional teams to understand end-to-end data flows and integration requirements.
• Build and maintain data ingestion and egress pipelines capable of handling large volumes of data.
• Develop data transformation functions using technologies such as Python, PySpark, SQL, and SSIS.
• Integrate data from various sources including Teradata, SAP ERP, SQL Server, Oracle, Sybase, ODBC connectors, and flat files using API or batch integrations.
• Assist in issue triaging, testing, and root cause analysis.
• Resolve batch automation disruptions in a timely manner to ensure customer SLA compliance and accurate data delivery.

Technical Leadership

• Provide technical guidance to junior consultants and review their code to ensure alignment with best practices and quality standards.
• Follow and promote o9 ways of working and industry standards for efficient project execution.

What you’ll have...

• 3+ years of experience in Data Architecture, Data Engineering, or related fields.
• Strong experience in data modelling, ETL processes, and cloud-based data platforms.
• Hands-on experience with: Python, PySpark, SQL
• Experience with workflow management tools such as Airflow and SSIS.
• Experience working with Parquet, JSON, RESTful APIs, HDFS, Delta Lake, and query frameworks such as Hive and Presto.
• Strong SQL expertise, including query writing and working with relational databases.
• Experience with version control platforms such as GitHub or Azure DevOps.
• Familiarity with Agile development methodologies.
• Strong problem-solving skills and a proactive learning mindset.
• Excellent verbal and written communication skills.

Good to Have

• Hands-on experience with Delta Lake.
• Experience working with Supply Chain Planning applications.
• Exposure to cloud platforms such as AWS, Azure, or Google Cloud.

What we’ll do for you:

• Flat organization: With a very strong entrepreneurial culture (and no corporate politics).
• Great people and unlimited fun at work.
• Possibility to really make a difference in a scale-up environment.
• Support network: Work with a team you can learn from every day.
• Diversity: We pride ourselves on our international working environment.
• DavOs Insight (Turning VUCA into value with Neuro-Symbolic AI): https://l1nk.dev/ngQoe
• Feel part of A team: https://www.youtube.com/watch?v=O6B2lEJl2oI