Data Engineer

Apex Systems; Bengaluru Urban; On-site; INR 800,000 - 1,500,000; Full time

Job summary

A technology solutions provider is seeking a Mid-Senior level professional in Bengaluru to build and manage data pipelines using Snowflake and Databricks. This full-time position involves working with data scientists to convert raw data into structured products, ensuring alignment with business processes. Ideal candidates will have experience in Azure Data Factory and data modeling.

Responsibilities

• Build and manage data pipelines in Snowflake and Databricks using PySpark.
• Convert raw data into structured data products for analytics.
• Collaborate with stakeholders to fulfill data sourcing needs.

Job description

Responsibilities

• Build and manage data pipelines in Snowflake and Databricks using PySpark, Snowflake procedures and tasks, Azure Data Factory, and Datastage.
• Sourcing from Snowflake (classic system), transforming, writing back to Snowflake, and then ingesting to downstream Databricks structural foundation (AI/ML work, feature engineering) and Operational System (primarily modeling, in Angular, Tomcat, Postgres likely).
• Convert raw data into structured data products to support internal analytics and modeling.
• Work with data scientists and analysts to fulfill data sourcing needs and support experimentation.
• Format and wrangle data for modeling and inference tasks. Write data to and from Snowflake, and support rules-based or API-based integrations with downstream systems.
• Support a lead modeling use case by preparing and integrating analytical outputs into operational systems.
• Ensure data solutions are aligned with business processes and system requirements.
• Learn and maintain pipelines and data artifacts created in Azure Data Factory (ADF), IBM DataStage, and Snowflake stored procedures/tasks.
• Promote process adoption across the team (e.g., release/change management, documentation, resource procurement).
• Collaborate with downstream engineering teams (working on systems like Angular, Tomcat, PostgreSQL) to integrate ML outputs into business workflows.

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Consulting

Industries

• IT Services and IT Consulting