Data Engineer- Databricks

Omnicom Global Solutions; Hyderabad; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Omnicom Global Solutions is seeking a Data Engineer to design and test data pipelines in Databricks, handling ingestion, transformation, and Gold-layer modeling. The role targets early-to-mid career engineers who will learn rapidly by asking questions and raising blockers promptly.

The candidate should have 2–4 years of data engineering experience with Spark/PySpark and SQL, plus Databricks Delta Lake expertise and an understanding of lakehouse concepts.

Qualifications

• 2–4 years of data engineering experience with Spark/PySpark and SQL.
• Hands-on Databricks experience (Delta Lake, notebooks, jobs/workflows).
• Understanding of data warehousing/lakehouse concepts (Bronze/Silver/Gold).
• Experience in an agile team with daily standups and sprint delivery.
• Strong written and verbal English communication.

Responsibilities

• Build and test Databricks pipelines (PySpark/SQL, Delta Lake) for ingestion, transformation, and modeling tasks.
• Write and maintain data quality checks and tests for pipelines.
• Participate in daily standups, report progress, and flag blockers promptly.
• Follow data security model and coding/documentation standards set by leads.
• Support integration testing for Boomi-based data flows (e.g., JDE-Concur, JDE-UKG).
• Apply feedback from code reviews and document pipelines clearly for ownership transfer.

Skills

Data engineering experience

Spark / PySpark

SQL

Agile team working

English communication

Questioning and blockers handling

Tools

Databricks

Delta Lake

Notebooks

Jobs/Workflows

Boilerplate data testing

Job description

Overview

The Data Engineer builds and tests data pipelines in Databricks, covering ingestion, transformation, and Gold-layer modeling, against tasks assigned by the Lead Engineer at the start of each sprint and refined daily. This role suits an early-to-mid career engineer seeking rapid growth. The Data Engineer is expected to ask questions when requirements are unclear and to raise blockers promptly rather than delay progress.

Responsibilities

• Build and test Databricks pipelines (PySpark/SQL, Delta Lake) for ingestion, transformation, and modeling tasks assigned during sprint planning.


• Write and maintain data quality checks and tests for pipelines built.


• Participate in daily standups, reporting progress accurately, asking questions when requirements are unclear, and flagging blockers promptly.


• Follow the platform's data security model (RLS/CLS) and coding/documentation standards set by the Lead and Senior Engineers.


• Support integration testing and reconciliation checks for Boomi-based data flows (e.g., JDE-Concur, JDE-UKG) as assigned.


• Apply feedback from code reviews to subsequent work.


• Document pipelines clearly enough for another engineer to assume ownership.

Qualifications

Required Qualifications

• 2-4 years of data engineering experience, including hands-on work with Spark/PySpark and SQL.


• Hands-on Databricks experience (Delta Lake, notebooks, jobs/workflows).


• Working understanding of data warehousing/lakehouse concepts (Bronze/Silver/Gold or comparable layered modeling).


• Experience working in an agile team with daily standups and sprint-based delivery.


• Strong written and verbal English communication skills.

Preferred Qualifications

• Databricks Certified Data Engineer Associate.


• Exposure to financial or ERP source data — JD Edwards (JDE), EcoSys, or Microsoft Dynamics — or general ledger/cost data.


• Exposure to an EPC, construction, energy, or other asset-heavy industry.


• Familiarity with Unity Catalog-based security (RLS/CLS) or Power BI/Genie reporting on governed data.


• Exposure to Boomi or another iPaaS/integration middleware platform.

Professional Attributes:

• Asks questions early when a task is unclear rather than proceeding on assumptions.


• Demonstrates visible progress daily rather than only at sprint boundaries.


• Applies feedback from the Lead and Senior Engineers quickly and consistently.


• Seeks increasing ownership and more complex problems over time.