Azure Data Engineer

Omnicom Global Solutions; Hyderabad, Chennai, Bengaluru; Ibrido; INR 1.200.000 - 2.100.000; Tempo pieno

Descrizione del lavoro

Omnicom Global Solutions in Hyderabad is seeking a data engineer to design, build, test and deploy scalable data systems that handle large volumes of data. You’ll collaborate with product owners and data scientists to build new data products and ensure data quality and reliability across pipelines.

The role requires strong Linux, Bash, Python and SQL skills, practical PySpark experience, and familiarity with cloud services in Azure.

Competenze

• Experience designing and managing data flows.
• Experience designing systems and APIs to integrate data into applications.
• 4+ years of Linux, Bash, Python, and SQL experience.
• 3+ years using PySpark and other frameworks to process large volumes of data.
• 3+ years using Parquet, ORC, or other columnar file formats.
• 3+ years of experience in Azure Databricks.
• 3+ years using Azure cloud services, esp. services that are used for data processing e.g. Glue, Dataflow, Data Factory, EMR, Dataproc, HDInsights , Athena, Redshift, BigQuery etc.

Mansioni

• Design, build, test and deploy scalable and reusable systems that handle large amounts of data.
• Collaborate with product owners and data scientists to build new data products.
• Ensure data quality and reliability.

Conoscenze

Data flow design

API data integration

Linux Bash Python SQL

PySpark

Parquet ORC formats

Azure Databricks

Azure cloud services

Strumenti

Azure Databricks

Azure Data Factory

HDInsight

EMR

Dataproc

Descrizione del lavoro

Role & responsibilities

• Design, build, test and deploy scalable and reusable systems that handle large amounts of data.
• Collaborate with product owners and data scientists to build new data products.
• Ensure data quality and reliability.

Preferred candidate profile

• Experience designing and managing data flows.
• Experience designing systems and APIs to integrate data into applications.
• 4+ years of Linux, Bash, Python, and SQL experience
• 3+ years using PySpark and other frameworks to process large volumes of data.
• 3+ years using Parquet, ORC, or other columnar file formats.
• 3+ years of experience in Azure Databricks
• 3+ years using Azure cloud services, esp. services that are used for data processing e.g. Glue, Dataflow, Data Factory, EMR, Dataproc, HDInsights , Athena, Redshift, BigQuery etc.

Passion for Technology: Excitement for new technology, bleeding edge applications, and a positive attitude towards solving real world challenges.