Azure Data Engineer

Inference Labs; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Inference Labs is seeking an experienced data engineer to design and implement data pipelines using Azure Data Factory and Databricks in Bengaluru. The role focuses on ETL, API integration, and large-scale data modeling.

The candidate should be proficient in SQL, Python/PySpark, and Azure data services, with hands-on experience in ADLS Gen2, Delta Lake, and Synapse. Collaboration and communication skills are essential.

Qualifications

• Proficient in SQL with data warehouse concepts.
• Experience with Azure data services in large-scale deployments.
• Strong Python and PySpark programming skills.

Responsibilities

• Excellent in SQL, ADF. A must.
• Build data pipelines in Azure Data Factory and call REST APIs using ADF.
• ETL and API data integration into ADLS Gen2/Azure Synapse Analytics and data migration.
• Ingest batch and streaming data with complex transformations.
• Handle semi-structured data (Parquet, JSON) and complex data types.
• Azure Synapse/data warehouse experience and data modeling.
• Azure Data Bricks experience writing in and using Databricks with Python.
• Azure Data Lake/Delta Lake, Blob Storage.
• Knowledge of data modeling, design, and optimization best practices.
• Work with very large volumes of log data to build analytical insights.

Skills

SQL

Databricks

Azure

Azure Data Factory

Azure Synapse

Azure Data Lake/Delta Lake

Blob Storage

Python

PySpark

REST API

Education

BE/BTech

BCA

MTech

MCA

Tools

Azure Data Factory

Azure Synapse

Azure Data Lake/Delta Lake

Blob Storage

Databricks

Job description

Job Experience:5-8 years

Responsibilities for the job ---

Technical (Primary) –

• Excellent in SQL, ADF. A must.
• Strong experience in building data pipelines in Azure Data Factory. Experience calling rest APIs using ADF
• ETL, API (Rest, OAuth, SOAP, etc) data integration into ADLS Gen2/ Azure Synapse Analytics and data migration.
• Experience in Ingestion of batch and Streaming data with complex transformations.
• Experience in handling Semi-structured data in various data formats (Parquet, JSON) and manipulating data in complex data types.
• Azure Synapse/data warehouse experience using synapse/data warehouse to present data securely and to build & manage data models.
• Azure Data Bricks experience writing in and using data bricks Using Python to transform, manipulate data.
• Azure Data Lake/ Delta Lake, Blob Storage
• Knowledge of data modeling, design, manipulation, optimization, and best practices.
• Experience in working with very large volume of log data and building analytical insights based on user requirements

Technical (Secondary) Skills:

• Knowledge of Dev-Ops tools and experience in building CI/CD pipelines on Azure.
• Write programs to pull data from External Applications and Services using REST API with different authentication methods.
• Knowledge on NoSQL Database types like Document and Graph DB is a plus.

Behavioral:

• Good Communication Skills Ability to work in a cooperative team environment

Eligibility Criteria for the Job

Education: -

B.E/B.Tech in any specialization, BCA, M.Tech in any specialization, MCA.

Primary Skill: -

1. SQL
2. Databricks
3. Cloud Azure

4. Azure Data Factory, Azure Synapse, Azure Data Lake/ Delta Lake, Blob Storage

5. Python, Pyspark

Management Skills: -

1. Ability to handle given tasks and projects simultaneously in an organized and timely manner.

Soft Skills: -

1. Good communication skills, verbal and written.
2. Attention to details.
3. Positive attitude and confidence.