Data Engineer- SQL+PySpark

Fragma Data Systems; Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A technology company in India is seeking a Software Development Engineer II proficient in Python, PySpark, and SQL. The ideal candidate will have strong Big Data programming and ELT architecture skills. This full-time role requires excellent communication and analytical skills. The position is based in Bengaluru and aims to develop large-scale enterprise data solutions using Azure services.

Qualifications

• Good experience in PySpark, including DataFrame core functions and Spark SQL.
• Strong experience in SQL databases with the ability to write complex queries.
• Proven ability in Big Data programming for data transformation.

Skills

PySpark

SQL

Big Data programming

ELT architecture

Customer communication

Analytical skills

Tools

Azure Synapse

Azure SQL

Databricks

Job description

Base pay range

This range is provided by Fragma Data Systems. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Must-Have Skills

• Good experience in PySpark - Including DataFrame core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity
• Excellent experience in Big Data programming for data transformation and aggregations
• Strong ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication
• Good analytical skill

Technology Skills (Good To Have)

• Building and operationalizing large-scale enterprise data solutions and applications using one or more of Azure data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub
• Experience in migrating on-premise data warehouses to data platforms on Azure cloud
• Designing and implementing data engineering, ingestion, and transformation functions
• Azure Synapse or Azure SQL data warehouse
• Spark on Azure is available in HD Insights and Data Bricks

Skills: Python, SQL, Spark, Windows Azure and PySpark

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology

Referrals increase your chances of interviewing at Fragma Data Systems by 2x

Software Development Engineer II - Python, PySpark, SQL

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.