Data Engineer_Bangalore_7yrs_15 October Joiners Only_Hybrid_F2F

RiDiK; Bengaluru; Hybrid; INR 1,400,000 - 2,200,000; Full time

Benefits offered by this job

3 days WFO

Job summary

RiDiK is seeking a senior Data Engineer in Bengaluru to design, develop, and maintain scalable data pipelines and backend services using Python. You will build ETL/ELT workflows with PySpark and Spark SQL on Databricks and implement Delta Lake architectures for high-performance data platforms.

The role requires strong Python skills, Databricks expertise, advanced SQL, and cloud experience with Azure services and ADF.

Qualifications

• Strong programming experience in Python.
• Hands-on experience with PySpark and Spark SQL.
• Experience with Databricks and Delta Lake in data pipelines.
• Advanced SQL development and query optimization.
• Experience with Microsoft Azure services and Azure Data Factory (ADF).
• CI/CD implementation using Azure DevOps and DevOps practices.
• Strong understanding of data modeling, ETL/ELT concepts, and distributed data processing.
• Experience with large-scale data platforms and cloud-based solutions.

Responsibilities

• Design, develop, and maintain scalable data pipelines and backend services using Python.
• Build and optimize ETLELT workflows using PySpark, Spark SQL, and Azure Data Factory ADF.
• Develop and manage data ingestion, transformation, and processing solutions on Databricks.
• Implement Delta Lake architectures to support reliable, high-performance data platforms.
• Write efficient SQL queries for large datasets.
• Collaborate with stakeholders and cross-functional teams to deliver robust solutions.
• Work with Azure cloud services to deploy, monitor, and maintain data applications.
• Implement CI/CD pipelines and deployment automation using Azure DevOps.
• Troubleshoot, tune, and optimize data workloads.
• Adhere to coding standards, security guidelines, and best practices.
• Participate in design discussions, code reviews, and documentation.

Skills

Python

PySpark

Spark SQL

Databricks

Delta Lake

Advanced SQL

Azure Data Factory

Azure

CI/CD

DevOps

Data modeling

ETL/ELT

Distributed data processing

Data warehousing

Lakehouse

Data governance

REST APIs

Agile/Scrum

Monitoring tools

Education

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field

Tools

Azure DevOps

SQL

Git

Job description

Experience:7years

Loction: Banglore

3 days wfo

Responsibilities

• Design, develop, and maintain scalable data pipelines and backend services using Python.
• Build and optimize ETLELT workflows using PySpark, Spark SQL, and Azure Data Factory ADF.
• Develop and manage data ingestion, transformation, and processing solutions on Databricks.
• Implement and maintain Delta Lake architectures to support reliable and high-performance data platforms.
• Write efficient and optimized SQL queries for large-scale datasets.
• Collaborate with business stakeholders, architects, data analysts, and cross-functional teams to understand requirements and deliver robust solutions.
• Work with Azure cloud services to deploy, monitor, and maintain data applications.
• Implement CICD pipelines and deployment automation using Azure DevOps.
• Perform troubleshooting, performance tuning, and optimization of data workloads.
• Ensure adherence to coding standards, security guidelines, and best engineering practices.
• Participate in solution design discussions, code reviews, and technical documentation.

Required Skills

• Must-Have Technical Skills Strong programming experience in Python
• Hands-on experience with PySpark and Spark SQL
• Strong experience working with Databricks
• Good understanding of Delta Lake Advanced SQL development and query optimization skills
• Experience with Microsoft Azure services Hands-on experience with Azure Data Factory ADF
• Experience in CICD implementation using Azure
• DevOps Strong understanding of data modeling, ETL ELT concepts, and distributed data processing
• Experience working with large-scale data platforms and cloud-based solutions
• Preferred Skills Experience with data warehousing and lakehouse architectures
• Understanding of data governance and data quality frameworks
• Experience with REST APIs and backend service development
• Exposure to AgileScrum methodologies
• Knowledge of monitoring and observability tools

Soft Skills

• Excellent verbal and written communication skills
• Ability to interact confidently with business stakeholders and technical teams
• Strong analytical and problem-solving abilities Self-driven with a proactive approach to ownership and delivery
• Strong collaboration and teamwork mindsetAbility to clearly articulate technical concepts to both technical and non-technical audiences

Qualifications Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

6 years of relevant industry experience in Data Engineering, Data Platform Engineering, or Backend Python Development.