Data Engineer

Cyberidentitysolutions; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Cyberidentitysolutions is seeking an experienced Data Engineer in Bengaluru to design and optimize data pipelines. The role focuses on building scalable batch and real-time architectures using Python, SQL, PySpark, and Databricks.

You will implement robust data models, ensure quality and governance, and collaborate across teams to integrate streaming and batch processes. The ideal candidate has 7+ years of data engineering experience, strong knowledge of Parquet/Delta Lake formats, and hands-on

Qualifications

• Expertise in Python and SQL for data engineering.
• Experience with PySpark or similar big data framework.
• Proficiency with Databricks or Spark-based pipelines.
• Orchestration with Airflow or equivalent.
• Experience with Parquet and Delta Lake formats.
• Data modeling and performance optimization.
• Kafka or similar streaming pipelines.
• Unified batch and real-time architectures.
• Docker, Git and CI/CD for data pipelines deployment.
• Data governance, MDM concepts, PII compliance in AI/ML contexts.

Skills

Python

SQL

PySpark

Databricks

Airflow

Parquet/Delta Lake

Data modeling

Kafka

Batch & real-time

Docker

Git

CI/CD

Data governance

Job description

Role: Data Engineer
Location: Bangalore

Experience- 7+ Years

REQUIRED SKILLS & EXPERTISE:

• Python and SQL for data engineering

• PySpark or equivalent big data processing framework

• Databricks or Spark-based platform for pipeline execution

• Airflow or equivalent for orchestration

• Parquet and Delta Lake (or equivalent columnar/transactional formats)

• Data modeling and performance optimization

• Kafka or equivalent for streaming pipelines

• Experience with unified batch and real-time architectures

• Docker, Git, and CI/CD practices for data pipeline engineering and deployment

• Data governance, MDM concepts, PII compliance, and responsible data handling in AI/ML contexts.