Sr. Data Engineer

Wissen; Pune District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Wissen Technology in Pune seeks a Sr. Data Engineer to build and optimize scalable data pipelines using Python, PySpark, and Databricks. The role emphasizes production-grade data processing with real-time streaming via Kafka and robust CI/CD integration.

The candidate should have 5–10 years of experience, a B.Tech/B.E. in CS or related field, and hands-on expertise with Azure Databricks, Kafka, and CI/CD tools.

Qualifications

• Candidates should have strong SQL and Python skills with hands-on PySpark experience.
• Experience building scalable data pipelines using PySpark, Databricks, and Kafka for real-time processing.
• Familiarity with CI/CD practices and production-grade data pipelines is required.

Responsibilities

• Design, develop, and optimize scalable data pipelines for production.
• Work with PySpark, Pandas, and SQL to transform raw data into clean, usable data sets.
• Implement and maintain data architectures using Azure Databricks and Apache Kafka.
• Ensure reliability and performance of streaming data pipelines in production environments.
• Collaborate with DevOps to manage CI/CD for data engineering workflows.
• Handle production data pipelines end-to-end, not just PoCs.

Skills

SQL

Python

PySpark

Streaming

Education

B.Tech/B.E. in Computer Science or related field

Tools

Azure Databricks

Apache Kafka

CI/CD (Jenkins)

Pandas

Job description

Sr. Data Engineer

At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our core philosophy is built around a strong product engineering mindset—ensuring every solution is architected and delivered right the first time. Today, Wissen Technology has a global footprint with 2000+ employees across offices in the US, UK, UAE, India, and Australia. Our commitment to excellence translates into delivering 2X impact compared to traditional service providers. How do we achieve this? Through a combination of deep domain knowledge, cutting‑edge technology expertise, and a relentless focus on quality. We don’t just meet expectations—we exceed them by ensuring faster time‑to‑market, reduced rework, and greater alignment with client objectives. We have a proven track record of building mission‑critical systems across industries, including financial services, healthcare, retail, manufacturing, and more. Wissen stands apart through its unique delivery models. Our outcome‑based projects ensure predictable costs and timelines, while our agile pods provide clients with the flexibility to adapt to their evolving business needs. Wissen leverages its thought leadership and technology prowess to drive superior business outcomes. Our success is powered by top‑tier talent. Our mission is clear: to be the partner of choice for building world‑class custom products that deliver exceptional impact—the first time, every time.

Job Summary

We are seeking a skilled Data Engineer with strong expertise in Python, PySpark, Databricks, and SQL to build, optimize, and maintain scalable data pipelines. The ideal candidate will have hands‑on experience with Azure Databricks, Apache Kafka, CI/CD practices, and production‑grade data pipeline development, ensuring reliable and high‑performance data processing solutions.

Experience

5–10 Years

Location

Pune

Mode of Work

Full Time

Education

B.Tech/M.Tech in Computer Science or related field

Key Responsibilities

• Candidate should be good with SQL base.
• Python, PySpark and Databricks come first in order of priority.
• Proficiency in PySpark, Pandas, and SQL.
• Experience with Azure Databricks.
• Strong hands‑on experience with Apache Kafka (real‑time streaming).
• Experience with workflow orchestration tools.
• Exposure to CI/CD tools such as Jenkins.
• Experience handling production data pipelines; candidates must have deployed data pipelines in production, not just a POC.

Required Skills

• Data observability, monitoring, or data quality frameworks.
• Streaming analytics or event‑driven architecture experience.
• DevOps or infrastructure‑as‑code exposure.