Data Engineer - Python

IntraEdge; Bengaluru; On-site; INR 600,000 - 1,200,000; Full time

Job summary

A leading technology company in Bengaluru is seeking a skilled Data Engineer. Responsibilities include designing and maintaining data pipelines, ensuring data quality, and optimizing performance using Python and SQL. The ideal candidate will work with cloud platforms and collaborate closely with data teams. A bachelor's or master's degree in a relevant field is required, along with strong analytical skills and experience with ETL tools. This position offers competitive compensation and opportunities for professional growth.

Qualifications

• Strong proficiency in Python (Pandas, NumPy, PySpark preferred).
• Advanced SQL skills (complex joins, CTEs, window functions, query optimization).
• Experience with relational databases (PostgreSQL, MySQL, SQL Server).
• Hands-on experience with ETL tools (Airflow, dbt, Talend).
• Knowledge of data modeling (star schema, snowflake schema).

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Ensure data quality, integrity, and security across systems.
• Optimize database performance and troubleshoot data-related issues.
• Collaborate with stakeholders to gather and translate business requirements into technical solutions.
• Work with cloud platforms (AWS/GCP/Azure) for data processing and storage.

Skills

Python

SQL

Data pipeline design

ETL/ELT processes

Data modeling

Cloud platforms (AWS/GCP/Azure)

Data quality assurance

CI/CD pipelines

Git

Analytical skills

Education

Bachelor’s or master’s degree in computer science, Engineering, Information Systems

Tools

PostgreSQL

MySQL

SQL Server

Snowflake

Redshift

Big Query

Airflow

Docker

Kubernetes

Spark

Job description

We are looking for a skilled Data Engineer with strong expertise in Python and SQL to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate will work closely with data analysts, data scientists, and cross-functional teams to ensure reliable data flow and optimized performance across systems.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines.

Build and maintain data warehouses and data lakes.

Ensure data quality, integrity, and security across systems.

Optimize database performance and troubleshoot data-related issues.

Work with cloud platforms (AWS/GCP/Azure) for data processing and storage.

Implement data models to support analytics and reporting.

Automate workflows using orchestration tools.

Collaborate with stakeholders to gather and translate business requirements into technical solutions.

Required Skills & Qualifications

Technical Skills

Strong proficiency in Python (Pandas, NumPy, PySpark preferred).

Advanced SQL skills (complex joins, CTEs, window functions, query optimization).

Experience with relational databases (PostgreSQL, MySQL, SQL Server).

Experience with data warehousing solutions (Snowflake, Redshift, Big Query).

Hands‑on experience with ETL tools (Airflow, dbt, Talend, etc.).

Knowledge of data modeling (star schema, snowflake schema).

Familiarity with big data tools (Spark, Hadoop) is a plus.

Experience with version control (Git).

Understanding of CI/CD pipelines.

AWS (S3, Glue, Redshift, Lambda)

Docker & Kubernetes (nice to have).

Soft Skills

Strong analytical and problem‑solving skills.

Excellent communication and stakeholder management.

Ability to work independently and in a team.

Attention to detail and commitment to data accuracy.

Education

Bachelor’s or master’s degree in computer science, Engineering, Information Systems, or related field.

Good to Have

Experience with real‑time streaming (Kafka).

Exposure to data governance and compliance frameworks.