Senior Data Engineer

Data Economy; Hyderabad, Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Data Economy in Hyderabad is seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure for analytics, reporting, and ML initiatives.

You will work with analysts, data scientists, and engineers to ensure reliable, high-quality data across the organization, focusing on performance, governance, and scalable architectures.

Qualifications

• Core Data Engineering & Architecture experience.
• End-to-end ETL/ELT pipeline design and build.
• Scalable data platforms for terabytes of data.
• Strong data modeling, warehousing, and lakehouse skills.
• Hands-on with Spark, Hadoop, Kafka, Flink (batch + streaming).
• Experience with AWS/Azure/GCP and data services (lakes/warehouses).
• Proficient in Python and SQL; knowledge of Scala is a plus.
• Experience with CI/CD and infrastructure-as-code.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines.
• Build and optimize data warehouses, lakes, and data models.
• Integrate data from multiple sources including APIs and databases.
• Ensure data quality, governance, security, and lineage.
• Monitor and troubleshoot data pipelines and performance issues.
• Collaborate with cross-functional teams to meet data requirements.

Skills

SQL

Python

Spark

Hadoop

Kafka

Flink

Airflow

Databricks

Snowflake

Redshift

Synapse

BigQuery

ETL/ELT

Data Modeling

Data Warehousing

Data Lakes

AWS/Azure/GCP

CI/CD

DevOps

Education

B.Tech/M.Tech/MCA

Tools

Databricks

Snowflake

Redshift

Synapse

BigQuery

Airflow

Job description

Notice Period: 0-30 days

Qualification: B.Tech/M.Tech/MCA

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support analytics, reporting, and machine learning initiatives. The ideal candidate has strong experience with SQL, Python, cloud platforms, and modern data engineering tools. You will collaborate with data analysts, data scientists, and software engineers to ensure reliable, high-quality data is available across the organization.

Job Description:

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize data warehouses, data lakes, and data models.
• Integrate data from multiple sources, including APIs, databases, and third-party systems.
• Ensure data quality, consistency, security, and governance.
• Monitor and troubleshoot data pipelines and resolve performance issues.
• Optimize SQL queries and database performance.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.

Requirements

• Core Data Engineering & Architecture

• Design and build end-to-end data pipelines (ETL/ELT) for structured and unstructured data
• Develop scalable data platforms handling terabytes of data with high reliability and low latency
• Strong expertise in data modeling, warehousing, and lakehouse architectures
• Own data quality, lineage, governance, and observability frameworks
• Hands-on experience with Spark, Hadoop, Kafka, Flink (batch + real-time processing)
• Build and optimize high-throughput, distributed data systems
• Experience in streaming + event-driven architectures for large-scale financial data
• Deep expertise in AWS / Azure / GCP (Data Lakes, Warehouses, Compute, Storage)
• Tools: Databricks, Snowflake, Redshift, Synapse, BigQuery
• Pipeline orchestration using Airflow, Prefect, or similar frameworks
• Strong coding in Python, SQL, Scala
• Focus on performance optimization, reliability, and production-grade systems
• Experience with CI/CD, DevOps, and infrastructure-as-code
• Ability to design architectures and make technology decisions at scale
• Strong understanding of:
• Trade lifecycle, transactions, risk, compliance, regulatory reporting
• Customer 360, payments, lending, capital markets data and Wealth Management Business
• GenAI / LLM integration (RAG, embeddings, vector stores)
• Lead legacy-to-cloud data platform migrations
• Drive data re-engineering, system decomposition, and modernization initiatives
• Drive resilience, failover, and incident response frameworks
• Work across engineering, data science, risk, compliance, and product teams
• Ability to translate business needs (risk, reporting, revenue) into data solutions
• Mentor engineers and drive engineering best practices and standards.