Senior Data Engineer

DATAECONOMY Inc; Hyderabad; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

DATAECONOMY Inc in Hyderabad is seeking a Data Engineer to design, build, and maintain scalable data pipelines and infrastructure for analytics, reporting, and ML initiatives. The ideal candidate has strong SQL, Python, cloud experience, and familiarity with modern data tools.

You will design ETL/ELT pipelines, data warehouses and lakes, ensure data quality and governance, monitor performance, and collaborate with analysts, scientists, and engineers using Spark, Hadoop, Kafka, Flink, and Airflow

Qualifications

• Design and build end-to-end ETL/ELT pipelines for structured and unstructured data.
• Develop scalable data platforms handling terabytes of data with high reliability and low latency.
• Strong expertise in data modeling, warehousing, and lakehouse architectures.
• Own data quality, lineage, governance, and observability frameworks.
• Hands-on experience with Spark, Hadoop, Kafka, Flink (batch + real-time).

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize data warehouses, data lakes, and data models.
• Integrate data from multiple sources, including APIs, databases, and third-party systems.
• Ensure data quality, consistency, security, and governance.
• Monitor and troubleshoot data pipelines and resolve performance issues.
• Optimize SQL queries and database performance.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.

Skills

Python

SQL

Scala

Data modeling

Data warehousing

Tools

Spark

Hadoop

Kafka

Flink

Databricks

Snowflake

Redshift

Synapse

BigQuery

Airflow

Prefect

Job description

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support analytics, reporting, and machine learning initiatives. The ideal candidate has strong experience with SQL, Python, cloud platforms, and modern data engineering tools. You will collaborate with data analysts, data scientists, and software engineers to ensure reliable, high-quality data is available across the organization.

Job Description

• Design, develop, and maintain scalable ETL/ELT data pipelines.
  
• Build and optimize data warehouses, data lakes, and data models.
  
• Integrate data from multiple sources, including APIs, databases, and third-party systems.
  
• Ensure data quality, consistency, security, and governance.
  
• Monitor and troubleshoot data pipelines and resolve performance issues.
  
• Optimize SQL queries and database performance.
  
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
  

Requirements

Core Data Engineering& Architecture

• Designand build end-to-end data pipelines (ETL/ELT) for structured andunstructured data
• Developscalable data platforms handling terabytes of data with high reliabilityand low latency
• Strongexpertise in data modeling, warehousing, and lakehouse architectures
• Owndata quality, lineage, governance, and observability frameworks
• Hands-onexperience with Spark, Hadoop, Kafka, Flink (batch + real-time processing)
• Buildand optimize high-throughput, distributed data systems
• Experiencein streaming + event-driven architectures for large-scale financial data
• Deepexpertise in AWS / Azure / GCP (Data Lakes, Warehouses, Compute, Storage)
• Tools:Databricks, Snowflake, Redshift, Synapse, BigQuery
• Pipelineorchestration using Airflow, Prefect, or similar frameworks
• Strongcoding in Python, SQL, Scala
• Focuson performance optimization, reliability, and production‑grade systems
• Experiencewith CI/CD, DevOps, and infrastructure-as-code
• Abilityto design architectures and make technology decisions at scale
• Strongunderstanding of:
• Tradelifecycle, transactions, risk, compliance, regulatory reporting
• Customer360, payments, lending, capital markets data and Wealth ManagementBusiness
• GenAI/ LLM integration (RAG, embeddings, vector stores)
• Leadlegacy-to-cloud data platform migrations
• Drivedata re‑engineering, system decomposition, and modernization initiatives
• Driveresilience, failover, and incident response frameworks
• Workacross engineering, data science, risk, compliance, and product teams
• Abilityto translate business needs (risk, reporting, revenue) into data solutions
• Mentorengineers and drive engineering best practices and standards