Senior Data Engineer

InFynd

Coimbatore

On-site

INR 900,000 - 1,500,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ProminentContact India Private Limited is hiring a Data Engineer in Coimbatore to design and maintain scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks. You will develop data ingestion workflows with Python, SQL, and PySpark, and work on Spark-based processing for large-scale data.

You will optimize data models and warehouse structures, implement CDC and batch processes, and ensure robust production monitoring and CI/CD practices.

Qualifications

  • Strong hands-on experience with Azure Data Factory, Azure Databricks, PySpark, Python, and SQL.
  • Experience building scalable ETL/ELT pipelines and data warehouses.
  • Familiarity with cloud-based data engineering concepts and data modeling.

Responsibilities

  • Design, develop, and maintain scalable ETL/ELT data pipelines.
  • Develop data ingestion and transformation workflows using Python, SQL, PySpark.
  • Work with Apache Spark for large-scale data processing and CDC mechanisms.
  • Integrate data from sources including APIs and cloud storage.
  • Develop and optimize data models and data warehouse structures.
  • Implement error handling, logging, monitoring, and alerting for production pipelines.
  • Manage version control and CI/CD pipelines with Git, GitHub, and Azure DevOps.
  • Troubleshoot production issues and provide production support.
  • Maintain technical documentation and collaborate with cross-functional teams.

Skills

Python
SQL
PySpark
Apache Spark
Airflow
Azure Data Factory
Azure Databricks
Delta Lake
Git
CI/CD

Tools

Azure Synapse
Azure DevOps
GitHub

Job description

Coimbatore South, India | Posted on 11/13/2026

ProminentContact India Private Limited was incorporated in August 2020 with just fouremployees. We have envisioned creating a B2B database solution forbusiness accelerators, offering customized data solutions in niche ICP, andnamed it "InFynd." InFynd serves as a one-stop solution for data intelligence.We have since expanded our operations to Coimbatore and Gateshead, UK. Today,we proudly uphold our Vision, Mission, and Core Values as we continue to grow and innovate.

Job Description
  • Design,develop, and maintain scalable ETL/ELT data pipelines using Azure DataFactory and Azure Databricks.
  • Developdata ingestion and transformation workflows using Python, SQL, andPySpark.
  • Workwith Apache Spark and PySpark for large-scale distributed data processing.
  • Implementbatch processing, incremental loading, full loading, and Change DataCapture (CDC) mechanisms.
  • Integratedata from multiple sources, including relational databases, APIs, andcloud storage.
  • Workwith Azure Data Lake Storage Gen2 and Azure Blob Storage.
  • Developand optimize data models, fact tables, dimension tables, and datawarehouse structures.
  • Performdata cleansing, validation, deduplication, and aggregation.
  • Implementand manage workflows using Azure Data Factory and Apache Airflow.
  • OptimizeSQL queries, Spark jobs, and data pipelines for better performance.
  • Workwith Azure Synapse Analytics and relational databases.
  • Implementerror handling, logging, monitoring, and alerting for productionpipelines.
  • Manageversion control and CI/CD pipelines using Git, GitHub, and Azure DevOps.
  • Troubleshootproduction issues, perform root cause analysis, and provide productionsupport.
  • Maintaintechnical documentation and collaborate with cross-functional teams.
Requirements
Programming Languages
  • Python,SQL, PySpark.
  • Pandas,NumPy, OOP concepts, exception handling, and API integration.
  • ApacheSpark and PySpark.
  • SparkDataFrames, Spark SQL, transformations, actions, joins, aggregations, andwindow functions.
  • Partitioning,caching, and performance optimization.
  • Dataingestion, transformation, and migration.
  • ChangeData Capture (CDC).
  • Incrementalloading and full loading.
  • Batchand distributed processing.
Database Technologies
  • MicrosoftSQL Server, PostgreSQL, MySQL, and Oracle.
  • ComplexSQL queries, indexing, stored procedures, and query optimization.
Data Modeling & Warehousing
  • Dimensionaland relational data modeling.
  • Factand dimension tables.
  • Datanormalization and denormalization.
  • Schemadesign and data warehouse development.
Workflow Orchestration & DevOps
  • AzureData Factory and Apache Airflow.
  • Git,GitHub, Azure DevOps, and CI/CD pipelines.
File Formats
  • CSV,JSON, XML, Parquet, and Delta Lake.
  • 4-5years of relevant experience in Data Engineering.
  • Stronghands-on experience with Azure Data Factory, Azure Databricks, PySpark,Python, and SQL.
  • Experiencedeveloping and optimizing scalable data pipelines.
  • Goodunderstanding of cloud-based data engineering and data warehousingconcepts.
  • Stronganalytical, debugging, and problem-solving skills.
  • Goodcommunication and collaboration skills.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Wissen • Bengaluru

Hybrid
INR 2,800,000 - 4,800,000
Senior Data Engineer
Senior Data Engineer

Kumaran Systems • Chennai

Hybrid
INR 900,000 - 1,600,000
Data Engineer
Data Engineer

Cnx • Bengaluru

On-site
INR 1,200,000 - 1,500,000
Senior Data Engineer
Senior Data Engineer

Vserve • Coimbatore District

Hybrid
INR 1,800,000 - 3,200,000
Data Engineer
Data Engineer

Weekday • Mumbai

On-site
INR 500,000 - 2,000,000
Senior Data Engineer
Senior Data Engineer

Vserve Ebusiness Solutions India • Coimbatore District

Hybrid
INR 1,500,000 - 2,600,000
Senior Data Engineer
Senior Data Engineer

Heineken • Hyderabad

On-site
INR 1,200,000 - 2,400,000
Senior Data Engineer · 5+ years · Hyderabad, India (Hybrid)
Senior Data Engineer · 5+ years · Hyderabad, India (Hybrid)

G360 Technologies • Hyderabad

On-site
INR 3,600,000 - 5,400,000
Senior Data Engineer
Senior Data Engineer

Kanini Software Solutions • Pune District, Chennai, Bengaluru

Hybrid
INR 3,500,000 - 7,000,000
Data Engineer
Data Engineer

Kumaran • Hyderabad

On-site
INR 1,000,000 - 1,400,000