Data Engineer

Techdome; Hyderabad; On-site; INR 1,500,000 - 2,200,000; Full time

Job summary

Techdome in Hyderabad is seeking a Data Engineer to design and implement data pipelines, connectors, and data stores that support AI agents and analytics. You will work with Python, SQL, and modern platforms to enable robust data foundations for dashboards and scorecards.

You will own data modeling, metadata extraction, and quality rules while collaborating across teams to ensure secure, governed data workflows and scalable data architectures.

Qualifications

• 3 to 6 years in data engineering; senior data engineer role involves ownership of data foundation.
• Strong Python and SQL and solid data modeling (dimensional and normalized).
• Experience with modern data platforms and pipeline orchestration.

Responsibilities

• Design and build ingestion pipelines for structured and unstructured data and metadata.
• Build reusable connectors to enterprise systems and third-party sources via APIs.
• Model and build raw-to-mart data layers for analytics and AI use cases.
• Implement metadata extraction, enrichment, and semantic/vector indexing for RAG.
• Manage vector databases and knowledge repositories used by LLM agents.
• Apply data quality rules, profiling, scoring, and exception handling.
• Register lineage, maintain registries, and manage refresh controls.
• Design data stores for signals, audit trails, and user feedback loops.
• Apply RBAC, PII flags, and data handling governance; support SIT/UAT.

Skills

Python

SQL

Data modeling

Databricks

Snowflake

Cloud platforms

Spark

Airflow

dbt

API integration

JSON handling

Git

CI/CD

Tools

Databricks

Snowflake

Azure data services

AWS data services

Spark tooling

dbt

Airflow

REST APIs

Job description

Role summary: The Data Engineer builds the data foundation that AI agents, scorecards, and dashboards run on: ingestion pipelines, connectors, metadata and search indexes, vector stores, data marts, and quality instrumentation.

Experience: 3 to 6 years in data engineering. Senior Data Engineer: 2+ years, including ownership of data foundation architecture and connector frameworks.

Key responsibilities

• Design and build ingestion pipelines for structured data, unstructured content (documents, PDFs), and metadata.
• Build reusable connectors to enterprise systems, catalogs, content repositories, and third-party or licensed sources via APIs.
• Model and build raw-to-mart data layers that serve analytics and AI use cases.
• Implement metadata extraction, enrichment, and search indexing, including semantic and vector indexes for RAG.
• Set up and manage vector databases and knowledge repositories used by LLM agents.
• Implement data quality rules, profiling, scoring outputs, and exception handling.
• Register lineage and maintain source registries, version tracking, and refresh controls.
• Design data stores for signals, findings, audit trails, and user feedback loops.
• Apply security and governance controls: RBAC, PII/sensitivity flagging, and approved data handling.
• Support SIT/UAT data validation, defect fixes, and production release activities.

Required skills

• Strong Python and SQL; solid data modeling (dimensional and normalized).
• Hands-on experience with a modern data platform such as Databricks, Snowflake, or Azure/AWS data services.
• Pipeline orchestration and transformation (Spark, Airflow, ADF, dbt, or similar).
• API-based integration (REST), JSON handling, and incremental/CDC ingestion patterns.
• Data quality frameworks and testing practices for pipelines.
• Version control (Git) and CI/CD for data workloads.

Preferred skills

• RAG data preparation: chunking, embeddings, vector databases (Azure AI Search, pgvector, Pinecone, or similar).
• Metadata management, data catalogs, ontologies, or knowledge graphs (for example Neptune or other graph databases).
• Experience supporting LLM or agentic applications with grounded, traceable data.
• Life sciences data exposure (commercial, medical, regulatory, or launch data) and regulated-data handling.
• Cloud certification (Azure Data Engineer, Databricks, AWS, or Snowflake).