Senior Associate - Data Engineer

Acuity Analytics; Gurugram District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Acuity Analytics in Gurugram invites a hands-on Data Engineer to design, build, and automate data pipelines that bring new datasets onto the platform quickly and reliably as they are identified by the investment team.

The role emphasizes core data engineering: ingesting, transforming, and organizing data so it lands in the right shape, with the future AI roadmap including vector databases and RAG workflows to enable insightful reporting.

Qualifications

• Hands-on data engineer with 4–5+ years of experience in data pipelines and data platforms.
• Experience designing, building, and automating data workflows with emphasis on reliability and maintainability.
• Proficiency in Python and SQL with production-quality code and data modeling.
• Familiarity with AI/LLM enablement concepts like vector databases and retrieval augmented generation.

Responsibilities

• Pipeline development: design, build, and maintain data pipelines to ingest datasets onto the platform.
• Automation: streamline ingestion, transformation, and data-quality checks.
• Data platform: organize datasets for efficient integration and querying across the platform.
• AI enablement: prepare documents for AI consumption and index content in vector databases.
• Insight & reporting: enable cross-dataset querying to support reporting.
• Stakeholder collaboration: work with hiring manager and analysts to scope requirements and prioritize work.

Skills

Data engineering

Airflow

Spark

Python

SQL

Data modeling

Data quality & monitoring

Stakeholder collaboration

Tools

Apache Airflow

Apache Spark

Vector databases

RAG workflows

Cloud data platforms

Job description

Seeking a hands-on Data Engineer to design, build, and automate data pipelines that bring new datasets onto the platform quickly and reliably as they are identified by the investment team.

Seeking a hands-on Data Engineer to design, build, and automate data pipelines that bring new datasets onto the platform quickly and reliably as they are identified by the investment team.

Key priority is core data engineering: ingesting, transforming, and organizing data so it lands in the right place, in the right shape. Over time, the role will extend into the firm's AI roadmap — preparing large document sets for retrieval-augmented generation (RAG), managing vector databases, and enabling effective querying across integrated datasets to generate insight and reporting. This is not a fixed-scope project role. The engineer will operate as a responsive, standing capability — picking up new pipeline builds as analysts surface new data requirements, while maintaining and improving what is already in place.

Key Responsibilities

• Pipeline development — Design, build, and maintain robust data pipelines to ingest new datasets onto the firm's data platform as demand arises from the investment and analyst teams.
• Automation — Automate ingestion, transformation, and data-quality workflows to reduce manual effort and speed up onboarding of new sources.
• Data platform — Structure and organize datasets so they can be integrated, joined, and queried effectively across the platform.
• AI enablement (roadmap) — Process large volumes of documents into formats suitable for AI consumption; store and index content in vector databases; support retrieval workflows (RAG) that surface the right information reliably.
• Insight & reporting (roadmap) — Enable effective querying across multiple integrated datasets to combine information and support sensible, decision-ready reporting.
• Stakeholder collaboration — Work directly with the hiring manager and analysts to scope new requirements, priorities the pipeline backlog, and respond quickly as new needs emerge.

Required Skills

• 4–5+ years of hands-on data engineering experience, ideally within financial services or data intensive environments.
• Strong experience building and orchestrating data pipelines with Apache Airflow.
• Solid working knowledge of Apache Spark for large-scale data processing.
• Strong Python and SQL skills, with a focus on clean, maintainable, production-quality code.
• Experience designing data models and organizing datasets for downstream integration and querying.
• Track record of automating data workflows and building for reliability, monitoring, and data quality.
• Comfortable operating with loosely defined, evolving scope — able to self-organize, scope work with stakeholders, and deliver iteratively.

Nice to Have

• Exposure to AI/LLM engineering: document processing at scale, embeddings, and vector databases (e.g. pgvector, Pinecone, Weaviate, or similar).
• Experience building or supporting retrieval-augmented generation (RAG) workflows.
• Familiarity with cloud data platforms and modern data stack tooling.
• Prior experience supporting investment management, hedge fund, or capital markets data environments.
• Experience working as a dedicated remote resource embedded with a client team.