Senior Data Engineer

Factspan Analytics; Bengaluru; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

Factspan Analytics is seeking a hands-on Python and AWS data engineering leader to architect, build, and own production features for our data platform. You will mentor an engineering team, perform design and code reviews, and ensure performance, security, and cost efficiency on AWS.

The role requires deep experience with Python services, FastAPI, and Polars for data processing, with strong API design skills.

Qualifications

• 10-14 years in software or data engineering, including 6+ years building production Python services on AWS.
• Expert Python and SQL with experience building APIs in FastAPI and data processing in Polars.
• Strong knowledge of AWS cloud infrastructure for deploying, securing, scaling and monitoring applications.
• Expertise in relational databases: schema design, query tuning, backup and security.
• Proven ability to design JSON APIs and data contracts for frontend integration.
• Proven ability to design secure, high-performance architectures documented in design documents.
• Hands-on use of AI coding assistants in production development with verification of outputs.
• Strong code review skills and experience with CI/CD, containers, testing and observability.
• Has owned features end-to-end and led teams through feature releases.
• Able to quickly translate business features into technical designs.
• Bachelors in engineering/compsci or equivalent.

Responsibilities

• Lead engineering for product with hands-on Python and AWS development.
• Mentor engineers, conduct design and code reviews, and raise engineering standards.
• Architect and maintain backend services and data pipelines on AWS using Python, FastAPI, and Polars.
• Design clear JSON APIs and data contracts for frontend integration.
• Drive releases with feature ownership and production support, including root cause analysis.

Skills

Python

AWS

SQL

API design

Team leadership

CI/CD

Education

Bachelor's degree in Engineering/CS

Tools

FastAPI

Polars

PostgreSQL

MySQL

Terraform / CDK

Job description

About the role

We are looking for a hands‑on Python and AWS data engineering expert to lead engineering for our product. The platform is built on AWS with FastAPI services, Polars‑based data processing and application services that serve the frontend through JSON APIs. You will understand the business feature, design the architecture to industry standards for security and performance, build and verify the production code yourself, and own the feature through to release.

You will lead the engineering and development team: mentoring engineers, running design and code reviews, raising the engineering bar, and leading feature releases.

Key responsibilities

Build and own features

• Take requirements and acceptance criteria from the product manager and turn them into technical designs and a development plan.
• Own each assigned feature through design, build, testing, release, production support and root‑cause analysis.
• Use AI coding assistants and agents in your own design, coding and testing work, and verify their output before it is merged.
• Architect, build and maintain backend services and data pipelines in Python, using FastAPI and Polars, backed by relational databases on AWS.
• Design clear, versioned JSON API contracts that the frontend can consume reliably, and keep them as the reference that engineers and coding agents both build against.

Design to industry standard

• Produce architecture decisions that weigh cost, performance, and operability, and record them in design documents and decision records.
• Design for availability, scalability, fault tolerance and disaster recovery on AWS.
• Use framework‑based, reusable and configurable components instead of one‑off code.
• Evaluate where other AWS services add value and introduce them where the trade‑off is clear.

Security, performance and cost

• Apply secure‑by‑default design: API authentication and authorisation, input validation, least‑privilege access, network isolation, encryption, secrets management and audit logging.
• Set baselines and SLOs for API latency, data‑processing time and cost; tune data pipelines, database queries and infrastructure sizing.

Lead AI‑assisted engineering

• Define how the team uses AI coding assistants and agents across design, coding, testing, documentation and code review.
• Set guardrails on agent access. Agents get no production credentials, changes to infrastructure as code, database migrations and security configuration go through controlled review, and higher‑risk changes need human approval.
• Set review and testing standards for AI‑generated code, with particular attention to SQL correctness, schema and contract drift, performance regressions and insecure defaults.
• Keep what agents work from current and explicit: design documents, decision records, API and data contracts, coding standards and repository conventions.
• Track whether AI tooling is improving delivery, using measures such as lead time, defect rates and rework, and adjust team practice based on the results.

Lead the team and releases

• Mentor engineers, including on working effectively and safely with AI coding tools. Run design and code reviews, set engineering standards for CI/CD, testing and infrastructure as code, and raise the engineering bar.
• Lead development activities and feature releases: break scope into tasks, allocate work, confirm release readiness, run rollout and rollback.

Must‑have skills and experience

• 10-14 years in software or data engineering, including 6+ years building production Python services and data applications on AWS.
• Expert Python and SQL with strong experience building APIs in FastAPI (or a comparable framework) and production data processing in Polars (or comparable libraries such as pandas or Spark).
• Strong working knowledge of AWS and cloud infrastructure for deploying, securing, scaling and monitoring applications.
• Expertise in relational databases (PostgreSQL or MySQL): schema design, query tuning, scaling, backup and security.
• Proven ability to design JSON APIs and data contracts for frontend integration.
• Proven ability to design secure, high‑performance architectures to industry standards, documented in design documents and decision records.
• Hands‑on use of AI coding assistants or agents in production development, with a clear view of where they help, where they fail and how to verify their output.
• Strong code review skills, including spotting subtle defects in code you did not write.
• Familiarity with CI/CD, containers, automated testing and observability.
• Has owned features from design to production support and has led an engineering team through feature releases.
• Quickly understands the business feature behind a requirement and designs to it.
• Bachelors degree in engineering, computer science or equivalent.

Good‑to‑have skills

• Wider AWS data and serverless services, such as S3, Lambda, Glue, Athena, Step Functions and managed analytics.
• Infrastructure as code (Terraform or CDK) and container orchestration.
• Experience introducing AI‑assisted development practices or standards to an engineering team.
• Experience configuring coding agents for a codebase: repository instructions, tool permissions and automated checks on agent output.
• AWS Professional or Specialty certification.
• Experience in a product or platform team.

How we assess

• System design: design a data service on AWS for a given business feature, covering security, cost, failure handling and scale.
• Code review: review an AI‑generated pull request with planted defects in SQL, API contract handling and security.
• Hands‑on build with AI tools allowed, followed by a walkthrough of how you directed the tools and checked their work.
• A release you led, including a production issue and how you found its root cause.