Senior Data Engineer – Full Stack

Webhosting; India; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

DigiCert is seeking a Sr. Data Engineer - Full Stack to design and deliver end-to-end data solutions on Databricks, spanning ingestion, Delta Lake storage, transformation, APIs, and user-facing experiences.

You will collaborate with business and tech teams to translate needs into robust data products, and build scalable pipelines for analytics, AI/ML, and operational decision-making.

Qualifications

• 5+ years of experience in data engineering, software engineering, or a related role with production data solutions.
• Strong proficiency in SQL, Python, and PySpark, building reliable end-to-end data products.
• Hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog or similar governance.

Responsibilities

• Design, build, and maintain end-to-end data products on Databricks, including ingestion, storage, transformation and APIs.
• Develop and operate reliable batch, incremental, and streaming pipelines using SQL, Python, PySpark, Kafka, and Databricks features.
• Create scalable data models and datasets supporting analytics, ML, and operational decisions.
• Lead technical design reviews, mentor engineers, and ensure governance, security, and testing across data products.

Skills

SQL

Python

PySpark

REST APIs

AWS/Azure/GCP

CI/CD

Data modeling

Data warehousing

Stakeholder collaboration

End-to-end ownership

Tools

Databricks

Unity Catalog

Kafka

Kinesis

Job description

DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, and certificate lifecycle management, to secure infrastructure, software, devices, messages, AI content and agents. Learn why more than 100,000 organizations, including 90% of the Fortune 500, choose DigiCert to stop today's threats and prepare for a quantum-safe future at www.digicert.com

Job summary

We are looking for a Sr. Data Engineer - Full Stack to design and deliver end-to-end data solutions on Databricks - from stakeholder discovery and data ingestion through modeling, APIs, applications, and production operations. This hands-on role combines deep data engineering expertise with strong stakeholder partnership, product thinking, and end-to-end solution ownership. You will work directly with business and technical teams to clarify ambiguous problems, rapidly prototype solutions, and turn validated concepts into secure, scalable, and reliable data products. You will bridge data engineering, analytics, software engineering, and business teams to make trusted data actionable.

What you will do

• Partner directly with business and technical stakeholders to understand workflows, define desired outcomes, and translate ambiguous needs into technical requirements and delivery plans.
• Design, build, and maintain end-to-end data products on Databricks, spanning ingestion, Delta Lake storage, transformation, serving, APIs, and user-facing experiences.
• Develop and operate reliable batch, incremental, and streaming pipelines using SQL, Python, PySpark, Kafka, and Databricks-native capabilities.
• Design event-driven and near-real-time data solutions that integrate with operational systems and downstream consumers.
• Build backend services, APIs, and integrations that make governed data available to applications and operational workflows.
• Develop lightweight applications, dashboards, and interfaces in partnership with product, analytics, BI, and user-experience teams.
• Rapidly prototype solutions, validate them through stakeholder feedback, and prepare successful concepts for production use.
• Create scalable data models and curated datasets that support analytics, reporting, AI/ML, and operational decision-making.
• Implement data-quality, security, lineage, and governance controls using Databricks and Unity Catalog.
• Establish automated testing, CI/CD, monitoring, alerting, and documentation across the data-product lifecycle.
• Optimize pipelines, streaming workloads, queries, services, and applications for reliability, performance, scalability, and cost.
• Diagnose and resolve issues across source systems, streaming platforms, pipelines, data models, APIs, applications, and downstream consumers.
• Collaborate with platform and product engineering teams to turn recurring stakeholder needs into reusable capabilities.
• Lead technical design and code reviews, mentor engineers, and help elevate full-stack data-engineering practices.

What you will have

• 5+ years of experience in data engineering, software engineering, or a related role, including ownership of production data solutions.
• Strong proficiency in SQL, Python, and PySpark, with experience building reliable, production-grade pipelines and data products.
• Hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog or comparable data-governance capabilities.
• Experience designing and supporting streaming or near-real-time data pipelines using Kafka, Kinesis, Event Hubs, or similar event-streaming technologies.
• Strong understanding of event-driven architecture, message processing, schema evolution, data consistency, and streaming reliability.
• Experience delivering full-stack solutions that include data pipelines, backend services or APIs, and lightweight user-facing applications.
• Experience building REST APIs, services, and integrations using Python frameworks such as FastAPI, Flask, or comparable technologies.
• Experience with AWS, Azure, or GCP and cloud-native architecture patterns.
• Strong understanding of data modeling, data warehousing, distributed processing, and analytics-friendly data design.
• Experience with Git, automated testing, CI/CD, monitoring, and production-deployment practices.
• Demonstrated ability to work directly with stakeholders, navigate ambiguity, and translate business problems into practical technical solutions.
• Strong communication, technical leadership, problem-solving, and end-to-end ownership skills.
• Ability to balance rapid delivery with maintainability, security, governance, and operational reliability.