Data Architect

Infosys; Bengaluru Urban; On-site; INR 1,200,000 - 2,000,000; Full time

Benefits offered by this job

Cutting-edge cloud and data technologies

Collaborative culture focusing on innovation

Cross-functional project visibility

Job summary

Infosys is seeking experienced Data Engineers to lead the design, architecture, and implementation of end-to-end data pipelines primarily using Python, Databricks, and Spark. The ideal candidate will have over 8 years of experience in data engineering, showcasing deep expertise in building cloud-native data solutions. They will architect data lakes, ensure data quality and compliance, and collaborate with various teams to deliver scalable data solutions, all within a culture emphasizing innovation and continuous improvement.

Qualifications

• 8+ years of experience in Data Engineering with strong command of Python.
• Deep expertise in Databricks, Spark (PySpark), Delta Lake.
• Proven experience with cloud data ecosystems.

Responsibilities

• Lead the design and implementation of data pipelines using Python and Databricks.
• Architect optimized data lake environments and manage cloud-native data environments.
• Partner with teams to translate requirements into scalable data solutions.

Skills

Data Engineering

Python

Databricks

Spark

SQL

Azure

Kafka

MLOps

Education

Bachelor’s or Master’s degree in Computer Science or Engineering

Tools

Apache Spark

Databricks Workflows

Azure DevOps

GitHub Actions

Job description

Key Responsibilities

Technical Leadership

• Lead the design, architecture, and implementation of end-to-end data pipelines using Python, Databricks, Spark, and Delta Lake.
• Provide technical direction on data modeling, ETL/ELT frameworks, and best practices.
• Mentor and guide junior and mid-level engineers, conduct code reviews, and enforce coding standards.

Advanced Data Engineering

• Architect optimized data lake and lakehouse environments, including multi-layered data models (bronze/silver/gold).
• Implement high-performance batch and streaming pipelines using Apache Spark and Databricks Workflows.
• Build scalable ingestion frameworks for structured, semi-structured, and unstructured data from diverse sources.
• Manage and optimize cloud-native data environments (Azure preferred):
• Databricks cluster tuning, job orchestration, cost optimization
• Integrate Databricks with enterprise systems, APIs, and CI/CD pipelines (Azure DevOps / GitHub Actions).

Data Quality, Security & Governance

• Implement enterprise-grade data quality frameworks and automated validation pipelines.
• Ensure compliance with security, data governance, and privacy standards (Unity Catalog / Purview).
• Define and maintain metadata, lineage, and documentation across data assets.

Cross-functional Collaboration

• Partner with data scientists, ML engineers, BI teams, and business stakeholders to translate requirements into scalable data solutions.
• Collaborate with product owners to prioritize data engineering roadmaps.
• Communicate technical decisions and trade-offs to technical and non-technical audiences.

Required Skills & Qualifications

Technical Expertise

• 8+ years of experience in Data Engineering with strong hands-on command of Python.
• Deep expertise in Databricks, Spark (PySpark), Delta Lake, job orchestration, and cluster tuning.
• Strong SQL experience, including performance tuning and complex transformations.
• Proven experience with cloud data ecosystems (Azure preferred):
• Strong understanding of distributed systems, data partitioning, caching, and performance optimization in Spark.

Preferred Skills

• Experience with streaming frameworks: Kafka, Event Hub, Spark Streaming.
• Data warehousing and dimensional modeling experience.
• Exposure to MLOps or ML lifecycle workflows in Databricks is an advantage.
• Experience implementing CI/CD for data pipelines and infra-as-code for data platforms.

Soft Skills

• Strong problem-solving, analytical thinking, and decision-making abilities.
• Excellent communication and leadership skills.
• Comfortable working in agile environments and managing multiple parallel initiatives.
• Ability to influence architecture and strategy decisions with strong technical judgment.

Education & Experience

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related fields.
• 8+ years of professional data engineering experience working on enterprise-scale data projects.

Why Join Us?

• Opportunity to work on cutting-edge cloud and data technologies.
• Influence data architecture and contribute to long-term data strategy.
• Lead impactful projects with cross-functional visibility.
• A collaborative culture with strong focus on innovation and continuous improvement.

Job Location

Multiple Locations in India