Senior Data Engineer (Databricks & AI Solutions)

Cognizant; São Paulo; Hybrid; BRL 240,000 - 360,000; Full time

Job summary

Cognizant is seeking a Senior Data Engineer (Databricks & AI Solutions) to build, operate, and support modern data platforms and AI-enabled data engineering solutions. You will work with architects, data teams, and stakeholders to translate requirements into production-ready data products, ensuring quality, observability, and reliability.

The role is hybrid in São Paulo, Brazil, and requires 4+ years in Data Engineering with strong Databricks, Spark, PySpark, SQL, and Python expertise.

Qualifications

• 4+ years of hands-on experience in Data Engineering.
• Strong experience with Databricks, Apache Spark, PySpark, SQL, and Python.
• Experience building and maintaining ETL/ELT pipelines in enterprise environments.
• Experience working with cloud platforms, preferably Microsoft Azure.
• Knowledge of Delta Lake, Delta Live Tables, and Unity Catalog.
• Experience implementing data quality, reconciliation, and validation processes.
• Experience with Git-based CI/CD practices for data engineering solutions.
• Exposure to AI-assisted development tools, code generation platforms, or GenAI-enabled engineering workflows.
• Experience supporting production environments, incident management, and root cause analysis.
• Strong analytical, troubleshooting, and problem-solving skills.
• Advanced English proficiency.
• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.

Responsibilities

• Design, develop, and deploy scalable data pipelines using Databricks, PySpark, SQL, and cloud-native technologies.
• Implement and support data quality, lineage, governance, and validation processes across enterprise data products.
• Operate and continuously optimize AI-assisted and agent-based data engineering solutions to improve delivery efficiency and platform performance.
• Lead troubleshooting, root cause analysis, and support activities for data pipelines, platforms, and production incidents.
• Maintain technical documentation, knowledge repositories, and operational runbooks while collaborating with cross-functional teams throughout the data product lifecycle.

Skills

Databricks

Apache Spark

PySpark

SQL

Python

Advanced English

Analytical skills

Troubleshooting

Problem solving

Education

Bachelor’s degree in Computer Science/Engineering/IT

Tools

Delta Lake

Delta Live Tables

Unity Catalog

Git-based CI/CD

Job description

About the role

As a Senior Data Engineer (Databricks & AI Solutions), you will make an impact by building, operating, and supporting modern data platforms and AI-enabled data engineering solutions. You will work closely with architects, data teams, and business stakeholders to transform requirements into production-ready data products while ensuring data quality, reliability, observability, and operational excellence.

In this role, you will

• Design, develop, and deploy scalable data pipelines using Databricks, PySpark, SQL, and cloud-native technologies.
• Implement and support data quality, lineage, governance, and validation processes across enterprise data products.
• Operate and continuously optimize AI-assisted and agent-based data engineering solutions to improve delivery efficiency and platform performance.
• Lead troubleshooting, root cause analysis, and support activities for data pipelines, platforms, and production incidents.
• Maintain technical documentation, knowledge repositories, and operational runbooks while collaborating with cross-functional teams throughout the data product lifecycle.

Work Model

Hybrid | São Paulo, Brazil

This role follows Cognizant’s hybrid work model, requiring periodic onsite collaboration while supporting global teams and stakeholders.

What you need to have to be considered

• 4+ years of hands‑on experience in Data Engineering.
• Strong experience with Databricks, Apache Spark, PySpark, SQL, and Python.
• Experience building and maintaining ETL/ELT pipelines in enterprise environments.
• Experience working with cloud platforms, preferably Microsoft Azure.
• Knowledge of Delta Lake, Delta Live Tables, and Unity Catalog.
• Experience implementing data quality, reconciliation, and validation processes.
• Experience with Git‑based CI/CD practices for data engineering solutions.
• Exposure to AI‑assisted development tools, code generation platforms, or GenAI‑enabled engineering workflows.
• Experience supporting production environments, incident management, and root cause analysis.
• Strong analytical, troubleshooting, and problem‑solving skills.
• Advanced English proficiency.
• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.

These will help you stand out

• Experience with ICEDQ or similar Data Quality frameworks.
• Familiarity with Data Validation Suite (DVS) or automated data testing frameworks.
• Experience with column‑level data lineage and data governance initiatives.
• Familiarity with agentic AI platforms and AI‑powered data engineering tools.
• Experience with Power BI, Tableau, or other analytics and visualization platforms.
• Knowledge of modern Data Mesh, Data Product, and Data Fabric concepts.
• Azure, Databricks, or cloud‑related certifications.
• Experience working in large‑scale enterprise data transformation programs.

We're excited to meet people who share our mission.