Senior Databricks Data Engineer | AWS, PySpark & SQL

Infosys

São Paulo

Hybrid

BRL 200,000 - 320,000

Full time

7 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Transportation allowance
Meal voucher
Childcare assistance
Life insurance
Funeral assistance
Medical insurance
Dental insurance

Job summary

Infosys Brazil is seeking a Senior Databricks Data Engineer with AWS, PySpark and SQL expertise to join our team. The ideal candidate will design, build and optimize scalable data pipelines using Databricks and Delta Lake in a hybrid Work Model.

The role requires strong Python, PySpark, SQL skills and experience with AWS services and Azure Synapse. Collaboration with global teams and stakeholders is essential, with CLT full-time employment in a hybrid environment.

Qualifications

  • Strong experience in Data Engineering and Databricks environments.
  • Advanced proficiency in Python, PySpark, and SQL.
  • Experience with Databricks and Delta Lake, including Delta Live Tables/Lakeflow, and Asset Bundles.
  • Strong knowledge of relational and dimensional data modeling.
  • Experience developing ETL/ELT, CDC, and batch data pipelines.
  • Hands-on experience with AWS services (S3, Glue, EMR, Athena, Lambda, DMS, Step Functions).
  • Experience with Azure Synapse, including SQL Pools and data pipelines.
  • Knowledge of Git and CI/CD practices applied to Data Engineering.
  • Experience with data quality, testing, reconciliation, and automated validation processes.
  • Ability to work independently and collaborate with stakeholders.

Responsibilities

  • Design, develop, test, and maintain scalable batch data pipelines using PySpark and Python.
  • Build and operate Databricks and Delta Lake solutions based on Bronze, Silver, and Gold architecture layers.
  • Develop data transformations and support relational and dimensional data modeling.
  • Migrate and rebuild legacy ETL pipelines using Spark, including CDC and batch ingestion from relational databases.
  • Support the reverse engineering of Azure Synapse pipelines and SQL Pools.
  • Implement automated testing, data quality controls, reconciliation processes, and validation between legacy and new environments.
  • Apply Git and CI/CD practices to data pipeline development and deployment.
  • Work with AWS data services to build reliable and scalable data solutions.
  • Collaborate with technical teams and business stakeholders to ensure successful solution delivery.

Skills

Databricks
PySpark
Python
SQL
AWS
Delta Lake
Data Modeling
ETL/ELT
CI/CD
Git
Spark
Azure Synapse

Education

Bachelor’s degree

Tools

Azure Synapse
Git

Job description

Infosys Brazil is looking for professionals with a Senior Databricks Data Engineer | AWS, PySpark & SQL profile to join our team.

Required Qualifications
  • Strong experience in Data Engineering and Databricks environments.
  • Advanced proficiency in Python, PySpark, and SQL.
  • Experience with Databricks and Delta Lake, including jobs, workflows, Medallion Architecture, Delta Live Tables/Lakeflow, and Asset Bundles.
  • Strong knowledge of relational and dimensional data modeling.
  • Experience developing ETL/ELT, CDC, and batch data pipelines.
  • Hands-on experience with AWS services such as Amazon S3, AWS Glue, Amazon EMR, Amazon Athena, AWS Lambda, AWS DMS, and AWS Step Functions.
  • Experience with Azure Synapse, including SQL Pools and data pipelines.
  • Knowledge of Git and CI/CD practices applied to Data Engineering.
  • Experience with data quality, testing, reconciliation, and automated validation processes.
  • Ability to work independently and collaborate effectively with technical and business stakeholders.
  • Strong communication and interpersonal skills.
Preferred Qualifications
  • Experience with Data Warehouse-to-Lakehouse migration programs.
  • Knowledge of Unity Catalog, including permissions, data lineage, governance, and data contracts.
  • Experience with DataStage, particularly analyzing existing jobs for reverse engineering.
  • Knowledge of automated testing, Data Quality gates, and data validation between legacy and modern environments.
  • Experience translating business rules from legacy ETL tools into Spark-based solutions.
Roles and Responsibilities
  • Design, develop, test, and maintain scalable batch data pipelines using PySpark and Python.
  • Build and operate Databricks and Delta Lake solutions based on Bronze, Silver, and Gold architecture layers.
  • Develop data transformations and support relational and dimensional data modeling.
  • Migrate and rebuild legacy ETL pipelines using Spark, including CDC and batch ingestion from relational databases.
  • Support the reverse engineering of Azure Synapse pipelines and SQL Pools.
  • Implement automated testing, data quality controls, reconciliation processes, and validation between legacy and new environments.
  • Apply Git and CI/CD practices to data pipeline development and deployment.
  • Work with AWS data services to build reliable and scalable data solutions.
  • Collaborate with technical teams and business stakeholders to ensure successful solution delivery.
Employment Type
  • Full-time, permanent position (CLT).
Work Model
  • Hybrid.
Benefits
  • Transportation allowance.
  • Meal voucher.
  • Childcare assistance.
  • Life insurance.
  • Funeral assistance.
  • Medical insurance.
  • Dental insurance.
Career Opportunities
  • Opportunity to work on global projects and international environments.
  • Exposure to Software Quality Assurance and project governance practices.
  • Collaboration with multicultural and cross-functional teams.
  • Professional development within a global technology organization.
Language Requirements
  • Advanced English proficiency required for interaction with international teams and stakeholders.
Additional Required Qualifications
  • Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
  • This position may require relocation and/or travel to work/project location.
  • Candidates authorized to work for any employer in the United States without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role now or in the future.
About Us

Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem.

EEO

Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer | Databricks, AWS, PySpark & SQL
Data Engineer | Databricks, AWS, PySpark & SQL

Infosys Limited • São Paulo

Hybrid
BRL 180,000 - 300,000
Life insurance
Medical insurance
Global project exposure
+1
Senior Data Engineering Tech Lead
Senior Data Engineering Tech Lead

Infosys • São Paulo

Hybrid
BRL 180,000 - 240,000
Transportation allowance
Meal voucher
Childcare assistance
+4
Databricks Data Engineer | Senior
Databricks Data Engineer | Senior

Jobgether • Brasília

Remote
BRL 180,000 - 320,000
Healthcare: Medical and dental
Remote work support: Home office
Welcome kit
+3
ADM-Dev Analytics Engineer / Data Analyst - Databricks
ADM-Dev Analytics Engineer / Data Analyst - Databricks

Infosys • Região Geográfica Intermediária de São Paulo

On-site
BRL 180,000 - 280,000
Staff Technical Solutions Engineer, Data & AI
Staff Technical Solutions Engineer, Data & AI

WinsAbove • São Paulo

Hybrid
BRL 380,000 - 550,000
Staff Technical Solutions Engineer, Data & Ai
Staff Technical Solutions Engineer, Data & Ai

Neon • São Paulo

Hybrid
BRL 300,000 - 540,000
Staff Technical Solutions Engineer, Data & AI
Staff Technical Solutions Engineer, Data & AI

Databricks Inc. • São Paulo

Hybrid
BRL 230,000 - 360,000
Senior Data Engineer (Databricks & AI Solutions)
Senior Data Engineer (Databricks & AI Solutions)

Cognizant Technology Solutions Corporation • São Paulo

Hybrid
BRL 240,000 - 360,000
Solutions Engineer
Solutions Engineer

Databricks Inc. • São Paulo

On-site
BRL 60,000 - 90,000
Technology Architect
Technology Architect

Infosys • São Paulo

Hybrid
BRL 260,000 - 520,000
Transportation allowance
Meal voucher
Childcare assistance
+4