Senior Data Engineer: Databricks, Spark & Azure

Ciandt; United States; Remote; BRL 110,000 - 190,000; Full time

Benefits offered by this job

Health insurance

Meal allowance

Childcare support

Paternity leave

Gym partnerships

Profit sharing

Life insurance

CI&T University

Language learning

Employee discounts

Job summary

CI&T is looking for a Data Engineer to design, build, and optimize scalable data solutions using Databricks, Apache Spark, SQL, Python, and Azure. You will collaborate with data scientists, analysts, engineers, and business stakeholders to transform large volumes of data into reliable data products.

You will design robust pipelines, implement data quality checks, monitor performance, and maintain documentation while working in a bilingual environment (English and Portuguese).

Qualifications

• Solid professional experience in Data Engineering, designing and developing data pipelines.
• Hands-on experience with Databricks and Apache Spark.
• Strong knowledge of SQL and experience with large datasets.
• Proficiency in Python or Scala for data engineering.
• Experience with Microsoft Azure and cloud-based data solutions.
• Strong analytical and problem-solving skills, including troubleshooting complex data issues.
• Excellent communication skills in English and Portuguese, with collaboration across teams.
• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.

Responsibilities

• Design, develop, and maintain robust and scalable data pipelines using Databricks, Apache Spark, SQL, and Python or Scala.
• Build solutions to process and transform large datasets efficiently.
• Monitor and optimize data pipelines for performance, scalability, throughput and reliability.
• Collaborate with data scientists, analysts, engineers and stakeholders to translate data needs into technical solutions.
• Implement data quality checks and governance to ensure data accuracy and compliance.
• Troubleshoot complex data and pipeline issues and propose sustainable solutions.
• Maintain clear technical documentation and contribute to engineering standards.

Skills

Data Engineering

Databricks

Apache Spark

SQL

Python

Azure

English

Portuguese

Education

Bachelor's degree in Computer Science or related

Tools

Databricks

Apache Spark

Azure

Python

Scala

Job description

CI&T is looking for a Data Engineer to design, build, and optimize scalable data solutions using Databricks, Apache Spark, SQL, Python, and Azure. You will collaborate with data scientists, analysts, engineers, and business stakeholders to transform large volumes of data into reliable data products.

You will design robust pipelines, implement data quality checks, monitor performance, and maintain documentation while working in a bilingual environment (English and Portuguese).