Senior Databricks Data Engineer: AI DataOps & Pipelines

Ciandt; United States; Remote; BRL 180,000 - 330,000; Full time

Benefits offered by this job

Health insurance

Meal allowance

Life insurance

Continuous learning (CI&T University)

Wellness program

PLR / Profit sharing

Gym partnerships

Discount club

Well-being resources

Job summary

CI&T is seeking a Senior Data Developer to design and maintain robust data pipelines using Databricks, PySpark, and Delta Lake. You will integrate data from APIs, databases, and files, collaborating with data scientists, analysts, and engineers to meet strategic needs.

You will ensure data quality, security, and governance while applying DataOps practices and CI/CD. Strong SQL, Python, and cloud experience are required.

Qualifications

• Proven experience building and managing large-scale data pipelines in Databricks (PySpark, Delta Lake, SQL)
• Strong programming skills in Python and SQL for data processing and transformation
• Deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing
• Hands-on experience with DataOps practices: Git version control, CI/CD, automated testing, IaC
• Familiarity with cloud platforms (AWS, Azure, or GCP) and related data services
• Strong problem-solving skills for performance, scalability, and reliability issues
• Proficiency in Git
• Advanced English is essential

Responsibilities

• Develop and maintain data pipelines.
• Integrate data from various sources (APIs, relational databases, files, etc.).
• Collaborate with business and analytics teams to understand data requirements.
• Ensure quality, reliability, security and governance of ingested data.
• Follow modern DataOps practices such as Code Versioning, Data Tests and CI/CD.
• Document processes and best practices in data engineering.

Skills

Databricks

PySpark

Delta Lake

Python

SQL

ETL/ELT

DataOps

Git

Cloud platforms (AWS/Azure/GCP)

English

Tools

Power BI

Job description

CI&T is seeking a Senior Data Developer to design and maintain robust data pipelines using Databricks, PySpark, and Delta Lake. You will integrate data from APIs, databases, and files, collaborating with data scientists, analysts, and engineers to meet strategic needs.

You will ensure data quality, security, and governance while applying DataOps practices and CI/CD. Strong SQL, Python, and cloud experience are required.