Senior Databricks Data Engineer — Build Scalable Pipelines

Logoslabs; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Health and dental insurance

Meal and food allowance

Childcare assistance

Extended paternity leave

Well-being programs

Job summary

CI&T is seeking a Senior Data Developer in San Francisco to design and build robust data pipelines using Databricks, PySpark, and Delta Lake. You will transform data from APIs, databases, and files, while guiding peers and collaborating with data scientists, analysts, and platform engineers.

You will ensure data quality, governance, and security and contribute to DataOps practices, CI/CD, and testing. English fluency is essential for global collaboration.

Qualifications

• Proven experience building large-scale data pipelines in Databricks (PySpark, Delta Lake, SQL).
• Strong Python and SQL programming for data processing and transformation.
• Deep understanding of ETL/ELT frameworks and data warehousing.
• Hands-on experience with DataOps practices: Git, CI/CD, automated testing, IaC.
• Familiarity with cloud platforms (AWS, Azure, or GCP).
• Strong problem-solving for performance, scalability and reliability.
• Proficiency in Git.
• Advanced English is essential.

Responsibilities

• Develop and maintain data pipelines across diverse sources (APIs, databases, files).
• Integrate and transform data for business and analytics needs.
• Collaborate with data scientists, analysts, and engineers to deliver high-quality data products.
• Ensure data quality, governance, security, and reliability of ingested data.
• Follow DataOps practices including version control, testing, and CI/CD.

Skills

Databricks

PySpark

Delta Lake

SQL

Python

ETL/ELT

DataOps

Git

Cloud platforms (AWS/Azure/GCP)

English

CI/CD

Data quality

Tools

Databricks

Power BI

CI/CD pipelines

Job description

CI&T is seeking a Senior Data Developer in San Francisco to design and build robust data pipelines using Databricks, PySpark, and Delta Lake. You will transform data from APIs, databases, and files, while guiding peers and collaborating with data scientists, analysts, and platform engineers.

You will ensure data quality, governance, and security and contribute to DataOps practices, CI/CD, and testing. English fluency is essential for global collaboration.