Master Data Developer (Databricks), Brazil

CI&T; Brasil; Híbrido; BRL 120 000 - 180 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health insurance

Meal allowance

Extended paternity leave

Wellness partnerships

Profit Sharing and Results Participa

Life insurance

Continuous learning platform (CI&T Uni

Discount club

Well-being platform

Pregnancy and parenting course

Resumo da oferta

CI&T is seeking a Data Developer to design, build, and maintain scalable data infrastructure and pipelines for enterprise analytics. You will implement and optimize data architectures, ensure data quality, and collaborate with analytics teams to translate business needs into robust technical solutions.

Responsibilities include developing scalable ETL/ELT processes, building data lakes and warehouses, enforcing data governance, and delivering performance-tuned data flows.

Qualificações

• Experience with designing and implementing scalable data pipelines and ETL/ELT processes.
• Experience building data lakes and warehouses and integrating data sources.
• Knowledge of data quality, governance, and security practices.
• Proficiency with Spark, PySpark, Python, SQL, and Scala.
• Experience with cloud data services (AWS/GCP/Azure) and modern data stack.
• Agile/Dedicated collaboration with analytics/business teams.

Responsabilidades

• Data Pipeline Development: Build, optimize, and maintain scalable data pipelines and ETL/ELT processes to support analytics and BI needs.
• Data Architecture Implementation: implement data architectures, including data lakes, warehouses, and integration layers that support the data strategy.
• Data Quality & Governance: Implement data quality checks, validation processes, and monitoring systems for data integrity.
• Integration & API Development: Develop and maintain integrations between data sources, tools, and analytics platforms.
• Collaboration: Work with Data & Analytics managers and stakeholders to translate requirements into technical solutions.
• Performance Optimization: Monitor and optimize pipeline performance and storage costs.
• Documentation: Maintain technical documentation for data architectures and processes.
• Data Security: Ensure compliance with data privacy regulations.

Conhecimentos

Data engineering

ETL/ELT

Data quality

SQL

Python

PySpark

Scala

Cloud platforms

Ferramentas

Databricks migration tool

Unity Catalog to Glue Catalog

AWS Secrets Manager

S3

Glue

LakeFormation

EMR

Spark SQL

Descrição da oferta de emprego

At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions.

With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy.

We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day.

As CI&T grows, we seek a talented, motivated, and experienced Data Developer to join our team and build robust data-driven solutions for our customers. This role is pivotal in designing, developing, and maintaining scalable data infrastructure and pipelines that enable our data initiatives to thrive.

The Data Developer will be responsible for implementing and optimizing data architectures, ensuring data quality and reliability, and collaborating closely with analytics and business teams to deliver high-performance data solutions. This position requires strong technical expertise in data engineering, cloud platforms, and modern data stack technologies, combined with the ability to translate business requirements into effective technical solutions.

Responsibilities

• Data Pipeline Development: Build, optimize, and maintain scalable data pipelines and ETL/ELT processes to support analytics and business intelligence needs
• Data Architecture Implementation: implement data architectures, including data lakes, warehouses, and integration layers that support the organization's data strategy and modernization
• Data Quality & Governance: Implement data quality checks, validation processes, and monitoring systems to ensure data accuracy, consistency, and reliability
• Integration & API Development: Develop and maintain integrations between various data sources, third-party tools, and analytics platforms
• Collaboration: Work closely with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions that enable data-driven decision making
• Performance Optimization: Monitor and optimize data pipeline performance, query efficiency, and storage costs to ensure scalable and cost-effective solutions
• Documentation: Create and maintain comprehensive technical documentation for data architectures, pipelines, and processes
• Data Security: Implement security best practices and ensure compliance with data privacy regulations across all data systems
• Powered by AI: Leverage AI tools and automation to enhance data engineering workflows and productivity

Requirements

• Hands-on Databricks migration with the databricks/migrate tool (notebooks,
  jobs, clusters, secrets, ACLs), Unity catalog to Glue Catalog
• Notebook and job migration: notebooks, DAG pipelines, DLT pipelines
• Storage path refactoring: /mnt/ DBFS references to S3-backed locations
• Data migration execution: Deltalake format to Iceberg format on S3 (Delta tables, raw files), incremental sync
• Streaming: Auto Loader / DLT checkpoint re-bootstrap, event_id dedup
• Secrets: recreate secret scopes in AWS Secrets Manager
• AWS services: S3, Glue, Secrets Manager, IAM basics, Athena, LakeFormation and
  EMR
• Languages: Spark SQL, PySpark, Java, Python and Scala
• Agile pod delivery: daily ticket updates, standups, code review

Nice to Have

Databricks Certified Consultant (Associate/Professional); DBRto EMR on AWS migration experience

• -Health and dental insurance
• -Meal and food allowance
• -Extended paternity leave
• -Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass;
• - Profit Sharing and Results Participation (PLR);
• -Life insurance
• - Continuous learning platform (CI&T University);
• -Discount club
• -Free online platform dedicated to physical, mental, and overall well-being
• -Pregnancy and responsible parenting course
• -Partnerships with online learning platforms
• -Language learning platform

And many more!

More details about our benefits here: https://ciandt.com/br/pt-br/carreiras

At CI&T, inclusion starts at the first contact. If you are a person with a disability, it is important to present your assessment during the selection process. See which data needs to be included in the report by clicking here. This way, we can ensure the support and accommodations that you deserve. If you do not yet have the assessment, don't worry: we can support you in obtaining it.

We have a dedicated Health and Well-being team, inclusion specialists, and affinity groups who will be with you at every stage. Count on us to make this journey side by side.