Master Data Developer (Databricks), Colombia

Ciandt

Colombia

À distance

COP 298 815 000 - 398 420 000

Plein temps

Il y a 13 heures
Soyez parmi les premiers à postuler

Une candidature complète en une minute

Un CV et une lettre de motivation personnalisés, prêts à envoyer.

Passez les filtres ATS

Avantages offerts par ce poste

Premium Healthcare
Meal voucher
Maternity and Parental leaves
Mobile services subsidy
Sick pay-Life insurance
CI&T University
Colombian Holidays
Paid Vacations

Résumé du poste

CI&T is seeking a Master Data Developer (Databricks) in Colombia to design, develop, and maintain scalable data infrastructures and pipelines. You will implement data architectures, ensure data quality, and collaborate with analytics teams to deliver high-performance data solutions.

The role requires strong experience in Databricks, data migration, and modern data stack technologies, with a focus on security and governance across cloud platforms.

Qualifications

  • Hands-on Databricks migration with migration tools and Unity catalog.
  • Experience migrating notebooks and DAG/ETL pipelines.
  • Experience with AWS services: S3, Glue, Secrets Manager, IAM, Athena.
  • Proficiency in Spark SQL, PySpark, Java, Python and Scala.

Responsabilités

  • Build and optimize scalable data pipelines and ETL/ELT processes.
  • Implement and optimize data architectures: data lakes, warehouses, integration layers.
  • Ensure data quality, governance, and security across systems.
  • Develop integrations between data sources and analytics platforms.
  • Collaborate with analytics managers to translate requirements into technical solutions.
  • Monitor and optimize pipeline performance and storage costs.
  • Create technical documentation for architectures and processes.
  • Leverage AI tools to enhance data engineering workflows.

Connaissances

Databricks migration
Unity catalog to Glue Catalog
Notebook migration
DLT pipelines
Delta lake to Iceberg
AWS S3 & Glue
Spark SQL
PySpark
Java / Python / Scala

Outils

S3
Glue
Athena
LakeFormation
EMR

Description du poste

Master Data Developer (Databricks), Colombia

At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions.

With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy.

We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day.

As CI&T grows, we seek a talented, motivated, and experienced Data Developer to join our team and build robust data-driven solutions for our customers. This role is pivotal in designing, developing, and maintaining scalable data infrastructure and pipelines that enable our data initiatives to thrive.

The Data Developer will be responsible for implementing and optimizing data architectures, ensuring data quality and reliability, and collaborating closely with analytics and business teams to deliver high-performance data solutions. This position requires strong technical expertise in data engineering, cloud platforms, and modern data stack technologies, combined with the ability to translate business requirements into effective technical solutions.

Responsibilities:

  • Data Pipeline Development: Build, optimize, and maintain scalable data pipelines and ETL/ELT processes to support analytics and business intelligence needs.
  • Data Architecture Implementation: implement data architectures, including data lakes, warehouses, and integration layers that support the organization's data strategy and modernization.
  • Data Quality & Governance: Implement data quality checks, validation processes, and monitoring systems to ensure data accuracy, consistency, and reliability.
  • Integration & API Development: Develop and maintain integrations between various data sources, third-party tools, and analytics platforms.
  • Collaboration: Work closely with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions that enable data-driven decision making.
  • Performance Optimization: Monitor and optimize data pipeline performance, query efficiency, and storage costs to ensure scalable and cost-effective solutions.
  • Documentation: Create and maintain comprehensive technical documentation for data architectures, pipelines, and processes.
  • Data Security: Implement security best practices and ensure compliance with data privacy regulations across all data systems.
  • Powered by AI: Leverage AI tools and automation to enhance data engineering workflows and productivity.

Requirements:

  • Hands-on Databricks migration with the databricks/migrate tool (notebooks,
    jobs, clusters, secrets, ACLs), Unity catalog to Glue Catalog
  • Notebook and job migration: notebooks, DAG pipelines, DLT pipelines
  • Storage path refactoring: /mnt/ DBFS references to S3-backed locations
  • Data migration execution: Deltalake format to Iceberg format on S3 (Delta tables, raw files), incremental sync
  • Streaming: Auto Loader / DLT checkpoint re-bootstrap, event_id dedup
  • Validation: row counts, checksums, sample queries, ETL runtime comparison
  • Secrets: recreate secret scopes in AWS Secrets Manager
  • AWS services: S3, Glue, Secrets Manager, IAM basics, Athena, LakeFormation and
    EMR
  • Languages: Spark SQL, PySpark, Java, Python and Scala
  • Agile pod delivery: daily ticket updates, standups, code review

Nice to Have: Databricks Certified Consultant (Associate/Professional); DBRto EMR on AWS migration experience #LI-LO1

Our benefits include:
  • Premium Healthcare
  • Meal voucher
  • Maternity and Parental leaves
  • Mobile services subsidy
  • Sick pay-Life insurance
  • CI&T University
  • Colombian Holidays
  • Paid Vacations And many others.

Collaboration is our superpower, diversity unites us, and excellence is our standard. We value diverse identities and life experiences, fostering a diverse, inclusive, and safe work environment. We encourage applications from diverse and underrepresented groups to our job positions.

Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Master Data Developer (Databricks), Colombia
Master Data Developer (Databricks), Colombia

CI&T • Colombie

Sur place
COP 60 000 000 - 120 000 000
Maternity and Parental leaves
Mobile services subsidy
Sick pay-Life insurance
+3
[Job-32426] Data Architect (Databricks), Colombia
[Job-32426] Data Architect (Databricks), Colombia

Ciandt • Colombie

À distance
COP 100 000 000 - 180 000 000
Premium Healthcare
Meal voucher
Maternity and Parental leaves
+5
Data Architect (Databricks), Colombia
Data Architect (Databricks), Colombia

Ciandt • Colombie

À distance
COP 120 000 000 - 180 000 000
Premium Healthcare
Meal voucher
Maternity and Parental leaves
+6
Data Architect (Databricks), Colombia
Data Architect (Databricks), Colombia

CI&T • Colombie

Sur place
COP 120 000 000 - 240 000 000
Maternity and Parental leaves
Mobile services subsidy
Sick pay-Life insurance
+3
[Job -31821] Master Data Architect (Databricks), Colombia
[Job -31821] Master Data Architect (Databricks), Colombia

Ciandt • Colombie

Sur place
COP 180 000 000 - 240 000 000
Premium Healthcare
Meal Voucher
Maternity and Parental leaves
+5
[Job - 32043] Senior Data Architect, Colombia
[Job - 32043] Senior Data Architect, Colombia

Ciandt • Colombie

Sur place
COP 180 000 000 - 300 000 000
Premium Healthcare
Meal voucher
Maternity and Parental leaves
+4
Master Data Architect, Databricks & Modernization Lead
Master Data Architect, Databricks & Modernization Lead

Ciandt • Colombie

Sur place
COP 180 000 000 - 240 000 000
Premium Healthcare
Meal Voucher
Maternity and Parental leaves
+5
Senior Data Developer, Colombia
Senior Data Developer, Colombia

CI&T • Colombie

Sur place
COP 162 680 982 - 325 361 964
Maternity and parental leaves
Mobile services subsidy
Sick pay
+3
Senior Data Architect — Lakehouse & DataOps Lead
Senior Data Architect — Lakehouse & DataOps Lead

Ciandt • Colombie

À distance
COP 180 000 000 - 300 000 000
Premium Healthcare
Meal voucher
Maternity and Parental leaves
+4
Data Manager, Colombia
Data Manager, Colombia

Ciandt • Colombie

À distance
COP 120 000 000 - 180 000 000
Premium Healthcare
Meal voucher
Maternity and Parental leaves
+5