Tech Lead

Jobgether

United States

Remote

USD 55,000 - 90,000

Full time

6 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Meal allowance
Medical and dental insurance
Home office allowance
Fully remote, full-time employment
Learning and certification programs
Wellness and fitness benefits

Job summary

Jobgether seeks a senior Tech Lead – Data Engineering to drive the data layer for an enterprise AI product in a remote setup. You will lead scalable pipelines with Databricks, PySpark, and SQL, shaping medallion architecture and governance frameworks. collaboration with product teams, and delivery of incremental AI-ready datasets.

You will mentor engineers, enforce best practices, and partner with data governance to ensure secure, observable data operations across landing to gold layers.

Qualifications

  • Strong hands-on Databricks, PySpark, and Spark SQL in production environments.
  • Practical knowledge of medallion architecture (landing, bronze, silver, gold).
  • Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
  • Experience preparing data for generative AI, including embeddings and vector indexes.
  • Ability to integrate data with APIs while ensuring security and observability.
  • Strong data modeling skills across silver and gold analytical layers.
  • Familiarity with Git, CI/CD, automated testing, and data quality validation.
  • Experience in agile product teams and delivering incremental improvements.
  • Excellent communication with technical and business stakeholders.
  • Bachelor's degree in IT or related field.

Responsibilities

  • Lead data pipelines using PySpark and SQL within Databricks.
  • Implement and evolve medallion architecture across data layers.
  • Work with Delta Lake, Unity Catalog, and Databricks Workflows to ensure reliability.
  • Prepare datasets for generative AI services including embeddings and vector indexes.
  • Integrate data with APIs and external services while maintaining security and observability.
  • Design and maintain gold data layer for product, Power BI, and analytics.
  • Develop reusable engineering frameworks and automated data quality controls.
  • Apply software delivery best practices: Git, CI/CD, testing, and documentation.
  • Collaborate with product teams for incremental, valuable deliveries.
  • Support governance teams on data classification, lineage, and access controls.

Skills

Databricks
PySpark
Spark SQL
Medallion architecture
Delta Lake
Unity Catalog
APIs
Git & CI/CD
Data governance
Agile teams

Education

Bachelor's degree in IT

Tools

Databricks Genie
Power BI
Oracle
GoldenGate/CDC
SQL Server
Confluence

Job description

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Tech Lead – Data Engineering based in Brazil.

In this senior, hands-on role, you will lead the development and evolution of the data layer powering an enterprise artificial intelligence product within the mobility infrastructure sector. You will design and maintain scalable data pipelines, prepare reliable datasets for generative AI services, and connect data platforms with APIs and analytical applications. Working primarily with Databricks, PySpark, and SQL, you will help shape modern data architecture while ensuring performance, quality, security, and governance. You will collaborate closely with product teams, guide technical decisions, and promote engineering best practices through reusable frameworks and incremental delivery. This is an opportunity to combine technical leadership with practical data engineering expertise in a remote, AI-focused environment.

Accountabilities
  • Understand the product, its users, and underlying data sources, taking technical ownership of requirements and delivery.
  • Build, maintain, and optimize data pipelines using PySpark and SQL within Databricks.
  • Implement and evolve medallion architecture across landing, bronze, silver, and gold layers.
  • Work with Delta Lake, Unity Catalog, and Databricks Workflows to support reliable and scalable data operations.
  • Prepare and manage datasets for generative AI services, including embeddings, vector indexes, data quality, and versioning.
  • Integrate data with AI services and Databricks Genie.
  • Consume and expose data through APIs, ensuring appropriate security, monitoring, and observability.
  • Design and maintain the gold data layer, making trusted datasets available to the product, Power BI, and analytical applications.
  • Develop reusable engineering frameworks, automated validations, and data quality controls.
  • Apply software delivery best practices, including Git versioning, CI/CD, automated testing, code reviews, and technical documentation.
  • Collaborate with product teams through short development cycles, prioritizing incremental and valuable deliveries.
  • Support technical estimates, guide squad members, and share engineering standards and best practices.
  • Work with Data Governance teams to implement data classification, anonymization, lineage, and access controls.
Requirements:
  • Solid professional experience in data engineering, including acting as a technical reference or lead within an engineering squad.
  • Strong hands‑on experience with Databricks, PySpark, and Spark SQL in production environments.
  • Practical knowledge of medallion architecture, including landing, bronze, silver, and gold layers.
  • Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
  • Experience preparing and managing data for generative AI applications, including embeddings, vector indexes, data quality, and version control.
  • Ability to integrate data with APIs and external services while maintaining security and observability.
  • Strong data modeling skills, particularly across silver and gold analytical layers.
  • Familiarity with Git, CI/CD pipelines, code reviews, automated testing, and data quality validation.
  • Experience working in agile product teams and delivering improvements incrementally.
  • Strong communication skills, with the ability to collaborate effectively with both technical and business stakeholders.
  • Degree or formal education in Information Technology or a related field.
  • Nice to have: Experience with Databricks Genie and enterprise generative AI implementation standards.
  • Nice to have: Knowledge of data governance for AI applications and Brazilian data protection requirements (LGPD), including classification, anonymization, lineage, and access management.
  • Nice to have: Experience with Oracle and GoldenGate/CDC replication, SAP data extraction through Open Hub, SQL Server, or Microsoft Graph (SharePoint/Microsoft 365).
  • Nice to have: Familiarity with Power BI, service identities, secrets management solutions, and technical documentation using Confluence.
Benefits:
  • Meal and/or food allowance.
  • Medical and dental insurance.
  • Health coverage for pets.
  • Home office allowance for remote employees.
  • Welcome kit during onboarding.
  • Life insurance.
  • Access to psychological and nutritional support through Zenklub.
  • TotalPass wellness and fitness benefits.
  • Certification and continuous learning programs with partners including Adobe, Anthropic, AWS, Google, Microsoft, and Salesforce.
  • Professional development partnerships and discounts.
  • Access to UOL digital content.
  • Maternity welcome kit.
  • Extended maternity and paternity leave.
  • Employee referral program.
  • Fully remote, full-time employment.
  • Opportunity to work with advanced AI, generative AI, and modern data engineering technologies.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Scientist / ML Engineer
Senior Data Scientist / ML Engineer

Addepto • United States

Remote
USD 150,000 - 210,000
Remote work flexibility
Career development & training
Equipment provided
Data Engineer (Spark)
Data Engineer (Spark)

Addepto • Town of Poland (NY)

Hybrid
USD 110,000 - 170,000
Flexible remote or office work
Professional training and conferences
Paid time off
+2
AI Data Solutions Engineer New
AI Data Solutions Engineer New

Binariks Inc. • United States

Hybrid
USD 120,000 - 180,000
Vacation days
Sick leave
Medical insurance
+4
Data Engineer Remote Latin America
Data Engineer Remote Latin America

Fractal River • United States

Hybrid
USD 70,000 - 120,000
Personal development plan
Access to a reference library
Unlimited access to AI tools
+3
Senior Agentic Data Management Advisor
Senior Agentic Data Management Advisor

Lingaro • Town of Poland (NY)

Hybrid
USD 120,000 - 160,000
Stable employment
Flexibility regarding working hours
Unlimited access to the Udemy learning platform
Senior Data & AI Engineer
Senior Data & AI Engineer

encora10 • United States

Remote
USD 140,000 - 180,000
Data Engineer
Data Engineer

Free resume • United States

Remote
USD 120,000 - 180,000
Senior Data Engineer / AI Specialist
Senior Data Engineer / AI Specialist

Codebridge Technology, Inc. • Dover (DE)

Remote
USD 140,000 - 210,000
Remote work
Flexible hours
20 vacation days + 15 sick days
+3
Databricks Subject Matter Expert
Databricks Subject Matter Expert

Jobgether • United States

Remote
USD 135,000 - 165,000
Health insurance
401(k) matching
Paid time off
+1
[ Job - 31855] Data Product Manager, Brazil
[ Job - 31855] Data Product Manager, Brazil

Ciandt • United States

Remote
USD 140,000 - 190,000
Health and dental insurance
Meal and food allowance
Childcare assistance
+5