Data Engineering Team Lead: Cloud, Spark & Data Solutions

DataArt; United States; Hybrid; USD 140,000 - 200,000; Full time

Benefits offered by this job

Vacation days up to 26 days per year

Health and life insurance

English language classes from day one

Flexible hybrid work options

Certification reimbursement (AWS, GCP)

Job summary

DataArt in the United States is seeking a senior Data Engineer Lead to drive the Data Engineering team, set technical direction, and coordinate delivery across architectures, product and client work.

You will translate client requirements into scalable data pipelines using AWS, Spark, SQL and modern CI/CD practices, review solutions, mentor engineers, and manage risks and priorities to ensure high-quality execution.

Qualifications

• Strong experience in data engineering, technical leadership, and delivery coordination.
• Hands-on with at least one major cloud platform (AWS, Azure, or GCP).
• Experience with batch processing, object storage, relational databases, messaging, containers, and serverless computing.
• Experience with Apache Spark.
• Strong SQL knowledge, relational databases, and data modelling.
• Experience with Git-based development and CI/CD (GitHub Actions, Azure DevOps or equivalent).
• Experience with web scraping and web parsing.
• Understanding of Master Data Management.
• Ability to translate client requirements into technical scope, estimates, and delivery plans.
• Ability to evaluate technical and architectural proposals and provide constructive feedback.
• Strong ownership and proactive blockers resolution.
• Clear communication and collaboration skills.
• Nice to have: experience on other cloud platforms and engineering tools.

Responsibilities

• Lead the day-to-day work of the Data Engineering team, including technical direction, planning, prioritization, and delivery.
• Translate client requirements into clear technical scope, implementation activities, estimates, and delivery plans.
• Contribute to solution design by proposing approaches, reviewing architectural decisions, and constructively challenging them when appropriate.
• Work closely with the Data Architect to ensure proposed solutions are practical, scalable, and aligned with implementation needs.
• Coordinate team priorities, task ownership, dependencies, and progress.
• Review technical solutions, code changes, database changes, and implementation approaches.
• Support engineers with technical decisions and blockers.
• Take ownership of problems and drive them through investigation, resolution, and validation.
• Identify risks and dependencies early and coordinate with the relevant teams to resolve them.
• Collaborate with Backend, AI, QA, DevOps, Product, and Architecture teams.
• Communicate progress, risks, estimates, and technical decisions clearly to stakeholders.

Skills

Data engineering leadership

Delivery coordination

Cloud platforms (AWS/Azure/GCP)

Batch processing and object storage

Relational databases & SQL

Apache Spark

Git-based development & CI/CD

Web scraping / parsing

Master Data Management understanding

Requirements translation into scope &

Architecture evaluation & feedback

Problem ownership & blockers

Communication & collaboration

Experience across cloud platforms

Tools

GitHub

CI/CD tools

Job description

DataArt in the United States is seeking a senior Data Engineer Lead to drive the Data Engineering team, set technical direction, and coordinate delivery across architectures, product and client work.

You will translate client requirements into scalable data pipelines using AWS, Spark, SQL and modern CI/CD practices, review solutions, mentor engineers, and manage risks and priorities to ensure high-quality execution.