Lead Data Engineer

System Soft Technologies; United States; Remote; USD 140,000 - 200,000; Full time

Job summary

System Soft Technologies is seeking a Lead Data Engineer to design and build enterprise-scale data platforms and AI-ready analytics capabilities from a remote location. You will lead data product development, establish best practices, and mentor junior engineers across cross-functional teams.

The role emphasizes advanced Databricks usage, semantic modeling, and integration with marketing platforms to enable conversational BI and GenAI-powered insights.

Qualifications

• 7+ years of experience in Data Engineering and cloud-based analytics solutions.
• 3+ years hands-on experience with Azure Databricks and modern data platforms.
• Experience delivering enterprise-scale data, analytics, or AI solutions.
• Experience in Marketing Analytics, Customer Analytics, or Digital Marketing domain is highly preferred.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products using Azure Databricks and modern Data & AI engineering practices.
• Implement and optimize data ingestion, transformation, and orchestration frameworks using PySpark, SQL, Python, and Azure Data Factory.
• Build and manage curated data layers following Medallion Architecture (Bronze, Silver, Gold).
• Develop semantic models, business-ready datasets, and governed data products to support analytics and AI use cases.
• Implement and support Databricks Genie for conversational analytics and self-service business intelligence.
• Collaborate with architects, business analysts, and stakeholders to translate business requirements into scalable technical solutions.
• Integrate and model data from marketing platforms including Salesforce, Marketo, Google Ads, LinkedIn Ads, and Bing Ads.
• Ensure data quality, performance optimization, governance, and security using Unity Catalog and industry best practices.
• Support AI-powered analytics, GenAI, and conversational BI initiatives by enabling AI-ready data foundations.
• Mentor junior engineers, conduct code reviews, and promote engineering best practices across the team.
• Participate in Agile ceremonies, sprint planning, backlog refinement, and release activities.

Skills

Data engineering

Azure Databricks

AI analytics

PySpark

SQL

Python

Unity Catalog

Marketing analytics

Agile ceremonies

Mentoring

Tools

Databricks Genie

Azure Data Factory

Salesforce

Marketo

Google Ads

LinkedIn Ads

Bing Ads

Job description

Location: Remote

Duration: 6 Months

We are seeking a highly skilled Lead Data Engineer / Senior Data Engineer with strong hands‑on experience in Data Engineering, Azure Databricks, and AI‑enabled analytics to design, develop, and support enterprise‑scale Marketing Intelligence and Conversational BI solutions. The ideal candidate will be responsible for building scalable data platforms, developing data products, and enabling AI‑driven analytics capabilities using modern Databricks technologies.

Experience with Databricks Genie, Semantic Modeling, Salesforce, Marketo, and digital marketing platforms is highly desirable.

Qualification

• Design, develop, and maintain scalable data pipelines and data products using Azure Databricks and modern Data & AI engineering practices.
• Implement and optimize data ingestion, transformation, and orchestration frameworks using PySpark, SQL, Python, and Azure Data Factory.
• Build and manage curated data layers following Medallion Architecture (Bronze, Silver, Gold).
• Develop semantic models, business‑ready datasets, and governed data products to support analytics and AI use cases.
• Implement and support Databricks Genie for conversational analytics and self‑service business intelligence.
• Collaborate with architects, business analysts, and stakeholders to translate business requirements into scalable technical solutions.
• Integrate and model data from marketing platforms including Salesforce, Marketo, Google Ads, LinkedIn Ads, and Bing Ads.
• Ensure data quality, performance optimization, governance, and security using Unity Catalog and industry best practices.
• Support AI‑powered analytics, GenAI, and conversational BI initiatives by enabling AI‑ready data foundations.
• Mentor junior engineers, conduct code reviews, and promote engineering best practices across the team.
• Participate in Agile ceremonies, sprint planning, backlog refinement, and release activities.

Experience

• 7+ years of experience in Data Engineering and cloud‑based analytics solutions.
• 3+ years of hands‑on experience with Azure Databricks and modern data platforms.
• Experience delivering enterprise‑scale data, analytics, or AI solutions.
• Experience in Marketing Analytics, Customer Analytics, or Digital Marketing domain is highly preferred.