Senior Data Engineer

Merci Technologies, Inc.; San Diego (CA); Remote; USD 140,000 - 210,000; Full time

Job summary

Merci Technologies is hiring a Senior Data Engineer to design, build, and optimize data infrastructure for AI and analytics workloads in a remote-capable role. You’ll work with data engineers, scientists, and business stakeholders to deliver reliable data platforms and pipelines that scale for ML and generative AI use cases.

The role emphasizes hands-on execution, system-thinking, clean code, and mentoring junior engineers.

Qualifications

• 5+ years in data or software engineering with data focus.
• Advanced SQL and Python; Scala/Java a plus.
• Hands-on with AWS, Azure, or GCP.
• Strong experience with Snowflake or Databricks, plus a cloud data platform.
• Experience with Spark and distributed processing.
• Knowledge of data modeling and medallion architecture.
• Experience with orchestration and dbt.
• Familiar with Git, CI/CD, and IaC tools like Terraform.
• Bachelor’s degree in CS/IS/Engineering or equivalent.

Responsibilities

• Design, build, and maintain scalable batch and real-time data pipelines using modern ELT/ETL patterns.
• Develop and optimize data models in cloud data warehouses and lakehouses such as Snowflake, Databricks, BigQuery, Redshift, or Azure Synapse.
• Build streaming data solutions using Kafka, Kinesis, Event Hubs, or Spark Structured Streaming.
• Prepare and curate high-quality datasets that support ML, analytics, and generative AI use cases, including RAG pipelines and vector data stores.
• Orchestrate workflows using Airflow, Dagster, Prefect, or equivalent tools.
• Implement transformation layers using dbt and SQL best practices.
• Establish data quality, validation, and observability frameworks to ensure accuracy and reliability.
• Apply data governance, lineage, cataloging, and access controls in line with security and compliance requirements.
• Lead migration of legacy on-premises data systems to modern cloud platforms.
• Use infrastructure-as-code and CI/CD practices to automate deployment of data infrastructure.
• Partner with data scientists, ML engineers, analysts, and stakeholders to translate requirements into technical solutions.
• Mentor junior engineers and contribute to engineering standards and documentation.

Skills

SQL

Python

Cloud platforms

Snowflake

Databricks

Spark

dbt

Airflow

Terraform

Git / CI/CD

Education

Bachelor’s degree in CS/IS/Engineering

Tools

Snowflake

Databricks

BigQuery

Redshift

Azure Synapse

Kafka

Kinesis

Event Hubs

dbt

Airflow

Dagster

Prefect

Terraform

Git

Job description

Category: Data & AI | Type: Full-Time or Contract | Condition: Permanent or Contract | Location: Remote

About the Role

Merci Technologies is partnering with a leading client organization to hire a Senior Data Engineer. Our client is investing heavily in AI and advanced analytics and needs an experienced engineer to design, build, and optimize the data infrastructure behind those efforts. You will work directly with the client’s data, analytics, and engineering teams to build reliable pipelines, modernize legacy data environments, and deliver platforms ready for machine learning and generative AI workloads. This is a high-impact, hands-on role for an engineer who enjoys solving complex data problems and seeing the work turn directly into business outcomes.

Responsibilities

• Design, build, and maintain scalable batch and real-time data pipelines using modern ELT/ETL patterns
• Develop and optimize data models in cloud data warehouses and lakehouses such as Snowflake, Databricks, BigQuery, Redshift, or Azure Synapse
• Build streaming data solutions using Kafka, Kinesis, Event Hubs, or Spark Structured Streaming
• Prepare and curate high-quality datasets that support machine learning, analytics, and generative AI use cases, including retrieval augmented generation (RAG) pipelines and vector data stores
• Orchestrate workflows using Airflow, Dagster, Prefect, or equivalent tools
• Implement transformation layers using dbt and SQL best practices
• Establish data quality, validation, and observability frameworks to ensure accuracy and reliability
• Apply data governance, lineage, cataloging, and access controls in line with security and compliance requirements
• Lead migration of legacy on-premises data systems to modern cloud platforms
• Use infrastructure-as-code and CI/CD practices to automate deployment of data infrastructure
• Partner with the client’s data scientists, ML engineers, analysts, and business stakeholders to translate requirements into technical solutions
• Mentor junior engineers and contribute to engineering standards and documentation

Qualifications

Required

• 5+ years of experience in data engineering, software engineering, or a related field
• Advanced proficiency in SQL and Python (Scala or Java a plus)
• Hands-on experience with at least one major cloud platform (AWS, Azure, or GCP)
• Strong experience with Snowflake, Databricks, or a comparable cloud data platform
• Experience with Apache Spark and distributed data processing
• Solid understanding of data modeling, including dimensional modeling and medallion architecture
• Experience with workflow orchestration and dbt
• Familiarity with Git, CI/CD pipelines, and infrastructure-as-code tools such as Terraform
• Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent experience

Preferred

• Experience supporting ML pipelines, feature stores, or MLOps workflows
• Exposure to vector databases (Pinecone, Weaviate, pgvector) and LLM data preparation
• Knowledge of data governance tools such as Unity Catalog, Collibra, Alation, or Purview
• Experience in regulated environments (financial services, healthcare, CPG, or public sector)
• Certifications such as Databricks Certified Data Engineer, SnowPro, AWS Data Engineer, Azure Data Engineer Associate, or Google Professional Data Engineer

What You Will Bring

You are a builder who takes pride in data that just works. You think in systems, write clean and maintainable code, and understand that reliable data is the foundation for every AI and analytics initiative. You communicate clearly with technical and non-technical stakeholders, take ownership of outcomes, and keep up with the fast-moving data and AI ecosystem. Above all, you want your work to make a measurable difference.