Sr. Data Engineer

WebSenor Ltd; United States; Remote; USD 110,000 - 150,000; Full time

Job summary

A leading company is seeking a Senior Data Engineer to enhance their data engineering team. This role involves designing scalable data pipelines, collaborating with stakeholders for effective data integration, and ensuring high standards of data quality and governance. Ideal candidates will possess deep expertise in Azure technologies, strong scripting abilities in Python and C#, and extensive data engineering experience.

Qualifications

• 6+ years of experience as a Data Engineer or similar role.
• Strong experience with Azure Data Factory, Databricks, and SQL Server.
• Proficiency in Python and C#.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory and Databricks.
• Collaborate with teams to understand data requirements.
• Implement and maintain CI/CD pipelines for data workflows.

Skills

Azure Data Factory

Databricks

SQL

Python

C#

CI/CD

Job description

Job Title: Senior Data Engineer
Experience: 6+ Years
Location: Remote
Employment Type: Contract

Job Summary

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic data engineering team. The ideal candidate will bring deep expertise in C#, Azure Data Factory (ADF), Databricks, SQL Server, and Python, along with a strong understanding of modern CI/CD practices. You will be responsible for designing, developing, and maintaining scalable, efficient data pipelines and solutions to support analytics, reporting, and operational systems.

Key Responsibilities

Design, develop, and maintain robust and scalable ETL/ELT pipelines using Azure Data Factory, Databricks, and SQL Server.

Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver optimal solutions.

Develop and deploy data integration and transformation workflows that ensure high performance and data integrity.

Write optimized, efficient SQL queries for data extraction, manipulation, and reporting purposes.

Leverage Python and C# for data processing, automation tasks, and custom tool development.

Implement and maintain CI/CD pipelines for data engineering workflows using tools like Azure DevOps or GitHub Actions.

Monitor and troubleshoot data pipelines to ensure high availability and performance.

Ensure data quality, consistency, and governance in all aspects of pipeline development.

Participate in code reviews, mentoring, and best practice sharing within the data engineering team.

Required Skills and Qualifications

6+ years of experience as a Data Engineer or in a similar role.

Strong experience with Azure Data Factory, Databricks, and SQL Server.

Proficiency in Python and C# for scripting and automation.

Solid understanding of data warehousing concepts and ETL/ELT best practices.

Experience with CI/CD tools and practices, especially in cloud data environments.

Familiarity with data governance, data security, and regulatory compliance.

Strong problem-solving skills, attention to detail, and a proactive mindset.

Excellent communication and collaboration skills in a remote team environment.

Preferred Qualifications

Experience with other Azure data services (e.g., Azure Synapse Analytics, Azure Data Lake Storage).

Familiarity with big data technologies and distributed computing concepts.

Background in financial services, healthcare, or another data-intensive industry (optional but preferred).