Senior Azure Databricks Developer | Charlotte, NC, USA

Tech Junction Ltd; Charlotte (NC); On-site; USD 140,000 - 180,000; Full time

Job summary

VDart seeks a Senior Azure Databricks Developer in Charlotte, NC to own end-to-end data pipeline design and cloud data warehousing initiatives. You will build and optimize PySpark notebooks and Python scripts in Azure Databricks, orchestrate complex workflows with ADF, and manage data across Azure Data Lake and SQL/Synapse environments.

You will collaborate within Agile teams, perform design and code reviews, and ensure compliance with data governance and security protocols, while delivering

Qualifications

• Bachelor's degree in Computer Science, Data Science, IT, Software Engineering, or related field.
• 8–10 years of progressive experience in data engineering, software development, and cloud data warehousing.
• Strong hands-on experience with Azure Databricks, PySpark, and Python.
• Deep expertise in Azure Data Factory, Azure Data Lake storage generations, and cloud data ingestion patterns.
• Strong knowledge of Azure SQL, Synapse Analytics, and MS SQL Server.
• Understanding of ETL/ELT concepts, data warehousing principles, and relational DBMS.
• Experience with design reviews, code reviews, and technical documentation.
• Legal work eligibility in the United States; availability for ETZ in Charlotte, NC.

Responsibilities

• Design, develop, test, and deploy scalable big data pipelines using Azure Databricks, PySpark, and Python.
• Architect and orchestrate ETL/ELT workflows with Azure Data Factory and Azure Data Lake.
• Manage relational data models, queries, and procedures across Azure SQL, Synapse, and MS SQL Server.
• Lead design reviews, peer code reviews, and maintain architectural documentation.
• Implement data governance, security, and performance tuning for large-scale data lakehouse environments.
• Collaborate in Agile teams, participate in sprint planning and stand-ups using JIRA.
• Work with insurance domain stakeholders to translate business requirements into data solutions.
• Ensure high availability SLAs, data latency targets, and data quality standards.

Skills

Azure Databricks

PySpark

Python

Azure Data Factory

Azure Data Lake

Azure SQL

Azure Synapse Analytics

MS SQL Server

ETL/ELT concepts

Data warehousing

JIRA

Confluence

Education

Bachelor's degree in Computer Science / related field

Tools

Azure Databricks

PySpark

Python

Azure Data Factory

Azure Data Lake

Azure SQL

Azure Synapse Analytics

MS SQL Server

JIRA

Confluence

Job description

Position Summary:

We are actively seeking an experienced, high-caliber, and technical Senior Azure Databricks Developer with 8 to 10 years of professional experience to join our client engineering team in Charlotte, NC, USA on a full-time contract basis. In this critical data engineering role, you will be responsible for building robust cloud data pipelines, architecting scalable data processing solutions, and integrating advanced analytics workflows. You will leverage your deep specialist expertise in Azure Databricks, PySpark, Python, Azure Data Factory, and Azure Data Lake to deliver uncompromised data integration, transformation, and reliability. This opportunity offers competitive compensation, executive technical visibility, and exceptional career growth prospects within a premier data-driven enterprise.

Detailed Job Description:

As a Senior Azure Databricks Developer supporting enterprise data engineering initiatives in Charlotte, NC, you will take full operational, tactical, and technical ownership of ETL pipeline design, big data processing, and cloud repository architecture. Your day-to-day responsibilities include developing optimized PySpark notebooks and Python scripts in Azure Databricks, orchestrating complex workflows using Azure Data Factory (ADF), managing data storage in Azure Data Lake, and maintaining relational integrity across Azure SQL, Azure Synapse Analytics, and MS SQL Server. You will also participate in rigorous design and code reviews, ensure adherence to technical documentation standards, and collaborate closely with Agile project teams and business stakeholders in the insurance domain. Furthermore, you will manage application requirements via direct email to Najma Haseen at [e-mail] or via LinkedIn direct messaging, ensuring strict compliance with 8-10 years of data engineering experience standards.

Key Responsibilities:

• Design, develop, test, and deploy scalable big data processing pipelines utilizing Azure Databricks, PySpark, and Python.
• Architect and orchestrate complex ETL/ELT workflows and data integration pipelines using Azure Data Factory (ADF) and Azure Data Lake.
• Manage and optimize relational data models, queries, and stored procedures across Azure SQL, Azure Synapse Analytics, and MS SQL Server.
• Lead comprehensive technical design reviews, peer code reviews, and maintain meticulous architectural documentation.
• Implement enterprise data governance, security protocols, and performance tuning for large-scale data lakehouse environments.
• Collaborate within Agile software development frameworks, participating in sprint planning, daily stand-ups, and backlog grooming via JIRA.
• Partner closely with insurance domain business stakeholders and data analysts to translate business requirements into technical data solutions.
• Ensure strict adherence to high-availability service level agreements (SLAs), data pipeline latency targets, and data quality standards.

Required Qualifications & Skills:

• Bachelor's degree in Computer Science, Data Science, Information Technology, Software Engineering, or a related technical discipline.
• Minimum of 8 to 10 years of progressive professional experience in data engineering, software development, and cloud data warehousing.
• Strong hands-on operational proficiency with Azure Databricks, PySpark, and Python programming.
• Deep expertise in Azure Data Factory (ADF), Azure Data Lake storage generations, and cloud data ingestion patterns.
• Strong practical knowledge of Azure SQL, Azure Synapse Analytics, and MS SQL Server database administration.
• Excellent understanding of ETL/ELT concepts, data warehousing principles, and relational database management systems.
• Proven experience conducting design reviews, code reviews, and authoring comprehensive technical documentation.
• Mandatory legal work eligibility in the United States with availability for work in the Eastern Time Zone (Charlotte, NC).

Nice-to-Have Skills:

• Working knowledge of data serialization formats (XML, JSON) and enterprise legacy migration (Informatica PowerCenter).
• Prior professional working experience in Agile methodologies using JIRA and Confluence.
• Deep domain knowledge and data modeling experience within the insurance or financial services sector.
• Strong client stakeholder management and technical advisory capabilities.

Application Information:

• Salary/Rate: Market Competitive / US Staffing Contract Package
• Deadline: Open until filled (Urgent Charlotte, NC Contract Opening)
• Notice Period: Immediate to 14 Days
• Contract Duration: Contract / Full-Time (Charlotte, NC, USA - Eastern Time Zone)

Recruitment Pro Tip:

When applying for this Senior Azure Databricks Developer position via VDart, ensure your resume explicitly highlights your PySpark/Databricks pipeline architecture, your ADF/Synapse integration history, and your insurance domain expertise.