Senior Azure Databricks Architect — Cloud Data Pipelines
Tech Junction Ltd
A complete application in a minute
Tailored resume and cover letter, ready to send.
Job summary
VDart seeks a Senior Azure Databricks Developer in Charlotte, NC to own end-to-end data pipeline design and cloud data warehousing initiatives. You will build and optimize PySpark notebooks and Python scripts in Azure Databricks, orchestrate complex workflows with ADF, and manage data across Azure Data Lake and SQL/Synapse environments.
You will collaborate within Agile teams, perform design and code reviews, and ensure compliance with data governance and security protocols, while delivering
Qualifications
- Bachelor's degree in Computer Science, Data Science, IT, Software Engineering, or related field.
- 8–10 years of progressive experience in data engineering, software development, and cloud data warehousing.
- Strong hands-on experience with Azure Databricks, PySpark, and Python.
- Deep expertise in Azure Data Factory, Azure Data Lake storage generations, and cloud data ingestion patterns.
- Strong knowledge of Azure SQL, Synapse Analytics, and MS SQL Server.
- Understanding of ETL/ELT concepts, data warehousing principles, and relational DBMS.
- Experience with design reviews, code reviews, and technical documentation.
- Legal work eligibility in the United States; availability for ETZ in Charlotte, NC.
Responsibilities
- Design, develop, test, and deploy scalable big data pipelines using Azure Databricks, PySpark, and Python.
- Architect and orchestrate ETL/ELT workflows with Azure Data Factory and Azure Data Lake.
- Manage relational data models, queries, and procedures across Azure SQL, Synapse, and MS SQL Server.
- Lead design reviews, peer code reviews, and maintain architectural documentation.
- Implement data governance, security, and performance tuning for large-scale data lakehouse environments.
- Collaborate in Agile teams, participate in sprint planning and stand-ups using JIRA.
- Work with insurance domain stakeholders to translate business requirements into data solutions.
- Ensure high availability SLAs, data latency targets, and data quality standards.
Skills
Education
Tools
Job description
VDart seeks a Senior Azure Databricks Developer in Charlotte, NC to own end-to-end data pipeline design and cloud data warehousing initiatives. You will build and optimize PySpark notebooks and Python scripts in Azure Databricks, orchestrate complex workflows with ADF, and manage data across Azure Data Lake and SQL/Synapse environments.
You will collaborate within Agile teams, perform design and code reviews, and ensure compliance with data governance and security protocols, while delivering
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
