Azure Databricks Architect

VBeyond Corporation; Richardson (TX); On-site; USD 148,000 - 287,500; Full time

Job summary

A leading technology firm is seeking an experienced AWS Cloud Architect and Advisory to drive innovation in data solutions. The role requires a Bachelor's degree and at least 11 years of IT experience, with proven expertise in architecting solutions on Azure, along with proficiency in data engineering tools like Spark, Kafka, and Python. This position is critical for leading teams, ensuring data privacy, and implementing comprehensive data strategies. Competitive salary and full-time employment offered in Richardson, Texas.

Qualifications

• At least 11 years of experience with Information Technology.
• Proven hands-on experience in architecting solutions on Azure, Databricks, and Spark.
• Strong understanding of retail or CPG data domains.

Skills

Data engineering tools: Spark

Data engineering tools: Kafka

Data engineering tools: Airflow

Data engineering tools: SQL

Data engineering tools: Python

Architecting solutions on Azure

Experience designing distributed systems

Data privacy and compliance expertise

End-to-end implementation experience

Experience in mentoring teams

Education

Bachelor’s degree or foreign equivalent

Tools

CI/CD platforms: GitLab CI

CI/CD platforms: GitHub Actions

CI/CD platforms: AWS CodeBuild

CI/CD platforms: Azure Pipelines

CI/CD platforms: Jenkins

Job description

MO-3-12-AWS Cloud Architect and Advisory (759545)

Location: Durham, NC

Salary: USD 148,000.00-287,500.00

Posted: 2 weeks ago

Seniority level: Mid‑Senior level

Employment type: Full‑time

Job function: Information Technology

Industries: Staffing and Recruiting and Information Services

Required Qualifications

• Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 11 years of experience with Information Technology.
• Proven hands‑on experience in architecting solutions on Azure, Databricks and Spark.
• Proficiency in data engineering tools: Spark, Kafka, Airflow, SQL, Python.
• Strong understanding of retail or CPG data domains.
• Familiarity with vector search and semantic retrieval for retail use cases.
• Expertise in data privacy and compliance in consumer‑facing environments.
• Good experience in end‑to‑end implementation of data warehouse and data marts.
• Strong understanding of Data Warehousing concepts of Data Vault, Dimensional modelling, OLAP design.
• Extensive experience with CI/CD platforms such as GitLab CI, GitHub Actions, AWS CodeBuild, Azure Pipelines, and Jenkins.
• Experience designing and implementing complex solutions for distributed systems.
• Experience in leading and mentoring teams.

Preferred Qualifications

• Experience in Relational Modeling, Dimensional Modeling and Modeling of Unstructured Data
• Familiarity with log and monitoring solutions
• Experience in utilizing development containers, unit testing/code quality review/linting - standard best practice
• Good understanding of Agile software development frameworks
• Strong communication and Analytical skills
• Ability to work in teams in a diverse, multi-stakeholder environment comprising of Business and Technology teams
• Experience and desire to work in a global delivery environment
• Experience with Generative AI frameworks: OpenAI, Hugging Face, LangChain
• Knowledge of Model Context Protocol (MCP) or similar AI integration standards.
• Experience and detailed knowledge with Master Data Management, ETL, Data Quality, metadata management, data profiling, micro-batches, streaming data loads