Databricks Lakehouse Architect | AI & Cloud

HCL Technologies Limited

St. Louis (MO)

On-site

USD 128,000 - 250,000

Full time

23 hours ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

HCL Technologies Limited in Missouri, St Louis is seeking a Data Architect to lead Databricks Lakehouse modernization and AI-driven data platforms. You will design scalable data foundations, govern data assets, and collaborate with executives to drive enterprise transformation.

The role requires hands-on experience with Databricks, Delta Lake, Unity Catalog, and Python, plus a track record of delivering production-grade data solutions across cloud platforms. Onsite work model.

Qualifications

Architect and deliver modern Databricks Lakehouse solutions using Delta Lake, Medallion architecture, and scalable cloud-native design patterns.
Lead enterprise data modernization initiatives, assess current-state environments, define target-state architectures, and guide implementation roadmaps.
Design, develop, and optimize batch and real-time data pipelines using Auto Loader, Structured Streaming, Delta Live Tables, and Databricks Workflows.
Establish and enforce data governance, security, lineage, and quality frameworks using Unity Catalog and industry best practices.
Collaborate with business stakeholders, product owners, engineering teams, and executive leadership to align technical solutions with strategic business objectives.
Lead architecture reviews, solution design workshops, and technical governance processes across client engagements.
Drive the adoption of AI-ready data architectures by developing model-ready data assets, vector-enabled pipelines, and scalable retrieval frameworks.
Design and implement production-grade AI/ML solutions using Python, PyTorch, and Databricks AI capabilities.
Define AI governance standards, security guardrails, prompt safety controls, and evaluation frameworks to support responsible AI deployment.
Provide technical leadership, mentorship, and oversight to engineering teams while ensuring solution quality, scalability, performance, and operational excellence.
Own end-to-end solution delivery, including architecture, implementation oversight, risk management, stakeholder communication, and successful production rollout.

Responsibilities

Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, Data Science, or a related field.
10+ years of experience in data architecture, data engineering, cloud platforms, or enterprise technology consulting.
5+ years of hands‑on experience with Databricks, including Delta Lake, Unity Catalog, Delta Live Tables, and Structured Streaming.
Strong expertise designing and implementing enterprise‑scale Lakehouse architectures and modern data platforms.
Proven experience leading data modernization, migration, and digital transformation initiatives.
Hands‑on experience with cloud platforms such as Azure, AWS, or Google Cloud Platform (GCP).
Strong programming skills in Python and experience developing AI/ML solutions using frameworks such as PyTorch.
Experience building scalable data pipelines, data integration solutions, and analytics platforms supporting large‑scale workloads.
Deep understanding of data governance, metadata management, data quality, security, compliance, and privacy best practices.
Experience with AI/GenAI architectures, vector databases, retrieval‑augmented generation (RAG), and intelligent application development is highly preferred.
Proven ability to engage with executive stakeholders, influence technical strategy, and translate business requirements into scalable solutions.
Excellent communication, presentation, consulting, and leadership skills.
Databricks certifications (Data Engineer, Solutions Architect, or related certifications) are strongly preferred.