AI Architect

Conch Technologies, Inc.; Minnesota; Remote; USD 150,000 - 190,000; Full time

Job summary

Conch Technologies, Inc. is seeking an experienced AI Architect for a remote, contract-to-hire engagement lasting 6–12 months. The candidate will lead hands-on development across PySpark, SQL, Python, and Databricks, shaping scalable data architectures and governance models.

The role requires deep expertise in Unity Catalog, Delta Lake, Delta Live Tables, MLflow, and cloud platforms (Azure, AWS, GCP), with strong focus on performance, cost optimization, and CI/CD for data platforms.

Qualifications

• Hands-on development in the past 12 months with production PySpark/SQL code, debugging and data pipelines.
• Advanced SQL and Python/PySpark skills with complex transformations and performance optimization.
• 7+ years in data architecture/engineering/platform engineering with 3+ years on Databricks.

Skills

PySpark

SQL

Python

Databricks

Unity Catalog

Delta Lake

Delta Live Tables

MLflow

Data governance

Cloud platforms (Azure AWS GCP)

Data architecture

Data mesh

Infrastructure as Code

DevOps

CI/CD

Cost optimization

Performance tuning

Tools

Terraform

ARM templates

CloudFormation

Job description

AI Arcitect

LOCATION REMOTE DURATION 6-12 Month Contract-to-Hire INTERVIEW TYPE 2 Rounds VISA RESTRICTIONS SPONSORSHIP NOT PROVIDED

Required Skills & Experience

• Current, hands-on development experience is essential. This role requires the ability to read, write, debug, and lead code reviews of production PySpark and SQL code-not just architectural oversight. Candidates must demonstrate recent (within past 12 months) hands-on development work including debugging live code, explaining business logic and technical implementation, optimizing queries, and implementing data pipelines. Ability to comprehend and explain unfamiliar code samples is a core requirement.
• Advanced proficiency in SQL and Python/PySpark with demonstrated ability to write complex transformations, optimize query performance, explain code logic at both business and technical levels, and troubleshoot production issues. Must be comfortable with CTEs, window functions, table-valued functions, APPLY operators, advanced SQL operators, and PySpark DataFrame/RDD operations. Current, active coding skills required-not aspirational or theoretical knowledge.
• 7+ years of experience in data architecture, data engineering, or platform engineering roles, with at least 3+ years focused on Databricks platform architecture.
• Expert-level knowledge of Databricks platform components: Unity Catalog, Delta Lake, Delta Live Tables, Workflows, SQL Warehouses, MLflow, and Databricks SQL.
• Deep expertise in Unity Catalog governance, including metastore design, catalog/schema strategies, permission models, data lineage, and multi-workspace/multi-cloud patterns.
• Strong architectural background in cloud platforms (Azure, AWS, or GCP), including storage services, identity management (Azure AD, AWS IAM), networking, and security best practices.
• Proven experience designing enterprise-scale data architectures, including medallion/multi-hop architectures, data mesh patterns, domain-driven design, and data product frameworks.
• Hands-on experience with infrastructure-as-code (Terraform, ARM templates, CloudFormation) for platform configuration and governance automation.
• Strong understanding of DevOps practices, CI/CD pipelines, version control strategies, and automated testing for data platforms.
• Experience with performance tuning, cost optimization, and capacity planning for large-scale data platforms.