Senior Data Engineer: Real-Time AI Pipelines & Databricks

DataJobs; Amarillo (TX); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Comprehensive health coverage

Robust retirement planning

Education reimbursement

Opportunities for continuous learning

Job summary

DataJobs is seeking an IT Applications Data Engineer to design and optimize high-performance data pipelines and infrastructure for enterprise AI and analytics. You will build scalable batch and real-time data flows using Databricks, Snowflake, and cloud services, writing production-grade Python/Scala code to transform data for models.

Responsibilities include data quality, lineage, monitoring, MLOps support, and secure ingestion.

Qualifications

• Bachelor’s degree in engineering/science/IT with 2+ years of experience.
• Master’s degree in engineering/science/IT is alternative.
• 10 years education/experience combination acceptable.
• At least 2 years in enterprise data pipelines incl. ELT/ETL.
• Expert SQL and Python/Scala for data manipulation.
• Experience with Spark/Databricks/Snowflake and cloud data services.
• MS degree plus 3+ years of relevant experience.
• Certifications such as Databricks/Snowflake/Azure Data Engineer are a plus.
• Experience with streaming (Kafka, Event Hubs).
• Data modeling concepts: Dimensional Modeling, Data Vault.

Responsibilities

• Build, test, and maintain scalable data pipelines for batch and real-time use.
• Manage systems in the modern data stack (Databricks, Snowflake, cloud warehouses).
• Develop production-grade Python/Scala code to ETL data into models.
• Implement data quality, lineage, and monitoring tooling.
• Optimize database performance and cost across cloud storage/services.
• Support Data Science with feature-ready data and MLOps integration.
• Enforce DataOps and DevSecOps for testing, deployment, and security.
• Maintain data flow docs, dictionaries, and runbooks for production infra.
• Collaborate with API teams to secure ingestion from sources.
• Model new sources with Data Architects to align with standards.

Skills

SQL

Python

Scala

Databricks

Snowflake

Apache Spark

Azure Data Factory

AWS Glue

Kafka

Azure Event Hubs

MLOps

ELT/ETL

DataOps

DevSecOps

Dimensional Modeling

Data Vault

Telemetry and data governance

Education

Bachelor’s degree in engineering/science/IT

Minimum 2 years of relevant experience

Master’s degree in engineering/science/IT

10 years education/experience combination

MS degree with 3+ years experience

Relevant certifications (Databricks/Snowflake/Azure Data Engineer)

Tools

Databricks

Snowflake

Apache Spark

Azure Data Factory

AWS Glue

Kafka

Azure Event Hubs

Job description

DataJobs is seeking an IT Applications Data Engineer to design and optimize high-performance data pipelines and infrastructure for enterprise AI and analytics. You will build scalable batch and real-time data flows using Databricks, Snowflake, and cloud services, writing production-grade Python/Scala code to transform data for models.

Responsibilities include data quality, lineage, monitoring, MLOps support, and secure ingestion.