Senior Data Engineer: Data Lakehouse & AI Pipelines

Vericast; San Antonio (TX); On-site; USD 130,000 - 150,000; Full time

Job summary

Vericast is seeking a Senior Data Engineer to build big data pipelines, automate marketing campaigns, and support data requirements for our analytics, reporting, and AI applications. This role will expand our data infrastructure and pipeline architecture while optimizing data flow into our Data Lake environment.

The ideal candidate is self-directed and comfortable supporting the data needs of multiple teams, systems, and products in an agile environment, delivering production-grade data

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field with 5+ years of experience
• Master’s degree is preferred

Responsibilities

• Develop scalable data pipelines and build new integrations to support growing data volume and complexity
• Build a robust Data Lakehouse that supports Vericast's marketing solutions and FI clients
• Collaborate with AI, Analytics, and business teams to improve data models
• Implement processes to monitor data quality and ensure availability and accuracy of production data
• Perform data analysis to troubleshoot data-related issues and support resolutions
• Provide post-deployment support to resolve production issues
• Partner with business units and engineering teams to shape long-term data platform strategy

Skills

Python

PySpark

Data pipelines

Data observability

Linux

REST APIs

AI/ML readiness

CI/CD

Airflow

Spark

Education

Bachelor’s degree in CS/IT

Master’s degree (preferred)

Tools

Apache Airflow

Iceberg

Hive

S3

Trino

Kubernetes

GitLab CI/CD

Ranger

Hadoop

Job description

Vericast is seeking a Senior Data Engineer to build big data pipelines, automate marketing campaigns, and support data requirements for our analytics, reporting, and AI applications. This role will expand our data infrastructure and pipeline architecture while optimizing data flow into our Data Lake environment.

The ideal candidate is self-directed and comfortable supporting the data needs of multiple teams, systems, and products in an agile environment, delivering production-grade data