Data Engineer

Resolve Tech Solutions; Irving (TX); On-site; USD 110,000 - 150,000; Full time

Job summary

Resolve Tech Solutions seeks a data engineer to build scalable batch and near-real-time data pipelines. You will collaborate with AI/ML teams, translate business requirements into robust data solutions, and ensure data quality and lineage across cloud platforms.

The role centers on Python, SQL, Spark, and modern data lake architectures, with emphasis on production readiness, CI/CD, and cost-efficient processing in Azure or AWS.

Qualifications

• Bachelor-level degree or equivalent practical experience in a quantitative field.
• 3–5 years of data engineering or closely related software engineering experience.
• Strong Python and SQL proficiency with complex queries and debugging.
• Experience with Spark/PySpark and distributed data processing.

Responsibilities

• Build, maintain, and optimize scalable batch and near-real-time data pipelines using Python, SQL, Spark.
• Integrate data from enterprise systems, APIs, and event streams into analysis-ready datasets.
• Collaborate with AI/ML teams to deliver data interfaces for inference workflows and model outputs.
• Ensure data quality, lineage, and governance across pipelines and data stores.
• Participate in CI/CD, code reviews, and documentation of data schemas and runbooks.

Skills

Python

SQL

Spark/PySpark

ETL/ELT

Data modeling

Data quality

CI/CD

Git

Education

Bachelor's degree in Computer Science, Software Engineering, Information Systems, Engineering, Mathematics, Statistics, or related field

Tools

Databricks

Airflow

Delta Lake

Kafka

Azure

AWS

Spark Structured Streaming

Databricks Workflows

Job description

• Build, maintain, and enhance scalable batch and near-real-time data pipelines using Python, SQL, and Spark/PySpark.
• Integrate data from enterprise applications, operational systems, APIs, event streams, and AI inference outputs into reliable, analysis-ready datasets.
• Develop transformations that combine structured and semi-structured data across sources using consistent identifiers and timestamps; handle time zones, duplicate records, and late-arriving events.
• Create and maintain data models and curated datasets for business analytics, AI model development and evaluation, and AI-enabled application workflows.
• Partner with AI/ML engineers to deliver data interfaces for inference workflows and capture model outputs, confidence scores, and user feedback for analysis and continuous improvement.
• Implement data quality checks, reconciliation, pipeline monitoring, and alerting to ensure completeness, accuracy, freshness, and traceability.
• Troubleshoot pipeline failures and performance issues; implement reliable retries, backfills, and incremental processing while optimizing compute and storage costs.
• Collaborate with product managers, data scientists, analysts, and engineering partners to translate business requirements into practical data solutions.
• Follow data security, privacy, access-control, and retention requirements when handling enterprise data and AI-generated outputs.
• Write maintainable, tested code; participate in code reviews and CI/CD practices; document schemas, data lineage, dependencies, and operational runbooks.

Basic Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, Information Systems, Engineering, Mathematics, Statistics, or a related quantitative discipline, or equivalent practical experience.
• 3–5 years of relevant experience in data engineering or a closely related software engineering role.
• Strong hands-on proficiency in Python and SQL, including complex queries, transformations, and troubleshooting.
• Experience with Spark/PySpark and distributed data processing.
• Experience building and supporting ETL/ELT pipelines on a cloud platform such as Azure or AWS.
• Understanding of data modeling, relational databases, data lakes, data warehouses, and incremental processing patterns.
• Experience with workflow orchestration tools such as Apache Airflow or Databricks Workflows.
• Familiarity with event-driven or streaming data processing concepts, including event time, deduplication, and reliable delivery.
• Experience with Git, automated testing, and production monitoring or troubleshooting.
• Ability to independently deliver scoped work, communicate technical concepts clearly, and collaborate with technical and business stakeholders.
• Hands-on experience with Databricks, Delta Lake, or lakehouse architectures.
• Experience with Databricks Asset Bundles / Declarative Automation Bundles to define, version, and deploy Databricks jobs and pipelines across environments through CI/CD.
• Experience with streaming technologies such as Kafka, Azure Event Hubs, or Spark Structured Streaming.
• Experience with retail, customer, inventory, or operational datasets.
• Experience preparing datasets for machine learning, anomaly detection, or AI model evaluation.
• Familiarity with multimodal data, unstructured data processing, or time-based event correlation.
• Exposure to edge-to-cloud data integration, containerized workloads, or infrastructure as code.