Data Engineer (W2 Only)

Yorkshire Global Solutions Inc.; United States; On-site; USD 110,000 - 160,000; Full time

Job summary

Yorkshire Global Solutions Inc. is hiring a Data Engineer to design, develop, and maintain scalable data pipelines that support enterprise analytics and reporting.

You will work with Python, SQL, Spark, PySpark, Databricks, Snowflake, Azure Data Factory, and cloud services to build robust data platforms and data warehouses.

Collaborate with data scientists, BI teams, and software engineers to deliver high-quality data solutions with a strong emphasis on data quality, governance, and automation.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Strong understanding of data modeling and warehousing concepts.
• Experience with cloud data platforms and big data technologies.

Responsibilities

• Design, develop, and maintain scalable data pipelines to support enterprise data integration, analytics, and reporting initiatives.
• Build and optimize ETL/ELT workflows using Python, SQL, Spark, PySpark, Azure Data Factory, Databricks, Airflow, or similar tools.
• Develop and manage cloud-based data platforms using Azure, AWS, Snowflake, Microsoft Fabric, and Delta Lake technologies.
• Design, implement, and optimize data warehouses, data lakes, and dimensional data models to support BI and analytics.
• Collaborate with Data Scientists, Data Analysts, BI teams, Product Owners, and Software Engineers to deliver reliable data solutions.
• Optimize SQL queries, Spark jobs, and data processing frameworks for performance and scalability.
• Implement data quality, governance, security, and monitoring processes for enterprise data.
• Develop real-time and batch data ingestion using Kafka, streaming tech, APIs, and cloud-native services.
• Automate deployment, testing, version control, and monitoring using Git, CI/CD pipelines, and IaC best practices.
• Create and maintain documentation, architecture diagrams, data lineage, runbooks, and support production environments.

Skills

Design scalable data pipelines

Cross-functional collaboration

Cloud data engineering

Tools

Python

SQL

Apache Spark

PySpark

Databricks

Snowflake

Azure Data Factory

Azure

AWS

Microsoft Fabric

ETL/ELT

Data Pipelines

Data Warehousing

Data Modeling

Delta Lake

Kafka

Airflow

Git

CI/CD

Job description

This is Prashant, a Lead Recruiter from Yorkshire Global Solutions Inc.

We are currently hiring for the below role.

Note

Job Title

Data Engineer (W2 Candidates Only)

Must-Have Technologies

Python | SQL | Apache Spark | PySpark | Databricks | Snowflake | Azure Data Factory | Azure | AWS | Microsoft Fabric | ETL | ELT | Data Pipelines | Data Warehousing | Data Modeling | Delta Lake | Kafka | Airflow | Git | CI/CD

Job Description

• Design, develop, and maintain scalable data pipelines to support enterprise data integration, analytics, and reporting initiatives.
• Build and optimize ETL/ELT workflows using Python, SQL, Apache Spark, PySpark, Azure Data Factory, Databricks, Airflow, or similar data engineering tools.
• Develop and manage modern cloud-based data platforms using Azure, AWS, Snowflake, Microsoft Fabric, and Delta Lake technologies.
• Design, implement, and optimize data warehouses, data lakes, and dimensional data models to support business intelligence and advanced analytics.
• Collaborate with Data Scientists, Data Analysts, Business Intelligence teams, Product Owners, and Software Engineers to deliver reliable and high-quality data solutions.
• Optimize SQL queries, Spark jobs, and data processing frameworks to improve performance, scalability, and overall system efficiency.
• Implement data quality, validation, governance, security, and monitoring processes to ensure the accuracy, consistency, and reliability of enterprise data.
• Develop real-time and batch data ingestion solutions using Kafka, streaming technologies, APIs, and cloud-native integration services.
• Automate deployment, testing, version control, and monitoring of data engineering solutions using Git, CI/CD pipelines, and Infrastructure as Code best practices.
• Create and maintain technical documentation, architecture diagrams, data lineage, operational runbooks, and support production environments while driving continuous process improvements.