Python SQL Developer

Compunnel, Inc.; Irving (TX); On-site; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking an experienced Python SQL Developer to support a critical data warehouse migration project. This role involves developing new data pipelines, leading data migrations, and implementing transformations in Azure. You'll work with cutting-edge technologies like Azure Data Factory and Synapse, ensuring smooth transitions from GCP to Azure. If you have a passion for cloud technologies and a strong background in Python and SQL, this opportunity offers a chance to make a significant impact on data integrity and performance optimization in a dynamic environment.

Qualifications

• 10+ years of experience in Python and SQL for data manipulation.
• Hands-on experience with Azure cloud technologies for 5+ years.

Responsibilities

• Develop and optimize new data pipelines in Azure using Azure Data Factory.
• Lead data migration efforts and implement necessary transformations.

Skills

Python Programming

SQL

Azure Cloud Technologies

Communication Skills

Problem-Solving Skills

Tools

Azure Data Factory

Azure Synapse

Event Hub

AKS

Kafka

Spark

Job description

We are looking for experienced Python SQL Developer with expertise in Python, SQL, and Azure cloud technologies to support a critical data warehouse migration project. The selected candidates will work on new data pipelines, data migration, transformations, query conversion, and validation tools in Azure. This is an exciting opportunity to work on cutting-edge technologies and contribute to the migration of data and systems from GCP to Azure.

Key Responsibilities

• Data Pipeline Development: Develop and optimize new data pipelines in Azure using Azure Data Factory, Azure Synapse, Event Hub, and AKS.
• Data Migration & Transformation: Lead data migration efforts and implement necessary data transformations to ensure smooth transition from GCP to Azure.
• Query Conversion & Optimization: Convert and optimize queries for the Azure environment.
• Data Validation Tools: Build and implement new data validation tools to ensure data integrity post-migration.
• API Integration: Develop and integrate data APIs in Azure AKS to support the new data platform.
• Performance Tuning: Conduct performance tuning to ensure optimal data processing performance in the new environment.
• Post-MVP Support: Provide documentation and support after the MVP (Minimum Viable Product) release, including additional scope and optimization.

Required Qualifications

Experience:

• Python: Minimum of 10 years of experience with Python programming.
• SQL: Minimum of 10 years of experience in SQL for data manipulation and querying.
• Cloud Experience: Hands-on experience with Azure cloud technologies for 5+ years (e.g., Azure Data Factory, Azure Synapse, Event Hub, AKS).
• Distributed Processing Frameworks: Experience with Kafka and Spark for distributed data processing.

Skills:

• Strong communication and problem-solving skills to effectively collaborate with cross-functional teams.
• Proven ability to work on large-scale data migration projects in cloud environments.

Preferred Qualifications

• API Development: Experience with GraphQL API and REST API.
• Microservices: Knowledge of data microservices architecture.
• Shell Scripting: Proficiency in Shell scripting to automate tasks and streamline processes.

Certifications

AWS Certified Solutions Architect (preferred)