Senior Azure Databricks PySpark Data Engineer

Intone Inc; Jersey City, Northern (NJ, KY); Hybrid; USD 96,000 - 165,000; Part time

Job summary

Intone Inc. is seeking a Senior PySpark/Databricks Data Engineer for a 12-month contract role based in Jersey City, New Jersey. You will design and deploy data applications on Azure cloud solutions and build scalable data pipelines using PySpark in Databricks.

The role requires hands-on experience with Azure Databricks, ADLS, Azure Data Factory and Kafka, and will involve collaborating in an Agile project environment with cross-functional teams.

Qualifications

• Proficiency in Python scripting, PySpark, and Spark SQL.
• Hands-on experience with Azure Databricks and ADLS.
• Experience with Azure SQL Database, EventHub, and Kafka.
• Strong experience implementing Azure Data Factory and Azure Data Analytics.
• Knowledge of Azure Storage services (ADLS and Storage Accounts).
• Solid SQL skills (T-SQL / PL-SQL) and data tooling.
• Experience in Agile projects using Jira.

Responsibilities

• Design and deploy data applications on Azure cloud solutions.
• Build and optimize data pipelines using PySpark and Spark SQL in Databricks.
• Apply transformations to create efficient data pipelines.
• Perform performance tuning within Databricks environments.
• Work on data ingestion projects using Azure services.
• Collaborate with team members in an Agile project environment.

Skills

Python scripting

PySpark

Spark SQL

Analytical problem solving

Agile methodologies

Tools

Azure Databricks

Azure Data Lake Storage (ADLS)

Azure Data Factory

Azure Data Analytics

Azure SQL Database

EventHub

Kafka

Azure Storage services

SQL/T-SQL/PL-SQL

Jira

Job description

Intone Inc. is seeking a Senior PySpark/Databricks Data Engineer for a 12-month contract role based in Jersey City, New Jersey. You will design and deploy data applications on Azure cloud solutions and build scalable data pipelines using PySpark in Databricks.

The role requires hands-on experience with Azure Databricks, ADLS, Azure Data Factory and Kafka, and will involve collaborating in an Agile project environment with cross-functional teams.