Sr Data Engineer

United Software Group Inc.; Plano (TX); Remote; USD 60,000 - 80,000; Full time

Job summary

An established industry player is on the lookout for a seasoned Data Engineer to join their innovative team. This role involves designing and implementing scalable data pipelines and ETL processes, with a focus on real-time data applications. You will work extensively with Azure technologies, including Azure Databricks and Data Factory, to build robust data platforms. If you thrive in a dynamic environment and have a passion for data engineering, this opportunity offers a chance to make a significant impact while working remotely from anywhere in the U.S.

Qualifications

• 7+ years of experience in data engineering and software development roles.
• Expertise in building scalable data pipelines and ETL processes.

Responsibilities

• Design and implement robust data platforms for real-time applications.
• Develop APIs for data access and integration.

Skills

Data Engineering

Software Development

Medallion Architecture

Databricks Development

PySpark

Python

Agile Development

Database Technologies (SQL/NoSQL)

ETL Processes

API Development

Tools

Azure Data Factory

Azure Databricks

Azure SQL DB

Azure Data Lake Storage

Azure Cosmos DB

Azure DevOps

Spark

Git

Job description

Description:
Client : Mars Incorporated
Location : Plano, TX (Permanent remote work accepted from anywhere in US)
Rate : $70/hour


Qualifications:
• 7+ years of professional experience in data engineering and software development roles
• Experience in Medallion Architecture, Databricks Development, and building robust data platforms for streaming / real time applications
• Hands on experience with PySpark and Python, including multiple libraries
• Great understanding of how data moves throughout an organization, data validation, cleansing, etc.
• Proven track record in Agile software product development
• 5+ years in programming languages like Python, Java, or Scala
• Extensive experience with Databricks and Azure are a must
• Experience with other data technologies such as Databricks and Hadoop, Spark, or Kafka are a plus
• Strong understanding of database technologies (SQL and NoSQL) and data modeling
• Expertise in designing and implementing scalable data pipelines and ETL processes
• Experience designing and developing APIs for data access and integration
• Strong understanding of software engineering principles, including design patterns and version controls


Critical technologies for this role:
- Azure Data Factory (ADF)
- Azure Databricks
- Azure SQL DB
- Azure Data Lake Storage (ADLS)
- Azure Cosmos DB
- Azure DevOps
- Python
- SQL
- Spark / PySpark
- ETL Pipelines
- Data Modeling


Other technologies
- Scala
- Advanced Scripting with Bash/Powershell
- Object-Oriented Programming (OOP)
- Functional Programming
- Test-Driven Development (TDD)
- Clean Code Principles
- Version Control (Git)


Minimum Skills Required:
3+ years of python development
3+ years databricks
2+ years ETL pipeline development ICIMS RR ID #

Additional Details

• Skill Category : Regular
• ICIMS RR ID # : NA
• Client Name : Mars Incorporated
• Engagement Type : T&M Competitive 1-3
• Vertical : MFGCI
• Must-Have Primary Skill : Data and Intelligence-Data Management-ETL Data Integration
• Primary Skill: Yrs Experience : Expert (5+ Years Experience)
• RC- Domain : Data & Analytics
• RC- Subdomain : 29 - Database Management(SQL Database, Cloudera Impala, DAMA BOK, Microsoft SSIS, Informatica, Integration Services, SSRS, SSAS, SSMS, Transparent Data Encryption, No SQL Database,HADR, Replication, O/S Clustering, IBM Pure Scale, Informix, Log Shipping, Mirroring, Data Modeling and Design, AS/400 DB2)
• RC- Role : Developer
• RC- Experience Level : IV
• RC- Geo Tier : US-3
• RC- Exception Required : No
• COVID-19 Vaccine Required? : (No Value)
• Client-Accepted Visa Types : (No Value)