Spark Developer (CLT)

DEF; Charlotte (NC); On-site; USD 100,000 - 140,000; Full time

Job summary

DEF is seeking a Senior Software Engineer to join the production support/DevOps team in Charlotte. The role requires independent work, proactive troubleshooting, and opportunities to optimize and automate complex data pipelines.

Ideal candidates will have strong Spark development experience, ETL with Hadoop, and proficiency in SQL, Unix, and data warehousing. Collaboration with multiple teams and business stakeholders is essential.

Qualifications

• Mid to senior level Software Engineer with strong experience in Platform / Production Support / DevOps.
• Experience in ETL, Bigdata- Hadoop, SQL; Spark knowledge is essential.
• Strong data and code analysis skills (troubleshooting data quality/integrity, root causes).

Responsibilities

• Collaborate with internal and external tech, production support, and middleware teams.
• Self-manage work and take initiative to troubleshoot and resolve issues independently.
• Identify opportunities for optimization and automation in the production environment.
• Engage with the business team to ensure alignment and timely delivery.

Skills

Spark development

ETL / Hadoop

SQL

Unix

Data warehousing

Scripting

Troubleshooting

Root cause analysis

Performance optimization

Production Support

Tools

Bitbucket

Gradle

Jenkins

Ansible

Job description

Person will be part of the production support / devops team and will interact with internal as well as external tech and production support and middleware teams. Person will be working closely with the business team.

The ideal candidate must be able to self-manage work. This is a deal breaker. This is a production support environment and perfect candidate must be able to take initiative to troubleshoot and figure out issues and resolutions independently and identify opportunities for optimization and automation. Please screen candidates accordingly.

Required Qualifications

• Mid to senior level Software Engineer with strong experience working in Platform / Production Support / DevOps team. Experience in ETL, Bigdata- Hadoop, SQL. Good knowledge of Spark, Unix, Data warehousing, scheduling tools and scripting.
• Must have strong experience in Spark development, performance optimization and troubleshooting
• Strong data and code analysis skills (troubleshooting data quality/integrity, root causes)

Desired Qualifications

• Comfortable working with all levels of management and comfortable with bringing attention to risks before they become issues.
• Experience working in Agile teams.
• Good understanding of Bit Bucket, Gradle, Jenkins, Ansible and CI-CD pipeline