Data Engineer (Azure)

Kforce Inc; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Kforce Inc is hiring for a data engineering role in New York. The candidate will work with architects and data teams to identify data sources, design scalable data pipelines, and develop ML-powered insights across cloud and on-premise environments.

Required experience includes 2-5 years in data app development, 2+ years in ETL/ELT on columnar MPP databases, and strong Python/PySpark skills. Familiarity with streaming and distributed data ecosystems is essential.

Qualifications

• 2-5 years of progressive data application development experience.
• 2+ years of experience modeling and implementing ETL/ELT on columnar MPP database technologies
• Strong Experience with Python and/or PySpark needed
• Experience working in large scale/distributed SQL, NoSQL, and/or Hadoop environments
• Experience with streaming architectures e.g. Kafka, Stream, PubSub

Responsibilities

• Identify, engage, and integrate data sources for discovery and profiling and, where necessary and define data services that empower business processes
• Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, as well as cloud-based), distributed/elastic environments, and downstream applications and/or self-service solutions
• Develop solutions combining data blending, profiling, mining, statistical analysis, and machine learning, to better define and curate models, test hypothesis, and deliver key insights
• Collaborate in establishing and evolving development, testing, and documentation standards, as well as related code reviews
• Partner with business analysts, application engineers, data scientists, leveraging the appropriate tools, solutions, and/or processes as part of their data mining, profiling, blending, and analytical activities
• Collaborate in establishing and evolving development, testing, and documentation standards, as well as related code reviews

Skills

Python

PySpark

Kafka

NoSQL

ETL/ELT

Distributed SQL

Hadoop

Streaming

Tools

Hadoop ecosystem

Spark

Job description

Responsibilities

• Work with architects, other data engineers and analysts to identify, engage, and integrate data sources for discovery and profiling and, where necessary and define data services that empower business processes
• Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, as well as cloud-based), distributed/elastic environments, and downstream applications and/or self-service solutions
• Develop solutions combining data blending, profiling, mining, statistical analysis, and machine learning, to better define and curate models, test hypothesis, and deliver key insights
• Collaborate in establishing and evolving development, testing, and documentation standards, as well as related code reviews
• Partner with business analysts, application engineers, data scientists, leveraging the appropriate tools, solutions, and/or processes as part of their data mining, profiling, blending, and analytical activities
• Collaborate in establishing and evolving development, testing, and documentation standards, as well as related code reviews

Requirements

• 2-5 years of progressive data application development experience
• 2+ years of experience modeling and implementing ETL/ELT on columnar MPP database technologies
• Strong Experience with Python and/or PySpark needed
• Experience working in large scale/distributed SQL, NoSQL, and/or Hadoop environments
• Experience with streaming architectures e.g. Kafka, Stream, PubSub

The pay range is the lowest to highest compensation we reasonably in good faith believe we would pay at posting for this role. We may ultimately pay more or less than this range. Employee pay is based on factors like relevant education, qualifications, certifications, experience, skills, seniority, location, performance, union contract and business needs. This range may be modified in the future.

We offer comprehensive benefits including medical/dental/vision insurance, HSA, FSA, 401(k), and life, disability & ADD insurance to eligible employees. Salaried personnel receive paid time off. Hourly employees are not eligible for paid time off unless required by law. Hourly employees on a Service Contract Act project are eligible for paid sick leave.

Note: Pay is not considered compensation until it is earned, vested and determinable. The amount and availability of any compensation remains in Kforce's sole discretion unless and until paid and may be modified in its discretion consistent with the law.

This job is not eligible for bonuses, incentives or commissions.

Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.