Data Engineer

NexDimension; San Francisco, Northern (CA, KY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

NexDimension in San Francisco, CA is seeking a Data Engineer for a hybrid role with 3 days per week at the Bay Area client’s office.

You will design, build, and maintain scalable Big Data solutions, crafting pipelines with Spark and integrating SQL/NoSQL databases. Collaborate in Agile ceremonies and mentor junior engineers to drive technical excellence in a fast-paced environment.

Qualifications

• Extensive experience as a Big Data engineer with cloud or on-prem data platforms.
• Proficient in at least one modern programming language (Java, Scala, or Python).
• Strong CS fundamentals: data structures, algorithms, and complexity analysis.
• Deep experience with Spark and distributed data processing.
• Hands-on with SQL and NoSQL databases.
• Familiarity with data orchestration and streaming tools (Kafka, Airflow, NIFI, etc.).
• Comfortable in Linux/Unix environments with shell scripting.
• Proven ability to design and develop software systems at scale.
• Strong Agile experience and cross-team collaboration.
• Excellent communication to explain complex concepts to diverse audiences.

Responsibilities

• Design, develop, test, deploy, and maintain scalable Big Data solutions.
• Manage project priorities, deadlines, and deliverables across multiple initiatives.
• Build and optimize data pipelines using Spark and distributed processing frameworks.
• Develop software using Java, Scala, or Python.
• Work with relational and NoSQL databases (e.g., MySQL, Postgres; NoSQL).
• Integrate and manage data workflows with Hive, Impala, OOZIE, Airflow, NIFI, Kafka.
• Write shell scripts and manage deployments in Linux/Unix.
• Collaborate with cross-functional teams in Agile ceremonies.
• Mentor junior engineers and foster continuous improvement.

Skills

Big Data

Java/Scala/Python

Spark

SQL

NoSQL

Kafka

Airflow

NIFI

Hive/Impala

Linux shell

Agile

Communication

Education

Bachelor's degree in CS/related

Tools

Hive

Impala

OOZIE

Airflow

NIFI

Kafka

Job description

We have a full-time job opportunity of Data Engineer in San Francisco, California (Hybrid).

Location: San Francisco, California (Hybrid, 3 times a week at client’s office in the Bay Area)

Responsibilities

• Design, develop, test, deploy, and maintain scalable Big Data solutions.
• Manage project priorities, deadlines, and deliverables across multiple initiatives.
• Build and optimize data pipelines using Spark and other distributed processing frameworks.
• Develop software using modern programming languages such as Java or Scala.
• Work with both relational (e.g., MySQL, Postgres) and NoSQL databases.
• Integrate and manage data workflows using tools like Hive, Impala, OOZIE, Airflow, NIFI, and Kafka.
• Write shell scripts and manage deployments in Linux/Unix environments.
• Collaborate with cross-functional teams in agile ceremonies including daily stand-ups, story pointing, and retrospectives.
• Mentor junior engineers and foster a culture of continuous improvement and technical excellence.

Skills/Experience

• Extensive experience as a Big Data engineer, with expertise in cloud or on-premise data platforms.
• Proficient in at least one modern programming language (Java, Scala, or Python).
• Strong foundation in computer science principles including object-oriented design, data structures, algorithms, and complexity analysis.
• Deep experience with Spark and distributed data processing.
• Hands-on experience with both SQL and NoSQL databases.
• Familiarity with data orchestration and streaming tools (Kafka, Airflow, NIFI, etc.).
• Comfortable working in Linux/Unix environments with basic shell scripting skills.
• Proven ability to design and develop software systems at scale.
• Strong understanding of Agile methodologies, with the ability to drive iterative delivery and cross-team collaboration.
• Strong communicator with the ability to explain complex concepts to both technical and non-technical audiences, and to influence stakeholders across product, engineering, and acquisition teams.

Bachelor’s degree in Computer Science, Data Analytics, Mathematics, Software Engineering, or a related field or equivalent practical experience

Ready to architect a data platform built to last?

“Zbis is here… Wow!!! It’s better than what we expected. This is transformational to our company and how we drive our business forward. It’s customer‑centric, fact‑driven, and captures margins and market‑share across all dimensions. Zbis is truly a game changer!”
– Frederick Howard