Java Developer

TechDigital Group; San Jose (CA); On-site; USD 80,000 - 120,000; Full time

Job summary

An innovative company is seeking a skilled data engineer to design and maintain data pipelines across cloud platforms. This role involves building integration points for seamless data flow, ensuring data quality, and collaborating with cross-functional teams. Candidates should have expertise in Java, SQL, and data processing technologies like Apache Beam and Hadoop. If you are passionate about creating scalable solutions in a dynamic environment, this is the perfect opportunity for you to make a significant impact.

Qualifications

• Expertise in Java and SQL is essential for this role.
• Experience with Apache Beam and real-time data processing is mandatory.

Responsibilities

• Design and maintain data pipelines for cloud platforms.
• Collaborate with teams to ensure data quality and integrity.

Skills

Java

SQL

ETL Tools

Apache Beam

Apache NiFi

Talend

Informatica

Hadoop

Spark

Kafka

Pub/Sub

Job description

Experience working in Google environment and building applications and integrations using tools such as Replicator (built on Apache Beam) is preferred.

Responsibilities

1. Design, develop, and maintain data pipelines to integrate data from various cloud platforms (e.g., Google Cloud) and systems.
2. Build integration points between data in one environment and software in another to ensure seamless data flow and interoperability.
3. Collaborate with cross-functional teams to understand data requirements and ensure data quality and integrity.

Qualification and Experience

1. Expertise with programming languages such as Java is mandatory.
2. SQL and ETL tools (e.g., Apache NiFi, Talend, Informatica).
3. Experience with data processing technologies and platforms (e.g., Apache Beam, Dataflow, Hadoop, Spark) - Streaming and batch pipeline creation is mandatory.
4. Experience with real-time data processing and streaming technologies (e.g., Kafka, Pub/Sub).
5. API design principles and best practices for performance and scalability.
6. Familiarity with cloud platforms (GCP) and associated data services.
7. Experience in building scalable solutions in large distributed systems.