Software Engineer

Corsogen; Houston (TX); On-site; USD 100,000 - 150,000; Full time

Job summary

Corporate Solutions General Inc. is hiring a Software Engineer to work with distributed computing platforms and cloud technologies such as Big Data, Spark, and GCP. You will optimize Spark jobs on large data volumes, develop Python automation scripts, and design efficient BigQuery queries.

Role requires a Master’s in CS/Engineering and 2 years’ experience, including hands-on GCP, BigQuery, Dataproc, Airflow, SQL, Python, and Spark. Houston, TX-based with potential travel and relocation support.

Qualifications

• Masters in Computer Science, Engineering, or related field.
• Minimum 2 years of experience; at least 2 years with GCP, BigQuery, Dataproc, Airflow, SQL, Python and Spark.

Responsibilities

• Work with distributed computing platforms/cloud tech (Big Data) and optimize Spark jobs.
• Develop Python scripts for automation and data output generation.
• Analyze customer data and build models to derive insights.
• Design and optimize BigQuery queries to reduce processing time.
• Migrate existing processes to GCP for better performance.
• Monitor Spark jobs/logs, fix issues, and optimize performance.
• Build scalable data pipelines using BigQuery and GCP (Airflow, Dataproc).

Skills

GCP

BigQuery

Dataproc

Airflow

SQL

Python

Spark

Big Data

Education

Master's degree in Computer Science or Engineering

Tools

BigQuery

Dataproc

Airflow

Python

Spark

SQL

GCP

Job description

Current Openings

Job Title: Software Engineer

Job Description

(Corporate Solutions General Inc has multiple openings in Houston, TX) Software Engineer: Work with distributed computing platforms/cloud technologies such as Big Data and associated technologies such as Operationalization and performance tuning of Spark jobs on a large volume of data. Develop python scripts for automation and to generate the output response. Develop code to analyze the customer data by building models on the data. Design and optimize queries in BigQuery so that it reduces data processing time. Migrating the current existing process into GCP cloud for better performance. Analyze the spark jobs and logs if there is any job failure and fix the code accordingly including optimization. Build scalable data pipelines using BigQuery and GCP. Utilize GCP, Big query, Dataproc, Airflow, SQL, Python, Spark. Will work in various unanticipated locations throughout the U.S. Requires Masters in Computer Science, Engineering, or related & 2 years experience. 2 years must include experience with GCP, Big query, Dataproc, Airflow, SQL, Python, and Spark.

6201 Bonhomme Rd, Ste 290N-H, Houston, Texas 77036

INDIA