Python + SQL PySpark + SQL PySpark Data Modeler/Data Architect

SFJ Business Solutions Pvt. Ltd.; Bengaluru; On-site; INR 800,000 - 1,200,000; Full time

Job summary

A technology solutions provider in Bengaluru is seeking a Mid-Senior Python + SQL PySpark Data Modeler/Data Architect. You will design data solutions, optimize ETL processes, and collaborate in Agile environments while utilizing your Python and SQL expertise. The role requires knowledge in data modeling and cloud technologies, offering competitive compensation and a full-time position.

Qualifications

• 2+ years in Python and/or PySpark development.
• Experience with SQL and analytical functions.
• Experience in CI/CD and Build Management.

Responsibilities

• Design and implement data solutions and data warehousing concepts.
• Develop and optimize data pipelines and ETL processes.
• Collaborate with developers and analysts to understand requirements.

Skills

Python

PySpark

SQL

Data modeling

ETL processes

Agile methodology

Tools

Scala

AWS

Jira

Confluence

Job description

Python + SQL PySpark Data Modeler/Data Architect

Overview

Pyspark Data Engineer / Python developer with experience in PySpark, SQL, and data modeling. The role involves designing and implementing data solutions, data warehousing concepts, and working with various data formats and platforms.

Responsibilities

• Hands-on experience in Scala/Spark development; hands-on SQL writing on RDBMS (DB2) databases.
• Develop and optimize data pipelines, ETL processes, and data integration using Python, PySpark, SQL, Hive, Impala, and other scripting tools.
• Design and implement CI/CD, build management, and development strategy.
• Experience with data modeling, data profiling, data lineage, and data warehousing concepts (Star schema, Snowflake, or Data Vault).
• Work with different file formats such as JSON, Parquet, AVRO, ORC, and XML; projections for data in HDFS or cloud storage.
• Ensure data integrity, consistency, and security within databases (e.g., Oracle, DB2, SQL Server, PostgreSQL, MySQL).
• Develop PL/SQL procedures, functions, triggers, and packages; write efficient SQL queries, joins, and subqueries for data retrieval.
• Collaborate with application developers, business analysts, and system architects to understand database requirements.
• Participate in Agile/Scrum/SAFe environments; use Jira/Confluence for project tracking.
• Proficiency in data analysis, data profiling, and data lineage; knowledge of calculus, linear algebra, and machine learning concepts for modeling.
• Maintain documentation of database structures, stored procedures, and coding best practices; stay up-to-date with Oracle database technologies and industry trends.

Qualifications

• 2+ years in Python and/or PySpark development; strong programming experience with Python, PySpark, and Scala is preferred.
• Experience with SQL and SQL analytical functions; experience participating in key business, architectural, and technical decisions.
• Experience in designing and implementing CI/CD, Build Management, and Development strategy.
• Experience working with data modeling tools and concepts; knowledge of ER modeling and data warehousing.
• Experience with AWS cloud technologies (training available on scope to be trained).

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Other

Industries

• IT Services and IT Consulting

Referrals increase your chances of interviewing at SFJ Business Solutions Pvt. Ltd. by 2x

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.