Data Engineer | Noida
DigitalXNode
Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.
Job summary
DigitalXNode in Noida, India is seeking a Data Engineer to design, develop, and maintain scalable data pipelines and big data solutions. You will collaborate with data scientists, software engineers, business analysts, and product teams to enable analytics, reporting, and BI.
Responsibilities include building ETL/ELT pipelines with Scala, Spark, Hadoop and SQL, optimizing jobs, ensuring data quality, and supporting cloud-ready data architectures.
Qualifications
- Hands-on experience with Scala, Spark, Hadoop, and SQL.
- Experience developing ETL/ELT pipelines and data modeling knowledge.
- Understanding of distributed computing and large-scale data processing.
- Experience with cloud platforms is a plus.
Responsibilities
- Design, develop, test, deploy, and maintain scalable data pipelines using Scala, Spark, Hadoop, and SQL.
- Build efficient ETL/ELT workflows for structured and unstructured data.
- Develop reusable, optimized data processing components and automate ingestion.
- Ensure high-quality, reliable data pipelines and monitor performance.
- Collaborate with data scientists, analysts, and product teams; participate in Agile sprints.
Skills
Education
Tools
Job description
We are seeking a motivated and detail-oriented Data Engineer to design, develop, and maintain scalable data pipelines and big data solutions. The ideal candidate should have hands‑on experience with Scala, Apache Spark, Hadoop, SQL, and modern data engineering practices. You will work closely with data scientists, software engineers, business analysts, and product teams to build reliable data platforms that enable analytics, reporting, and business intelligence.
In this role, you will be responsible for developing high‑performance data pipelines, optimizing large‑scale data processing workflows, ensuring data quality, and supporting enterprise data initiatives. This position offers an excellent opportunity to work with modern big data technologies and cloud‑ready data architectures.
- Design, develop, test, deploy, and maintain scalable data pipelines using Scala, Apache Spark, Hadoop, and SQL.
- Build efficient ETL/ELT workflows for processing structured and unstructured data.
- Develop reusable and optimized data processing components.
- Automate data ingestion, transformation, and loading processes.
- Ensure high-quality, reliable, and maintainable data pipelines.
- Process large-scale datasets using distributed computing frameworks.
- Optimize Spark jobs for performance, scalability, and resource utilization.
- Work with Hadoop ecosystem components for distributed data storage and processing.
- Support batch and near real‑time data processing workflows.
- Monitor and improve big data platform performance.
- Design and optimize SQL queries for efficient data retrieval.
- Work with relational databases such as MySQL and PostgreSQL.
- Ensure data integrity, consistency, and accuracy across systems.
- Support database optimization and performance tuning.
- Manage data storage and lifecycle processes.
- Collaborate with cross‑functional teams to understand business and technical requirements.
- Translate business needs into scalable data engineering solutions.
- Work closely with data scientists and analysts to prepare datasets for analytics and machine learning.
- Participate in Agile development processes and sprint planning.
- Maintain technical documentation and implementation records.
- Troubleshoot complex issues related to data processing, storage, and retrieval.
- Ensure data platform scalability, security, reliability, and availability.
- Perform data validation and quality assurance activities.
- Optimize data workflows for improved efficiency and reduced processing time.
- Follow data engineering best practices and coding standards.
- Strong experience in Data Engineering concepts and practices.
- Hands‑on experience with:
- Scala
- Apache Spark
- Hadoop
- SQL
- Understanding of distributed computing and large‑scale data processing.
- Experience developing ETL/ELT pipelines.
- Knowledge of data modeling and data architecture principles.
- Hadoop Ecosystem
- HDFS
- MapReduce
- Apache Spark
- Distributed Data Processing
- Batch Processing
- MySQL
- PostgreSQL
- SQL Query Optimization
- Database Performance Tuning
- Data Modeling
- Scala
- SQL
- Python (Preferred)
- Git & Version Control
- Linux Basics
- Strong analytical and problem‑solving skills.
- Excellent communication and collaboration abilities.
- Ability to work independently and in Agile teams.
- Strong attention to detail and commitment to quality.
- Willingness to learn emerging data engineering technologies.
- Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Familiarity with Apache Kafka or streaming technologies.
- Knowledge of Airflow or workflow orchestration tools.
- Experience with Docker and Kubernetes.
- Exposure to Data Warehousing concepts.
- Understanding of CI/CD pipelines and DevOps practices.
- Basic knowledge of machine learning data pipelines.
- Apache Spark
- Hadoop
- HDFS
- MapReduce
- Scala
- SQL
- Python
- MySQL
- PostgreSQL
- AWS (Preferred)
- Azure (Preferred)
- Docker
- Kubernetes
- Git
- Linux
- IntelliJ IDEA
- Maven
- Bachelor's degree in Computer Science, Information Technology, Data Science, Software Engineering, or a related field.
- B.Tech / BE in Information Technology, Computer Science, Electronics, or equivalent disciplines preferred.
- MCA, M.Tech, M.Sc. (Computer Science, Data Science, Information Technology), or equivalent qualifications are an added advantage.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
