Big Data Engineer (relocate to Middle East) in Erbis

About the Role:

We are looking for talented Big Data Engineers to join our dynamic team for an exciting new project. This is a unique opportunity to work on a cutting-edge initiative, building a team from scratch in a fast-paced environment. If you are passionate about big data technologies and ready for a new challenge, we want to hear from you!

This long-term project requires engineers to join ASAP and relocate to the Middle East.

Requirements:

• Proficient in Hadoop and the HDFS ecosystem.
• At least 3 years of experience in Data Engineering.
• Extensive familiarity with Cloudera Data Platform and its runtime components.
• Strong expertise with Apache Spark.
• Hands-on experience with one or more programming languages such as Python, Go, or Java.
• Solid foundational knowledge of computer software, including Linux operating systems, storage, network I/O, and related principles.
• Experience with computing, distributed, or big data systems (e.g., Nginx, Kubernetes, Docker, OpenStack, Kafka) is advantageous.
• Highly preferred to have good data structure and system design skills.

Responsibilities:

• Oversee the setup, configuration, and maintenance of Hadoop clusters and the HDFS ecosystem.
• Ensure data integrity and availability across the Hadoop Distributed File System.
• Design, develop, and maintain scalable data pipelines.
• Implement data ingestion processes to capture and store large volumes of data efficiently.
• Collaborate with data scientists and analysts to support their data needs.
• Manage and optimize the Cloudera Data Platform and its runtime components.
• Monitor system performance and troubleshoot issues to ensure high availability and reliability.
• Develop and maintain Apache Spark applications for data processing.
• Write and maintain code in programming languages such as Python, Go, or Java.
• Apply solid foundational knowledge of computer software, including Linux operating systems, storage, network I/O, and related principles.
• Utilize experience with computing, distributed, or big data systems (e.g., Nginx, Kubernetes, Docker, OpenStack, Kafka).
• Apply strong data structure and system design skills to develop reliable solutions.