Data Engineer (Spark) @ GetInData | Part of Xebia in GetInData | Part of Xebia

We are working on developing a modern platform for the collection and analysis of lineage metadata. As a Data Engineer, you will work closely with the GetInData and client teams on building and maintaining specifications and integrations for tools like Apache Airflow, Flink, or Spark, as well as developing Marquez service for the collection, aggregation, and visualization of lineage metadata. Public GitHub environments and discussions open to the rest of the world limit the number of meetings to a minimum. Very good knowledge of Java orPython, or both Experience working with databases or data processing systems Good knowledge of SQL Experience working with Spark Knowledge of big data landscape Ability to work independently and focus on impactful features Strong B2/C1 English - ability to express complex ideas both in text and speech Experience as a contributor to an open-source community is a plus Ability to actively participate/lead discussions with clients to identify and assess concrete and ambitious avenues for improvement We are working on developing a modern platform for the collection and analysis of lineage metadata. As a Data Engineer, you will work closely with the GetInData and client teams on building and maintaining specifications and integrations for tools like Apache Airflow, Flink, or Spark, as well as developing Marquez service for the collection, aggregation, and visualization of lineage metadata. Public GitHub environments and discussions open to the rest of the world limit the number of meetings to a minimum. ,[Working with the GetInData and client teams and open source community to identify requirements and execute them, Writing simple, understandable, testable, and efficient code with a focus on solid technical stability and great user experience, Collaborating with stakeholders to understand project functional requirements and objectives from a data processing perspective, Ensuring compliance with industry standards and regulations in terms of security, data privacy applied in the data processing layer, Defining and implementing security measures according to best practices, Ensuring compliance with industry standards and regulations] Requirements: Java, SQL, Spark, Python Tools: Jira, GitLab, GIT, Jenkins / GitLab, Agile. Additionally: Sport subscription, Private healthcare, Flat structure, Small teams, International projects, Team Events, Training budget, Free coffee, Gym, Bike parking, Playroom, Free snacks, Free beverages, In-house trainings, Startup atmosphere, No dress code, Kitchen.