Senior Python Data Engineer in Nortal

OverviewNortalwelcomes an experiencedSenior Python (Data) Engineerto join the project team of our client — the international mining industry giant! The company is re-imagining and digitalizing the mining process, using the latest and most innovative IT technologies, to create sustainable value across an integrated value chain.About Our ClientOur client is a global mining company with a world-class portfolio of products that are used by a billion consumers every day. Our client relies on data and cutting-edge data science to enhance their operations. They also use advanced machine learning and artificial intelligence to optimize their processes.About The ProjectOur client is a giant corporation with 90000 employees globally. They have their own innovative IT department handling the whole cycle of development of the products used by the organization. You will work in an international expert team to develop sophisticated software solutions that have a direct impact on almost every aspect of modern life. You will use the innovative practices and the latest technologies to deliver safe and sustainable products to their customers around the world. Data Analytics is a new discipline in the Technical and Sustainability function of our client. The company is going to generate more data than ever before, and it needs to build the systems to support this and for making better decisions.ResponsibilitiesWork closely with the Data Engineering Lead to devise effective data ingestion strategies aligned with business objectivesDefine and implement robust data ingestion patterns and processes to ensure efficient and reliable data flow into the organization’s data platformDevelop and maintain reusable data engineering or ETL pipelines and codebase using Python, PySpark, Databricks, Airflow, REST APIs, and the Azure cloud platformDesign and implement robust data APIs using Python frameworks such as FastAPI or Flask and deploy them on Azure App ServiceCollaborate with cross-functional teams to understand data requirements and provide scalable data engineering solutionsDesign and implement batch and streaming data architectures leveraging Azure cloud services like Azure Data Factory and Azure DatabricksContribute to the development and deployment of Apache Airflow on Azure Kubernetes Service (AKS)Ensure adherence to software engineering best practices, including version control, testing, and continuous integration/continuous deployment (CI/CD) processesParticipate in code reviews, technical discussions, and knowledge-sharing sessions within the teamQualificationsGood experience in Python programming languageProven experience with Apache Airflow and its deployments on Azure Kubernetes ServiceHands-on experience with Python frameworks like FastAPI or Flask and deploying REST APIs on Azure App ServiceStrong understanding of core data engineering concepts and principlesSolid experience with Azure cloud services, including Azure Data Factory and Azure DatabricksProficiency in writing PySpark code for data processing and transformationAbility to design and implement reusable and scalable data engineering pipelines and codebaseSound understanding of software engineeringdevelopment practices, including version control, testing, and continuous integration/continuous deployment (CI/CD)Experience in designing batch and streaming data architectures using Azure cloud platform servicesEffective problem-solving skills and the ability to troubleshoot complex data engineering issuesCommitment to continuous learning and staying updated with industry trends and best practices in data engineeringStrong communication and collaboration skills with English language proficiencyAbility to work effectively in a team environment, provide training, and document processesNice to have:Familiarity with Python tools like Pytest, tox, poetryFamiliarity with Python testing frameworks like pytest and build tools like tox and poetryExperience in building CI/CD pipelines and working with Git-based version control systemsKnowledge of containerization technologies such as Docker and KubernetesNortal offersFamily-like environment and personal attention to each specialistDirect cooperation with European and USA clients and their innovative productsCompetitive salary and regular reviewsThe work-life balance you deserve: 24 working days of paid vacationsEducational reimbursement, funded language classes, certifications"Benefit Cafe with various categories like sport, leisure, books, fuel, etc.Health care: 10 paid sick leaves, on-demand medical insurance, vaccinationsCozy workplace and WFH opportunitiesExciting events and lovely gifts for your family Show more Show less Посадовий рівень Старший середній рівень Тип зайнятості Повний робочий день Посадові обов’язки Інформаційні технології Галузі ІТ-послуги та ІТ Консалтинг