Senior Python Data Engineer in Nortal

Overview

Nortal   welcomes an experienced   Senior Python (Data) Engineer   to join the project team of our client — the international mining industry giant! The company is re-imagining and digitalizing the mining process, using the latest and most innovative IT technologies, to create sustainable value across an integrated value chain.

About our client

Our client is a global mining company with a world-class portfolio of products that are used by a billion consumers every day. Our client relies on data and cutting-edge data science to enhance their operations. They also use advanced machine learning and artificial intelligence to optimize their processes.

About the project

Our client is a giant corporation with 90000 employees globally. They have their own innovative IT department handling the whole cycle of development of the products used by the organization. You will work in an international expert team to develop sophisticated software solutions that have a direct impact on almost every aspect of modern life. You will use the innovative practices and the latest technologies to deliver safe and sustainable products to their customers around the world. Data Analytics is a new discipline in the Technical and Sustainability function of our client. The company is going to generate more data than ever before, and it needs to build the systems to support this and for making better decisions.

Responsibilities

• Work closely with the Data Engineering Lead to devise effective data ingestion strategies aligned with business objectives
• Define and implement robust data ingestion patterns and processes to ensure efficient and reliable data flow into the organization’s data platform
• Develop and maintain reusable data engineering or ETL pipelines and codebase using Python, PySpark, Databricks, Airflow, REST APIs, and the Azure cloud platform
• Design and implement robust data APIs using Python frameworks such as FastAPI or Flask and deploy them on Azure App Service
• Collaborate with cross-functional teams to understand data requirements and provide scalable data engineering solutions
• Design and implement batch and streaming data architectures leveraging Azure cloud services like Azure Data Factory and Azure Databricks
• Contribute to the development and deployment of Apache Airflow on Azure Kubernetes Service (AKS)
• Ensure adherence to software engineering best practices, including version control, testing, and continuous integration/continuous deployment (CI/CD) processes
• Participate in code reviews, technical discussions, and knowledge-sharing sessions within the team

Qualifications

• Good experience in Python programming language
• Proven experience with Apache Airflow and its deployments on Azure Kubernetes Service
• Hands-on experience with Python frameworks like FastAPI or Flask and deploying REST APIs on Azure App Service
• Strong understanding of core data engineering concepts and principles
• Solid experiencewith Azure cloud services, including Azure Data Factory and Azure Databricks
• Proficiency in writing PySpark code for data processing and transformation
• Ability to design and implement reusable and scalable data engineering pipelines and codebase
• Sound understanding of software engineering development practices, including version control, testing, and continuous integration/continuous deployment (CI/CD)
• Experience in designing batch and streaming data architectures using Azure cloud platform services
• Effective problem-solving skills and the ability to troubleshoot complex data engineering issues
• Commitment to continuous learning and staying updated with industry trends and best practices in data engineering
• Strong communication and collaboration skills with English language proficiency
• Ability to work effectively in a team environment, provide training, and document processes

Nice to have:

• Familiarity with Python tools like Pytest, tox, poetry
• Familiarity with Python testing frameworks like pytest and build tools like tox and poetry
• Experience in building CI/CD pipelines and working with Git-based version control systems
• Knowledge of containerization technologies such as Docker and Kubernetes

Nortal offers

• Family-like environment and personal attention to each specialist
• Direct cooperation with European and USA clients and their innovative products
• Competitive salary and regular reviews
• The work-life balance you deserve: 24 working days of paid vacations
• Educational reimbursement, funded language classes, certifications
• "Benefit Cafe with various categories like sport, leisure, books, fuel, etc.
• Health care: 10 paid sick leaves, on-demand medical insurance, vaccinations
• Cozy workplace and WFH opportunities
• Exciting events and lovely gifts for your family