Data Engineer (Azure Databricks) in LimestoneDigital

Requirements:- Strong proficiency in Azure Data Factory (ADF) and Azure Synapse for designing and implementing data pipelines and workflows;- Expertise in SQL for querying and manipulating data within relational databases and data warehouses;- Experience with Databricks for big data processing, data exploration, and advanced analytics;- Knowledge of Azure Events Hub for ingesting and processing real-time data streams;- Familiarity with Azure Functions for developing serverless computing solutions and event-driven architectures;- Understanding of Azure PureView for data quality management and data profiling;- Proficiency in Azure Machine Learning for building and deploying machine learning models in a cloud environment;- Ability to design and develop scalable, reliable, and efficient data processing solutions on the Azure cloud platform;- Proficient in data modeling and database design principles to ensure data integrity and optimize query performance;- Knowledge of data governance and data security best practices to ensure compliance and data privacy;- Experience with data integration and data transformation techniques to extract, transform, and load (ETL) data from various sources into target data repositories;- Familiarity with machine learning concepts and techniques to support data scientists in model development and deployment;- Understanding of cloud computing platforms (Azure and/or AWS) and their services for data engineering and data processing tasks.Responsibilities:- Work closely with our engineers and client to develop a high-quality result for our customers; - Communicating with external web services and collecting date; - Develop and write efficient, readable code in (ADF/Synapse, SQL, Databricks, Events Hub, Azure Functions, PureView);- Work with third-party libraries; - Self-improvement development processes. Nice to have:- Proficient in programming Python for scripting and automation in data engineering workflows;- Strong communication and collaboration skills to work effectively with cross-functional teams and stakeholders;- Ability to stay up-to-date with the latest trends and advancements in data engineering and cloud technologies;- Strong problem-solving skills and the ability to troubleshoot and debug data engineering processes and workflows.