Senior Data Engineer (IT Solutions) in Sigma Software

Seeking Senior Data Engineer to lead a small team, modernize a simple data platform into a cloud-based self-service analytics project.

CUSTOMER
Sigma Software provides top-quality software development, graphic design, testing, and support services. Systems developed to deliver customer benefits in automotive, telecommunications, aviation, advertising, gaming, banking, real estate, and healthcare industries.

PROJECT
Within an internal project at Sigma Software, we have two primary goals: to automate reporting and to create a self-service data analytics system for our company.
You will be responsible for designing and modeling data into DWH and Data Marts using Spark and MS SQL, exposing this data to, and collaborating with, BI engineers making reports in Apache Superset, and other roles consuming such data. Additionally, you’ll work with various other team members who rely on this data. Part of your task will involve transitioning our on-premises solution to a cloud-based environment.

RESPONSIBILITIES
• Our goal is to create an efficient system for collecting, processing, analyzing, and visualizing large amounts of data from various sources
• Lay down the foundation and model the data
• Develop a platform for building and maintaining data pipelines that collect data from different sources
• Create Data Warehouse
• Analyze and prepare domain descriptions in collaboration with Business analytics
• Create Data Marts
• Design a permission model with flexible control over vertical and horizontal access to data

REQUIREMENTS
• Conceptual knowledge of data analytics fundamentals, e.g., dimensional modeling, ETL/ELT, reporting tools, data governance, data warehousing, structured and unstructured data
• Strong SQL knowledge and experience with RDBMS, confident knowledge of database fundamentals
• Experience in database development and data modeling, ideally with Databricks/Spark
• Experience with Python
• Experience with Azure
• Working knowledge of serialization formats and their trade-offs (columnar vs. row-based)
• Experience debugging and optimizing Spark jobs
• Strong written and verbal communication skills
• At least an Upper-Intermediate level of English
• BS in Computer Science or a related field
• Experience with a Business Intelligence tool is a nice-to-have

Ми шукаємо старшого інженера з обробки даних, який очолюватиме невелику команду, що перетворить просту платформу даних на сучасний аналітичний проєкт із самообслуговуванням на основі хмарних технологій.

ЗАМОВНИК
Sigma Software надає високоякісні послуги з розробки програмного забезпечення, графічного дизайну, тестування та підтримки. Системи, розроблені для забезпечення потреб клієнтів в автомобільній, телекомунікаційній, авіаційній, рекламній, ігровій, банківській сферах, нерухомості та охороні здоров’я.

ПРОЄКТ
У рамкAs an internal project at Sigma Software, we have two main goals: to automate reporting and to create a self-service data analysis system for our company.

You will be responsible for the development and modeling of data in DWH and Data Marts using Spark and MS SQL, providing this data to BI engineers and collaborating with them, creating reports in Apache Superset, and other tasks related to the use of this data. In addition, you will work with other team members who use this data. Also part of your work will be the transfer of our local solution to the cloud environment.

DUTIES
< span style="font-style: normal; font-weight: 400">Our goal is to create an efficient system for collecting, processing, analyzing and visualizing large amounts of data from various sources

• Create a base and model data
• Develop a platform to build and support for data pipelines that collect data from different sources
• Create data store
• Analyze and prepare domain descriptions in cooperation with business analysts
• Create data showcases
• Develop a permissions model with flexible management of vertical and horizontal data access

REQUIREMENTS
• Conceptual knowledge of data analytics fundamentals such as dimensional modeling, ETL/ELT, reporting tools, data management, data warehouses, structured and unstructured data
• Deep knowledge of SQL and experience with RDBMS, solid knowledge of database fundamentals
• Database development and data modeling experience, ideally with Databricks/Spark
• Experience with Python
• Working knowledge of serialization formats and their trade-offs (column vs. row)
• Experience debugging and optimizing Spark jobs
• Strong written and verbal communication skills
• The level of English language proficiency is not lower than Upper-Intermediate
• Bachelor's degree in computer science or related fields
Experience with a business analytics tool would be a plus