Senior Big Data Engineer (Web and Mobile Unit) in Sigma Software

Looking for a team of professionals who work with the most modern technologies? Looking for a place to grow your career in Big Data? We invite you to join our Big Data Competence Center, which is part of the complex organizational structure of Sigma Software, which combines numerous clients, interesting projects and events aimed at developing your professional skills.

We work in different areas of business and with a wide range of customers (those without an NDA can be found here: sigma.software/case-studies).

Our team highly values ​​the freedom of employees, independence in decision-making and the desire to deeply understand the client's request and see the root of the problem. We believe that people who strive to think in a similar way will definitely succeed, becoming recognized professionals and the driving force of Big Data development in Ukraine.

PROJECT

The Big Data Competence Center is a place where we collect engineering best practices and combine them into a single knowledge base, providing the best technical excellence services to our customers.

We are not just a team assembled to write code . We all want to contribute to this field by participating in the life of the Big Data department or constantly developing our own skills. In our department, you can be a Big Data engineer/team leader/architect, or you can become a mentor, or a person who implements all new technologies in the team, or just an active listener if you wish. Whatever you decide, rest assured that you are not alone. Whether you are faced with a difficult task, an unusual request from a client or the next project to choose from, you will always have a mentor and like-minded colleagues who will help you find the best solution.

We use an unconventional approach, hiring people not for a specific project, and first of all to our team. This gives us the opportunity to get to know you better and ensure that we provide a perfect match between the client's needs and your professional interests.

If you are ready to become part of the first-class Big Data community in Ukraine, take your chance and let's start making history together!

REQUIREMENTS

• 5+ years of experience with Python and SQL
• Experience with AWS, including API Gateway, Kinesis, Athena, RDS, and Aurora
• Experience in building ETL pipelines for analytics and internal operations
• Experience in creating internal APIs and their integration into external
• Working with the Linux operating system
• Effective communication skills, especially for explaining technical concepts to non-technical business leaders
• Willingness to work in a dynamic, research-oriented team
• Experience working with distributed application concepts and DevOps tools
• Excellent written and communication skills
• Ability to troubleshoot and troubleshoot

ADVANTAGE WILL BE:

• 2+ years of experience with Hadoop, Spark, and Airflow
• Experience with DAG and orchestration tools
• Hands-on data warehouse development experience based on Snowflake
• Experience rdevelopment of event pipelines of data

RESPONSIBILITIES

• Contributing to the research of new technologies and the development of complex solutions, supporting the culture of innovation, taking into account security issues, scalability and reliability with an emphasis on building our ETL processes
• Working with a modern data stack, developing well-thought-out technical solutions and writing reliable code, implementing data management processes
• Working and communicating professionally with the team of the customer
• Responsibility for the implementation of the main functions of the solution
• Participation in the process of gathering and specifying requirements, proposing optimal architectural strategies, leading the implementation of data architecture
• Development of main modules and functions , designing scalable and cost-effective solutions
• Executing code reviews, writing unit and integration tests
• Scaling a distributed system and infrastructure to the next level
• Building a data platform using capabilities of modern cloud providers (AWS/GCP/Azure)

ADDITIONAL DUTIES:

• These responsibilities will help you grow professionally and can change depending on the project and the desire to expand their role in the company
• Be part of the AWS cloud team and actively promote partnerships
• Development of Micro Batch/Real-Time streaming pipelines (Lambda architecture)
• Work on POC to validate proposed solutions and migrations
• Manage migration to modern technology platforms, provide technical guidance
• Follow CI/CD practices, help implement best practices practitioner in the team
• Facilitating the development of the department, mentoring other team members (optional)
• Owning the entire pipeline and optimizing engineering processes
• Developing complex ETL processes for analytics and data management, mass implementation management

Are you looking for a team of professionals on top of cutting-edge technologies? Are you looking for a place to boost your Big Data career? We invite you to join our Big Data Competence Center, part of Sigma Software's complex organizational structure that combines various clients, interesting projects, and activities to grow your professional skills.

CUSTOMER
We're acting in various business domains and working with the top range of clients (those without NDAs can be found here: sigma.software/case-studies).

Our team highly supports employees' freedom, independence in decision-making, and the desire to deeply understand client's request and see the root of the problem. We believe that people striving toward this mindset are bound to succeed as acclaimed professionals & driving force of Big Data development in Ukraine.

PROJECT
Big Data Competence Center is where we collect the best engineering practices and unite them into one knowledge base to provide the best technical excellence services for our clients.

We're not just a team gathered to write code. We all are willing to contribute to the field by participating in the life of Big Data Unit or by constantly growing our skills. In our unit, you can be a Big Data engineer/team lead/architect, or you can become a mentor, or a person behind all the new technologies in the team, or an active listener if you will. Whatever you decide, be sure that you’re not alone. Whether it’s a difficult task, an unordinary client request, or your next project to choose, you’ll always have a mentor & teammates with likewise mindsets to come up with the best solution.

We’re using an unusual approach, hiring people not for a specific project, but, first and foremost, to join our team. This gives us a chance to know you better and ensure that we’ll provide a perfect match between the client’s needs & your professional interests.

If you have the courage to become a part of the top-notch Big Data community in Ukraine, grab your chance, and let’s make some history together!

REQUIREMENTS

• 5+ years of experience with Python and SQL
• Experience with AWS, specifically API Gateway, Kinesis, Athena, RDS, and Aurora
• Experience building ETL pipelines for analytics and internal operations
• Experience building internal APIs and integrating with external APIs
• Working with Linux operational system
• Effective communication skills, especially for explaining technical concepts to nontechnical business leaders
• Desire to work in a dynamic, research-oriented team
• Experience with distributed application concepts and DevOps tooling
• Excellent writing and communication skills
• Troubleshooting and debugging ability

WILL BE A PLUS

• 2+ years of experience with Hadoop, Spark, and Airflow
• Experience with DAGs and orchestration tools
• Practical experience with developing Snowflake-driven data warehouses
• Experience with developing event-driven data pipelines

RESPONSIBILITIES

• Contributing to new technologies investigations and complex solutions design, supporting a culture of innovation considering matters of security, scalability, and reliability with the focus on building out our ETL processes
• Working with a modern data stack, producing well-designed technical solutions and robust code, and implementing data governance processes
• Working and professionally communicating with the customer’s team
• Taking up responsibility for delivering major solution features
• Participating in requirements gathering & clarification process, proposing optimal architecture strategies, leading the data architecture implementation
• Developing core modules and functions, designing scalable and cost-effective solutions
• Performing code reviews, writing unit and integration tests
• Scaling the distributed system and infrastructure to the next level
• Building data platform using the power of modern cloud providers (AWS/GCP/Azure)

EXTRA RESPONSIBILITIES:

• These responsibilities will help you to grow professionally and can vary depending on the project and the desire to extend your role in the company
• Being in the AWS cloud team and actively contributing to the partnership
• Developing the Micro Batch/Real-Time streaming pipelines (Lambda architecture)
• Working on POCs for validating proposed solutions and migrations
• Leading the migration to modern technology platforms, providing technical guidance
• Adhering to CI/CD methods, helping to implement best practices in the team
• Contributing to unit growth, mentoring other members of the team (optional)
• Owning the whole pipeline and optimizing the engineering processes
• Designing complex ETL processes for analytics and data management, driving the massive implementation