Senior Data Engineer in Smart Arbitrage Technologies

About

We are looking for a strong Data Engineer to join our development team and help us build the new highly scalable and highly available version of an Algorithmic Trading Platform.

As a Data Engineer you will participate in platform’s architecture as well as in implementation of ETL services for it.

You will also develop data warehousing and data processing systems, tools to empower our research team in accessing the huge amounts of data efficiently.

The ideal candidate should have a strong background in Python, SQL, ETL, distributed data analytics, HA and HL systems.

This is a new project with a new foundation, giving the ability to fully utilise your experience without looking back at any legacy or previous technical choices.

Stack

• Python, Java, Scala, Go
• Spark, Flink, Pandas, Numpy, FastAPI
• Kafka, NATS, gRPC
• Postgres, Redis, ClickHouse
• Zookeeper, Consul, etcd
• Docker, Ansible, Vault, GHA
• Prometheus, AM, Grafana, ELK
• AWS

Requirements

• 5+ years of work experience as a Data Engineer.
• Proficiency in Python and using it as the primary development language in recent years.
• Practical knowledge of at least one other programming language: Java, Scala, Go, Rust, etc.
• Experience working with Big Data.
• Proficiency in SQL and relational databases: Postgres, MySQL, etc.
• Proficiency in NoSQL: Redis, Zookeeper, Hadoop, Consul, etc.
• Proficiency with column-oriented DBMS: ClickHouse, Vertica, Snowflake, etc.
• Ability to design complex data models: normalized, multidimensional, etc.
• Experience implementing distributed, scalable and fault-tolerant systems.
• Experience developing, maintaining, and debugging problems in large server-side code bases.
• Good knowledge of engineering best practices and testing.
• Ability to share knowledge and mentor others.

Responsibilities

• Improve existing Data Engineering infrastructure to make it scalable and prepare it for anticipated Big Data volumes.
• Create maintainable and fault-tolerant ETL pipelines, data services.
• Solve complicated technical problems regarding stability, availability, latency and throughput.
• Conduct R&D work with new technology.
• Maintain high-quality coding standards within the team.
• Identify, design and implement process improvements.
• Maintain good engineering culture, help others to learn and grow.
• Take a high degree of ownership and freedom to experiment with new technologies to improve our software.

Terms

• Minimum bureaucracy, the ability to easily communicate with all levels of management. We strive to make decisions quickly, without months of discussions.
• An opportunity to be at the forefront of trading technology, impacting major investment decisions.
• A collaborative, innovative environment where your contributions make a difference.