Senior Data Engineer in Impressit

Senior Data Engineer віддалено Impressit islooking for a Senior Data Engineer who will join anAI/ML team working onafully autonomousAI trading platform. What doweexpect?—Proficiency inPython for software development. —Experience with SQL and NoSQLdatabases. —Familiarity with data management techniques such asMapReduce, partitioning, and Spark. —Experience with data ingestion tools like Kafka, Flume, orSqoop. —Knowledge ofMapReduce for distributed data processing. —Understanding ofdatabase partitioning techniques. —Experience with Apache Spark for large-scale data processing. —Familiarity with Feature Store for managing machine learning features. —Experience inensuring engineering reproducibility indata pipelines. —Proficiency inwriting and executing unit tests. —Experience inerror handling toensure fault tolerance and robustness oftrading systems. —Proficiency inDirected Acyclic Graphs (DAGs) using Airflow orPrefect for workflow orchestration. —Familiarity with MLFlow for managing machine learning experiments. —Proficiency inDocker for containerization. —Experience with AWS services such asS3and Lambda for cloud deployment. —Upper-intermediate orhigher level ofEnglish. —Bachelor’s degree orhigher, preferably inSoftware Engineering orrelated field. What will youdo?—Focuses onDataOps from ingesting todeployment. —Maintain and enhance trading pipelines. —Combine various data sources via APIor scraping into asingular source oftruth. —Develop feature store and automate features cleaning, processing, engineering, and monitoring ofdrifts. —Develop data lineage ofeach feature and also ensure feature integrity with data fingerprinting techniques. —Work with batch and streaming data and reduce anomalies indata orgaps indifferent financial data sources. —Build, maintain, and enhance data pipelines using DataOps, MLOps, and AIOps frameworks. —Work with afull suite ofdata and model infrastructure and integration i.e. SageMaker Studio What about our benefits?—Generous vacation time: 24days per year; —Professional education and training budget; —Full tax compensation; —WOW team-building events; —Office inthe Lviv city center; —Paid sick leaves; Weare waiting for you tojoin our WOW team!