Remote worker in Krakow, Poland with 6+ years of experience - Last seen 1 year ago

Data Engineer who worked in industries: medical, e-commerce, education, retail. Former finance student, who is passionate about automating routine and delivering good data for decision-making.

Details concerning my experience / achievements are presented below.
1010data (May 2021 - June 2022)
• Moved existing production computation to AWS EMR, Redshift
• Migrated corporate processing system to Airflow 2 / Google Api v4
• Supported, optimised distributed ETL workflow in Scala, Spark
• Code review and workload distribution among peers

First Line Software (Dec 2020 - Nov 2021)
• Memory & performance bottleneck optimization in Spark, graphX, cats
• Delivering graph processing algorithms for VAT tax fraud detection
• International medical product development support for sustainable code
• End-to-end OMOP CDM medical data processing delivery

Uchi (Apr 2020 – Sep 2020)
• Migrated over 25 data pipelines from 4 departments to SparkSQL
• Engaged in architecture planning, design and implementation for DWH
• Delivered code optimization for streaming / disributed processing Scala
• Active monitoring with Ansible, Swarm to support continuous data delivery

OutOfCloud (Nov 2019 – Apr 2020)
• Ad-hoc research for customer journey, segmentation, cohort analysis
• Designed DWH architecture to support Analysts with baseline metrics
• Responsible for designing ETL solution for communication campaigns through Python libs (bigquery, numpy, pandas, gspread)

X5 Retail Group (Jul 2018 - May 2019)
• Optimized \& contributed to corporate ETL solution (python, sqlalchemy)
• Engaged with descriptive report automation via pyspark, matplotlib
• Hive metastore datamart maintenance for batch processing
• Sources quality amelioration with ML practices (xgboost, clustering)
• Ad-hoc research for statistical analysis on time-series data

🛂 Nationality	🇵🇱 Poland
🏡 Residency	🇵🇱 Poland
📍 Location	🇵🇱 Poland
Remote OK	rok.co/@lithiferous
GitHub	lithiferous
LinkedIn	https://www.linkedin.com/in/degterev/
Skilled in	scala python rust sql bash c plus plus hdfs hive spark git avro emr docker airflow postgres kafka clickhouse s3 vertica mongodb redshift bigquery azure cloud databricks
Fluent in	englishfrenchrussian
Preferred annual pay (min)	$100,000/year
Last seen	1 year ago
Signed up	1 year ago
Badges	👨‍💻 Remote worker 🎖 Early adopter