Projects
A collection of personal projects showcasing my work in data engineering and software development.
Filter by:
AWS S3
Apache Airflow
Bash Scripts
Celery
Cosmos
DBT
Dagster
DigitalOcean
Django
Docker
Pandas
Plotly Dash
Polars
Pydantic
Python
RabbitMQ
Regex
SQL
Snowflake
fastAPI
matplotlib
posgresql
Data Pipeline Orchestrator (DBT+Airflow+Snowflake)
Scalable ETL pipeline for processing large datasets utilising DBT+Airflow+Snowflake, includes data quality checks and monitoring.
PythonDBTApache AirflowSQLAWS S3CosmosSnowflake
Data Pipeline Orchestrator (Dagster)
Scalable ETL pipeline for processing large datasets from Dagster, includes data quality checks and monitoring.
PythonDagsterSQLDockerAWS S3Polars
Log Analysis Tool
CLI tool for parsing and analyzing application logs. Features pattern matching, anomaly detection, and automated report generation.
PythonRegexBash Scripts
Taiwan Stock Exchange Data
Extract daily information from TWSE and pre-process before ingesting the information to database.
PythonposgresqlRabbitMQCeleryPydanticmatplotlibfastAPIDigitalOceanPandas
Trading Journal Application
Day trading journal platform where users can log, track, analyze trade performance with visual dashboard using plotly Dash and tagging trade comments feature.
PythonPlotly DashSQLDjango