I am a big data solution consultant/data engineer, in Auckland New Zealand
I have years of experience of designing and developing data solutions on hadoop/spark/kafka stack both on-prem and in cloud, I have dealt with many variant types of spark such as EMR, databricks, Glue etc. I have successfully helped organizations implementing their big data strategies for data analytics and AI, some of them like banks in highly regulated environments, which require high standard data quality, data lineage and security
Recent years, I have also been involved in projects of migrating on prem hadoop solutions on to much consolidated platforms in cloud.
This is moved from previous wordpress backend to hexo
The image which was done by my son when he was about 9 is also moved to here.