Shawn Ng

Data Engineer

Hi, I'm Shawn Ng from Singapore 🇸🇬. With a background spanning all data roles – from analyst to scientist to engineer – my expertise lies in backend software engineering. Currently, I'm working as a Data Engineer in the banking sector.

My passion extends beyond my full-time role, as I dedicate my free time to building my tech company, MaverickX.

This website is where I document my data and software projects, and I sincerely hope you find something useful within.

View Projects View Resume

What I do

Here is an overview of my technical skill sets

Data Engineering / Science

Big data (Spark, Hadoop)
Data pipeline (Airflow)
Database (SQL, NoSQL)
API
Data mining
Data visualization
Machine learning
Natural language processing

Cloud Engineering

Linux (Bash)
Amazon Web Service
Google Cloud Platform

Backend Engineering

Django/Flask/FastAPI
Express.js

Frontend Engineering

Bootstrap
JavaScript

Featured Projects

Obama VS Trump Twitter Analysis

In this project, I scraped and analyzed President Obama and President Trump tweets using natural language processing (NLP).

Published: 2018-04-01

View Case Study

Y Combinator Startups Analysis

In this project, I scraped and analysed 1317 Y Combinator Startups and share my insights.

Published: 2017-10-24

View Case Study

View Projects

Latest Blog Posts

What I am looking for

Published: 2021-12-10

Migrate to Unraid from Synology

How I migrate to Unraid from Synology

Published: 2021-12-03

Apache Spark VS Pandas VS Koalas

Apache Spark is an open-source unified analytics engine for large-scale data processing. Pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language. Koalas is Pandas API on Apache Spark.

Published: 2021-06-13

View Blogs