Experimentation with different techniques for tracking optical flow in a video sequence. Optical flow is the apparent motion of objects, either due to the motion of the objects themselves or the motion of the camera.
Techiniques implemented include:

• Lucas-Kanade method for estimating optical flow by assuming that the flow is constant in a local neighborhood.
• Lucas-Kanade inverse-compositional method for estimating optical flow by assuming that the flow is constant in a local neighborhood and that the flow is smooth across the image.
• Matthews-Baker inverse-compositional method for estimating optical flow by assuming that the flow is constant in a local neighborhood and that the flow is smooth across the image.

You can view the report here.

Experimentation with using planar homographies to overlay images on top of other images. The goal is to create an augmented reality effect by transforming the perspective of the overlay image to match the perspective of the background image.
Techiniques implemented include:

• Image feature descriptors, such as BRIEF and ORB, for detecting and describing keypoints in images.
• RANSAC algorithm for estimating the homography between two images.
• Perspective Transform for transforming the perspective of an image to match the perspective of another image.

You can view the report here.

Artificial Intelligence (AI) is a hot topic, especially in light of recent improvements in its capabilities. This research project studies the societal attitudes toward AI, both present and how they have evolved over the years, as a way to understand how different events have shaped the public's perception of AI. We use topic modeling, sentiment analysis, and procrustes analysis to analyze relationships across time periods and extract insight into the changing story of artificial intelligence. Here's the studied dataset and the project report.

Collaborative project with Aimen Abdulaziz and Angelic McPherson.

I wrote a high-performant web scraper in Haskell to scrape 17000+ articles from online technology websites such as DeepMind, MIT Tech Review, OpenAI, Singularity Hub, and TechCrunch

The scraper uses Arrows and other functional programming patterns to ensure concurrency and efficiency. The dataset is open-source and available on HuggingFace.

Collaborative project with Aimen Abdulaziz and Angelic McPherson.

For fun, I implemented the Generative Pre-trained Transformer (GPT) architecture in PyTorch. GPT is a language model that uses a transformer architecture to generate text. I trained the model on the tiny Shakespeare dataset and used it to generate text in the style of Shakespeare.

Transformers are a type of neural network architecture that employes techniques such as:

• Self-attention, which allows the model to learn the relationships between different parts of the input data.
• Positional encoding, which allows the model to learn the relative positions of the input data.
• Multi-head attention, which allows the model to learn different relationships between different parts of the input data.
• Residual connections, which allows the model to learn the difference between the input data and the output data.

Experimentation with various adversarial training techniques for neural networks. Adversarial training is useful to improve the robustness of neural networks to adversarial attacks — which often happen as noise in the input data. Techniques explored include:

• Data augmentation, which helps the model have more data to learn from. New samples are generated by randomly cropping and/or flipping some images. We also add pertubations, to a random subset of images, which helps the model learn to be robust to noise.
• Dropout, which entails randomly blocking a subset of the neurons in the network from transmitting information during training. This helps huge models avoid overfitting, thus generalize better.
• Ensemble learning, which entails training multiple models on the same dataset and then averaging their predictions. This helps the model generalize better by mitigating the effects of a single model overfitting.

A relational database with a Python frontend and a MySQL backend, including various triggers and SQL automations.

Collaborative project with Ke Lou.

A fully functional 16-bit CPU implemented in Logisim. All CPU components including the ALU, registers, control unit, program counter, RAM, micro-sequencer, finite-state machine, and IO are implemented from basic gates.

A chess bot that uses various strategies including minimax, alpha-beta pruning, iterative deepening, transposition tables, move ordering, null-move pruning, aspiration windows, and quiescence search to maximize outcome against an opponent in chess.

A hyper-efficient search engine that crawls webpages (whose domain can be restricted to a given subset) and indexes them, then handles user queries on the contents of the collection of pages, with results ranked by frequency. It also supports query modifiers such as AND, OR, and NOT.