Research

Undergraduate Research

REU at North Carolina State University, sponsored by the National Science Foundation:

Continual Learning via Learning a Continual Memory in Vision Transformer, advised by Dr. Tianfu Wu

Abstract:

This paper studies task-incremental continual learning (TCL) using Vision Transformers (ViTs). Our goal is to improve the overall streaming-task performance without catastrophic forgetting by learning task synergies (e.g., a new task learns to automatically reuse/adapt modules from previous similar tasks, or to introduce new modules when needed, or to skip some modules when it appears to be an easier task). One grand challenge is how to tame ViTs at streaming diverse tasks in terms of balancing their plasticity and stability in a task-aware way while overcoming the catastrophic forgetting. To address the challenge, we propose a simple yet effective approach that identifies a lightweight yet expressive ``sweet spot'' in the ViT block as the task-synergy memory in TCL. We present a Hierarchical task-synergy Exploration-Exploitation (HEE) sampling based neural architecture search (NAS) method for effectively learning task synergies by structurally updating the identified memory component with respect to four basic operations (reuse, adapt, new and skip) at streaming tasks. The proposed method is thus dubbed as CHEEM (Continual Hierarchical-Exploration-Exploitation Memory). In experiments, we test the proposed CHEEM on the challenging Visual Domain Decathlon (VDD) benchmark and the 5-Dataset benchmark. It obtains consistently better performance than the prior art with sensible CHEEM learned continually.

Senior Thesis:

Pricing American Options Through Model-Guided Machine Learning Incorporating Asset Characteristics and Market Environments, advised by Dr. Jianqing Fan

Abstract:

In the face of volatile financial markets, a sufficiently accurate estimation and prediction of stock option prices is crucial for researchers, traders, and investors. Since the development of the influential Black-Scholes model in 1973, many parametric option pricing models have been proposed to better capture the implied volatility surface, but they all incur significant pricing errors when applied to real world data. We expect that a parametrically-guided machine learning approach can achieve both improved accuracy and enhanced interpretability; in a recent work [Almeida et al., 2022], feedforward neural networks are employed to further correct the residual errors of various parametric attempts to model the implied volatility curve of European-style S&P 500 index options. In this thesis, we apply a similar methodology on American-style equity options by using parametric models, nonparametric neural networks, and parametrically-guided neural networks to determine prediction errors and neural network correction performance on individual stock components of the S&P 100. We also conduct a cross-sectional analysis of prediction error to determine the market conditions and company types that generate the best modeling results. As in previous literature, nonparametric correction from neural networks results in substantial gains, which vary dramatically across stock sectors. Error levels are most influenced by regressors dependent on market trends, and the level of error correction possible also varies based on the responsiveness of each stock sector to market changes. These discoveries are useful for determining market conditions and company characteristics that are conducive to low modeling error and finding improved techniques for options less receptive to error correction.