ATCS ¨C Selected Topics in Learning, Prediction, and Optimization (with application in Finance)
Lecturer: Jian Li ( lapordge at gmail dot com)
TA: Tianping Zhang ( ztp18 at mails.tsinghua.edu.cn )
time: every Monday 9:50am-12:15am
Room: TBD (we use ZOOM for now)
We intend to cover a subset of the following topics (tentative):
(1) I assume you already know all basics (convex optimization and machine learning, stochastic gradient descent, gradient boosting, deep learning basics, CNN, RNN, please see my undergrad course). If you don't know any machine learning, I do not suggest you to take this course. I will recall some concepts briefly when necessary.
(2) online learning, multi-armed bandit, statistical learning theory, theory of deep learning
I won't stickly follow the above order....I may skip something mentioned above and cover something not mentioned above...It is a graduate course.
I will be talking about several applications of ML and optimization in Finance (trading, pricing derivatives etc), and of course in typical CS areas like vision, nlp, social networks as well...
I will teach about 2/3 of the classes. For the rest, I will choose some topics and students need to do class presentation.
Tentative topics for class presentation: GAN, adverserial training, federated learning, AutoML, various financial applications.
knowledge about convex optimization may be useful. See
this course (by
S. Boyd) and
a previous course by myself. But it will be fine if you didn't take those
courses. Basic machine learning knowledge is a must.
Andrew Ng's undergrad lecture notes
The course is a blending of theory and practice. We will cover both the underlying mathematics as well as interesting heuristics.
A similar previous course.
|Feb 17||Basics of Online learning
The expert problem.
Multiplicative weights method
Online gradient descent
|Feb 24||Basics of Stock and Future Markets.
Cover's universal portfolio.
2nd order bound
Online Mirror Descent: Fenchel conjugate function
(closely related to other notion of duality, such as polar dual, dual norm)
Universal Portfolios With and Without Transaction Costs
|Mar 2||Online Mirror Descent
Interval regret, Sleeping Expert, Adaptive Regret
Pure exploration: Median Elimination
|Mar 16||Thompson Sampling
Contextual bandit: EXP4
The Epoch-Greedy Algorithm for Contextual Multi-armed Bandits
|Mar 23||Hyperparameter optimization
Introduction to Stock and Future Market
Efficient Market Hypothesis
|Mar 30||Digress to
Martingale convergence theorem
Brief Intro to Future and Option
No arbitrage argument
Basics of Brownian Motion
|See corresponding chapters in Options, Futures and Other Derivatives|
|Aril 6||Break. No class.||¡¡|
|Apr 13.||Pricing Derivatives
Binomial Tree model
Black-Scholes Merton (BSM) model
Pricing derivative via no regret online learning
Relation between SDE and PDE
Risk neutral valuation representation of BS formula
Robust pricing options using online learning
|Apr 27||Quick review of Classical Statistical
Maxima of (sub)Gaussian process
There are many good refereces for these classical results
I mainly followed the very well written (by van Handel) [lecture note] Probability in High Dimension (chapter 5 and 7)
[Book] Neural Network Learning: Theoretical Foundations is also a very good source.
|May 4||Labor day break, class moved to May 9||¡¡|
Pseudo-dimension, Fat-Shattering dimension
Relation between these dimensions and covering/packing numbers.
arxiv.org › cs
|May 11||Classical VC
Bounds of Neural Networks
(overview) generalization bounds in deep learning
Spectral norm generalization bound
Generalization via Compression
MMD, Universality, integral operator
generalization bound for kernel method
(overview) why existing (kernel) generalization bounds are not enough¡¡
|May 25||student presentation group 1||¡¡|
|Jun 1||student presentation group 2||¡¡|
|Jun 8||Project presentation||¡¡|
[Book] Introduction to online convex optimization
[Book] Learning, Prediction and Games
[Book] Options, Futures and Other Derivatives
[Book] Advances in Financial Machine Learning
[Book] Foundation of Machine Learning
Python is the default programming language we will use in the course.
If you haven't use it before, don't worry. It is very easy to learn (if you know any other programming language), and is a very efficient language, especially
for prototyping things related scientific computing and numerical optimization. Python codes are usually much shorter than C/C++ code (the lauguage has done a lot for you). It is also more flexible and generally faster than matlab.
A standard combination for this class is Python+numpy (a numeric lib for python)+scipy (a scientific computing lib for python)+matplotlab (for generating nice plots)
Another somewhat easier way is to install Anaconda (it is a free Python distribution with most popular packages).