Courses Detail Information
ECE7606J – Stochastic Control and Reinforcement Learning
Instructors:
Credits: 3 credits
Pre-requisites: Undergraduate probability course.
Description:
Control and optimization of discrete-time and continuous-time Markov processes. Learning-based methods with exact or approximate solutions. Probability model, convergence of random variables. Countable-state Markov chains, continuous-state Markov chains, Foster-Lyapunov stability theory, Markov decision processes, dynamic programming. Continuous-time Markov processes, Poisson processes, queuing theory, infinitesimal generator, piecewise-deterministic Markov processes. Monte-Carlo method, temporal-difference method, approximate dynamic programming, Q learning, learning-based adaptive control. Applications include connected and autonomous vehicles, intelligent transportation systems, computer and communication systems, social networks, epidemics, and finance.
Course Topics:
Probability models
Bernoulli processes & birth-death chains
Finite-state Markov chains
lMarkov decision processes
Dynamic programming
Reinforcement learning methods for finite DPsApproximate methods for finite DPsCountable-state Markov chainsReview for quiz 1
Foster’s theorem and drift criteria
Continuous-state lVarkov chains
Discrete-time linear system with noise
Discrete-time linear quadratic regulation
Approximate dynamic programming
Countable-state Markov processes
Markovian queues
Foster-Lyapunov criterion
Continuous -time linear quadratic regulation