Skip to McMaster Navigation Skip to Site Navigation Skip to main content
McMaster logo

Faculty of Science

School Of Computational Science & Engineering




	

		

			

				

					
Accelerated Planning and Reinforcement Learning Algorithms
					
 Nov 23, 2022 
 3:30PM to 5:00PM

									

				

					Event Categories   2022 Seminars and   CSE Seminars Page
				

			


	

	

		

			

				

					

						

														

															

						

						

	

		


	Date/Time

	Date(s) - 23/11/2022
3:30 pm - 5:00 pm







Speaker: Amir-massoud Farahmand


Topic: Accelerated Planning and Reinforcement Learning Algorithms


Date of Presentation: Wednesday, Nov 23, 2022


Location:  IWC 224 & Online


Abstract: Solving sequential decision-making problems with long planning horizons is computationally challenging. In this talk, He focuses on the Value Iteration (VI) algorithm, a fundamental algorithm in dynamic programming and the basis of many reinforcement learning algorithms, and asks: Can we accelerate VI? He proposes two ideas. The first is based on the realization that VI itself defines a dynamical system. This suggests that we can use control theoretic tools to modify and accelerate it. The resulting algorithm is called PID VI [ICML 2021]. The second idea assumes that in addition to the true but expensive model of the environment, we have access to an inaccurate but cheap model too. Inspired by the matrix splitting technique in numerical linear algebra, we design an Operator Splitting Value Iteration algorithm that has a significantly faster convergence rate compared to VI [NeurIPS 2022].


[ICML 2021] Farahmand and Ghavamzadeh, “PID Accelerated Value Iteration Algorithm,” International Conference on Machine Learning, 2021.


[NeurIPS 2022] Rakhsha, Wang, Ghavamzadeh, & Farahmand, “Operator Splitting Value Iteration,” Neural Information Processing Systems, 2022.


Bio: Amir-massoud Farahmand is a research scientist, faculty member, and CIFAR AI Chair at the Vector Institute since 2018, and an assistant professor at the Department of Computer Science, the University of Toronto since 2019. He received his Ph.D. from the University of Alberta in 2011, followed by postdoctoral fellowships at McGill University (2011–2014) and Carnegie Mellon University (CMU) (2014).


Amir-massoud’s research interests are in reinforcement learning (RL) and machine learning (ML), with a focus on developing algorithms with theoretical guarantees. He has experience in developing RL and ML methods to solve industrially-motivated problems through his three years of experience at Mitsubishi Electric Research Laboratories in Cambridge, USA.


Amir-massoud has extensively published in the top and selective venues in machine learning. He has served as a member of the editorial board of Machine Learning Journal and Transactions on Machine Learning Research, as senior program committee of flagship conferences in ML, and has won many best reviewer awards.