Ciamac Moallemi
Graduate School of Business
Columbia University
Room 337 Towne
11:00 am - 12:00 pm
"Approximate Dynamic Programming via a Smoothed Linear Program"
Abstract
Problems involving the optimization and control of large-scale stochastic systems are plagued by the prohibitive computational burdens that arise from the curse of dimensionality. Approximate dynamic programming (ADP) seeks to obtain solutions to such problems by fitting the dynamic programming cost-to-go function. We present a novel ADP algorithm which we call smoothed approximate linear programming (SALP). SALP is related to the class of linear programming methods for ADP. However, we demonstrate that stronger theoretical guarantees are available as compared to existing LP-based methods. Further, we demonstrate the experimental effectiveness of the SALP method in the game of Tetris. Here, SALP outperforms a broad selection of ADP methods include LP-based methods, temporal difference learning, and policy gradient methods.
This is joint work with Vijay Desai (Columbia) and Vivek Farias (MIT).
BIOGRAPHY:
Ciamac C. Moallemi is an Assistant Professor at the Graduate School of Business of Columbia University, where he has been since 2007. He received SB degrees in Electrical Engineering & Computer Science and in Mathematics from the Massachusetts Institute of Technology (1996). He studied at the University of Cambridge, where he earned a Certificate of Advanced Study in Mathematics, with distinction (1997). He received a PhD in Electrical Engineering from Stanford University (2007). Prior to his doctoral studies, he developed quantitative methods in a number of entrepreneurial ventures, as a founder of a computer security software startup, in an early stage drug discovery startup, and as a partner of a fixed-income arbitrage hedge fund. He is a member of the IEEE and INFORMS. He is the recipient of a British Marshall Scholarship (1996) and a Benchmark Stanford Graduate Fellowship (2003).
Wednesday, March 4, 2009