ORBi: Castronovo Michaël - Learning exploration/exploitation strategies for single trajectory reinforcement learning

Reference : Learning exploration/exploitation strategies for single trajectory reinforcement learning


	Document type :	Scientific congresses and symposiums : Paper published in a book
	Discipline(s) :	Engineering, computing & technology : Computer science
	To cite this reference:	http://hdl.handle.net/2268/127985

Title :	Learning exploration/exploitation strategies for single trajectory reinforcement learning
Language :	English
Author, co-author :	Castronovo, Michaël [Université de Liège - ULg > > > 2e an. master sc. infor., fin. appr.]
	Maes, Francis [Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation >]
	Fonteneau, Raphaël [Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation >]
	Ernst, Damien [Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids >]
Publication date :	2012
Main document title :	Proceedings of the 10th European Workshop on Reinforcement Learning (EWRL 2012)
Collection and collection volume :	JMLR Workshop and Conference Proceedings 24
Pages :	1-9
On invitation :	No
Audience :	International
Event name :	10th European Workshop on Reinforcement Learning (EWRL 2012)
Event date :	June 30-July 1, 2012
Event place (city) :	Edinburgh
Event country :	UK
Keywords :	[en] reinforcement learning ; Exploration/Exploitation dilemma ; formula discovery
Abstract :	[en] We consider the problem of learning high-performance Exploration/Exploitation (E/E) strategies for finite Markov Decision Processes (MDPs) when the MDP to be controlled is supposed to be drawn from a known probability distribution pM( ). The performance criterion is the sum of discounted rewards collected by the E/E strategy over an in finite length trajectory. We propose an approach for solving this problem that works by considering a rich set of candidate E/E strategies and by looking for the one that gives the best average performances on MDPs drawn according to pM( ). As candidate E/E strategies, we consider index-based strategies parametrized by small formulas combining variables that include the estimated reward function, the number of times each transition has occurred and the optimal value functions V and Q of the estimated MDP (obtained through value iteration). The search for the best formula is formalized as a multi-armed bandit problem, each arm being associated with a formula. We experimentally compare the performances of the approach with R-max as well as with e-Greedy strategies and the results are promising.
Funders :	Fonds de la Recherche Scientifique (Communauté française de Belgique) - F.R.S.-FNRS
Target :	Researchers ; Professionals ; Students
Permalink :	http://hdl.handle.net/2268/127985
Other URL :	http://jmlr.csail.mit.edu/proceedings/papers/v24/

File(s) associated to this reference

Fulltext file(s):

	File	Commentary	Version	Size	Access
Open access	castronovo12a.pdf		Publisher postprint	264.51 kB	View/Open

All documents in ORBi are protected by a user license.

Dspace v1.5