Paper published in a book (Scientific congresses and symposiums)
How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies
François-Lavet, Vincent; Fonteneau, Raphaël; Ernst, Damien
2015In NIPS 2015 Workshop on Deep Reinforcement Learning
Peer reviewed
 

Files


Full Text
nips2015_Francois-Lavet_V.pdf
Author postprint (1.02 MB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
Reinforcement learning; Deep learning
Abstract :
[en] Using deep neural nets as function approximator for reinforcement learning tasks have recently been shown to be very powerful for solving problems approaching real-world complexity. Using these results as a benchmark, we discuss the role that the discount factor may play in the quality of the learning process of a deep Q-network (DQN). When the discount factor progressively increases up to its final value, we empirically show that it is possible to significantly reduce the number of learning steps. When used in conjunction with a varying learning rate, we empirically show that it outperforms original DQN on several experiments. We relate this phenomenon with the instabilities of neural networks when they are used in an approximate Dynamic Programming setting. We also describe the possibility to fall within a local optimum during the learning process, thus connecting our discussion with the exploration/exploitation dilemma.
Disciplines :
Computer science
Author, co-author :
François-Lavet, Vincent ;  Université de Liège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Dép. d'électric., électron. et informat. (Inst.Montefiore)
Fonteneau, Raphaël ;  Université de Liège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Ernst, Damien  ;  Université de Liège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Language :
English
Title :
How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies
Publication date :
December 2015
Event name :
NIPS 2015 Workshop on Deep Reinforcement Learning
Event date :
11 décembre 2015
Audience :
International
Main work title :
NIPS 2015 Workshop on Deep Reinforcement Learning
Peer reviewed :
Peer reviewed
Available on ORBi :
since 08 December 2015

Statistics


Number of views
245 (51 by ULiège)
Number of downloads
1120 (27 by ULiège)

Bibliography


Similar publications



Contact ORBi