References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailOptimized look-ahead tree policies: a bridge between look-ahead tree policies and direct policy search
Jung, Tobias ULg; Wehenkel, Louis ULg; Ernst, Damien ULg et al

in International Journal of Adaptive Control and Signal Processing (2014), 28(3-5), 255-289

Direct policy search (DPS) and look-ahead tree (LT) policies are two popular techniques for solving difficult sequential decision-making problems. They both are simple to implement, widely applicable ... [more ▼]

Direct policy search (DPS) and look-ahead tree (LT) policies are two popular techniques for solving difficult sequential decision-making problems. They both are simple to implement, widely applicable without making strong assumptions on the structure of the problem, and capable of producing high performance control policies. However, computationally both of them are, each in their own way, very expensive. DPS can require huge offline resources (effort required to obtain the policy) to first select an appropriate space of parameterized policies that works well for the targeted problem, and then to determine the best values of the parameters via global optimization. LT policies do not require any offline resources; however, they typically require huge online resources (effort required to calculate the best decision at each step) in order to grow trees of sufficient depth. In this paper, we propose optimized look-ahead trees (OLT), a model-based policy learning scheme that lies at the intersection of DPS and LT. In OLT, the control policy is represented indirectly through an algorithm that at each decision step develops, as in LT using a model of the dynamics, a small look-ahead tree until a prespecified online budget is exhausted. Unlike LT, the development of the tree is not driven by a generic heuristic; rather, the heuristic is optimized for the target problem and implemented as a parameterized node scoring function learned offline via DPS. We experimentally compare OLT with pure DPS and pure LT variants on optimal control benchmark domains. The results show that the LT-based representation is a versatile way of compactly representing policies in a DPS scheme (which results in OLT being easier to tune and having lower offline complexity than pure DPS); while at the same time, DPS helps to significantly reduce the size of the look-ahead trees that are required to take high-quality decisions (which results in OLT having lower online complexity than pure LT). Moreover, OLT produces overall better performing policies than pure DPS and pure LT and also results in policies that are robust with respect to perturbations of the initial conditions. [less ▲]

Detailed reference viewed: 94 (32 ULg)
Full Text
Peer Reviewed
See detailA learning procedure for sampling semantically different valid expressions
St-Pierre, David Lupien; Maes, Francis; Ernst, Damien ULg et al

in International Journal of Artificial Intelligence (2014), 12(1), 18-35

A large number of problems can be formalized as finding the best symbolic expression to maximize a given numerical objective. Most approaches to approximately solve such problems rely on random ... [more ▼]

A large number of problems can be formalized as finding the best symbolic expression to maximize a given numerical objective. Most approaches to approximately solve such problems rely on random exploration of the search space. This paper focuses on how this random exploration should be performed to take into account expressions redundancy and invalid expressions. We propose a learning algorithm that, given the set of available constants, variables and operators and given the target finite number of trials, computes a probability distribution to maximize the expected number of semantically different, valid, generated expressions. We illustrate the use of our approach on both medium-scale and large-scale expression spaces, and empirically show that such optimized distributions significantly outperform the uniform distribution in terms of the diversity of generated expressions. We further test the method in combination with the recently proposed nested Monte-Carlo algorithm on a set of benchmark symbolic regression problems and demonstrate its interest in terms of reduction of the number of required calls to the objective function. [less ▲]

Detailed reference viewed: 26 (5 ULg)
Full Text
See detailL'invité - Damien Ernst - "Nous allons vers une globalisation du marché de l'électricité"
Ernst, Damien ULg

Article for general public (2014)

En décembre 2013, Damien Ernst, Professeur à l’ULG, a donné une conférence au CESW intitulée : «Vers une globalisation du marché de l’électricité. Quel rôle pour les acteurs du secteur belge de ... [more ▼]

En décembre 2013, Damien Ernst, Professeur à l’ULG, a donné une conférence au CESW intitulée : «Vers une globalisation du marché de l’électricité. Quel rôle pour les acteurs du secteur belge de l’électricité?». Damien Ernst est un observateur privilégié du secteur énergétique belge, et plus particulièrement de tout ce qui concerne le secteur de l’électricité. Auteur de nombreuses publications et études, Damien Ernst s’est notamment interrogé sur les perspectives des énergies renouvelables en Belgique. Damien Ernst est l’invité de ce numéro 120 de la revue Wallonie. Dans son interview, il nous explique pourquoi la globalisation du marché de l’électricité est inéluctable et quelles en seront les conséquences, pour les entreprises du secteur et pour la Wallonie. [less ▲]

Detailed reference viewed: 39 (8 ULg)
Full Text
See detailActive Management of Low-Voltage Networks for Mitigating Overvoltages due to Photovoltaic Units
Olivier, Frédéric ULg; Aristidou, Petros ULg; Ernst, Damien ULg et al

Report (2014)

In this paper, the problem of integrating photo- voltaic panels into low-voltage distribution networks is addressed. A distributed scheme is proposed that modulates the active and reactive power output of ... [more ▼]

In this paper, the problem of integrating photo- voltaic panels into low-voltage distribution networks is addressed. A distributed scheme is proposed that modulates the active and reactive power output of inverters to prevent overvoltage problems. The proposed scheme is model-free and makes use of limited communication between the controllers, in the form of a distress signal, only during emergency conditions. It prioritizes the use or reactive power, while active power curtailment is performed only as a last resort. The performance of the scheme is tested using dynamic simulations, first on a single low-voltage feeder, then on a larger network composed of 14 low-voltage feeders. Its performance is compared to a centralized scheme based on the solution of an Optimal Power Flow problem, whose objective function is to minimize the active power curtailment. The proposed scheme successfully mitigates overvoltage situations due to high photovoltaic penetration and performs almost as well as the Optimal Power Flow based solution with significantly less information and communication requirements. [less ▲]

Detailed reference viewed: 29 (6 ULg)
Full Text
See detailActive network management for electrical distribution systems: problem formulation and benchmark
Gemine, Quentin ULg; Ernst, Damien ULg; Cornélusse, Bertrand ULg

E-print/Working paper (2014)

In order to operate an electrical distribution network in a secure and cost-efficient way, it is necessary, due to the rise of renewable energy-based distributed generation, to develop Active Network ... [more ▼]

In order to operate an electrical distribution network in a secure and cost-efficient way, it is necessary, due to the rise of renewable energy-based distributed generation, to develop Active Network Management (ANM) strategies. These strategies rely on short-term policies that control the power injected by generators and/or taken off by loads in order to avoid congestion or voltage problems. While simple ANM strategies would curtail the production of generators, more advanced ones would move the consumption of loads to relevant time periods to maximize the potential of renewable energy sources. However, such advanced strategies imply solving large-scale optimal sequential decision-making problems under uncertainty, something that is understandably complicated. In order to promote the development of computational techniques for active network management, we detail a generic procedure for formulating ANM decision problems as Markov decision processes. We also specify it to a 75-bus distribution network. The resulting test instance is available at http://www.montefiore.ulg.ac.be/~anm/ . It can be used as a test bed for comparing existing computational techniques, as well as for developing new ones. A solution technique that consists in an approximate multistage program is also illustrated on the test instance. [less ▲]

Detailed reference viewed: 14 (2 ULg)
Full Text
Peer Reviewed
See detailLipschitz robust control from off-policy trajectories
Fonteneau, Raphaël ULg; Ernst, Damien ULg; Boigelot, Bernard ULg et al

in Proceedings of the 53rd IEEE Conference on Decision and Control (IEEE CDC 2014) (2014)

We study the minmax optimization problem introduced in [Fonteneau et al. (2011), ``Towards min max reinforcement learning'', Springer CCIS, vol. 129, pp. 61-77] for computing control policies for batch ... [more ▼]

We study the minmax optimization problem introduced in [Fonteneau et al. (2011), ``Towards min max reinforcement learning'', Springer CCIS, vol. 129, pp. 61-77] for computing control policies for batch mode reinforcement learning in a deterministic setting with fixed, finite optimization horizon. First, we state that the $\min$ part of this problem is NP-hard. We then provide two relaxation schemes. The first relaxation scheme works by dropping some constraints in order to obtain a problem that is solvable in polynomial time. The second relaxation scheme, based on a Lagrangian relaxation where all constraints are dualized, can also be solved in polynomial time. We theoretically show that both relaxation schemes provide better results than those given in [Fonteneau et al. (2011)] [less ▲]

Detailed reference viewed: 24 (2 ULg)
Full Text
See detailPower system transient stability preventive and emergency control
Ruiz-Vega, Daniel; Wehenkel, Louis ULg; Ernst, Damien ULg et al

in Savulescu, Savu (Ed.) Real-Time Stability in Power Systems 2nd Edition (2014)

A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open loop emergency, and closed loop emergency controls. Recent ... [more ▼]

A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open loop emergency, and closed loop emergency controls. Recent progress in terms of a global transient stability constrained optimal power flow are presented, yielding in a scalable nonlinear programming formulation which allows to take near-optimal decisions for preventive control with a computing budget corresponding only to a few runs of standard optimal power flow and time domain simulations. These complementary techniques meet the stringent conditions imposed by the real-life applications. [less ▲]

Detailed reference viewed: 23 (2 ULg)
Full Text
Peer Reviewed
See detailApprentissage par renforcement batch fondé sur la reconstruction de trajectoires artificielles
Fonteneau, Raphaël ULg; Murphy, Susan A.; Wehenkel, Louis ULg et al

in Proceedings of the 9èmes Journées Francophones de Planification, Décision et Apprentissage (JFPDA 2014) (2014)

Cet article se situe dans le cadre de l’apprentissage par renforcement en mode batch, dont le problème central est d’apprendre, à partir d’un ensemble de trajectoires, une politique de décision optimisant ... [more ▼]

Cet article se situe dans le cadre de l’apprentissage par renforcement en mode batch, dont le problème central est d’apprendre, à partir d’un ensemble de trajectoires, une politique de décision optimisant un critère donné. On considère plus spécifiquement les problèmes pour lesquels l’espace d’état est continu, problèmes pour lesquels les schémas de résolution classiques se fondent sur l’utilisation d’approxima- teurs de fonctions. Cet article propose une alternative fondée sur la reconstruction de “trajectoires arti- ficielles” permettant d’aborder sous un angle nouveau les problèmes classiques de l’apprentissage par renforcement batch. [less ▲]

Detailed reference viewed: 31 (5 ULg)
Full Text
See detailThe Global Grid
Ernst, Damien ULg

Speech (2013)

Detailed reference viewed: 16 (0 ULg)
Full Text
Peer Reviewed
See detailOn periodic reference tracking using batch-mode reinforcement learning with application to gene regulatory network control
Sootla, Aivar; Strelkowa, Natajala; Ernst, Damien ULg et al

in Proceedings of the 52nd Annual Conference on Decision and Control (CDC 2013) (2013, December)

In this paper, we consider the periodic reference tracking problem in the framework of batch-mode reinforcement learning, which studies methods for solving optimal control problems from the sole knowledge ... [more ▼]

In this paper, we consider the periodic reference tracking problem in the framework of batch-mode reinforcement learning, which studies methods for solving optimal control problems from the sole knowledge of a set of trajectories. In particular, we extend an existing batch-mode reinforcement learning algorithm, known as Fitted Q Iteration, to the periodic reference tracking problem. The presented periodic reference tracking algorithm explicitly exploits a priori knowledge of the future values of the reference trajectory and its periodicity. We discuss the properties of our approach and illustrate it on the problem of reference tracking for a synthetic biology gene regulatory network known as the generalised repressilator. This system can produce decaying but long-lived oscillations, which makes it an interesting application for the tracking problem. [less ▲]

Detailed reference viewed: 13 (0 ULg)
Full Text
Peer Reviewed
See detailAn efficient algorithm for the provision of a day-ahead modulation service by a load aggregator
Mathieu, Sébastien ULg; Ernst, Damien ULg; Louveaux, Quentin ULg

in Proceedings of the 4th European Innovative Smart Grid Technologies (ISGT) (2013, October)

This article studies a decision making problem faced by an aggregator willing to offer a load modulation service to a Transmission System Operator. This service is contracted one day ahead and consists in ... [more ▼]

This article studies a decision making problem faced by an aggregator willing to offer a load modulation service to a Transmission System Operator. This service is contracted one day ahead and consists in a load modulation option, which can be called once per day. The option specifies the range of a potential modification on the demand of the loads within a certain time interval. The specific case where the loads can be modeled by a generic tank model is considered. Under this assumption, the problem of maximizing the range of the load modulation service can be formulated as a mixed integer linear programming problem. A novel heuristic-method is proposed to solve this problem in a computationally efficient manner. This method is tested on a set of problems. The results show that this approach can be orders of magnitude faster than CPLEX without significantly degrading the solution accuracy. [less ▲]

Detailed reference viewed: 117 (31 ULg)
Full Text
See detailGREDOR
Ernst, Damien ULg

Speech (2013)

Detailed reference viewed: 14 (1 ULg)
Full Text
Peer Reviewed
See detailThe global grid
Chatzivasileiadis, Spyros; Ernst, Damien ULg; Andersson, Göran

in Renewable Energy : An International Journal (2013), 57

This paper puts forward the vision that a natural future stage of the electricity network could be a grid spanning the whole planet and connecting most of the large power plants in the world: this is the ... [more ▼]

This paper puts forward the vision that a natural future stage of the electricity network could be a grid spanning the whole planet and connecting most of the large power plants in the world: this is the “Global Grid”. The main driving force behind the Global Grid will be the harvesting of remote renewable sources, and its key infrastructure element will be the high capacity long transmission lines. Wind farms and solar power plants will supply load centers with green power over long distances. This paper focuses on the introduction of the concept, showing that a globally interconnected network can be technologically feasible and economically competitive. We further highlight the multiple opportunities emerging from a global electricity network such as smoothing the renewable energy supply and electricity demand, reducing the need for bulk storage, and reducing the volatility of the energy prices. We also discuss possible investment mechanisms and operating schemes. Among others, we envision in such a system a global power market and the establishment of two new coordinating bodies, the “Global Regulator” and the “Global System Operator”. [less ▲]

Detailed reference viewed: 186 (31 ULg)
Full Text
Peer Reviewed
See detailThe global grid
Chatzivasileiadis, Spyros; Ernst, Damien ULg; Andersson, Göran

in Renewable Energy : An International Journal (2013), 57

This paper puts forward the vision that a natural future stage of the electricity network could be a grid spanning the whole planet and connecting most of the large power plants in the world: this is the ... [more ▼]

This paper puts forward the vision that a natural future stage of the electricity network could be a grid spanning the whole planet and connecting most of the large power plants in the world: this is the “Global Grid”. The main driving force behind the Global Grid will be the harvesting of remote renewable sources, and its key infrastructure element will be the high capacity long transmission lines. Wind farms and solar power plants will supply load centers with green power over long distances. This paper focuses on the introduction of the concept, showing that a globally interconnected network can be technologically feasible and economically competitive. We further highlight the multiple opportunities emerging from a global electricity network such as smoothing the renewable energy supply and electricity demand, reducing the need for bulk storage, and reducing the volatility of the energy prices. We also discuss possible investment mechanisms and operating schemes. Among others, we envision in such a system a global power market and the establishment of two new coordinating bodies, the “Global Regulator” and the “Global System Operator”. [less ▲]

Detailed reference viewed: 186 (31 ULg)
Full Text
Peer Reviewed
See detailMonte Carlo search algorithm discovery for single-player games
Maes, Francis; Lupien St-Pierre, David ULg; Ernst, Damien ULg

in IEEE Transactions on Computational Intelligence and AI in Games (2013), 5(3), 201-213

Much current research in AI and games is being devoted to Monte Carlo search (MCS) algorithms. While the quest for a single unified MCS algorithm that would perform well on all problems is of major ... [more ▼]

Much current research in AI and games is being devoted to Monte Carlo search (MCS) algorithms. While the quest for a single unified MCS algorithm that would perform well on all problems is of major interest for AI, practitioners often know in advance the problem they want to solve, and spend plenty of time exploiting this knowledge to customize their MCS algorithm in a problem-driven way. We propose an MCS algorithm discovery scheme to perform this in an automatic and reproducible way. We first introduce a grammar over MCS algorithms that enables inducing a rich space of candidate algorithms. Afterwards, we search in this space for the algorithm that performs best on average for a given distribution of training problems. We rely on multi-armed bandits to approximately solve this optimization problem. The experiments, generated on three different domains, show that our approach enables discovering algorithms that outperform several well-known MCS algorithms such as Upper Confidence bounds applied to Trees and Nested Monte Carlo search. We also show that the discovered algorithms are generally quite robust with respect to changes in the distribution over the training problems. [less ▲]

Detailed reference viewed: 169 (12 ULg)
Full Text
Peer Reviewed
See detailBatch mode reinforcement learning based on the synthesis of artificial trajectories
Fonteneau, Raphaël ULg; Murphy, Susan A.; Wehenkel, Louis ULg et al

in Annals of Operations Research (2013), 208(1), 383-416

Detailed reference viewed: 77 (21 ULg)
Full Text
See detailRisque majeur de blackout : que faire ?
Ernst, Damien ULg

Article for general public (2013)

Detailed reference viewed: 45 (9 ULg)
Full Text
See detailQuelles perspectives pour les énergies renouvelables en Wallonie ?
Ernst, Damien ULg

in LiègeU (2013), Eté 2013

Detailed reference viewed: 60 (7 ULg)
Full Text
See detailUnderstanding active network management in 30 minutes
Ernst, Damien ULg

Speech (2013)

Detailed reference viewed: 7 (1 ULg)