References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailContinuous-state reinforcement learning with fuzzy approximation
Busoniu, Lucian; Ernst, Damien ULg; Babuska, Robert et al

in Proceedings of the 7th European Symposium on Adaptive Learning Agents and Multi-Agent Systems (ALAMAS-07) (2007)

Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. Well-understood RL algorithms with good convergence and consistency properties exist. In their original form, these ... [more ▼]

Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. Well-understood RL algorithms with good convergence and consistency properties exist. In their original form, these algorithms require that the environment states and agent actions take values in a relatively small discrete set. Fuzzy representations for approximate, model-free RL have been proposed in the literature for the more difficult case where the state-action space is continuous. In this work, we propose a fuzzy approximation structure similar to those previously used for Q-learning, but we combine it with the model-based Q-value iteration algorithm. We show that the resulting algorithm converges. We also give a modif ed, serial variant of the algorithm that converges at least as fast as the original version. An illustrative simulation example is provided. [less ▲]

Detailed reference viewed: 14 (3 ULg)
Full Text
Peer Reviewed
See detailA comparison of Nash equilibria analysis and agent-based modelling for power markets
Krause, Thilo; Beck, Elena Vdovina; Cherkaoui, Rachid et al

in International Journal of Electrical Power & Energy Systems (2006), 28(9), 599-607

In this paper we compare Nash equilibria analysis and agent-based modelling for assessing the market dynamics of network-constrained pool markets. Power suppliers submit their bids to the market place in ... [more ▼]

In this paper we compare Nash equilibria analysis and agent-based modelling for assessing the market dynamics of network-constrained pool markets. Power suppliers submit their bids to the market place in order to maximize their payoffs, where we apply reinforcement learning as a behavioral agent model. The market clearing mechanism is based on the locational marginal pricing scheme. Simulations are carried out on a benchmark power system. We show how the evolution of the agent-based approach relates to the existence of a unique Nash equilibrium or multiple equilibria in the system. Additionally, the parameter sensitivity of the results is discussed. (C) 2006 Elsevier Ltd. All rights reserved. [less ▲]

Detailed reference viewed: 26 (6 ULg)
Full Text
Peer Reviewed
See detailExtremely randomized trees
Geurts, Pierre ULg; Ernst, Damien ULg; Wehenkel, Louis ULg

in Machine Learning (2006), 63(1), 3-42

This paper proposes anew tree-based ensemble method for supervised classification and regression problems. It essentially consists of randomizing strongly both attribute and cut-point choice while ... [more ▼]

This paper proposes anew tree-based ensemble method for supervised classification and regression problems. It essentially consists of randomizing strongly both attribute and cut-point choice while splitting a tree node. In the extreme case, it builds totally randomized trees whose structures are independent of the output values of the learning sample. The strength of the randomization can be tuned to problem specifics by the appropriate choice of a parameter. We evaluate the robustness of the default choice of this parameter, and we also provide insight on how to adjust it in particular situations. Besides accuracy, the main strength of the resulting algorithm is computational efficiency. A bias/variance analysis of the Extra-Trees algorithm is also provided as well as a geometrical and a kernel characterization of the models induced. [less ▲]

Detailed reference viewed: 366 (52 ULg)
Full Text
Peer Reviewed
See detailReference transmission network: A game theory approach
Minoia, Anna; Ernst, Damien ULg; Dicorato, Maria et al

in IEEE Transactions on Power Systems (2006), 21(1), 249-259

The transmission network plays a key role in an oligopolistic electricity market. In fact, the capacity of a transmission network determines the degree to which the generators in different locations ... [more ▼]

The transmission network plays a key role in an oligopolistic electricity market. In fact, the capacity of a transmission network determines the degree to which the generators in different locations compete with others and could also greatly influence the strategic behaviors of market participants. In such an oligopolistic framework, different agents may have distinct and sometimes opposite interests in urging or hindering certain transmission expansions. Therefore, the regulatory authority, starting from the existing grid, faces the challenge of defining an optimal network upgrade to be used as benchmark for approval or rejection of a given transmission expansion. The aim of this paper is to define the concept of reference transmission network (RTN) from an economic point of view and to provide a tool for the RTN assessment in a deregulated framework where strategic behaviors are likely to appear. A general game-theoretic model for the RTN evaluation is presented, and the solution procedure is discussed. The strategic behavior of market agents in the spot market is modeled according to a Supply Function Equilibrium approach. The impact of transmission capacity expansion on market participants' strategic behavior is studied on a three-bus test network. The RTN is computed and compared with the optimal expansion found when perfect competition among power producers is assumed. [less ▲]

Detailed reference viewed: 44 (3 ULg)
Full Text
Peer Reviewed
See detailModel predictive control and reinforcement learning as two complementary frameworks
Ernst, Damien ULg; Glavic, Mevludin; Capitanescu, Florin ULg et al

in Proceedings of the 13th IFAC Workshop on Control Applications of Optimisation (2006)

Model predictive control (MPC) and reinforcement learning (RL) are two popular families of methods to control system dynamics. In their traditional setting, they formulate the control problem as a ... [more ▼]

Model predictive control (MPC) and reinforcement learning (RL) are two popular families of methods to control system dynamics. In their traditional setting, they formulate the control problem as a discrete-time optimal control problem and compute a suboptimal control policy. We present in this paper in a unified framework these two families of methods. We run for MPC and RL algorithms simulations on a benchmark control problem taken from the power system literature and discuss the results obtained. [less ▲]

Detailed reference viewed: 94 (6 ULg)
Full Text
Peer Reviewed
See detailReinforcement learning with raw image pixels as input state
Ernst, Damien ULg; Marée, Raphaël ULg; Wehenkel, Louis ULg

in Advances in machine vision, image processing & pattern analysis (Lecture notes in computer science, Vol. 4153) (2006)

We report in this paper some positive simulation results obtained when image pixels are directly used as input state of a reinforcement learning algorithm. The reinforcement learning algorithm chosen to ... [more ▼]

We report in this paper some positive simulation results obtained when image pixels are directly used as input state of a reinforcement learning algorithm. The reinforcement learning algorithm chosen to carry out the simulation is a batch-mode algorithm known as fitted Q iteration. [less ▲]

Detailed reference viewed: 40 (8 ULg)
Full Text
Peer Reviewed
See detailApplications of security-constrained optimal power flows
Capitanescu, Florin ULg; Glavic, Mevludin; Ernst, Damien ULg et al

in In Proceedings of Modern Electric Power Systems Symposium, MEPS06 (2006)

This paper proposes to formulate security control as a sequential decision making problem and presents new developments in automatic learning of sequential decision making strategies from simulations and ... [more ▼]

This paper proposes to formulate security control as a sequential decision making problem and presents new developments in automatic learning of sequential decision making strategies from simulations and/or information collected from real-life system measurements. The exploitation of these methods for the design of decision making strategies and control policies in the context of preventive and emergency mode dynamic security assessment and control is discussed and further opportunities for research in this area are highlighted. [less ▲]

Detailed reference viewed: 81 (0 ULg)
Full Text
Peer Reviewed
See detailClinical data based optimal STI strategies for HIV: a reinforcement learning approach
Ernst, Damien ULg; Stan, Guy-Bart; Gonçalves, Jorge et al

in Proceedings of the 45th IEEE Conference on Decision and Control (CDC 2006) (2006)

This paper addresses the problem of computing optimal structured treatment interruption strategies for HIV infected patients. We show that reinforcement learning may be useful to extract such strategies ... [more ▼]

This paper addresses the problem of computing optimal structured treatment interruption strategies for HIV infected patients. We show that reinforcement learning may be useful to extract such strategies directly from clinical data, without the need of an accurate mathematical model of HIV infection dynamics. To support our claims, we report simulation results obtained by running a recently proposed batch-mode reinforcement learning algorithm, known as fitted Q iteration, on numerically generated data. [less ▲]

Detailed reference viewed: 22 (8 ULg)
Full Text
Peer Reviewed
See detailClinical data based optimal STI strategies for HIV: a reinforcement learning approach
Ernst, Damien ULg; Stan, Guy-Bart; Gonçalves, Jorge et al

in Proceedings of the 15th Annual Machine Learning Conference of Belgium and The Netherlands (Benelearn 2006) (2006)

This paper addresses the problem of computing optimal structured treatment interruption strategies for HIV infected patients. We show that reinforcement learning may be useful to extract such strategies ... [more ▼]

This paper addresses the problem of computing optimal structured treatment interruption strategies for HIV infected patients. We show that reinforcement learning may be useful to extract such strategies directly from clinical data, without the need of an accurate mathematical model of HIV infection dynamics. To support our claims, we report simulation results obtained by running a recently proposed batch-mode reinforcement learning algorithm, known as tted Q iteration, on numerically generated data. [less ▲]

Detailed reference viewed: 45 (18 ULg)
Full Text
Peer Reviewed
See detailMulti-area security assessment: results using efficient bounding method
Wehenkel, Louis ULg; Glavic, Mevludin; Ernst, Damien ULg

in Proceedings of the 38th North American Power Symposium (NAPS 2006) (2006)

We present our recent results on using previously introduced framework for multi-area security assessment in large interconnections. The basic idea of the framework is exchanging just enough information ... [more ▼]

We present our recent results on using previously introduced framework for multi-area security assessment in large interconnections. The basic idea of the framework is exchanging just enough information so that each operator can evaluate the impact in his control area of contingencies both internal and external to his area. We provide illustrations based on a localization concept known as efficient bounding method and recently introduced approximate DC model of the European interconnected system. In this paper we focus on four transmission system operators (within the approximate DC model): Belgium, France, Germany, and Netherlands (for both Summer and Winter-peak load conditions). [less ▲]

Detailed reference viewed: 22 (3 ULg)
Full Text
Peer Reviewed
See detailDamping control by fusion of reinforcement learning and control Lyapunov functions
Glavic, Mevludin; Wehenkel, Louis ULg; Ernst, Damien ULg

in Proceedings of the 38th North American Power Symposium (NAPS 2006) (2006)

The main idea behind the concept, proposed in the paper, is the opportunity to make control systems with increased capabilities by synergetic fusion of the domain-specific knowledge and the methodologies ... [more ▼]

The main idea behind the concept, proposed in the paper, is the opportunity to make control systems with increased capabilities by synergetic fusion of the domain-specific knowledge and the methodologies from control theory and artificial intelligence. The particular approach considered combines Control Lyapunov Functions (CLF), a constructive control technique, an Reinforcement Learning (RL) in attempt to optimize a mix of system stability and performance. Two control schemes are proposed and the capabilities of the resulting controllers are illustrated on a control problem involving a Thyristor Controlled Series Capacitor (TCSC) for damping oscillations in a four-machine power system. [less ▲]

Detailed reference viewed: 25 (2 ULg)
Full Text
Peer Reviewed
See detailAutomatic learning of sequential decision strategies for dynamic security assessment and control
Wehenkel, Louis ULg; Glavic, Mevludin; Geurts, Pierre ULg et al

in Proceedings of the IEEE Power Engineering Society General Meeting 2006 (2006)

This paper proposes to formulate security control as a sequential decision making problem and presents new developments in automatic learning of sequential decision making strategies from simulations and ... [more ▼]

This paper proposes to formulate security control as a sequential decision making problem and presents new developments in automatic learning of sequential decision making strategies from simulations and/or information collected from real-life system measurements. The exploitation of these methods for the design of decision making strategies and control policies in the context of preventive and emergency mode dynamic security assessment and control is discussed and further opportunities for research in this area are highlighted. [less ▲]

Detailed reference viewed: 24 (4 ULg)
Full Text
Peer Reviewed
See detailEnsembles of extremely randomized trees and some generic applications
Wehenkel, Louis ULg; Ernst, Damien ULg; Geurts, Pierre ULg

in Proceedings of Robust Methods for Power System State Estimation and Load Forecasting (2006)

In this paper we present a new tree-based ensemble method called “Extra-Trees”. This algorithm averages predictions of trees obtained by partitioning the inputspace with randomly generated splits, leading ... [more ▼]

In this paper we present a new tree-based ensemble method called “Extra-Trees”. This algorithm averages predictions of trees obtained by partitioning the inputspace with randomly generated splits, leading to significant improvements of precision, and various algorithmic advantages, in particular reduced computational complexity and scalability. We also discuss two generic applications of this algorithm, namely for time-series classification and for the automatic inference of near-optimal sequential decision policies from experimental data. [less ▲]

Detailed reference viewed: 51 (5 ULg)
Full Text
Peer Reviewed
See detailOn multi-area security assessment of large interconnected power systems
Wehenkel, Louis ULg; Glavic, Mevludin; Ernst, Damien ULg

in Proceedings of the Second Carnegie Mellon Conference in Electric Power Systems: Monitoring, Sensing, Software and its Valuation for the Changing electric Power Industry (2006)

The paper introduces a framework for information exchange and coordination of security assessment suitable for distributed multi-area control in large interconnections operated by a team of transmission ... [more ▼]

The paper introduces a framework for information exchange and coordination of security assessment suitable for distributed multi-area control in large interconnections operated by a team of transmission system operators. The basic idea of the proposed framework consists of exchanging just enough information so that each operator can evaluate the impact in his control area of contingencies both internal and external to his area. The framework has been thought out with the European perspective in mind where it is presently not possible to set up a transnational security coordinator that would have authority to handle security control over the whole or part of the European interconnection. Nevertheless, it can also be considered as an approach to handle security control in North-American Mega-RTOs, where it could help to circumvent problems of scalability of algorithms and maintainability of data by distributing them over the TSOs under the authority of the RTO. [less ▲]

Detailed reference viewed: 29 (0 ULg)
Full Text
Peer Reviewed
See detailAbout automatic learning for advanced sensing, monitoring and control of electric power systems
Wehenkel, Louis ULg; Glavic, Mevludin; Geurts, Pierre ULg et al

in Proceedings of the Second Carnegie Mellon Conference in Electric Power Systems: Monitoring, Sensing, Software and its Valuation for the Changing electric Power Industry (2006)

The paper considers the possible uses of automatic learning for improving power system performance by software methodologies. Automatic learning per se is first reviewed and recent developements of the ... [more ▼]

The paper considers the possible uses of automatic learning for improving power system performance by software methodologies. Automatic learning per se is first reviewed and recent developements of the field are highlighted. Then the authors’ views of its main actual or potential applications related to power system operation and control are described, and in each application present status and needs for further developments are discussed. [less ▲]

Detailed reference viewed: 51 (4 ULg)
Full Text
Peer Reviewed
See detailA reinforcement learning based discrete supplementary control for power system transient stability enhancement
Glavic, M.; Ernst, Damien ULg; Wehenkel, Louis ULg

in Engineering Intelligent Systems for Electrical Engineering and Communications (2005), 13(2 Sp. Iss. SI), 81-88

This paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in ... [more ▼]

This paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in the form of switching strategies. In particular, the paper focuses on the application of a model based RL method, known as prioritized sweeping, a method proven to be suitable in applications in which computation is considered to be cheap. The curse of dimensionality problem is resolved by the system state dimensionality reduction based on the One Machine Infinite Bus (OMIB) transformation. Results obtained by using a synthetic four-machine power system are given to illustrate the performances of the proposed methodology. [less ▲]

Detailed reference viewed: 41 (1 ULg)
Full Text
Peer Reviewed
See detailTree-based batch mode reinforcement learning
Ernst, Damien ULg; Geurts, Pierre ULg; Wehenkel, Louis ULg

in Journal of Machine Learning Research (2005), 6

Reinforcement learning aims to determine an optimal control policy from interaction with a system or from observations gathered from a system. In batch mode, it can be achieved by approximating the so ... [more ▼]

Reinforcement learning aims to determine an optimal control policy from interaction with a system or from observations gathered from a system. In batch mode, it can be achieved by approximating the so-called Q-function based on a set of four-tuples (x(t), u(t), r(t), x(t+1)) where x(t) denotes the system state at time t, u(t) the control action taken, r(t) the instantaneous reward obtained and x(t+1) the successor state of the system, and by determining the control policy from this Q-function. The Q-function approximation may be obtained from the limit of a sequence of (batch mode) supervised learning problems. Within this framework we describe the use of several classical tree-based supervised learning methods (CART, Kd-tree, tree bagging) and two newly proposed ensemble algorithms, namely extremely and totally randomized trees. We study their performances on several examples and find that the ensemble methods based on regression trees perform well in extracting relevant information about the optimal control policy from sets of four-tuples. In particular, the totally randomized trees give good results while ensuring the convergence of the sequence, whereas by relaxing the convergence constraint even better accuracy results are provided by the extremely randomized trees. [less ▲]

Detailed reference viewed: 336 (42 ULg)
Full Text
Peer Reviewed
See detailCombining a stability and a performance-oriented control in power systems
Glavic, M.; Ernst, Damien ULg; Wehenkel, Louis ULg

in IEEE Transactions on Power Systems (2005), 20(1), 525-526

This paper suggests that the appropriate combination of a stability-oriented and a performance-oriented control technique is a promising way to implement advanced control schemes in power systems. The ... [more ▼]

This paper suggests that the appropriate combination of a stability-oriented and a performance-oriented control technique is a promising way to implement advanced control schemes in power systems. The particular approach considered combines control Lyapunov functions (CLF) and reinforcement learning. The capabilities of the resulting controller are illustrated on a control problem involving a thyristor-controlled series capacitor (TCSC) device for damping oscillations in a four-machine power system. [less ▲]

Detailed reference viewed: 25 (1 ULg)
Full Text
See detailPreventive and emergency control of power systems
Wehenkel, Louis ULg; Ruiz-Vega, Daniel; Ernst, Damien ULg et al

in Real Time Stability in Power Systems - Techniques for Early Detection of the Risk of Blackout (2005)

A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open loop emergency, and closed loop emergency controls. The ... [more ▼]

A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open loop emergency, and closed loop emergency controls. The organization of the resulting control schemes is then revisited in order to make it able to cover static and voltage security, in addition to transient stability. Distinct approaches for preventive and emergency operating conditions are advocated. [less ▲]

Detailed reference viewed: 90 (3 ULg)
Full Text
Peer Reviewed
See detailApproximate value iteration in the reinforcement learning context. Application to electrical power system control
Ernst, Damien ULg; Glavic, Mevludin; Geurts, Pierre ULg et al

in International Journal of Emerging Electrical Power Systems (2005), 3(1),

In this paper we explain how to design intelligent agents able to process the information acquired from interaction with a system to learn a good control policy and show how the methodology can be applied ... [more ▼]

In this paper we explain how to design intelligent agents able to process the information acquired from interaction with a system to learn a good control policy and show how the methodology can be applied to control some devices aimed to damp electrical power oscillations. The control problem is formalized as a discrete-time optimal control problem and the information acquired from interaction with the system is a set of samples, where each sample is composed of four elements: a state, the action taken while being in this state, the instantaneous reward observed and the successor state of the system. To process this information we consider reinforcement learning algorithms that determine an approximation of the so-called Q-function by mimicking the behavior of the value iteration algorithm. Simulations are first carried on a benchmark power system modeled with two state variables. Then we present a more complex case study on a four-machine power system where the reinforcement learning algorithm controls a Thyristor Controlled Series Capacitor (TCSC) aimed to damp power system oscillations. [less ▲]

Detailed reference viewed: 38 (3 ULg)