References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailEnsembles of extremely randomized trees and some generic applications
Wehenkel, Louis ULg; Ernst, Damien ULg; Geurts, Pierre ULg

in Proceedings of Robust Methods for Power System State Estimation and Load Forecasting (2006)

In this paper we present a new tree-based ensemble method called “Extra-Trees”. This algorithm averages predictions of trees obtained by partitioning the inputspace with randomly generated splits, leading ... [more ▼]

In this paper we present a new tree-based ensemble method called “Extra-Trees”. This algorithm averages predictions of trees obtained by partitioning the inputspace with randomly generated splits, leading to significant improvements of precision, and various algorithmic advantages, in particular reduced computational complexity and scalability. We also discuss two generic applications of this algorithm, namely for time-series classification and for the automatic inference of near-optimal sequential decision policies from experimental data. [less ▲]

Detailed reference viewed: 48 (5 ULg)
Full Text
Peer Reviewed
See detailOn multi-area security assessment of large interconnected power systems
Wehenkel, Louis ULg; Glavic, Mevludin; Ernst, Damien ULg

in Proceedings of the Second Carnegie Mellon Conference in Electric Power Systems: Monitoring, Sensing, Software and its Valuation for the Changing electric Power Industry (2006)

The paper introduces a framework for information exchange and coordination of security assessment suitable for distributed multi-area control in large interconnections operated by a team of transmission ... [more ▼]

The paper introduces a framework for information exchange and coordination of security assessment suitable for distributed multi-area control in large interconnections operated by a team of transmission system operators. The basic idea of the proposed framework consists of exchanging just enough information so that each operator can evaluate the impact in his control area of contingencies both internal and external to his area. The framework has been thought out with the European perspective in mind where it is presently not possible to set up a transnational security coordinator that would have authority to handle security control over the whole or part of the European interconnection. Nevertheless, it can also be considered as an approach to handle security control in North-American Mega-RTOs, where it could help to circumvent problems of scalability of algorithms and maintainability of data by distributing them over the TSOs under the authority of the RTO. [less ▲]

Detailed reference viewed: 28 (0 ULg)
Full Text
Peer Reviewed
See detailAbout automatic learning for advanced sensing, monitoring and control of electric power systems
Wehenkel, Louis ULg; Glavic, Mevludin; Geurts, Pierre ULg et al

in Proceedings of the Second Carnegie Mellon Conference in Electric Power Systems: Monitoring, Sensing, Software and its Valuation for the Changing electric Power Industry (2006)

The paper considers the possible uses of automatic learning for improving power system performance by software methodologies. Automatic learning per se is first reviewed and recent developements of the ... [more ▼]

The paper considers the possible uses of automatic learning for improving power system performance by software methodologies. Automatic learning per se is first reviewed and recent developements of the field are highlighted. Then the authors’ views of its main actual or potential applications related to power system operation and control are described, and in each application present status and needs for further developments are discussed. [less ▲]

Detailed reference viewed: 49 (4 ULg)
Full Text
Peer Reviewed
See detailA reinforcement learning based discrete supplementary control for power system transient stability enhancement
Glavic, M.; Ernst, Damien ULg; Wehenkel, Louis ULg

in Engineering Intelligent Systems for Electrical Engineering and Communications (2005), 13(2 Sp. Iss. SI), 81-88

This paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in ... [more ▼]

This paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in the form of switching strategies. In particular, the paper focuses on the application of a model based RL method, known as prioritized sweeping, a method proven to be suitable in applications in which computation is considered to be cheap. The curse of dimensionality problem is resolved by the system state dimensionality reduction based on the One Machine Infinite Bus (OMIB) transformation. Results obtained by using a synthetic four-machine power system are given to illustrate the performances of the proposed methodology. [less ▲]

Detailed reference viewed: 39 (1 ULg)
Full Text
Peer Reviewed
See detailTree-based batch mode reinforcement learning
Ernst, Damien ULg; Geurts, Pierre ULg; Wehenkel, Louis ULg

in Journal of Machine Learning Research (2005), 6

Reinforcement learning aims to determine an optimal control policy from interaction with a system or from observations gathered from a system. In batch mode, it can be achieved by approximating the so ... [more ▼]

Reinforcement learning aims to determine an optimal control policy from interaction with a system or from observations gathered from a system. In batch mode, it can be achieved by approximating the so-called Q-function based on a set of four-tuples (x(t), u(t), r(t), x(t+1)) where x(t) denotes the system state at time t, u(t) the control action taken, r(t) the instantaneous reward obtained and x(t+1) the successor state of the system, and by determining the control policy from this Q-function. The Q-function approximation may be obtained from the limit of a sequence of (batch mode) supervised learning problems. Within this framework we describe the use of several classical tree-based supervised learning methods (CART, Kd-tree, tree bagging) and two newly proposed ensemble algorithms, namely extremely and totally randomized trees. We study their performances on several examples and find that the ensemble methods based on regression trees perform well in extracting relevant information about the optimal control policy from sets of four-tuples. In particular, the totally randomized trees give good results while ensuring the convergence of the sequence, whereas by relaxing the convergence constraint even better accuracy results are provided by the extremely randomized trees. [less ▲]

Detailed reference viewed: 326 (42 ULg)
Full Text
Peer Reviewed
See detailCombining a stability and a performance-oriented control in power systems
Glavic, M.; Ernst, Damien ULg; Wehenkel, Louis ULg

in IEEE Transactions on Power Systems (2005), 20(1), 525-526

This paper suggests that the appropriate combination of a stability-oriented and a performance-oriented control technique is a promising way to implement advanced control schemes in power systems. The ... [more ▼]

This paper suggests that the appropriate combination of a stability-oriented and a performance-oriented control technique is a promising way to implement advanced control schemes in power systems. The particular approach considered combines control Lyapunov functions (CLF) and reinforcement learning. The capabilities of the resulting controller are illustrated on a control problem involving a thyristor-controlled series capacitor (TCSC) device for damping oscillations in a four-machine power system. [less ▲]

Detailed reference viewed: 21 (1 ULg)
Full Text
See detailPreventive and emergency control of power systems
Wehenkel, Louis ULg; Ruiz-Vega, Daniel; Ernst, Damien ULg et al

in Real Time Stability in Power Systems - Techniques for Early Detection of the Risk of Blackout (2005)

A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open loop emergency, and closed loop emergency controls. The ... [more ▼]

A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open loop emergency, and closed loop emergency controls. The organization of the resulting control schemes is then revisited in order to make it able to cover static and voltage security, in addition to transient stability. Distinct approaches for preventive and emergency operating conditions are advocated. [less ▲]

Detailed reference viewed: 87 (2 ULg)
Full Text
Peer Reviewed
See detailApproximate value iteration in the reinforcement learning context. Application to electrical power system control
Ernst, Damien ULg; Glavic, Mevludin; Geurts, Pierre ULg et al

in International Journal of Emerging Electrical Power Systems (2005), 3(1),

In this paper we explain how to design intelligent agents able to process the information acquired from interaction with a system to learn a good control policy and show how the methodology can be applied ... [more ▼]

In this paper we explain how to design intelligent agents able to process the information acquired from interaction with a system to learn a good control policy and show how the methodology can be applied to control some devices aimed to damp electrical power oscillations. The control problem is formalized as a discrete-time optimal control problem and the information acquired from interaction with the system is a set of samples, where each sample is composed of four elements: a state, the action taken while being in this state, the instantaneous reward observed and the successor state of the system. To process this information we consider reinforcement learning algorithms that determine an approximation of the so-called Q-function by mimicking the behavior of the value iteration algorithm. Simulations are first carried on a benchmark power system modeled with two state variables. Then we present a more complex case study on a four-machine power system where the reinforcement learning algorithm controls a Thyristor Controlled Series Capacitor (TCSC) aimed to damp power system oscillations. [less ▲]

Detailed reference viewed: 38 (3 ULg)
Full Text
Peer Reviewed
See detailNew developments in the application of automatic learning to power system control
Wehenkel, Louis ULg; Glavic, Mevludin; Ernst, Damien ULg

in Proceedings of the 15th Power System Computation Conference (PSCC 2005) (2005)

In this paper we present the basic principles of supervised learning and reinforcement learning as two complementary frameworks to design control laws or decision policies within the context of power ... [more ▼]

In this paper we present the basic principles of supervised learning and reinforcement learning as two complementary frameworks to design control laws or decision policies within the context of power system control. We also review recent developments in the realm of automatic learning methods and discuss their applicability to power system decision and control problems. Simulation results illustrating the potentials of the recently introduced fitted Q iteration learning algorithm in controlling a TCSC device aimed to damp electro-mechanical oscillations in a synthetic 4-machine system, are included in the paper. [less ▲]

Detailed reference viewed: 27 (4 ULg)
Full Text
Peer Reviewed
See detailA comparison of Nash equilibria analysis and agent-based modelling for power markets
Krause, Thilo; Andersson, Goran; Ernst, Damien ULg et al

in Proceedings of the 15th Power System Computation Conference (PSCC 2005) (2005)

In this paper we compare Nash equilibria analysis and agent-based modelling for assessing the market dynamics of network-constrained pool markets. Power suppliers submit their bids to the market place in ... [more ▼]

In this paper we compare Nash equilibria analysis and agent-based modelling for assessing the market dynamics of network-constrained pool markets. Power suppliers submit their bids to the market place in order to maximize their payoffs, where we apply reinforcement learning as a behavioral agent model. The market clearing mechanism is based on the locational marginal pricing scheme. Simulations are carried out on a benchmark power system.We show how the evolution of the agent-based approach relates to the existence of a unique Nash equilibrium or multiple equilibria in the system. Additionally, the parameter sensitivity of the results is discussed. [less ▲]

Detailed reference viewed: 14 (2 ULg)
Full Text
Peer Reviewed
See detailApplication of an advanced transient stability assessment and control method to a realistic power system
Cirio, D.; Lucarella, D.; Vimercati, G. et al

in Proceedings of the 15th Power System Computation Conference (PSCC 2005) (2005)

The paper presents a technical overview of a large research project on Dynamic Security Assessment (DSA) supported by EU. Transient Stability Assessment and Control, which was one of the main goals of the ... [more ▼]

The paper presents a technical overview of a large research project on Dynamic Security Assessment (DSA) supported by EU. Transient Stability Assessment and Control, which was one of the main goals of the project, is taken into consideration by presenting the fundamental theoretical methodology and possible applications. A specific prototype installation for a realistic power system is then reported by presenting and commenting some of the obtained results. [less ▲]

Detailed reference viewed: 52 (1 ULg)
Full Text
Peer Reviewed
See detailOn multi-area control in electric power systems
Zima, Marek; Ernst, Damien ULg

in Proceedings of the 15th Power System Computation Conference (PSCC 2005) (2005)

In this paper we study the concept of electric power system control, when the responsibility for controlling the entire system is shared by agents controlling their assigned areas. Within this framework ... [more ▼]

In this paper we study the concept of electric power system control, when the responsibility for controlling the entire system is shared by agents controlling their assigned areas. Within this framework, we suggest to study the dynamics created by the interactions of agents. In particular, we discuss the relation that exists between the information available to the different agents and their optimisation objective, and the performance of the overall power system. Simulations results, carried out on a 39-node power system voltage control problem, are provided and analyzed. They highlight, among others, the sub-optimal performance level attained by the system when the different agents exchange information about their area dynamics without sharing a common control objective. [less ▲]

Detailed reference viewed: 40 (0 ULg)
Full Text
Peer Reviewed
See detailSelecting concise sets of samples for a reinforcement learning agent
Ernst, Damien ULg

in Proceedings of the 3rd International Conference on Computational Intelligence, Robotics and Autonomous Systems (CIRAS 2005) (2005)

We derive an algorithm for selecting from the set of samples gathered by a reinforcement learning agent interacting with a deterministic environment, a concise set from which the agent can extract a good ... [more ▼]

We derive an algorithm for selecting from the set of samples gathered by a reinforcement learning agent interacting with a deterministic environment, a concise set from which the agent can extract a good policy. The reinforcement learning agent is assumed to extract policies from sets of samples by solving a sequence of standard supervised learning regression problems. To identify concise sets, we adopt a criterion based on an error function defined from the sequence of models produced by the supervised learning algorithm. We evaluate our approach on two-dimensional maze problems and show its good performances when problems are continuous. [less ▲]

Detailed reference viewed: 41 (8 ULg)
Full Text
Peer Reviewed
See detailPower systems stability control: Reinforcement learning framework
Ernst, Damien ULg; Glavic, Mevludin; Wehenkel, Louis ULg

in IEEE Transactions on Power Systems (2004), 19(1), 427-435

In this paper, we explore how a computational approach to learning from interactions, called reinforcement learning (RL), can be applied to control power systems. We describe some challenges in power ... [more ▼]

In this paper, we explore how a computational approach to learning from interactions, called reinforcement learning (RL), can be applied to control power systems. We describe some challenges in power system control and discuss how some of those challenges could be met by using these RL methods. The difficulties associated with their application to control power systems are described and discussed as well as strategies that can be adopted to overcome them. Two reinforcement learning modes are considered: the online mode in which the interaction occurs with the real power system and the offline mode in which the interaction occurs with a simulation model of the real power system. We present two case studies made on a four-machine power system model. The first one concerns the design by means of RL algorithms used in offline mode of a dynamic brake controller. The second concerns RL methods used in online mode when applied to control a thyristor controlled series capacitor (TCSC) aimed to damp power system oscillations. [less ▲]

Detailed reference viewed: 58 (7 ULg)
Full Text
Peer Reviewed
See detailNash equilibria and reinforcement learning for active decision maker modelling in power markets
Krause, Thilo; Andersson, Goran; Ernst, Damien ULg et al

in Proceedings of the 6th IAEE European Conference: Modelling in Energy Economics and Policy (2004)

In this paper, we study the behavior of power suppliers who submit their bids to the market place in order to maximize their payoffs. The market clearing mechanism is based on the locational marginal ... [more ▼]

In this paper, we study the behavior of power suppliers who submit their bids to the market place in order to maximize their payoffs. The market clearing mechanism is based on the locational marginal price. To study the interaction of the power suppliers, we rely on two different approaches and compare the results obtained. One approach consists of computing the Nash equilibria of the market, and the other models each player’s behavior by using reinforcement learning algorithms. Simulations are carried out on a five node power system. [less ▲]

Detailed reference viewed: 43 (4 ULg)
Full Text
Peer Reviewed
See detailMarket dynamics driven by the decision-making of both power producers and transmission owners
Minoia, Anna; Ernst, Damien ULg; Ilic, Marija

in Proceedings of the IEEE Power Engineering Society General Meeting 2004 (2004)

In this paper we consider an electricity market in which not only the power producers but also the transmission owners can submit a bid. The market is cleared at each stage by minimizing the sum of the ... [more ▼]

In this paper we consider an electricity market in which not only the power producers but also the transmission owners can submit a bid. The market is cleared at each stage by minimizing the sum of the production prices and the transmission prices. A model of the strategic behavior is formulated for the different agents of the system. This strategic behavior modelling leads to a market dynamics that can be used to determine the different payoffs of the agents over a temporal horizon. Simulations are carried out for several configurations of this two node power system. The influence of the transfer capacity and the market structure on the payoffs of the different agents is discussed. [less ▲]

Detailed reference viewed: 13 (0 ULg)
Full Text
Peer Reviewed
See detailMarket dynamics driven by the decision-making power producers
Ernst, Damien ULg; Minoia, Anna; Marija, Ilic

in Proceedings of 2004 IREP Symposium - Bulk Power System Dynamics and Control - VI (2004)

In this paper we consider a tool for analyzing the market outcomes when competitive agents (power producers) interact through the market place. The market clearing mechanism is based on the locational ... [more ▼]

In this paper we consider a tool for analyzing the market outcomes when competitive agents (power producers) interact through the market place. The market clearing mechanism is based on the locational marginal price scheme. A model of the strategic behavior is formulated for the agents. Each agent chooses its bid in order to maximize its profit by assuming that the other agents will post the same bid as at the previous session of the market, and by knowing the network characteristics. The income of each agent over a certain temporal horizon for different power system configurations (the addition of new transmission capabilities, new power plants) is evaluated according to this market dynamics and by integrating this dynamics over the chosen temporal horizon. The mathematical formulation, for the sake of simplicity, is related to a two node power system. In the simulations, the influence of different conditions (line transfer capacity, the number and size of generators, the presence of portfolio) on market outcomes is analyzed, and interesting and sometimes counterintuitive results are found. [less ▲]

Detailed reference viewed: 39 (1 ULg)
Full Text
See detailNear Optimal Closed-Loop Control. Application to Electric Power Systems
Ernst, Damien ULg

Doctoral thesis (2003)

Detailed reference viewed: 42 (16 ULg)
Full Text
Peer Reviewed
See detailIteratively extending time horizon reinforcement learning
Ernst, Damien ULg; Geurts, Pierre ULg; Wehenkel, Louis ULg

in Machine Learning: ECML 2003, 14th European Conference on Machine Learning (2003)

Reinforcement learning aims to determine an (infinite time horizon) optimal control policy from interaction with a system. It can be solved by approximating the so-called Q-function from a sample of four ... [more ▼]

Reinforcement learning aims to determine an (infinite time horizon) optimal control policy from interaction with a system. It can be solved by approximating the so-called Q-function from a sample of four-tuples (x(t), u(t), r(t), x(t+1)) where x(t) denotes the system state at time t, ut the control action taken, rt the instantaneous reward obtained and x(t+1) the successor state of the system, and by determining the optimal control from the Q-function. Classical reinforcement learning algorithms use an ad hoc version of stochastic approximation which iterates over the Q-function approximations on a four-tuple by four-tuple basis. In this paper, we reformulate this problem as a sequence of batch mode supervised learning problems which in the limit converges to (an approximation of) the Q-function. Each step of this algorithm uses the full sample of four-tuples gathered from interaction with the system and extends by one step the horizon of the optimality criterion. An advantage of this approach is to allow the use of standard batch mode supervised learning algorithms, instead of the incremental versions used up to now. In addition to a theoretical justification the paper provides empirical tests in the context of the "Car on the Hill" control problem based on the use of ensembles of regression trees. The resulting algorithm is in principle able to handle efficiently large scale reinforcement learning problems. [less ▲]

Detailed reference viewed: 40 (6 ULg)
Full Text
Peer Reviewed
See detailA reinforcement learning based discrete supplementary control for power system transient stability enhancement
Glavic, Mevludin; Ernst, Damien ULg; Wehenkel, Louis ULg

in Proceedings of the 12th Intelligent Systems Application to Power Systems Conference (ISAP 2003) (2003)

This paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in ... [more ▼]

This paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in the form of switching strategies. In particular, the paper focuses on the application of a model based RL method, known as prioritized sweeping, a method proven to be suitable in applications in which computation is considered to be cheap. The curse of dimensionality problem is resolved by the system state dimensionality reduction based on the One Machine Infinite Bus (OMIB) transformation. Results obtained by using a synthetic four-machine power system are given to illustrate the performances of the proposed methodology. [less ▲]

Detailed reference viewed: 27 (1 ULg)