References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailReinforcement learning versus model predictive control: a comparison on a power system problem
Ernst, Damien ULg; Glavic, Mevludin; Capitanescu, Florin ULg et al

in IEEE Transactions on Systems, Man & Cybernetics : Part B (2009), 33(2), 517-519

This paper compares reinforcement learning (RL) with model predictive control (MPC) in a unified framework and reports experimental results of their application to the synthesis of a controller for a ... [more ▼]

This paper compares reinforcement learning (RL) with model predictive control (MPC) in a unified framework and reports experimental results of their application to the synthesis of a controller for a nonlinear and deterministic electrical power oscillations damping problem. Both families of methods are based on the formulation of the control problem as a discrete-time optimal control problem. The considered MPC approach exploits an analytical model of the system dynamics and cost function and computes open-loop policies by applying an interior-point solver to a minimization problem in which the system dynamics are represented by equality constraints. The considered RL approach infers in a model-free way closed-loop policies from a set of system trajectories and instantaneous cost values by solving a sequence of batch-mode supervised learning problems. The results obtained provide insight into the pros and cons of the two approaches and show that RL may certainly be competitive with MPC even in contexts where a good deterministic system model is available. [less ▲]

Detailed reference viewed: 111 (13 ULg)
Full Text
Peer Reviewed
See detailPseudo-geographical representations of power system buses by multidimensional scaling
Belmudes, Florence ULg; Ernst, Damien ULg; Wehenkel, Louis ULg

in Proceedings of the 15th International Conference on Intelligent System Applications to Power Systems (ISAP 2009) (2009)

Graphical representations of power systems are systematically used for planning and operation. The coordinate systems commonly used by Transmission System Operators are static and reflect the geographical ... [more ▼]

Graphical representations of power systems are systematically used for planning and operation. The coordinate systems commonly used by Transmission System Operators are static and reflect the geographical positions of each equipment of the system. We propose in this work to position on a twodimensional map the different buses of a power system in a way such that their coordinates also highlight some other physical information related to them. These pseudo-geographical representations are computed by formulating multidimensional scaling problems which aim at mapping a distance matrix combining both geographical and physical information into a vector of two-dimensional bus coordinates. We illustrate through examples that these pseudo-geographical representations can help to gain insights into the power system physical properties. [less ▲]

Detailed reference viewed: 56 (11 ULg)
Full Text
Peer Reviewed
See detailHVDC control strategies to improve transient stability in interconnected power systems
Hazra, Jagabondhu; Phulpin, Yannick; Ernst, Damien ULg

in Proceedings of the 2009 IEEE Bucharest PowerTech (2009)

This paper presents three HVDC modulation strategies to improve transient stability in an interconnected power system. AC variables such as rotor speeds, voltage phasors, and tieline power flows are used ... [more ▼]

This paper presents three HVDC modulation strategies to improve transient stability in an interconnected power system. AC variables such as rotor speeds, voltage phasors, and tieline power flows are used as input to the controller that modifies the power flow settings through the HVDC-links. The proposed techniques are tested on the IEEE 24-Bus reliability test system and critical clearing times obtained for several contingencies are analyzed. The paper shows that HVDC modulation can lead to substantial improvement in transient stability. [less ▲]

Detailed reference viewed: 135 (7 ULg)
Full Text
Peer Reviewed
See detailDecentralized reactive power dispatch for a time-varying multi-TSO system
Phulpin, Yannick; Begovic, Miroslav; Petit, Marc et al

in Proceedings of the 42nd Hawaii International Conference on System Sciences (HICSS-42) (2009)

This paper addresses the problem of reactive power dispatch in a power system partitioned into several areas controlled by different transmission system operators. Previous research has shown that nearly ... [more ▼]

This paper addresses the problem of reactive power dispatch in a power system partitioned into several areas controlled by different transmission system operators. Previous research has shown that nearly optimal performance could be achieved in a time-invariant system by using a specific iterative decentralized control scheme with no information exchange. At each iteration of this scheme, every transmission system operator concurrently schedules its own control settings for the next iteration while representing the neighboring areas with external network equivalents. This paper focuses on some parameter tracking techniques to extend the range of application of the decentralized control scheme to time-varying systems, where the time-varying nature of the system is modeled as a succession of steady-state operating conditions with variation of the load demand. Those new techniques are evaluated in the context of IEEE 118 bus system partitioned into three control areas. [less ▲]

Detailed reference viewed: 36 (8 ULg)
Full Text
Peer Reviewed
See detailResearch and Education Activities in Electric Power Systems at the University of Liège
Wehenkel, Louis ULg; Ernst, Damien ULg; Rousseaux, Patricia ULg et al

in Revue E Tijdschrift (2008), (4), 54-59

This paper presents research and education activities of the power systems group of the Department of Electrical Engineering and Computer Science of the University of Liège. These activities cover power ... [more ▼]

This paper presents research and education activities of the power systems group of the Department of Electrical Engineering and Computer Science of the University of Liège. These activities cover power systems stability, security, reliability, and markets, within the contexts of expansion planning, operation planning and real-time operation and automatic control. The paper also reviews the international collaborations of the team. [less ▲]

Detailed reference viewed: 126 (17 ULg)
Full Text
Peer Reviewed
See detailVariable selection for dynamic treatment regimes: a reinforcement learning approach
Fonteneau, Raphaël ULg; Wehenkel, Louis ULg; Ernst, Damien ULg

in The annual machine learning conference of Belgium and the Netherlands (BeNeLearn 2008) (2008, May)

Detailed reference viewed: 12 (1 ULg)
Full Text
Peer Reviewed
See detailModelling the influence of activation-induced apoptosis of CD4+ and CD8+ T-cells on the immune system response of a HIV-infected patient
Stan, Guy-Bart; Belmudes, Florence ULg; Fonteneau, Raphaël ULg et al

in IET Systems Biology (2008), 2(2), 94-102

On the basis of the human immunodeficiency virus (HIV) infection dynamics model proposed by Adams, the authors propose an extended model that aims at incorporating the influence of activation-induced ... [more ▼]

On the basis of the human immunodeficiency virus (HIV) infection dynamics model proposed by Adams, the authors propose an extended model that aims at incorporating the influence of activation-induced apoptosis of CD4+ and CD8+ T-cells on the immune system response of HIV-infected patients. Through this model, the authors study the influence of this phenomenon on the time evolution of specific cell populations such as plasma concentrations of HIV copies, or blood concentrations of CD4+ and CD8+ T-cells. In particular, this study shows that depending on its intensity, the apoptosis phenomenon can either favour or mitigate the long-term evolution of the HIV infection. [less ▲]

Detailed reference viewed: 63 (12 ULg)
Full Text
Peer Reviewed
See detailVariable selection for dynamic treatment regimes
Fonteneau, Raphaël ULg; Wehenkel, Louis ULg; Ernst, Damien ULg

in 27th Benelux Meeting on Systems and Control (2008)

Detailed reference viewed: 5 (0 ULg)
Full Text
Peer Reviewed
See detailRisk-aware decision making and dynamic programming
Defourny, Boris ULg; Ernst, Damien ULg; Wehenkel, Louis ULg

Conference (2008)

This paper considers sequential decision making problems under uncertainty, the tradeoff between the expected return and the risk of high loss, and methods that use dynamic programming to find optimal ... [more ▼]

This paper considers sequential decision making problems under uncertainty, the tradeoff between the expected return and the risk of high loss, and methods that use dynamic programming to find optimal policies. It is argued that using Bellman's Principle determines how risk considerations on the return can be incorporated. The discussion centers around returns generated by Markov Decision Processes and conclusions concern a large class of methods in Reinforcement Learning. [less ▲]

Detailed reference viewed: 96 (14 ULg)
Full Text
Peer Reviewed
See detailVariable selection for dynamic treatment regimes: a reinforcement learning approach
Fonteneau, Raphaël ULg; Wehenkel, Louis ULg; Ernst, Damien ULg

Conference (2008)

Dynamic treatment regimes (DTRs) can be inferred from data collected through some randomized clinical trials by using reinforcement learning algorithms. During these clinical trials, a large set of ... [more ▼]

Dynamic treatment regimes (DTRs) can be inferred from data collected through some randomized clinical trials by using reinforcement learning algorithms. During these clinical trials, a large set of clinical indicators are usually monitored. However, it is often more convenient for clinicians to have DTRs which are only defined on a small set of indicators rather than on the original full set. To address this problem, we analyse the approximation architecture of the state-action value functions computed by the fitted Q iteration algorithm - a RL algorithm - using tree-based regressors in order to identify a small subset of relevant ones. The RL algorithm is then rerun by considering only as state variables these most relevant indicators to have DTRs defined on a small set of indicators. The approach is validated on benchmark problems inspired from the classical ‘car on the hill’ problem and the results obtained are positive. [less ▲]

Detailed reference viewed: 64 (7 ULg)
Full Text
Peer Reviewed
See detailFuzzy partition optimization for approximate fuzzy Q-iteration
Busoniu, Lucian; Ernst, Damien ULg; Babuska, Robert et al

in Proceedings of the 17th IFAC World Congress (IFAC-08) (2008)

Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. Because exact RL can only be applied to very simple problems, approximate algorithms are usually necessary in practice ... [more ▼]

Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. Because exact RL can only be applied to very simple problems, approximate algorithms are usually necessary in practice. Many algorithms for approximate RL rely on basis-function representations of the value function (or of the Q-function). Designing a good set of basis functions without any prior knowledge of the value function (or of the Q-function) can be a difficult task. In this paper, we propose instead a technique to optimize the shape of a constant number of basis functions for the approximate, fuzzy Q-iteration algorithm. In contrast to other approaches to adapt basis functions for RL, our optimization criterion measures the actual performance of the computed policies in the task, using simulation from a representative set of initial states. A complete algorithm, using cross-entropy optimization of triangular fuzzy membership functions, is given and applied to the car-on-the-hill example. [less ▲]

Detailed reference viewed: 77 (2 ULg)
Full Text
Peer Reviewed
See detailConsistency of fuzzy model-based reinforcement learning
Busoniu, Lucian; Ernst, Damien ULg; Babuska, Robert et al

in Proceedings of the 2008 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE-08) (2008)

Reinforcement learning (RL) is a widely used paradigm for learning control. Computing exact RL solutions is generally only possible when process states and control actions take values in a small discrete ... [more ▼]

Reinforcement learning (RL) is a widely used paradigm for learning control. Computing exact RL solutions is generally only possible when process states and control actions take values in a small discrete set. In practice, approximate algorithms are necessary. In this paper, we propose an approximate, model-based Q-iteration algorithm that relies on a fuzzy partition of the state space, and a discretization of the action space. Using assumptions on the continuity of the dynamics and of the reward function, we show that the resulting algorithm is consistent, i.e., that the optimal solution is obtained asymptotically as the approximation accuracy increases. An experimental study indicates that a continuous reward function is also important for a predictable improvement in performance as the approximation accuracy increases. [less ▲]

Detailed reference viewed: 18 (3 ULg)
Full Text
Peer Reviewed
See detailHow compatible is perfect competition with transmission loss allocation methods?
Jing, Dai; Phulpin, Yannick; Rious, Vincent et al

in Proceedings of the 5th International Conference on the European Electricity Market (EEM-08) (2008)

This paper addresses the problem of transmission loss allocation in a power system where the generators, the demands and the system operator are independent. We suppose that the transmission losses are ... [more ▼]

This paper addresses the problem of transmission loss allocation in a power system where the generators, the demands and the system operator are independent. We suppose that the transmission losses are exclusively charged to the generators, which are willing to adopt a perfectly competitive behavior. In this context, their offers must reflect their production costs and their transmission loss costs, the latter being unknown beforehand and having to be predicted. We assume in this paper that the generators predict their loss costs from the past observations by using a weighted average of their past allocated costs. Under those assumptions, we simulate the market dynamics for different types of transmission loss allocation methods. The results show that the transmission loss allocation scheme can lead to a poorly efficient market in terms of social welfare. [less ▲]

Detailed reference viewed: 48 (6 ULg)
Full Text
Peer Reviewed
See detailCross-entropy based rare-event simulation for the identification of dangerous events in power systems
Belmudes, Florence ULg; Ernst, Damien ULg; Wehenkel, Louis ULg

in Proceedings of the 10th International Conference on Probabilistic Methods Applied to Power Systems (PMAPS-08) (2008)

We propose in this paper a novel approach for identifying rare events that may endanger power system integrity. This approach is inspired by the rare-event simulation literature and, in particular, by the ... [more ▼]

We propose in this paper a novel approach for identifying rare events that may endanger power system integrity. This approach is inspired by the rare-event simulation literature and, in particular, by the cross-entropy (CE) method for rare event simulation. We propose a general framework for exploiting the CE method in the context of power system reliability evaluation, when a severity function defined on the set of possible events is available. The approach is illustrated on the IEEE 30 bus test system when instability mechanisms related to static voltage security are considered. [less ▲]

Detailed reference viewed: 42 (5 ULg)
Full Text
Peer Reviewed
See detailAnalyzing transient instability phenomena beyond the classical stability boundary
Ali, Mahmoud; Glavic, Mevludin; Buisson, Jean et al

in Proceedings of the 40th North American Power Symposium (NAPS 2008) (2008)

We consider power systems for which the amount of power produced by their individual power plants is small with respect to the total generation of the system, and analyze how the transient instability ... [more ▼]

We consider power systems for which the amount of power produced by their individual power plants is small with respect to the total generation of the system, and analyze how the transient instability mechanisms of these systems change qualitatively when their size or the dispersion of their generators increases. Simulation results show that loss of synchronism will propagate more slowly and even stop propagating. Given the evolution of power systems towards more dispersed generation and geographically larger interconnections, we conclude that research in transient stability should focus more on the propagation of the loss of synchronism over longer time periods, so as to assess what happens to the overall system subsequently to the loss of synchronism of the first generators. We also argue that such studies might be very useful in order to provide guidelines for setting up power system control schemes to contain the propagation of instabilities, and we discuss some ideas for designing islanding based emergency control schemes for this. [less ▲]

Detailed reference viewed: 153 (2 ULg)
Full Text
Peer Reviewed
See detailLazy planning under uncertainty by optimizing decisions on an ensemble of incomplete disturbance trees
Defourny, Boris ULg; Ernst, Damien ULg; Wehenkel, Louis ULg

in Defourny, Boris; Ernst, Damien; Wehenkel, Louis (Eds.) Recent Advances in Reinforcement Learning (2008)

This paper addresses the problem of solving discrete-time optimal sequential decision making problems having a disturbance space W composed of a finite number of elements. In this context, the problem of ... [more ▼]

This paper addresses the problem of solving discrete-time optimal sequential decision making problems having a disturbance space W composed of a finite number of elements. In this context, the problem of finding from an initial state x0 an optimal decision strategy can be stated as an optimization problem which aims at finding an optimal combination of decisions attached to the nodes of a disturbance tree modeling all possible sequences of disturbances w0, w1, . . ., w(T−1) in W^T over the optimization horizon T. A significant drawback of this approach is that the resulting optimization problem has a search space which is the Cartesian product of O(|W|^(T−1)) decision spaces U, which makes the approach computationally impractical as soon as the optimization horizon grows, even if W has just a handful of elements. To circumvent this difficulty, we propose to exploit an ensemble of randomly generated incomplete disturbance trees of controlled complexity, to solve their induced optimization problems in parallel, and to combine their predictions at time t = 0 to obtain a (near-)optimal first-stage decision. Because this approach postpones the determination of the decisions for subsequent stages until additional information about the realization of the uncertain process becomes available, we call it lazy. Simulations carried out on a robot corridor navigation problem show that even for small incomplete trees, this approach can lead to near-optimal decisions. [less ▲]

Detailed reference viewed: 39 (3 ULg)
Full Text
Peer Reviewed
See detailContinuous-state reinforcement learning with fuzzy approximation
Busoniu, Lucian; Ernst, Damien ULg; De Schutter, Bart et al

in Tuyls, K.; Nowé, A.; Guessoum, Z. (Eds.) et al Adaptive Agents and Multi-Agent Systems III, Adaptation and Multi-Agent Learning (2008)

Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. There exist several convergent and consistent RL algorithms which have been intensively studied. In their original form ... [more ▼]

Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. There exist several convergent and consistent RL algorithms which have been intensively studied. In their original form, these algorithms require that the environment states and agent actions take values in a relatively small discrete set. Fuzzy representations for approximate, model-free RL have been proposed in the literature for the more difficult case where the state-action space is continuous. In this work, we propose a fuzzy approximation architecture similar to those previously used for Q-learning, but we combine it with the model-based Q-value iteration algorithm. We prove that the resulting algorithm converges. We also give a modified, asynchronous variant of the algorithm that converges at least as fast as the original version. An illustrative simulation example is provided. [less ▲]

Detailed reference viewed: 42 (7 ULg)
Full Text
Peer Reviewed
See detailOn the fairness of centralised decision-making strategies in multi-area power systems
Phulpin, Yannick; Begovic, Miroslav; Petit, Marc et al

in Proceedings of the 16th Power Systems Computation Conference (PSCC-08) (2008)

In this paper, we consider an interconnected power system, where the different Transmission System Operators (TSOs) have agreed to transferring some of their competences to a Centralised Control Center ... [more ▼]

In this paper, we consider an interconnected power system, where the different Transmission System Operators (TSOs) have agreed to transferring some of their competences to a Centralised Control Center (CCC). In such a context, a recurrent difficulty for the CCC is to define decision-making strategies which are fair enough to every TSO of the interconnected system. We address this multiobjective problem when the objective of every TSO can be represented by a real-valued function. We propose an algorithm to elect the solution that leads to the minimisation of the distance with the utopian minimum after having normalised the different objectives. We analyse the fairness of this solution in the sense of economics. We illustrate the approach with the IEEE 118 bus system partitioned in 3 areas having as local objective the minimisation of active power losses, the maximisation of reactive power reserves, or a combination of both criteria. [less ▲]

Detailed reference viewed: 17 (5 ULg)
Full Text
Peer Reviewed
See detailContingency filtering techniques for preventive security-constrained optimal power flow
Capitanescu, Florin ULg; Glavic, M.; Ernst, Damien ULg et al

in IEEE Transactions on Power Systems (2007), 22(4), 1690-1697

This paper focuses on contingency filtering to accelerate the iterative solution of preventive security-constrained optimal power flow (PSCOPF) problems. To this end, we propose two novel filtering ... [more ▼]

This paper focuses on contingency filtering to accelerate the iterative solution of preventive security-constrained optimal power flow (PSCOPF) problems. To this end, we propose two novel filtering techniques relying on the comparison at an intermediate PSCOPF solution of post-contingency constraint violations among postulated contingencies. We assess these techniques by comparing them with severity index-based filtering schemes, on a 60- and a 118-bus system. Our results show that the proposed contingency filtering techniques lead to faster solution of the PSCOPF, while being more robust and meaningful, than severity-index based ones. [less ▲]

Detailed reference viewed: 64 (12 ULg)
Full Text
Peer Reviewed
See detailEstimation of rotor angles of synchronous machines using artificial neural networks and local PMU-based quantities
Del Angel, A.; Geurts, Pierre ULg; Ernst, Damien ULg et al

in Neurocomputing (2007), 70(16-18), 2668-2678

This paper investigates a possibility for estimating rotor angles in the time frame of transient (angle) stability of electric power systems, for use in real-time. The proposed dynamic state estimation ... [more ▼]

This paper investigates a possibility for estimating rotor angles in the time frame of transient (angle) stability of electric power systems, for use in real-time. The proposed dynamic state estimation technique is based on the use of voltage and current phasors obtained from a phasor measurement unit supposed to be installed on the extra-high voltage side of the substation of a power plant, together with a multilayer perceptron trained off-line from simulations. We demonstrate that an intuitive approach to directly map phasor measurement inputs to the neural network to generator rotor angle does not offer satisfactory results. We found out that a good way to approach the angle estimation problem is to use two neural networks in order to estimate the sin(delta) and cos(delta) of the angle and recover the latter from these values by simple post-processing. Simulation results on a part of the Mexican interconnected system show that the approach could yield satisfactory accuracy for realtime monitoring and control of transient instability. (c) 2007 Elsevier B.V. All rights reserved. [less ▲]

Detailed reference viewed: 111 (8 ULg)