Supplementary damping control; reinforcement learning; model predictive control; extremely randomized trees
Abstract :
[en] This paper considers a trajectory-based approach to determine control signals superimposed to those of existing controllers so as to enhance the damping of electromechanical oscillations. This approach is framed as a discrete-time, multi-step optimization problem which can be solved by model-based and/or by learning-based methods. This paper proposes to apply a model-free tree-based batch mode Reinforcement Learning (RL) algorithm to perform such a supplementary damping control based only on information collected from observed trajectories of the power system. This RL-based supplementary damping control scheme is first implemented on a single generator and then several possibilities are investigated for extending it to multiple generators. Simulations are carried out on a 16-generators medium size power system model, where also possible benefits of combining this RL-based control with Model Predictive Control (MPC) are assessed.
Research center :
System and modeling, Montefiore institute
Disciplines :
Electrical & electronics engineering
Author, co-author :
Wang, Da ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Glavic, Mevludin ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Wehenkel, Louis ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Language :
English
Title :
Trajectory-Based Supplementary Damping Control for Power System Electromechanical Oscillations
G. Rogers, Power System Oscillations. Dordrecht, The Netherlands: Kluwer, 2000.
E. Grebe, J. Kabouris, S. L. Barba,W. Sattinger, andW.Winter, "Low frequency oscillations in the interconnected system of continental Europe," in Proc. IEEE Power and Energy Soc. General Meeting, Minneapolis, MN, USA, Jul. 2010, pp. 1-7.
J. Turunen et al., "Comparison of three electromechanical oscillation damping estimation methods," IEEE Trans. Power Syst., vol. 26, no. 4, pp. 2398-2407, Nov. 2011.
A. A. Hashmani and I. Erlich, "Mode selective damping of power system electromechanical oscillations using supplementary remote signals," IET Gener. Transm. Distrib., vol. 4, no. 10, pp. 1127-1138, 2010.
R. Preece, J. V. Milanovi?, A. M. Almutairi, and O. Marjanovic, "Damping of inter-area oscillations in mixed AC/DC networks using WAMS based supplementary controller," IEEE Trans. Power Syst., vol. 28, no. 2, pp. 1160-1169, May 2013.
R. Majumder, B. Chaudhuri, and B. C. Pal, "A probabilistic approach to model-based adaptive control damping of interarea oscillations in power system," IEEE Trans. Power Syst., vol. 20, no. 1, pp. 367-374, Feb. 2005.
R. Hadidi and B. Jeyasurya, "Reinforcement learning based real-time wide-area stabilizing control agents to enhance power system stability," IEEE Trans. Smart Grid, vol. 4, no. 1, pp. 489-497, Mar. 2013.
R.Hadidi and B. Jeyasurya, "Reinforcement learning approach for controlling power system stabilizers," Can. J. Electr. Comput. Eng., vol. 34, no. 3, pp. 99-103, 2009.
D. Ernst, M. Glavic, and L. Wehenkel, "Power system stability control: Reinforcement learning framework," IEEE Trans. Power Syst., vol. 19, no. 1, pp. 427-435, Feb. 2004.
M. Glavic, "Design of a resistive brake controller for power system stability enhancement using reinforcement learning," IEEE Trans. Control Syst. Technol., vol. 13, no. 5, pp. 743-751, Sep. 2005.
A. Karimi, S. Eftekarnejad, and A. Feliachi, "Reinforcement learning based backstepping control of power oscillations," Electr. Power Syst. Res., vol. 79, no. 11, pp. 1511-1520, Nov. 2009.
D. Ernst,M. Glavic, F. Capitanescu, and L.Wehenkel, "Reinforcement learning versus model predictive control: a comparison on a power system problem," IEEE Trans. Syst. Man. Cyber. B, Cybern., vol. 39, no. 2, pp. 517-529, Apr. 2009.
P. Geurts, D. Ernst, and L. Wehenkel, "Extremely randomised trees," Mach. Learning, vol. 63, no. 2, pp. 3-42, Apr. 2006.
D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning," J. Mach. Learning Res., vol. 6, pp. 503-556, Apr. 2005.
S. Russell and P. Norvig, Artificial Intelligence-A Modern Approach. Englewood Cliffs, NJ, USA: Prentice-Hall, 1995.
P. Stone andM.Veloso, "Multiagent systems:A survey fromamachine learning perspective," Autonomous Robots, vol. 8, no. 3, pp. 345-383, 2000.
A.G. Phadke and J. S. Thorp, Synchronized Phasor Measurements and Their Application. Berlin, Germany: Springer, 2008.
P. Korba and K. Uhlen, "Wide-area monitoring of electromechanical oscillations in the nordic power system: practical experience," IET Gener. Transm. Distrib., vol. 4, no. 10, pp. 1116-1126, 2010.
A. Chakrabortty, "Wide-area damping control of power systems using dynamic clustering and TCSC-based redesigns," IEEE Trans. Smart Grid, vol. 3, no. 3, pp. 1503-1514, Sep. 2012.
J. M. Maciejowski, Predictive Control with Constraints. Upper Saddle River, NJ, USA: Prentice-Hall, 2002.
R. Sutton and A. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT, 1998.
L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst, Reinforcement Learning and Dynamic Programming Using Function Approximators. Boca Raton, FL, USA: CRC, 2010.
D. Wang, M. Glavic, and L. Wehenkel, "A new MPC scheme for damping wide-area electromechanical oscillations in power systems," in Proc. IEEE PES PowerTech, Trondheim, Norway, Jun. 2011, pp. 1-7.
D.Wang, M. Glavic, and L.Wehenkel, "Distributed MPC of wide-area electromechanical oscillations of large-scale power systems," in Proc. 16th Intell. Syst. Applicat. to Power Syst., Crete, Greece, Sep. 2011, pp. 1-7.
J. H. Chow and K.W. Cheung, "A toolbox for power system dynamics and control engineering education and research," IEEE Trans. Power Syst., vol. 7, no. 4, pp. 1559-1564, Nov. 1992.
D. Wang, M. Glavic, and L. Wehenkel, "Comparison of centralized, distributed and hierarchical model predictive control schemes for electromechanical oscillations damping in large-scale power systems," Int. J. Electr. Power Energy Syst., vol. 58, pp. 32-41, Jun. 2014.
P. Kundur, Power System Stability and Control. New York, NY, USA: McGraw-Hill Professional, 1994.