References of "Ernst, Damien"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailDecision Making from Confidence Measurement on the Reward Growth using Supervised Learning: A Study Intended for Large-Scale Video Games
Taralla, David ULg; Qiu, Zixiao ULg; Sutera, Antonio ULg et al

in Proceedings of the 2016 International Conference on Agents and Artificial Intelligence (in press)

Video games have become more and more complex over the past decades. Today, players wander in visually and option- rich environments, and each choice they make, at any given time, can have a combinatorial ... [more ▼]

Video games have become more and more complex over the past decades. Today, players wander in visually and option- rich environments, and each choice they make, at any given time, can have a combinatorial number of consequences. However, modern artificial intelligence is still usually hard-coded, and as the game environments become increasingly complex, this hard-coding becomes exponentially difficult. Recent research works started to let video game autonomous agents learn instead of being taught, which makes them more intelligent. This contribution falls under this very perspective, as it aims to develop a framework for the generic design of autonomous agents for large-scale video games. We consider a class of games for which expert knowledge is available to define a state quality function that gives how close an agent is from its objective. The decision making policy is based on a confidence measurement on the growth of the state quality function, computed by a supervised learning classification model. Additionally, no stratagems aiming to reduce the action space are used. As a proof of concept, we tested this simple approach on the collectible card game Hearthstone and obtained encouraging results. [less ▲]

Detailed reference viewed: 78 (10 ULg)
Full Text
Peer Reviewed
See detailActive Management of Low-Voltage Networks for Mitigating Overvoltages due to Photovoltaic Units
Olivier, Frédéric ULg; Aristidou, Petros ULg; Ernst, Damien ULg et al

in IEEE Transactions on Smart Grid (in press)

In this paper, the overvoltage problems that might arise from the integration of photovoltaic panels into low-voltage distribution networks is addressed. A distributed scheme is proposed that adjusts the ... [more ▼]

In this paper, the overvoltage problems that might arise from the integration of photovoltaic panels into low-voltage distribution networks is addressed. A distributed scheme is proposed that adjusts the reactive and active power output of inverters to prevent or alleviate such problems. The proposed scheme is model-free and makes use of limited communication between the controllers, in the form of a distress signal, only during emergency conditions. It prioritizes the use of reactive power, while active power curtailment is performed only as a last resort. The behavior of the scheme is studied using dynamic simulations on a single low-voltage feeder and on a larger network composed of 14 low-voltage feeders. Its performance is compared to a centralized scheme based on the solution of an Optimal Power Flow problem, whose objective function is to minimize the active power curtailment. The proposed scheme successfully mitigates overvoltage situations due to high photovoltaic penetration and performs almost as well as the Optimal Power Flow based solution with significantly less information and communication requirements. [less ▲]

Detailed reference viewed: 553 (104 ULg)
Full Text
See detailIsraël, vainqueur involontaire des guerres islamistes
Ernst, Damien ULg; Hermans, Michel ULg

Article for general public (2016)

Detailed reference viewed: 29 (1 ULg)
Full Text
Peer Reviewed
See detailDSIMA: A testbed for the quantitative analysis of interaction models within distribution networks
Mathieu, Sébastien ULg; Louveaux, Quentin ULg; Ernst, Damien ULg et al

in Sustainable Energy, Grids and Networks (2016), 5

This article proposes an open-source testbed to simulate interaction models governing the exchange of flexibility services located within a distribution network. The testbed is an agent-based system in ... [more ▼]

This article proposes an open-source testbed to simulate interaction models governing the exchange of flexibility services located within a distribution network. The testbed is an agent-based system in which the distribution system operator, the transmission system operator, producers and retailers make their decisions based on mixed-integer linear programs. This testbed helps to highlight the characteristics of an interaction model, the benefits for the agents and eases the detection of unwanted or abusive behaviors which decreases the welfare. The testbed is implemented in Python and the optimization problems are encoded in the modeling language ZIMPL. A web interface is coupled with an instance generator using macroscopic parameters of the system such as the total power production. This testbed is, therefore, well suited to test the implemented interaction models on various scenarios and to extend the implementation to other models. Five interaction models developed with industrial partners are simulated over a year on a 75-bus test system. Simulations show that interaction models relying on active network management, as they have been developed, lead to substantial welfare even though they suffer from a lack of coordination between the DSO and the TSO. A conservative interaction model restricting grid users to an access range that is computed ahead of time to prevent any congestion, avoids shedding distributed generation but considerably restrains the amount of distributed production. [less ▲]

Detailed reference viewed: 59 (17 ULg)
Full Text
Peer Reviewed
See detailImitative Learning for Online Planning in Microgrids
Aittahar, Samy ULg; François-Lavet, Vincent ULg; Lodeweyckx, Stefan et al

in Woon, Wei Lee; Zeyar, Aung; Stuart, Madnick (Eds.) Data Analytics for Renewable Energy Integration (2015, December 15)

This paper aims to design an algorithm dedicated to operational planning for microgrids in the challenging case where the scenarios of production and consumption are not known in advance. Using expert ... [more ▼]

This paper aims to design an algorithm dedicated to operational planning for microgrids in the challenging case where the scenarios of production and consumption are not known in advance. Using expert knowledge obtained from solving a family of linear programs, we build a learning set for training a decision-making agent. The empirical performances in terms of Levelized Energy Cost (LEC) of the obtained agent are compared to the expert performances obtained in the case where the scenarios are known in advance. Preliminary results are promising. [less ▲]

Detailed reference viewed: 86 (40 ULg)
Full Text
Peer Reviewed
See detailHow to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies
François-Lavet, Vincent ULg; Fonteneau, Raphaël ULg; Ernst, Damien ULg

in NIPS 2015 Workshop on Deep Reinforcement Learning (2015, December)

Using deep neural nets as function approximator for reinforcement learning tasks have recently been shown to be very powerful for solving problems approaching real-world complexity. Using these results as ... [more ▼]

Using deep neural nets as function approximator for reinforcement learning tasks have recently been shown to be very powerful for solving problems approaching real-world complexity. Using these results as a benchmark, we discuss the role that the discount factor may play in the quality of the learning process of a deep Q-network (DQN). When the discount factor progressively increases up to its final value, we empirically show that it is possible to significantly reduce the number of learning steps. When used in conjunction with a varying learning rate, we empirically show that it outperforms original DQN on several experiments. We relate this phenomenon with the instabilities of neural networks when they are used in an approximate Dynamic Programming setting. We also describe the possibility to fall within a local optimum during the learning process, thus connecting our discussion with the exploration/exploitation dilemma. [less ▲]

Detailed reference viewed: 91 (14 ULg)
Full Text
See detailGlobal Grid(s) versus Microgrids
Ernst, Damien ULg

Speech/Talk (2015)

Detailed reference viewed: 52 (12 ULg)
Full Text
See detailLa Russie, ce dangereux ami pour lutter contre le djihadisme wahhabiste
Ernst, Damien ULg; Hermans, Michel ULg

Article for general public (2015)

Detailed reference viewed: 38 (2 ULg)
Full Text
Peer Reviewed
See detailSequential decision-making approach for quadrangular mesh generation
Johnen, Amaury ULg; Ernst, Damien ULg; Geuzaine, Christophe ULg

in Engineering with Computers (2015), 31(4), 729-735

A new indirect quadrangular mesh generation algorithm which relies on sequential decision-making techniques to search for optimal triangle recombinations is presented. In contrast to the state-of-art ... [more ▼]

A new indirect quadrangular mesh generation algorithm which relies on sequential decision-making techniques to search for optimal triangle recombinations is presented. In contrast to the state-of-art Blossom-quad algorithm, this new algorithm is a good candidate for addressing the 3D problem of recombining tetrahedra into hexahedra. [less ▲]

Detailed reference viewed: 273 (56 ULg)
Full Text
Peer Reviewed
See detailGlobal capacity announcement of electrical distribution systems: A pragmatic approach
Cornélusse, Bertrand ULg; Vangulick, David; Glavic, Mevludin ULg et al

in Sustainable Energy, Grids and Networks (2015), 4

We propose a pragmatic procedure to facilitate the connection process of Distributed Generation (DG) with reference to the European regulatory framework where Distribution System Operators (DSOs) are ... [more ▼]

We propose a pragmatic procedure to facilitate the connection process of Distributed Generation (DG) with reference to the European regulatory framework where Distribution System Operators (DSOs) are, except in specific cases, not allowed to own their generation. The procedure is termed Global Capacity ANnouncement (GCAN) and is intended to compute the estimates of maximum generation connection amount at appropriate substations in a distribution system, to help generation connection decisions. The pragmatism of the proposed procedure stems from its reliance on the tools that are routinely used in distribution systems planning and operation, and their use such that the possibilities of network sterilization are avoided. The tools involved include: long-term load forecasting, long-term planning of network extension/reinforcement, network reconfiguration, and power flow. Network sterilizing substations are identified through repeated power flow computations. The proposed procedure is supported by results using an artificially created 5-bus test system, the IEEE 33-bus test system, and a part of real-life distribution system of ORES (a Belgian DSO serving a large portion of the Walloon region in Belgium). [less ▲]

Detailed reference viewed: 50 (23 ULg)
Full Text
See detailArabie saoudite et Etats-Unis: un divorce annoncé
Hermans, Michel ULg; Ernst, Damien ULg

Article for general public (2015)

Detailed reference viewed: 121 (11 ULg)
Full Text
Peer Reviewed
See detailBenchmarking for Bayesian Reinforcement Learning
Castronovo, Michaël ULg; Ernst, Damien ULg; Couëtoux, Adrien ULg et al

E-print/Working paper (2015)

In the Bayesian Reinforcement Learning (BRL) setting, agents try to maximise the col- lected rewards while interacting with their environment while using some prior knowledge that is accessed beforehand ... [more ▼]

In the Bayesian Reinforcement Learning (BRL) setting, agents try to maximise the col- lected rewards while interacting with their environment while using some prior knowledge that is accessed beforehand. Many BRL algorithms have already been proposed, but even though a few toy examples exist in the literature, there are still no extensive or rigorous benchmarks to compare them. The paper addresses this problem, and provides a new BRL comparison methodology along with the corresponding open source library. In this methodology, a comparison criterion that measures the performance of algorithms on large sets of Markov Decision Processes (MDPs) drawn from some probability distributions is defined. In order to enable the comparison of non-anytime algorithms, our methodology also includes a detailed analysis of the computation time requirement of each algorithm. Our library is released with all source code and documentation: it includes three test prob- lems, each of which has two different prior distributions, and seven state-of-the-art RL algorithms. Finally, our library is illustrated by comparing all the available algorithms and the results are discussed. [less ▲]

Detailed reference viewed: 235 (18 ULg)
Full Text
Peer Reviewed
See detailReinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity Market
Vandael, Stijn; Claessens, Bert; Ernst, Damien ULg et al

in IEEE Transactions on Smart Grid (2015), 6(4), 1795-1805

This paper addresses the problem of defining a day-ahead consumption plan for charging a fleet of electric vehicles (EVs), and following this plan during operation. A challenge herein is the beforehand ... [more ▼]

This paper addresses the problem of defining a day-ahead consumption plan for charging a fleet of electric vehicles (EVs), and following this plan during operation. A challenge herein is the beforehand unknown charging flexibility of EVs, which depends on numerous details about each EV (e.g., plug-in times, power limitations, battery size, power curve, etc.). To cope with this challenge, EV charging is controlled during opertion by a heuristic scheme, and the resulting charging behavior of the EV fleet is learned by using batch mode reinforcement learning. Based on this learned behavior, a cost-effective day-ahead consumption plan can be defined. In simulation experiments, our approach is benchmarked against a multistage stochastic programming solution, which uses an exact model of each EVs charging flexibility. Results show that our approach is able to find a day-ahead consumption plan with comparable quality to the benchmark solution, without requiring an exact day-ahead model of each EVs charging flexibility. [less ▲]

Detailed reference viewed: 58 (7 ULg)
Full Text
See detailAn economic case for transnational and international transmission
Ernst, Damien ULg

Speech/Talk (2015)

Detailed reference viewed: 75 (11 ULg)
Full Text
Peer Reviewed
See detailGraph matching for reconciling SCADA and GIS of a distribution network
Cornélusse, Bertrand ULg; Leroux, Amandine; Glavic, Mevludin ULg et al

in Proceedings of the International Conference on Electricity Distribution, CIRED 2015 (2015, June)

This article deals with the problem of automatically es- tablishing a correspondence between two databases popu- lated independently over the years by a distribution com- pany, for instance a SCADA system ... [more ▼]

This article deals with the problem of automatically es- tablishing a correspondence between two databases popu- lated independently over the years by a distribution com- pany, for instance a SCADA system and a geographical information system. This problem is abstracted as a graph matching problem, well known in the combinatorial op- timisation community. It is then casted as an integer quadratic program. An idea of achievable results on a real system is provided, and needs for approximation or decom- position algorithms are discussed. [less ▲]

Detailed reference viewed: 227 (15 ULg)
Full Text
Peer Reviewed
See detailA process to address electricity distribution sector challenges: the GREDOR project approach
Cornélusse, Bertrand ULg; Vangulick, David; Glavic, Mevludin ULg et al

in Proceedings of the International Conference on Electricity Distribution, CIRED 2015 (2015, June)

This paper presents a general process set in the GREDOR (French acronym for “Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables”) project to address the challenges in distribution ... [more ▼]

This paper presents a general process set in the GREDOR (French acronym for “Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables”) project to address the challenges in distribution systems posed by the integration of renewable generation, changing load patterns, and the changes in the electricity market sector. A use case describing interactions among different players that fits the process is also presented. A pseudo-dynamic approach to Global Capacity Announcement as a way to increase penetration of Renewable Energy Sources in a distribution system is elaborated in more details. [less ▲]

Detailed reference viewed: 176 (10 ULg)
Full Text
Peer Reviewed
See detailMacroscopic analysis of interaction models for the provision of flexibility in distribution systems
Mathieu, Sébastien ULg; Ernst, Damien ULg; Cornélusse, Bertrand ULg

in Proceedings of the International Conference on Electricity Distribution, CIRED 2015 (2015, June)

To ease the transition towards the future of distribution grid management, regulators must revise the current interaction model, that is, the set of rules guiding the interactions between all the parties ... [more ▼]

To ease the transition towards the future of distribution grid management, regulators must revise the current interaction model, that is, the set of rules guiding the interactions between all the parties of the system. Five interaction models are proposed, three of them considering active network management. This paper evaluates the economic efficiency of each model using macroscopic representation of the system, by opposition to more techniques requiring a complete picture of the system. The interaction models are simulated on the horizon 2015-2030. Results show that for the first five years all the models provide similar economic efficiency. For the remaining ten years, interaction models implementing active network management provide up to a 10% higher economic efficiency. [less ▲]

Detailed reference viewed: 146 (22 ULg)
Full Text
See detailThe global grid for empowering renewable energy
Ernst, Damien ULg

Speech/Talk (2015)

Detailed reference viewed: 48 (1 ULg)