References of "Fonteneau, Raphaël"
     in
Bookmark and Share    
Full Text
See detailA Simulator to Explore Tarification Designs for Distribution Networks
Manuel de Villena Millan, Miguel ULiege; Gautier, Axel ULiege; Fonteneau, Raphaël ULiege et al

E-print/Working paper (2017)

This paper introduces a computational tool to help assessing the impact of regulation policies within distribution networks in the deployment of distributed renewable electricity generation. This tool is ... [more ▼]

This paper introduces a computational tool to help assessing the impact of regulation policies within distribution networks in the deployment of distributed renewable electricity generation. This tool is a comprehensive multi-agent simulator capable of handling the interaction between the users of a distribution network and their distribution system operator. With this simulator, it is possible to address the different regulatory constraints encountered by distribution system operators, for any regulation policy. In the simulator, we model individual electricity consumers as rational agents, that may invest in optimised distributed renewable energy installations, if they are cost-efficient compared to the network tariff. By modelling the cost recovery scheme of the distribution system operator, the simulator then computes the evolution of the network tariff in response to a change of the consumption and generation of the consumers in the distribution network, due to the deployment of distributed generation. The simulator is illustrated with various regulation policies. [less ▲]

Detailed reference viewed: 74 (6 ULiège)
Full Text
Peer Reviewed
See detailOn overfitting and asymptotic bias in batch reinforcement learning with partial observability
François-Lavet, Vincent ULiege; Ernst, Damien ULiege; Fonteneau, Raphaël ULiege

E-print/Working paper (2017)

This paper stands in the context of reinforcement learning with partial observability and limited data. In this setting, we focus on the tradeoff between asymptotic bias (suboptimality with unlimited data ... [more ▼]

This paper stands in the context of reinforcement learning with partial observability and limited data. In this setting, we focus on the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data), and theoretically show that while potentially increasing the asymptotic bias, a smaller state representation decreases the risk of overfitting. Our analysis relies on expressing the quality of a state representation by bounding L1 error terms of the associated belief states. Theoretical results are empirically illustrated when the state representation is a truncated history of observations. Finally, we also discuss and empirically illustrate how using function approximators and adapting the discount factor may enhance the tradeoff between asymptotic bias and overfitting. [less ▲]

Detailed reference viewed: 88 (7 ULiège)
Full Text
Peer Reviewed
See detailForeseeing New Control Challenges in Electricity Prosumer Communities
Olivier, Frédéric ULiege; Marulli, Daniele; Ernst, Damien ULiege et al

in Proc. of the 10th Bulk Power Systems Dynamics and Control Symposium – IREP’2017 (2017, August)

This paper is dedicated to electricity prosumer communities, which are groups of people producing, sharing and consuming electricity locally. This paper focuses on building a rigorous mathematical ... [more ▼]

This paper is dedicated to electricity prosumer communities, which are groups of people producing, sharing and consuming electricity locally. This paper focuses on building a rigorous mathematical framework in order to formalise sequen- tial decision making problems that may soon be encountered within electricity prosumer communities. After introducing our formalism, we propose a set of optimisation problems reflecting several types of theoretically optimal behaviours for energy exchanges between prosumers. We then discuss the advantages and disadvantages of centralised and decentralised schemes and provide illustrations of decision making strategies, allowing a prosumer community to generate more distributed electricity (compared to commonly applied strategies) by mitigating over- voltages over a low-voltage feeder. We finally investigate how to design distributed control schemes that may contribute reaching (at least partially) the objectives of the community, by resort in to machine learning techniques to extract, from centralised solution(s), decision making patterns to be applied locally. First empirical results show that, even after a post-processing phase so that it satisfies physical constraints, the learning approach still performs better than predetermined strategies targeting safety first, then cost minimisation. [less ▲]

Detailed reference viewed: 699 (31 ULiège)
Full Text
Peer Reviewed
See detailReinforcement Learning for Electric Power System Decision and Control: Past Considerations and Perspectives
Glavic, Mevludin ULiege; Fonteneau, Raphaël ULiege; Ernst, Damien ULiege

in The 20th World Congress of the International Federation of Automatic Control, Toulouse 9-14 July 2017 (2017, July)

In this paper, we review past (including very recent) research considerations in using reinforcement learning (RL) to solve electric power system decision and control problems. The RL considerations are ... [more ▼]

In this paper, we review past (including very recent) research considerations in using reinforcement learning (RL) to solve electric power system decision and control problems. The RL considerations are reviewed in terms of speci c electric power system problems, type of control and RL method used. We also provide observations about past considerations based on a comprehensive review of available publications. The review reveals the RL is considered as viable solutions to many decision and control problems across di erent time scales and electric power system states. Furthermore, we analyse the perspectives of RL approaches in light of the emergence of new-generation, communications, and instrumentation technologies currently in use, or available for future use, in power systems. The perspectives are also analysed in terms of recent breakthroughs in RL algorithms (Safe RL, Deep RL and path integral control for RL) and other, not previously considered, problems for RL considerations (most notably restorative, emergency controls together with so-called system integrity protection schemes, fusion with existing robust controls, and combining preventive and emergency control). [less ▲]

Detailed reference viewed: 807 (29 ULiège)
Full Text
Peer Reviewed
See detailAutomatic phase identification of smart meter measurement data
Olivier, Frédéric ULiege; Ernst, Damien ULiege; Fonteneau, Raphaël ULiege

in Proc. of CIRED 2017 (2017, June)

This paper highlights the importance of the knowledge of the phase identification for the different measurement points inside a low-voltage distribution network. Besides considering existing solutions, we ... [more ▼]

This paper highlights the importance of the knowledge of the phase identification for the different measurement points inside a low-voltage distribution network. Besides considering existing solutions, we propose a novel method for identifying the phases of the measurement devices, based exclusively on voltage measurement correlation. It relies on graph theory and the notion of maximum spanning tree. It has been tested on a real Belgian LV network, first with simulated unbalanced voltage for which it managed to correctly identify the phases of all measurement points, second, on preliminary data from a real measurement campaign for which it shows encouraging results. [less ▲]

Detailed reference viewed: 607 (44 ULiège)
Full Text
Peer Reviewed
See detailApproximate Bayes Optimal Policy Search using Neural Networks
Castronovo, Michaël ULiege; François-Lavet, Vincent ULiege; Fonteneau, Raphaël ULiege et al

in Proceedings of the 9th International Conference on Agents and Artificial Intelligence (ICAART 2017) (2017, February)

Bayesian Reinforcement Learning (BRL) agents aim to maximise the expected collected rewards obtained when interacting with an unknown Markov Decision Process (MDP) while using some prior knowledge. State ... [more ▼]

Bayesian Reinforcement Learning (BRL) agents aim to maximise the expected collected rewards obtained when interacting with an unknown Markov Decision Process (MDP) while using some prior knowledge. State-of-the-art BRL agents rely on frequent updates of the belief on the MDP, as new observations of the environment are made. This offers theoretical guarantees to converge to an optimum, but is computationally intractable, even on small-scale problems. In this paper, we present a method that circumvents this issue by training a parametric policy able to recommend an action directly from raw observations. Artificial Neural Networks (ANNs) are used to represent this policy, and are trained on the trajectories sampled from the prior. The trained model is then used online, and is able to act on the real MDP at a very low computational cost. Our new algorithm shows strong empirical performance, on a wide range of test problems, and is robust to inaccuracies of the prior distribution. [less ▲]

Detailed reference viewed: 587 (35 ULiège)
Full Text
Peer Reviewed
See detailAn App-based Algorithmic Approach for Harvesting Local and Renewable Energy Using Electric Vehicles
Dubois, Antoine; Wehenkel, Antoine; Fonteneau, Raphaël ULiege et al

in Proceedings of the 9th International Conference on Agents and Artificial Intelligence (ICAART 2017) (2017, February)

The emergence of electric vehicles (EVs), combined with the rise of renewable energy production capacities, will strongly impact the way electricity is produced, distributed and consumed in the very near ... [more ▼]

The emergence of electric vehicles (EVs), combined with the rise of renewable energy production capacities, will strongly impact the way electricity is produced, distributed and consumed in the very near future. This position paper focuses on the problem of optimizing charging strategies for a fleet of EVs in the context where a significant amount of electricity is generated by (distributed) renewable energy. It exposes how a mobile application may offer an efficient solution for addressing this problem. This app can play two main roles. Firstly, it would incite and help people to play a more active role in the energy sector by allowing photovoltaic (PV) panel owners to sell their electrical production directly to consumers, here the EVs’ agents. Secondly, it would help distribution system operators (DSOs) or transmission system operators (TSOs) to modulate more efficiently the load by allowing them to influence EV charging behaviour in real time. Finally, the present paper advocates for the introduction of a two-sided market-type model between EV drivers and electricity producers. [less ▲]

Detailed reference viewed: 920 (123 ULiège)
Full Text
Peer Reviewed
See detailOn the Dynamics of the Deployment of Renewable Energy Production Capacities
Fonteneau, Raphaël ULiege; Ernst, Damien ULiege

in Furze, James N.; Swing, Kelly; Gupta, Anil K. (Eds.) et al Mathematical Advances Towards Sustainable Environmental Systems (2017)

This chapter falls within the context of modeling the deployment of renewable en-ergy production capacities in the scope of the energy transition. This problem is addressed from an energy point of view, i ... [more ▼]

This chapter falls within the context of modeling the deployment of renewable en-ergy production capacities in the scope of the energy transition. This problem is addressed from an energy point of view, i.e. the deployment of technologies is seen as an energy investment under the constraint that an initial budget of non-renewable energy is provided. Using the Energy Return on Energy Investment (ERoEI) characteristics of technologies, we propose MODERN, a discrete-time formalization of the deployment of renewable energy production capacities. Be-sides showing the influence of the ERoEI parameter, the model also underlines the potential benefits of designing control strategies for optimizing the deployment of production capacities, and the necessity to increase energy efficiency. [less ▲]

Detailed reference viewed: 268 (17 ULiège)
Full Text
Peer Reviewed
See detailDeep Reinforcement Learning Solutions for Energy Microgrids Management
François-Lavet, Vincent ULiege; Taralla, David; Ernst, Damien ULiege et al

in European Workshop on Reinforcement Learning (EWRL 2016) (2016, December)

This paper addresses the problem of efficiently operating the storage devices in an electricity microgrid featuring photovoltaic (PV) panels with both short- and long-term storage capacities. The problem ... [more ▼]

This paper addresses the problem of efficiently operating the storage devices in an electricity microgrid featuring photovoltaic (PV) panels with both short- and long-term storage capacities. The problem of optimally activating the storage devices is formulated as a sequential decision making problem under uncertainty where, at every time-step, the uncertainty comes from the lack of knowledge about future electricity consumption and weather dependent PV production. This paper proposes to address this problem using deep reinforcement learning. To this purpose, a specific deep learning architecture has been designed in order to extract knowledge from past consumption and production time series as well as any available forecasts. The approach is empirically illustrated in the case of a residential customer located in Belgium. [less ▲]

Detailed reference viewed: 562 (34 ULiège)
Full Text
Peer Reviewed
See detailBenchmarking for Bayesian Reinforcement Learning
Castronovo, Michaël ULiege; Ernst, Damien ULiege; Couëtoux, Adrien ULiege et al

in PLoS ONE (2016)

In the Bayesian Reinforcement Learning (BRL) setting, agents try to maximise the col- lected rewards while interacting with their environment while using some prior knowledge that is accessed beforehand ... [more ▼]

In the Bayesian Reinforcement Learning (BRL) setting, agents try to maximise the col- lected rewards while interacting with their environment while using some prior knowledge that is accessed beforehand. Many BRL algorithms have already been proposed, but even though a few toy examples exist in the literature, there are still no extensive or rigorous benchmarks to compare them. The paper addresses this problem, and provides a new BRL comparison methodology along with the corresponding open source library. In this methodology, a comparison criterion that measures the performance of algorithms on large sets of Markov Decision Processes (MDPs) drawn from some probability distributions is defined. In order to enable the comparison of non-anytime algorithms, our methodology also includes a detailed analysis of the computation time requirement of each algorithm. Our library is released with all source code and documentation: it includes three test prob- lems, each of which has two different prior distributions, and seven state-of-the-art RL algorithms. Finally, our library is illustrated by comparing all the available algorithms and the results are discussed. [less ▲]

Detailed reference viewed: 605 (52 ULiège)
Full Text
Peer Reviewed
See detailA Gaussian mixture approach to model stochastic processes in power systems
Gemine, Quentin ULiege; Cornélusse, Bertrand ULiege; Glavic, Mevludin ULiege et al

in Proceedings of the 19th Power Systems Computation Conference (PSCC'16) (2016, June)

Probabilistic methods are emerging for operating electrical networks, driven by the integration of renewable generation. We present an algorithm that models a stochastic process as a Markov process using ... [more ▼]

Probabilistic methods are emerging for operating electrical networks, driven by the integration of renewable generation. We present an algorithm that models a stochastic process as a Markov process using a multivariate Gaussian Mixture Model, as well as a model selection technique to search for the adequate Markov order and number of components. The main motivation is to sample future trajectories of these processes from their last available observations (i.e. measurements). An accurate model that can generate these synthetic trajectories is critical for applications such as security analysis or decision making based on lookahead models. The proposed approach is evaluated in a lookahead security analysis framework, i.e. by estimating the probability of future system states to respect operational constraints. The evaluation is performed using a 33-bus distribution test system, for power consumption and wind speed processes. Empirical results show that the GMM approach slightly outperforms an ARMA approach. [less ▲]

Detailed reference viewed: 191 (46 ULiège)
Full Text
Peer Reviewed
See detailTowards the Minimization of the Levelized Energy Costs of Microgrids using both Long-term and Short-term Storage Devices
François-Lavet, Vincent ULiege; Gemine, Quentin ULiege; Ernst, Damien ULiege et al

in Smart Grid: Networking, Data Management, and Business Models (2016)

This chapter falls within the context of the optimization of the levelized energy cost (LEC) of microgrids featuring photovoltaic panels (PV) associated with both long-term (hydrogen) and short-term ... [more ▼]

This chapter falls within the context of the optimization of the levelized energy cost (LEC) of microgrids featuring photovoltaic panels (PV) associated with both long-term (hydrogen) and short-term (batteries) storage devices. First, we propose a novel formalization of the problem of building and operating microgrids interacting with their surrounding environment. Then we show how to optimally operate a microgrid using linear programming techniques in the context where the consumption and the production are known. It appears that this optimization technique can also be used to address the problem of optimal sizing of the microgrid, for which we propose a robust approach. These contributions are illustrated in two different settings corresponding to Belgian and Spanish data. [less ▲]

Detailed reference viewed: 261 (79 ULiège)
Full Text
See detailArtificial Intelligence and Energy
Cornélusse, Bertrand ULiege; Fonteneau, Raphaël ULiege

Conference (2016, February 02)

Detailed reference viewed: 147 (9 ULiège)
Full Text
Peer Reviewed
See detailDecision Making from Confidence Measurement on the Reward Growth using Supervised Learning: A Study Intended for Large-Scale Video Games
Taralla, David ULiege; Qiu, Zixiao ULiege; Sutera, Antonio ULiege et al

in Proceedings of the 8th International Conference on Agents and Artificial Intelligence (ICAART 2016) - Volume 2 (2016, February)

Video games have become more and more complex over the past decades. Today, players wander in visually and option- rich environments, and each choice they make, at any given time, can have a combinatorial ... [more ▼]

Video games have become more and more complex over the past decades. Today, players wander in visually and option- rich environments, and each choice they make, at any given time, can have a combinatorial number of consequences. However, modern artificial intelligence is still usually hard-coded, and as the game environments become increasingly complex, this hard-coding becomes exponentially difficult. Recent research works started to let video game autonomous agents learn instead of being taught, which makes them more intelligent. This contribution falls under this very perspective, as it aims to develop a framework for the generic design of autonomous agents for large-scale video games. We consider a class of games for which expert knowledge is available to define a state quality function that gives how close an agent is from its objective. The decision making policy is based on a confidence measurement on the growth of the state quality function, computed by a supervised learning classification model. Additionally, no stratagems aiming to reduce the action space are used. As a proof of concept, we tested this simple approach on the collectible card game Hearthstone and obtained encouraging results. [less ▲]

Detailed reference viewed: 551 (27 ULiège)
Full Text
Peer Reviewed
See detailImitative Learning for Online Planning in Microgrids
Aittahar, Samy ULiege; François-Lavet, Vincent ULiege; Lodeweyckx, Stefan et al

in Woon, Wei Lee; Zeyar, Aung; Stuart, Madnick (Eds.) Data Analytics for Renewable Energy Integration (2015, December 15)

This paper aims to design an algorithm dedicated to operational planning for microgrids in the challenging case where the scenarios of production and consumption are not known in advance. Using expert ... [more ▼]

This paper aims to design an algorithm dedicated to operational planning for microgrids in the challenging case where the scenarios of production and consumption are not known in advance. Using expert knowledge obtained from solving a family of linear programs, we build a learning set for training a decision-making agent. The empirical performances in terms of Levelized Energy Cost (LEC) of the obtained agent are compared to the expert performances obtained in the case where the scenarios are known in advance. Preliminary results are promising. [less ▲]

Detailed reference viewed: 156 (54 ULiège)
Full Text
Peer Reviewed
See detailHow to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies
François-Lavet, Vincent ULiege; Fonteneau, Raphaël ULiege; Ernst, Damien ULiege

in NIPS 2015 Workshop on Deep Reinforcement Learning (2015, December)

Using deep neural nets as function approximator for reinforcement learning tasks have recently been shown to be very powerful for solving problems approaching real-world complexity. Using these results as ... [more ▼]

Using deep neural nets as function approximator for reinforcement learning tasks have recently been shown to be very powerful for solving problems approaching real-world complexity. Using these results as a benchmark, we discuss the role that the discount factor may play in the quality of the learning process of a deep Q-network (DQN). When the discount factor progressively increases up to its final value, we empirically show that it is possible to significantly reduce the number of learning steps. When used in conjunction with a varying learning rate, we empirically show that it outperforms original DQN on several experiments. We relate this phenomenon with the instabilities of neural networks when they are used in an approximate Dynamic Programming setting. We also describe the possibility to fall within a local optimum during the learning process, thus connecting our discussion with the exploration/exploitation dilemma. [less ▲]

Detailed reference viewed: 159 (46 ULiège)
Full Text
See detailUne histoire d'énergie: équations et transition
Fonteneau, Raphaël ULiege

Speech/Talk (2015)

Detailed reference viewed: 30 (2 ULiège)
Full Text
Peer Reviewed
See detailArtificial Intelligence in Video Games: Towards a Unified Framework
Safadi, Firas ULiege; Fonteneau, Raphaël ULiege; Ernst, Damien ULiege

in International Journal of Computer Games Technology (2015), 2015

With modern video games frequently featuring sophisticated and realistic environments, the need for smart and comprehensive agents that understand the various aspects of complex environments is pressing ... [more ▼]

With modern video games frequently featuring sophisticated and realistic environments, the need for smart and comprehensive agents that understand the various aspects of complex environments is pressing. Since video game AI is often specifically designed for each game, video game AI tools currently focus on allowing video game developers to quickly and efficiently create specific AI. One issue with this approach is that it does not efficiently exploit the numerous similarities that exist between video games not only of the same genre, but of different genres too, resulting in a difficulty to handle the many aspects of a complex environment independently for each video game. Inspired by the human ability to detect analogies between games and apply similar behavior on a conceptual level, this paper suggests an approach based on the use of a unified conceptual framework to enable the development of conceptual AI which relies on conceptual views and actions to define basic yet reasonable and robust behavior. The approach is illustrated using two video games, Raven and StarCraft: Brood War. [less ▲]

Detailed reference viewed: 527 (39 ULiège)