Sequential decision-making approach for quadrangular mesh generation Johnen, Amaury ; Ernst, Damien ; Geuzaine, Christophe in Engineering with Computers (in press) A new indirect quadrangular mesh generation algorithm which relies on sequential decision-making techniques to search for optimal triangle recombinations is presented. In contrast to the state-of-art ... [more ▼] A new indirect quadrangular mesh generation algorithm which relies on sequential decision-making techniques to search for optimal triangle recombinations is presented. In contrast to the state-of-art Blossom-quad algorithm, this new algorithm is a good candidate for addressing the 3D problem of recombining tetrahedra into hexahedra. [less ▲] Detailed reference viewed: 144 (17 ULg)Blackouts: des vérités qui dérangent Ernst, Damien Speech (2014) Detailed reference viewed: 794 (121 ULg)Mathematical Modeling of HIV Dynamics After Antiretroviral Therapy Initiation: A Review ; ; et al in BioResearch Open Acces (2014), 3(5), 233-241 This review shows the potential ground-breaking impact that mathematical tools may have in the analysis and the understanding of the HIV dynamics. In the first part, early diagnosis of immunological ... [more ▼] This review shows the potential ground-breaking impact that mathematical tools may have in the analysis and the understanding of the HIV dynamics. In the first part, early diagnosis of immunological failure is inferred from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This method is supported by clinical research results from an original clinical trial: data just after 1 month following therapy initiation are used to carry out the model identification. The diagnosis is shown to be consistent with results from monitoring of the patients after 6 months. In the second part of this review, prospective research results are given for the design of individual anti-HIV treatments optimizing the recovery of the immune system and minimizing side effects. In this respect, two methods are discussed. The first one combines HIV population dynamics with pharmacokinetics and pharmacodynamics models to generate drug treatments using impulsive control systems. The second one is based on optimal control theory and uses a recently published differential equation to model the side effects produced by highly active antiretroviral therapy therapies. The main advantage of these revisited methods is that the drug treatment is computed directly in amounts of drugs, which is easier to interpret by physicians and patients. [less ▲] Detailed reference viewed: 16 (4 ULg)Mathematical Modeling of HIV Dynamics After Antiretroviral Therapy Initiation: A Review ; ; et al in BioResearch Open Acces (2014), 3(5), 233-241 This review shows the potential ground-breaking impact that mathematical tools may have in the analysis and the understanding of the HIV dynamics. In the first part, early diagnosis of immunological ... [more ▼] This review shows the potential ground-breaking impact that mathematical tools may have in the analysis and the understanding of the HIV dynamics. In the first part, early diagnosis of immunological failure is inferred from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This method is supported by clinical research results from an original clinical trial: data just after 1 month following therapy initiation are used to carry out the model identification. The diagnosis is shown to be consistent with results from monitoring of the patients after 6 months. In the second part of this review, prospective research results are given for the design of individual anti-HIV treatments optimizing the recovery of the immune system and minimizing side effects. In this respect, two methods are discussed. The first one combines HIV population dynamics with pharmacokinetics and pharmacodynamics models to generate drug treatments using impulsive control systems. The second one is based on optimal control theory and uses a recently published differential equation to model the side effects produced by highly active antiretroviral therapy therapies. The main advantage of these revisited methods is that the drug treatment is computed directly in amounts of drugs, which is easier to interpret by physicians and patients. [less ▲] Detailed reference viewed: 16 (4 ULg)Optimal Assignment of Off-Peak Hours to Lower Curtailments in the Distribution Network Merciadri, Luca ; Mathieu, Sébastien ; Ernst, Damien et al in Proceedings of the 5th European Innovative Smart Grid Technologies (ISGT) (2014, October) We consider a price signal with two settings: off-peak tariff and on-peak tariff. Some loads are connected to specific electricity meters which allow the consumption of power only in off-peak periods ... [more ▼] We consider a price signal with two settings: off-peak tariff and on-peak tariff. Some loads are connected to specific electricity meters which allow the consumption of power only in off-peak periods. Historically, off-peak periods were located during the night and on-peak periods during the day. Changing the assignment of off-peak periods is an easy method for distribution system operators to access to the flexibility of small consumers. This solution can be implemented quickly as the infrastructure needed already exists in some countries. We propose a mixed-integer linear model to assign optimally the off-peak hours so as to minimize a societal cost. This cost gathers together the cost of electricity, the financial losses due to energy curtailments of photovoltaic installations and the loads' wellbeing. Our model considers automatic tripping of inverters and constraints of the electrical distribution networks. Simulation results show that the new disposition of off-peak hours could reduce significantly the photovoltaic energy curtailed in the summer. [less ▲] Detailed reference viewed: 54 (14 ULg)Mathematical modeling of HIV dynamics after antiretroviral therapy initiation: A clinical research study ; ; et al in AIDS Research and Human Retroviruses (2014), 30(9), 831-834 Immunological failure is identified from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This identification is supported by clinical research results from an ... [more ▼] Immunological failure is identified from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This identification is supported by clinical research results from an original clinical trial. Standard clinical data were collected from infected patients starting Highly Active Anti-Retroviral Therapy (HAART), just after one month following therapy initiation and were used to carry out the model identification. The early diagnosis is shown to be consistent with the patients monitoring after six months. [less ▲] Detailed reference viewed: 28 (3 ULg)Global power grids for harnessing world renewable energy ; Ernst, Damien ; in Jones, Lawrence (Ed.) Renewable Energy Integration: Practical Management of Variability, Uncertainty and Flexibility in Power Grids (2014) The Global Grid advocates the connection of all regional power systems into one electricity transmission system spanning the whole globe. Power systems are currently forming larger and larger ... [more ▼] The Global Grid advocates the connection of all regional power systems into one electricity transmission system spanning the whole globe. Power systems are currently forming larger and larger interconnections. Environmental awareness and increased electricity consumption leads more investments towards renewable energy sources, abundant in remote locations (oﬀ-shore or in deserts). The Global Grid will facilitate the transmission of this “green” electricity to load centers, serving as backbone. This chapter elaborates on the concept presenting four stages that could gradually lead to the development of a globally interconnected power network. Quantitative analyses are carried out for all stages, demonstrating that a Global Grid is both technically feasible and economically competitive. Real price data from Europe and the USA are used to identify the potential of intercontinental electricity trade, showing that substantial proﬁts can be generated through such interconnections. [less ▲] Detailed reference viewed: 67 (10 ULg)A quantitative analysis of the effect of flexible loads on reserve markets Mathieu, Sébastien ; Louveaux, Quentin ; Ernst, Damien et al in Proceedings of the 18th Power Systems Computation Conference (PSCC) (2014, August) We propose and analyze a day-ahead reserve market model that handles bids from flexible loads. This pool market model takes into account the fact that a load modulation in one direction must usually be ... [more ▼] We propose and analyze a day-ahead reserve market model that handles bids from flexible loads. This pool market model takes into account the fact that a load modulation in one direction must usually be compensated later by a modulation of the same magnitude in the opposite direction. Our analysis takes into account the gaming possibilities of producers and retailers, controlling load flexibility, in the day-ahead energy and reserve markets, and in imbalance settlement. This analysis is carried out by an agent-based approach where, for every round, each actor uses linear programs to maximize its profit according to forecasts of the prices. The procurement of a reserve is assumed to be determined, for each period, as a fixed percentage of the total consumption cleared in the energy market for the same period. The results show that the provision of reserves by flexible loads has a negligible impact on the energy market prices but markedly decreases the cost of reserve procurement. However, as the rate of flexible loads increases, the system operator has to rely more and more on non-contracted reserves, which may cancel out the benefits made in the procurement of reserves. [less ▲] Detailed reference viewed: 148 (28 ULg)Relaxations for multi-period optimal power flow problems with discrete decision variables Gemine, Quentin ; Ernst, Damien ; Louveaux, Quentin et al in Proceedings of the 18th Power Systems Computation Conference (PSCC'14) (2014, August) We consider a class of optimal power flow (OPF) applications where some loads offer a modulation service in exchange for an activation fee. These applications can be modeled as multi-period formulations ... [more ▼] We consider a class of optimal power flow (OPF) applications where some loads offer a modulation service in exchange for an activation fee. These applications can be modeled as multi-period formulations of the OPF with discrete variables that define mixed-integer non-convex mathematical programs. We propose two types of relaxations to tackle these problems. One is based on a Lagrangian relaxation and the other is based on a network flow relaxation. Both relaxations are tested on several benchmarks and, although they provide a comparable dual bound, it appears that the constraints in the solutions derived from the network flow relaxation are significantly less violated. [less ▲] Detailed reference viewed: 140 (33 ULg)Simple connectome inference from partial correlation statistics in calcium imaging Sutera, Antonio ; Joly, Arnaud ; François-Lavet, Vincent et al in Proceedings of Connectomics 2014 (ECML 2014) (2014, June) In this work, we propose a simple yet effective solution to the problem of connectome inference in calcium imaging data. The proposed algorithm consists of two steps. First, processing the raw signals to ... [more ▼] In this work, we propose a simple yet effective solution to the problem of connectome inference in calcium imaging data. The proposed algorithm consists of two steps. First, processing the raw signals to detect neural peak activities. Second, inferring the degree of association between neurons from partial correlation statistics. This paper summarises the methodology that led us to win the Connectomics Challenge, proposes a simplified version of our method, and finally compares our results with respect to other inference methods. [less ▲] Detailed reference viewed: 369 (113 ULg)Distributed Model-free Control of Photovoltaic Units for Mitigating Overvoltages in Low-Voltage Networks Aristidou, Petros ; Olivier, Frédéric ; et al in Proc. of CIRED 2014 workshop (2014, June) In this paper, a distributed model-free control scheme to mitigate overvoltage problems caused by high photovoltaic generation in low-voltage feeders is proposed. The distributed controllers are ... [more ▼] In this paper, a distributed model-free control scheme to mitigate overvoltage problems caused by high photovoltaic generation in low-voltage feeders is proposed. The distributed controllers are implemented on the photovoltaic inverters and modulate the active and reactive power injected into the network. In particular, they direct photovoltaic units first to consume reactive power and, if necessary, curtail active power generation to reduce high voltages in the feeder. [less ▲] Detailed reference viewed: 87 (30 ULg)Bayes Adaptive Reinforcement Learning versus Off-line Prior-based Policy Search: an Empirical Comparison Castronovo, Michaël ; Ernst, Damien ; Fonteneau, Raphaël in Proceedings of the 23rd annual machine learning conference of Belgium and the Netherlands (BENELEARN 2014) (2014, June) This paper addresses the problem of decision making in unknown finite Markov decision processes (MDPs). The uncertainty about the MDPs is modeled using a prior distribution over a set of candidate MDPs ... [more ▼] This paper addresses the problem of decision making in unknown finite Markov decision processes (MDPs). The uncertainty about the MDPs is modeled using a prior distribution over a set of candidate MDPs. The performance criterion is the expected sum of discounted rewards collected over an infinite length trajectory. Time constraints are defined as follows: (i) an off-line phase with a given time budget can be used to exploit the prior distribution and (ii) at every time step of the on-line phase, decisions have to be computed within a given time budget. In this setting, we compare two decision-making strategies: OPPS, a recently proposed meta-learning scheme which mainly exploits the off-line phase to perform policy search and BAMCP, a state-of-the-art model-based Bayesian reinforcement learning algorithm, which mainly exploits the on-line time budget. We empirically compare these approaches in a real Bayesian setting by computing their performances over a large set of problems. To the best of our knowledge, it is the first time that this is done in the reinforcement learning literature. Several settings are considered by varying the prior distribution and the distribution from which test problems are drawn. The main finding of these experiments is that there may be a significant benefit of having an off-line prior-based optimization phase in the case of informative and accurate priors, especially when on-line time constraints are tight. [less ▲] Detailed reference viewed: 131 (55 ULg)Microgrids and their destructuring effects on the electrical industry Ernst, Damien Speech (2014) Detailed reference viewed: 38 (8 ULg)Apprentissage par renforcement bayésien versus recherche directe de politique hors-ligne en utilisant une distribution a priori: comparaison empirique Castronovo, Michaël ; Ernst, Damien ; Fonteneau, Raphaël in Proceedings des 9èmes Journée Francophones de Planification, Décision et Apprentissage (2014, May) Cet article aborde le problème de prise de décision séquentielle dans des processus de déci- sion de Markov (MDPs) finis et inconnus. L’absence de connaissance sur le MDP est modélisée sous la forme ... [more ▼] Cet article aborde le problème de prise de décision séquentielle dans des processus de déci- sion de Markov (MDPs) finis et inconnus. L’absence de connaissance sur le MDP est modélisée sous la forme d’une distribution de probabilité sur un ensemble de MDPs candidats connue a priori. Le cri- tère de performance utilisé est l’espérance de la somme des récompenses actualisées sur une trajectoire infinie. En parallèle du critère d’optimalité, les contraintes liées au temps de calcul sont formalisées rigoureusement. Tout d’abord, une phase « hors-ligne » précédant l’interaction avec le MDP inconnu offre à l’agent la possibilité d’exploiter la distribution a priori pendant un temps limité. Ensuite, durant la phase d’interaction avec le MDP, à chaque pas de temps, l’agent doit prendre une décision dans un laps de temps contraint déterminé. Dans ce contexte, nous comparons deux stratégies de prise de déci- sion : OPPS, une approche récente exploitant essentiellement la phase hors-ligne pour sélectionner une politique dans un ensemble de politiques candidates et BAMCP, une approche récente de planification en-ligne bayésienne. Nous comparons empiriquement ces approches dans un contexte bayésien, en ce sens que nous évaluons leurs performances sur un large ensemble de problèmes tirés selon une distribution de test. A notre connaissance, il s’agit des premiers tests expérimentaux de ce type en apprentissage par renforcement. Nous étudions plusieurs cas de figure en considérant diverses distributions pouvant être utilisées aussi bien en tant que distribution a priori qu’en tant que distribution de test. Les résultats obtenus suggèrent qu’exploiter une distribution a priori durant une phase d’optimisation hors-ligne est un avantage non- négligeable pour des distributions a priori précises et/ou contraintes à de petits budgets temps en-ligne. [less ▲] Detailed reference viewed: 48 (18 ULg)Gestion active d’un réseau de distribution d’électricité : formulation du problème et benchmark Gemine, Quentin ; Ernst, Damien ; Cornélusse, Bertrand in Proceedings des 9èmes Journées Francophones de Planification, Décision et Apprentissage (2014, May) Afin d’opérer un réseau de distribution d’électricité de manière fiable et efficace, c’est-à-dire de respecter les contraintes physiques tout en évitant des coûts de renforcement prohibitifs, il devient ... [more ▼] Afin d’opérer un réseau de distribution d’électricité de manière fiable et efficace, c’est-à-dire de respecter les contraintes physiques tout en évitant des coûts de renforcement prohibitifs, il devient nécessaire de recourir à des stratégies de gestion active du réseau. Ces stratégies, rendues nécessaires notamment par l’essor de la production distribuée, reposent sur des politiques de contrôle à court-terme du niveau de puissance des dispositifs producteurs ou consommateurs d’électricité. Alors qu’une solution simple consisterait à moduler à la baisse la production des générateurs, il paraît néan- moins plus intéressant de déplacer la consommation aux moments adéquats afin d’exploiter au mieux les sources d’énergie renouvelables sur lesquelles reposent généralement ces générateurs. Un tel moyen de contrôle introduit néanmoins un couplage temporel au problème, menant à un problème d’optimisation non-linéaire, séquentiel sous incertitude et à variables mixtes. Afin de favoriser la recherche dans ce domaine très complexe, nous proposons une formalisation générique du problème de ges- tion active d’un réseau de distribution moyenne tension (MT). Plus spécifiquement, cette formalisa- tion se présente sous la forme d’un processus de décision markovien. Dans cette article, nous pré- sentons également une spécification de ce modèle décisionnel à un réseau de 75 noeuds et pour un ensemble de services de modulation donnés. L’instance de test qui en résulte est disponible à l’adresse http://www.montefiore.ulg.ac.be/~anm/ et a pour objectif de mesurer et de comparer les performances des techniques de résolution qui seront développées. [less ▲] Detailed reference viewed: 144 (42 ULg)Toggling a genetic switch using reinforcement learning ; ; Ernst, Damien et al in Proceedings of the 9th French Meeting on Planning, Decision Making and Learning (2014, May) In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the ﬁtted Q ... [more ▼] In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the ﬁtted Q iteration. This algorithm infers the control law directly from the measurements of the system’s response to external control inputs without the use of a mathematical model of the system. The measurement data set can either be collected from wet-lab experiments or artiﬁcially created by computer simulations of dynamical models of the system. The algorithm is applicable to a wide range of biological systems due to its ability to deal with nonlinear and stochastic system dynamics. To illustrate the application of the algorithm to a gene regulatory network, the regulation of the toggle switch system is considered. The control objective of this problem is to drive the concentrations of two speciﬁc proteins to a target region in the state space. [less ▲] Detailed reference viewed: 27 (1 ULg)Toggling a genetic switch using reinforcement learning ; ; Ernst, Damien et al in Proceedings of the 9th French Meeting on Planning, Decision Making and Learning (2014, May) In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the ﬁtted Q ... [more ▼] In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the ﬁtted Q iteration. This algorithm infers the control law directly from the measurements of the system’s response to external control inputs without the use of a mathematical model of the system. The measurement data set can either be collected from wet-lab experiments or artiﬁcially created by computer simulations of dynamical models of the system. The algorithm is applicable to a wide range of biological systems due to its ability to deal with nonlinear and stochastic system dynamics. To illustrate the application of the algorithm to a gene regulatory network, the regulation of the toggle switch system is considered. The control objective of this problem is to drive the concentrations of two speciﬁc proteins to a target region in the state space. [less ▲] Detailed reference viewed: 27 (1 ULg)Toggling a genetic switch using reinforcement learning ; ; Ernst, Damien et al in Proceedings of the 9th French Meeting on Planning, Decision Making and Learning (2014, May) In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the ﬁtted Q ... [more ▼] In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting an established reinforcement learning algorithm called the ﬁtted Q iteration. This algorithm infers the control law directly from the measurements of the system’s response to external control inputs without the use of a mathematical model of the system. The measurement data set can either be collected from wet-lab experiments or artiﬁcially created by computer simulations of dynamical models of the system. The algorithm is applicable to a wide range of biological systems due to its ability to deal with nonlinear and stochastic system dynamics. To illustrate the application of the algorithm to a gene regulatory network, the regulation of the toggle switch system is considered. The control objective of this problem is to drive the concentrations of two speciﬁc proteins to a target region in the state space. [less ▲] Detailed reference viewed: 27 (1 ULg); ; Ernst, Damien et al in Proceedings of the 9th French Meeting on Planning, Decision Making and Learning (2014, May) Estimating the revenues of a hydrogen-based high-capacity storage device: methodology and results François-Lavet, Vincent ; Fonteneau, Raphaël ; Ernst, Damien in Proceedings des 9èmes Journée Francophones de Planification, Décision et Apprentissage (2014, May) This paper proposes a methodology to estimate the maximum revenue that can be generated by a company that operates a high-capacity storage device to buy or sell electricity on the day-ahead electricity ... [more ▼] This paper proposes a methodology to estimate the maximum revenue that can be generated by a company that operates a high-capacity storage device to buy or sell electricity on the day-ahead electricity market. The methodology exploits the Dynamic Programming (DP) principle and is specified for hydrogen-based storage devices that use electrolysis to produce hydrogen and fuel cells to generate electricity from hydrogen. Experimental results are generated using historical data of energy prices on the Belgian market. They show how the storage capacity and other parameters of the storage device influence the optimal revenue. The main conclusion drawn from the experiments is that it may be interesting to invest in large storage tanks to exploit the inter-seasonal price fluctuations of electricity. [less ▲] Detailed reference viewed: 52 (23 ULg) |
||