View source for Energy balance

[[op_fi:Energiatase]]
[[Category:Energy]]
[[Category:Energy balance]]
[[Category:Online model]]
[[Category:Contains R code]]
[[Category:Code under inspection]]
{{method|moderator=Jouni|stub=Yes}}

{{summary box | 
question = What is energy balance and how is it modelled? |
answer = 
Summing up the amount of energy produced and subtracting the amount of energy consumed within a time period gives the energy balance. Since the electricity grid and district heat network lack significant storage mechanics, the balance has to be virtually zero over short periods. When considering the balance of a particular area (e.g. Helsinki), we can make the assumption that electricity can be imported and exported in international markets. The energy in the district heat network, however, has to produced locally. This sets up the non-trivial problem of optimising production so that there are no significant deficits as well as minimising losses and maximising profits. This problem is solved (to some extent) by market forces in the real world. Our most recent energy balance model uses linear programming tools to solve an optimum for the activity of a given set of production units in simulated instances created by the main model. The main model is responsible for the decision making aspects, while the energy balance optimisation only functions as an approximation of real world market mechanics. 

The linear programming problem is set up as follows.
For each production unit: let x<sub>i</sub> be activity of the plant. Lets also have variables y<sub>j</sub> for deficits and excesses for each type of energy produced. 
The objective function is the function we are optimising. Each production unit has a unit profit per activity denoted by a<sub>i</sub> which is determined by the amount of different input commodities (e.g. coal) per amount of different output commodities (i.e. electricity and heat) and their market prices. Also, lets say we want to make sure that district heat demand is always met when possible and have a large penalty factor for each unit of heat demand not met (1M€ in the model). In addition, it must be noted that excess district heat becomes wasted so it counts as loss. Let these deficit and excess related losses be denoted by b<sub>j</sub>. The whole objective function then becomes: sum(x<sub>i</sub>a<sub>i</sub>) + sum(y<sub>j</sub>b<sub>j</sub>). 
The values of variables are constrained by equalities and inequalities: the sum of production of a commodity is equal to its demand minus deficit plus excess, activity is constrained by the maximum capacity and all variables are non-negative by definition.  
This can be efficiently solved by computers for each given instance. Production wind-up and wind-down is ignored, since time continuity is not considered. As a consequence fuel limits (e.g. diminishing hydropower capacity) are not modelled completely either.
}}
==Question==

How to calculate energy balances?

==Answer==

Press button to run the model. Get a free user account and edit the table below to change the model inputs. You may also copy the model to your own page for your own purposes. For examples, see e.g. [[Energy balance in Kuopio]] or [[Energy balance in Suzhou]].

<rcode name="answer" graphics="1">
library(OpasnetUtils)
library(ggplot2)

objects.latest("Op_en5141", code_name = "initiate")

N <- 10

balance <- Ovariable("balance", ddata = "Op_en5141.equations")

balance@data$Equation <- as.character(levels(balance@data$Equation)[balance@data$Equation]) # Change factor into character.
balance@data$Policy[balance@data$Policy == ""] <- NA # Prepare indices for fillna. This should be part of fillna.
balance@data$CHPcapacity[balance@data$CHPcapacity == ""] <- NA
balance@data <- fillna(balance@data, marginals = 1:2) # Fill empty slots in indices.

nonlinearity <- Ovariable("nonlinearity", ddata = "Op_en5141", subset = "Nonlinearity parameters")

directinput <- Ovariable("directinput", ddata = "Op_en5141", subset = "No modelled upstream variables")


eb <- EvalOutput(energy.balance) 

eb@marginal[colnames(eb@output) == "energy.balanceVar"] <- TRUE # This has to be done manually because CheckMarginals does not notice this as a marginal.
eb@marginal[colnames(eb@output) %in% nonlinearity@output$critIndex] <- FALSE # Nonlinear indices are demarginalised because only one of the two equations apply.

oprint(summary(eb))

ggplot(eb@output, aes_string(x = "energy.balanceVar", y = "energy.balanceResult", fill = "Policy")) + 
	geom_boxplot() +
	theme_grey(base_size = 24) + 
	theme(axis.text.x = element_text(angle = 90, hjust = 1))

</rcode>

==Rationale==

===Input===

; Example table for making matrices from text format equations. CHPcapacity describes which of the piecewise linear equations should be used. Policy is a decision option that alters the outcome. Dummy is only for compatibility but it is not used.

<t2b name="Equations" index="CHPcapacity,Policy,Equation" obs="Dummy" desc="Description" unit="GWh /a">
|Biofuel|CHP renewable = CHP peat|1|Biofuel policy contains half biofuels, half peat
|BAU|CHP renewable = 89.24|1|
||CHP peat + CHP renewable + CHP oil = CHP heat + CHP electricity|1|
||CHP peat = 90-98*CHP oil|1|
||CHP electricity = 0.689*CHP heat|1|
CHP<1000||H heat = 0.08*CHP heat|1|Small heat plants reflect the total heat need
CHP>1000||CHP heat + CHP electricity = 1000|1|But production capacity of CHP may be overwhelmed, decoupling CHP heat and H heat.
||H biogas + H oil = H heat|1|
||H oil = 18.973*H biogas|1|
||Bought electricity + CHP electricity = Cons electricity|1|
||CHP heat + H heat = Cons heat|1|
||Cons electricity = 900-1100|1|
||Cons heat = 900-1000|1|
</t2b>

; Example table to describe the details about nonlinear equations.

<t2b name="Nonlinearity parameters" index="critVar,critIndex,rescol,critLocLow,critLocHigh" obs="critValue" unit="GWh /a">
Cons heat|CHPcapacity|Result|CHP<1000|CHP>1000|1080
</t2b>


; This table is fetched if there are no nonlinearities. Therefore, there is no need to copy it to the case study page.

<t2b name="No nonlinearities" index="critVar,critIndex,rescol,critLocLow,critLocHigh" obs="critValue" unit="GWh /a">
|||||
</t2b>

; This table is fetched if there are no modelled upstream variables that would affect the equations.

<t2b name="No modelled upstream variables" index="energybalanceVars" obs="Result" unit="-">
|
</t2b>

===Output===

Output is an ovariable with column energy.balanceVar for the solved variables, a result column, and all other indices in the balance object used as input. This method has been used in several cases:
* [[Energy balance in Kuopio]]
* [[Energy balance in Basel]]
* [[Energy balance in Stuttgart]]
* [[Energy balance in Suzhou]]

===Data===

* Energy balances are described as input = output on a coarse level (called classes) where the structure is the same or similar to the OECD energy balance tables. If possible, this is described on the Energy balance method level and it is shared by all cities.
* On more detailed (variable level in the matrix), the fraction of each variable of the total class are described separately. Fractions are city specific and they are described on city level in a separate table.
* Based on the fraction table, detailed equations with variables are created. The format will be fraction * class total = variable.
* The last fraction has zero degrees of freedom when the class total is given. However, it must have a variable and thus a row in the fraction table. The result for that variable is an empty cell (which results in NA).
* Unlike in the previous version, all variables are given either as values or equations, and the user interface is not used for BAU. In contrast, user interface or decision table may be used to derive values for alternative scenarios.
* To make this work, the city-specific fraction data must be defined as ovariable (so that it can be changed with a decision table), and also the energy balance method must be described asa ovariable. How are we going to make the two interplay, as we may want to have several cities?
** Define one city ovariable and evaluate energy balance with that. The ovariable has a generic name. Then, define a new city ovariable with the same name and re-evaluate the energy balance ovariable; this must be done so that the two cities are appended rather than replaced.
** city ovariables are appended first into a large fraction table, and then that is used to create the large energy balance matrix. {{defend|# |This is clearly better.|--[[User:Jouni|Jouni]] 17:09, 21 February 2013 (EET)}}
* The city-specific ovariable may have Iter and other indices. A separate matrix is created and solved for each unique combination of indices. This makes it possible to have a very flexible approach.
* We should check if the energy balance matrix (see Matti's Excel) has city-specific equations. If possible, energy transformations are described as generic equations on the energy balance method.
* Structure of OECD Energy balance tables (data):
** Fuel (given as observation columns in OECD table)
** Activity (row in OECD table)
** Description
* Structure of the generic process table
** Equation, 
** Col, 
** Result, 
** Description? {{attack|# |This does not join up in a coherent way.|--[[User:Jouni|Jouni]] 17:09, 21 February 2013 (EET)}}
* Columns for fraction table
** Class
** Item
** Result (fraction)
** Indices as needed

''''Additional thoughts

Energy balance has these parts
* General table that describes balances and conversions from ole fuel to another. 
** Items and sums Are defined at generic level. 
* Data table that contains amounts from OECD table
* Fraction table thAt describes how detailSs Are derived from sums. 
* if there Are additions to sta dard items, they descrided at city level in an addition table. It must have the struxture of the matrix table in long format. 
* can nonlinearities ne handled simply by indices separating two linear parts of nonlinear equations?

====Old description====

{{attack|# |What are other energy sources,i wish it should be specific,how about air transport,does it included in the energy balance calculation?|--[[User:Sam0911|Sam0911]] 16:29, 10 February 2013 (EET)}}

Use a table where different fuel types are columns and different stocks, energy production processes, or consumption types are rows called ''Energy accounts'' (Ene.account for short). See also an example [[File:Energy supply in Europe.xls]]. All Ene.accounts and fuel types used are listed on [[Energy consumption classes]].

{| {{prettytable}}
|+ '''A part of a full energy balance sheet (ktoe/a).
|----
!
! Coal and peat
! Crude oil
! Petrochemical products
! Electricity
! Heat
|----
!colspan="6"|Production and supply of primary energy
|----
| Production || 129|| || || ||
|----
| Import || || 28|| 63|| || 
|----
| Export || || || || ||  
|----
!colspan="6"| Conversion of primary energy to use
|----
| Transfers || || || || ||
|----
| Electricity plants || || || || || 
|----
| CHP plants || -129|| -28|| || 61|| 96
|----
!colspan="6"|Final energy consumption
|----
| Industry || || || || || 
|----
| Road transport || || || 54|| || 
|----
| Heating || || || || 10|| 96
|----
| Other energy use || || || 9|| 51||
|}

Energy balance can be considered as double-entry bookkeeping. The production and conversion are typically credit (i.e., where the energy comes from), while consumption is debit (i.e., where the energy is used). In a typical case, both credit and debit are marked positive on the energy balance sheet. However, there are activities that convert one fuel to another, so that the energy is moved from one column to another. In this case, credit (the source) is marked negative and debit (the target) is marked positive. This is because the production and conversion rows together should reflect how much energy is actually available for final consumption. In other words, the sum of production and conversion rows should equal the sum of the consumption rows in every column.

Because production + conversion = consumption, also production = consumption - conversion. These equations are used to derive the supply that is needed to fulfil the demand.

=== Fuel use and fuel shares in generic processes ===

There is an alternative to energy for calculating fuel use. It is based on the idea that for each heating type, there is a constant share of fuels used. For some heating types, this is generic and is shown on this page. For some others, the constant is case-specific and is determined on a case-specific page.

The table below contains connections of heating types and fuel usage in generic situations. There may be case-specific differences, which must be handled separately.

<t2b name='Fuel use in different heating types' index='Heating,Burner,Fuel' obs='Fraction' desc='Description' unit='-'>
Wood|Household|Wood|1|
Oil|Household|Light oil|1|
Gas|Household|Gas|1|
Heating oil|Household|Light oil|1|
Coal|Household|Coal|1|
Other sources|Household|Other sources|1|
No energy source|Household|Other sources|1|
Geothermal|Grid|Electricity|0.3|Geothermal does not sum up to 1 because more heat is produced than electricity consumed.
Centrifuge, hydro-extractor|Grid|Electricity|0.3|Not quite clear what this is but presumably a heat pump.
Solar heater/ collector|Grid|Electricity|0.1|Use only; life-cycle impacts omitted.
Electricity|Grid|Electricity|1|
District|Undefined|Heat|1|
</t2b>
{{comment|# |District removed from fuelShares/Heating, but it is not clear why it was there in the first place. For completion? But for what?|--[[User:Jouni|Jouni]] ([[User talk:Jouni|talk]]) 13:41, 23 July 2015 (UTC)}}
:{{attack|# |Still necessary in model. It completes the description of flows related to different heating methods. In a descriptor of flows, "fuel" is a slight misnomer and should be reclassified as a "commodity" or similar throughout the model.|--[[User:Teemu R|Teemu R]] ([[User talk:Teemu R|talk]]) 12:23, 24 July 2015 (UTC)}}

<rcode name='fuelSharesgeneric' embed=1 label='Initiate fuelSharesgeneric (only for developers)'>

# This is code Op_en5141/fuelSharesgeneric (only generic) on page [[Energy balance]].
library(OpasnetUtils)

fuelSharesgeneric <- Ovariable("fuelSharesgeneric", ddata = "Op_en5141", subset = "Fuel use in different heating types") # [[Energy balance]]
colnames(fuelSharesgeneric@data) <- gsub("[ \\.]", "_", colnames(fuelSharesgeneric@data))
#fuelSharesgeneric@data <- merge(fuelSharesgeneric@data, data.frame(Time = 1900:2080))

objects.store(fuelSharesgeneric)
cat("Object fuelSharesgeneric initiated!\n")

</rcode>


<rcode name="fuelUse" label="Initiate fuelUse (developers only)" embed=1>
## This is code Op_en5141/fuelUse on page [[Energy balance]].
library(OpasnetUtils)

fuelUse <- Ovariable("fuelUse",
	dependencies = data.frame(
		Name = c("energyUse", "fuelShares")
	),
	formula = function(...) {
		out <- energyUse * fuelShares
		return(out)
	}
)

objects.store(fuelUse)
cat("Ovariable fuelUse stored.\n")
</rcode>

=== Calculations ===

:''Ovariable energyBalance below is used in [[Helsinki energy decision 2015]]. [[Prices of fuels in heat production]] are used as direct inputs in the optimising.

<rcode name="energyBalancetest" label="Initiate energyBalance (developers only)" embed=1>
## This code is  Op_en5141/energyBalance [[Energy balance]].
library(OpasnetUtils)

energyBalance <- Ovariable("energyBalance",
	dependencies = data.frame(
		Name = c(
			"energy", # Energy supply and demand in the system
			"nondynsupply", # Energy supply that is not optimised.
			"requiredName", # Fuel name for the energy type that is balanced for inputs and outputs.
			"fuelShares", # Shares of fuels for different heating types.
			"energyProcess", # Matrix showing inputs and outputs of the process of each plant.
			"fuelPrice", # Prices of fuels (by Time)
			"timelylimit", # Maximum energy supply in the given conditions at a particular time point
			"plantParameters" # Capacities and costs of running each plant
		),
		Ident = c(
			NA,
			NA,
			NA,
			NA,
			NA,
			"Op_en4151/fuelPrice",
			NA,
			NA
		)
	),
	formula = function(...) {
		
		# First, define the optimising function
		optfunction <- function(
			x, # Variables to optimise (activities of power plants).
			demand, # energy demand that must be met by supply (scalar)
			unitprice, # vector of prices per activity unit (by plant).
			unitreq # vector of required energy produced per activity uniit (by plant).
		) {

			out <- sum(x * unitprice) + (demand - sum(x * unitreq))^2 * 1E6
			
			return(out)
		}	

		# Simplify the energy demand and supply data and add index Fuel.
		ene <- unkeep(energy, prevresults = TRUE, sources = TRUE)
		ene <- oapply(ene, cols = c("Efficiency", "Renovation", "Building"), FUN = sum) # I: X, Heating, Consumamble, Fuel

		# Take the heating and look for fuel shares.
		eneheating <- unkeep(ene, cols = "Fuel")
		eneheating@output <- eneheating@output[eneheating@output$Heating != "Not heating" , ]
		eneheating <- oapply(eneheating * fuelShares, cols = "Heating", FUN = sum) # I: X, Consumable + Burner, Fuel

		# Then take the non-heating, which already has fuel shares.
		enenonheating <- ene * Ovariable(output = data.frame(Burner = "Unidentified", Result = 1), marginal = c(TRUE, FALSE))
		enenonheating@output <- enenonheating@output[enenonheating@output$Heating == "Not heating" , ]
		enenonheating <- unkeep(enenonheating, cols = "Heating") # I: X, Consumable, Fuel + Burner

		ene <- combine(eneheating, enenonheating) # I: X, Consumable, Fuel, Burner

		if(!is.na(requiredName)) { # Energy balance is only done if requiredName is given

			# Non-dynamic energy supply and demand, which is not optimised.
			# It is multiplied by -1 because energy production consumes fuels.
			nondynen <- ene 
			nondynen@output <- nondynen@output[nondynen@output$Fuel != requiredName , ]
			nondynen <- nondynen * Ovariable( # I: X, Consumable, Fuel, Burner, Plant
				output = data.frame(Plant = "Unidentified", Result = -1), 
				marginal = c(TRUE, FALSE)
			)

			# Dynamic energy supply and demand, which IS optmised.
			dynen <- ene
			dynen@output <- dynen@output[dynen@output$Fuel == requiredName , ]
			dynen <- oapply(dynen, cols = c("Fuel", "Burner", "Consumable"), FUN = sum) # I: X

			# Operational costs per unit of activity. Fuel price is excluded and calculated elsewhere.
			operationCost <- plantParameters
			operationCost@output <- operationCost@output[operationCost@output$Parameter == "Operation cost" , ]
			operationCost <- unkeep(operationCost, cols = "Parameter") # I: Y, Plant

			# Costs per unit of activity by plant. Fuel price included. Also revenues included.
			unitprice <- oapply(energyProcess * fuelPrice * -1, cols = "Fuel", FUN = sum) + operationCost
			colnames(unitprice@output)[colnames(unitprice@output) == "plantParametersResult"] <- "unitpriceResult"
			unitprice@output$unitpriceResult <- result(unitprice) # Save result into another column. I: Z, Plant, Burner - Fuel
			
			# Required energy produced per unit activity in each plant. Only take plants that affect required energy.
			unitreq <- energyProcess
			unitreq@output <- unitreq@output[unitreq@output$Fuel == requiredName & result(unitreq) != 0 , ]
			unitreq <- unkeep(unitreq, cols = "Fuel") # I: Plant, Burner - Fuel
			
			# Temporary limit to energy supply due to time-related particular constraints. Exclude unused plants.
			timlim <- plantParameters
			timlim@output <- timlim@output[timlim@output$Parameter == "Max" , ]
			timlim <- (
				unkeep((timelylimit >= timlim) * timlim, prevresults = TRUE, sources = TRUE) +
				(timelylimit < timlim) * timelylimit
			)
			timlim@output <- timlim@output[result(timlim) != 0 , ] # Exclude if max == 0
			colnames(timlim@output)[colnames(timlim@output) == "plantParametersResult"] <- "timelylimitResult"
			timlim@output$timelylimitResult <- result(timlim) # Save result into another column
			timlim <- unkeep(timlim, cols = "Parameter") # I: Y, Plant
			
			# Take the minimum from the plantParameters.
			ppar <- plantParameters
			ppar@output <- ppar@output[ppar@output$Parameter == "Min" , ]
			ppar <- unkeep(ppar, cols = "Parameter", sources = TRUE, prevresults = TRUE)

			# Put all inputs into one ovariable and sort by Plant. Note: important outputs in columns
				# unitpriceResult, energyProcessResult, plantParametersResult, timelylimitResult and Result (dynen).
			temp <- unitprice * unitreq * timlim * ppar * 0 + dynen # I: X + Y + Z, Plant, Burner

			indices <- unique(temp@output[temp@marginal & !colnames(temp@output) %in% c("Plant", "Burner", "Parameter")])

			out <- data.frame()
			
			for(i in 1:nrow(indices)) {

				temp2 <- merge(indices[i , , drop = FALSE], temp@output)
				temp2 <- temp2[order(temp2$Plant) , ]
				temp3 <- optim(
					par = temp2$plantParametersResult,
					fn = optfunction, 
					lower = temp2$plantParametersResult,
					upper = temp2$timelylimitResult,
					unitprice = temp2$unitpriceResult,
					demand = temp2$Result[1], # from dynen
					unitreq = temp2$energyProcessResult,
					method = "L-BFGS-B"
				)

				out <- rbind(out, data.frame(
					indices[i , , drop = TRUE],
					Plant = temp2$Plant,
					Result = temp3$par
				))
			}
			out <- Ovariable(output = out, marginal = c(rep(TRUE, ncol(out)-1), FALSE))
			out <- out * energyProcess # Optimised fuel usage
			out <- combine(out, nondynen) # Combine optimised and non-optimised energy demand
		} else {
			out <- ene # If optimisation is not done, take the simple version updated with fuelShares.
		}
		
		out <- unkeep(out, source = TRUE)
		marginals <- colnames(out@output)[out@marginal]
		out <- combine(out, nondynsupply) # Combine also non-dynamic energy supply
		
		# Fill in other indices (typically decisions) that are missing in nondynsupply
		marginals <- setdiff(marginals, colnames(nondynsupply@output)[nondynsupply@marginal])
		out@output <- fillna(out@output, marginals = match(marginals, colnames(out@output)))

		return(out)
	}
)

objects.store(energyBalance)
cat("Ovariable energyBalance stored.\n")
</rcode>

==== Newer version ====

<rcode name="EnergyNetworkOptim" label="Initiate EnergyNetworkOptim(developers only)" embed=1>
## This code is  Op_en5141/EnergyNetworkOptim [[Energy balance]].
library(OpasnetUtils)

EnergyConsumerDemandTotal <- Ovariable("EnergyConsumerDemandTotal",
	dependencies = data.frame(
		Name = c(
			"EnergyConsumerDemand"
		)
	),
	formula = function(...) {
		energy <- oapply(EnergyConsumerDemand, NULL, sum, c("Renovation", "Efficiency"))
		energy <- energy * 1e-6 # W to MW
		return(energy)
	}
)

EnergyFlowHeat <- Ovariable("EnergyFlowHeat",
	dependencies = data.frame(
		Name = c(
			"EnergyConsumerDemandTotal",
			"fuelShares"
		)
	),
	formula = function(...) {
		EnergyFlowHeat <- EnergyConsumerDemandTotal[EnergyConsumerDemandTotal$Consumable %in% c("Heating", "Hot water"),] * fuelShares
		return(EnergyFlowHeat)
	}
)

EnergyFlowOther <- Ovariable("EnergyFlowOther",
	dependencies = data.frame(
		Name = c(
			"EnergyConsumerDemandTotal",
			"temperene",
			"nontemperene"
		)
	),
	formula = function(...) {
		EnergyFlowOther <- merge(
				EnergyConsumerDemandTotal[!EnergyConsumerDemandTotal$Consumable %in% c("Heating", "Hot water"),], 
				unique(combine(temperene, nontemperene)[,c("Consumable", "Fuel")])
		)
		EnergyFlowOther@name <- EnergyConsumerDemandTotal@name
		return(EnergyFlowOther)
	}
)

EnergyNetworkDemand <- Ovariable("EnergyNetworkDemand",
	dependencies = data.frame(
		Name = c(
			"EnergyFlowHeat",
			"EnergyFlowOther",
			"EnergyConsumerDemandTotal"
		)
	),
	formula = function(...) {
		demand <- combine(
				EnergyFlowHeat[EnergyFlowHeat$Fuel %in% c("Heat", "Electricity"), colnames(EnergyFlowHeat@output) != "Burner"],
				EnergyFlowOther[EnergyFlowOther$Fuel %in% c("Cooling", "Electricity"),]
		)
		
		# EnergyFlowOther should not have other flows
		#assign("EnergyFlowOther", NULL, .GlobalEnv)
		
		#assign("EnergyFlowHeat", EnergyFlowHeat[!EnergyFlowHeat$Fuel %in% c("Heat", "Electricity"),], .GlobalEnv)
		
		# Total consumer demand also unnecessary?
		#assign("EnergyConsumerDemandTotal", NULL, .GlobalEnv)
		
		NAindex <- sapply(lapply(lapply(demand@output, unique), is.na), sum)
		NAindex <- names(NAindex)[NAindex != 0]
		
		demand@output <- fillna(demand@output , NAindex)
		demand <- oapply(demand, NULL, sum, c("Heating", "Consumable"))#, "Renovation", "Efficiency")) # Assumes no NA
		
		return(demand)
	}
)

EnergyNetworkOptim <- Ovariable("EnergyNetworkOptim",
	dependencies = data.frame(
		Name = c(
			#"energy", # Energy supply and demand in the system
			#"nondynsupply", # Energy supply that is not optimised.
			#"requiredName", # Fuel name for the energy type that is balanced for inputs and outputs.
			#"fuelShares", # Shares of fuels for different heating types.
			"energyProcess", # Matrix showing inputs and outputs of the process of each plant.
			"fuelPrice", # Prices of fuels (by Time)
			#"timelylimit", # Maximum energy supply in the given conditions at a particular time point
			"plantParameters", # Capacities and costs of running each plant
			#"temperene",
			#"nontemperene",
			"EnergyNetworkDemand"
		),
		Ident = c(
			#NA,
			#NA,
			#NA,
			#NA,
			NA,
			"Op_en4151/fuelPrice",
			#NA,
			NA,
			#"Op_en5488/temperene",
			#"Op_en5488/nontemperene",
			NA
		)
	),
	formula = function(...) {
		
		require(lpSolve)
		require(plyr)
		require(reshape2)
		
		# Prices are given as €/MWh while plant activity is given as MW while optimisation is done 
		# with time resolution = 1 day
		optim_time_period_adj <- 24
	
		# List non-interating indices
		
		exclude <- c("Fuel", "Plant", "Parameter", "Heating", "Consumable", "Burner")
		
		# Calculate unitprofit for plant activity energy flows
		
		unitp <- fuelPrice * energyProcess * optim_time_period_adj
		unitp <- oapply(unitp, NULL, sum, c("Fuel", "Burner"))
		colnames(unitp@output)[colnames(unitp@output) == "Result"] <- "UnitPrice"
		
		# Reshape crucial variables to reduce merging difficulty
		
		ePmargs <- colnames(energyProcess@output)[energyProcess@marginal]
		energyProcess@output <- dcast(
			energyProcess@output, 
			as.formula(paste(paste(ePmargs[ePmargs != "Fuel"], collapse = "+"), "~ Fuel")), 
			value.var = paste(energyProcess@name, "Result", sep = ""),
			fill = NA
		)
		energyProcess@marginal <- colnames(energyProcess@output) %in% ePmargs
		
		pPmargs <- colnames(plantParameters@output)[plantParameters@marginal]
		plantParameters@output <- dcast(
				plantParameters@output, 
				as.formula(paste(paste(pPmargs[pPmargs != "Parameter"], collapse = "+"), "~ Parameter")), 
				value.var = paste(plantParameters@name, "Result", sep = ""),
				fill = NA
		)
		plantParameters@marginal <- colnames(plantParameters@output) %in% pPmargs
		
		EnergyNetworkDemand$Fuel <- paste(EnergyNetworkDemand$Fuel, "Demand", sep = "")
		ENDmargs <- colnames(EnergyNetworkDemand@output)[EnergyNetworkDemand@marginal]
		EnergyNetworkDemand@output <- dcast(
				EnergyNetworkDemand@output, 
				as.formula(paste(paste(ENDmargs[ENDmargs != "Fuel"], collapse = "+"), "~ Fuel")), 
				value.var = paste(EnergyNetworkDemand@name, "Result", sep = ""),
				fill = NA
		)
		EnergyNetworkDemand@marginal <- colnames(EnergyNetworkDemand@output) %in% ENDmargs
		
		fuelPrice$Fuel <- paste(fuelPrice$Fuel, "Price", sep = "")
		fPmargs <- colnames(fuelPrice@output)[fuelPrice@marginal]
		fuelPrice@output <- dcast(
				fuelPrice@output, 
				as.formula(paste(paste(fPmargs[fPmargs != "Fuel"], collapse = "+"), "~ Fuel")), 
				value.var = paste(fuelPrice@name, "Result", sep = ""),
				fill = 0
		)
		fuelPrice@marginal <- colnames(fuelPrice@output) %in% fPmargs
		
		# Duplicates a couple of values, but provides better performance
		vars <- merge(EnergyNetworkDemand, plantParameters)
		vars <- merge(vars, energyProcess)
		vars <- merge(vars, fuelPrice)
		vars <- merge(vars, unitp)
		
		# Split into iterations
		#vars <- dlply(vars@output, colnames(vars@output)[vars@marginal & !colnames(vars@output) %in% exclude], I)

		#out <- data.frame()

		include <- union(ePmargs, pPmargs)
		include <- union(include, ENDmargs)
		include <- union(include, fPmargs)
		include <- include[!include %in% exclude]
		
		# Optimisation iteration function
		
		optf <- function(varsi) {
	
			# Acquire relevant sections of variables with respect to index iteration
			
			demandi <- unlist(varsi[1,grepl("Demand$", colnames(varsi))])
			names(demandi) <- gsub("Demand$", "", names(demandi))
			
			fuelPricei <- unlist(varsi[1,grepl("Price$", colnames(varsi))])
			names(fuelPricei) <- gsub("Price$", "", names(fuelPricei))
			fuelPricei <- fuelPricei[names(fuelPricei) != "Unit"]
			
			plants <- as.character(unique(varsi[["Plant"]])) # Used for ordering parameters
			if (nrow(varsi) != length(plants)) stop("Wrong subset given to energy production optimizer! Each row not unique with respect to Plant.")
			
			lower <- varsi[["Min"]]
			
			upper <- varsi[["Max"]]
			
			opcost <- varsi[["Operation cost"]]
	
			unitpi <- varsi[["UnitPrice"]]
			
			#commodities <- as.character(unique(energyProcessi$Fuel)) # Used for ordering parameters
			
			# Constraint matrix
			
			# Total number of variables:
			# * plant activity for each plant
			# * (commodity flow total)
			# * commodity deficit and excess with respect to demand
			# * opeartion costs
			nvars <- length(plants) + length(demandi) * 2 + 1
			
			# Rows:
			# * demand rows and their constraints
			# * (commodity flow rows)
			# * plant activity constraints
			# * operation costs row
			nrows <- 3 * length(demandi) + 2 * length(plants) + 1
			M <- matrix(
				0,
				nrows, 
				nvars
			)
			
			sign <- rep(">=", nrows)
			constant <- rep(0, nrows)
			
			for (j in 1:length(demandi)) {
				# Total flow (production - consumption between all processes) + deficit = demand + excess
				# which translates to:
				# total_commodity_flow + deficit - excess == constant ("static" demand)

				M[j, length(plants) + j*2 + 1:2 - 2] <- c(1, -1)
				sign[j] <- "=="
				constant[j] <- demandi[j]
				
				# Plant flows by activity
				M[j, 1:length(plants)] <- varsi[[names(demandi)[j]]]
				
			}
			
			rowi <- length(demandi)
			diag(M[
				(rowi + 1):(rowi + length(plants)), 
				1:length(plants)
			]) <- 1
			diag(M[
				(rowi + length(plants) + 1):(rowi + 2 * length(plants)), 
				1:length(plants)
			]) <- 1
			constant[rowi + 1:length(plants)] <- lower
			constant[rowi + length(plants) + 1:length(plants)] <- upper
			sign[rowi + length(plants) + 1:length(plants)] <- "<="
			
			rowi <- rowi + 2 * length(plants)
			# Excess and deficit must both be positive
			diag(M[
				rowi + 1:(2 * length(demandi)), 
				length(plants) + #length(commodities) + 
						1:(2 * length(demandi))
			]) <- 1
			
			# Operating cost
			M[nrows, 1:length(plants)] <- opcost * optim_time_period_adj
			M[nrows, nvars] <- -1
			sign[nrows] <- "=="
			
			# Objective function (profits - penalty)
			objective <- rep(0, nvars)
			# Commodity prices
			#objective[length(plants) + (1:length(commodities))[!is.na(fuelPricej)]] <- result(fuelPricei)[fuelPricej[!is.na(fuelPricej)]]
			# Penalize heat deficit heavily, 
			# electricity deficit has no impact on profit assuming company sells to customers at the purchase price
			# cooling deficit has no effect
			if ("Heat" %in% names(demandi)) {
				objective[length(plants) + match("Heat", names(demandi)) * 2 - 1] <- -1e6
			}
			
			# Electricity excess can be sold in the markets, 
			# heat and cooling (and other?) excess, however, cannot, hence they need to be deducted from profits
			objective[
				length(plants) + which(names(demandi) != "Electricity") * 2
			] <- - fuelPricei[match(names(demandi)[names(demandi) != "Electricity"], names(fuelPricei))] * optim_time_period_adj
			
			# Operation cost
			objective[nvars] <- -1

			# Unit profit
			objective[1:length(plants)] <- unitpi
			
			# Do actual linear programming 
			temp <- lp(
				"max",
				objective, 
				M,
				sign,
				constant
			)
			
			# Format results
			outi <- varsi[rep(1, nvars + 1), include, drop = FALSE]
			outi[["Process_variable_type"]] <- rep(
				c("Activity", "Flow", "Misc"), 
				c(length(plants), nvars - length(plants) - 1, 2)
			)
			outi[["Process_variable_name"]] <- c(
				as.character(plants), 
				paste(rep(names(demandi), each = 2), c("Deficit", "Excess"), sep = ""), 
				"Operation cost",
				"Profit"
			)
			outi[["Result"]] <- c(temp$solution, temp$objval)
			
			return(outi)
		}
		#total_time <- Sys.time() - total_time
		#aggregate(out$Result[out$Process_variable_name == "Profit"], out[out$Process_variable_name == "Profit",colnames(out) %in% c("Time", "Temperature") | grepl("Policy", colnames(out))], mean)
		#aggregate(out$Result[out$Process_variable_name == "HeatDeficit"], out[out$Process_variable_name == "HeatDeficit", colnames(out) %in% c("Time", "Temperature") | grepl("Policy", colnames(out))], mean)
		#aggregate(out$Result[out$Process_variable_name == "Hanasaari"], out[out$Process_variable_name == "Hanasaari", colnames(out) %in% c("Time", "Temperature") | grepl("Policy", colnames(out))], mean)
		
		out <- ddply(vars@output, colnames(vars@output)[vars@marginal & !colnames(vars@output) %in% exclude], optf)
		
		out <- Ovariable(output = out, marginal = !grepl("Result$", colnames(out)))

		return(out)
	}
)

fuelUse <- Ovariable("fuelUse",
	dependencies = data.frame(
		Name = c(
			"EnergyNetworkOptim",
			"EnergyNetworkDemand",
			"energyProcess",
			"EnergyFlowHeat",
			"temperdays"
		)
	),
	formula = function(...){
		
		# Calculate flows resulting from plant activity
		EnergyFlow <- EnergyNetworkOptim[EnergyNetworkOptim$Process_variable_type == "Activity", colnames(EnergyNetworkOptim@output) != "Process_variable_type"]
		colnames(EnergyFlow@output)[colnames(EnergyFlow@output) == "Process_variable_name"] <- "Plant"
		
		EnergyFlow <- EnergyFlow * energyProcess
		
		# Realised consumption of centrally produced commodities (demand - deficit or prod - excess)
		# NOTE: centralized flows which consume electricity for example should be separated
		# from consumer demand and consumption, so prod - excess is not accurate
		deficit <- EnergyNetworkOptim[
				grepl("Deficit$", EnergyNetworkOptim$Process_variable_name), 
				colnames(EnergyNetworkOptim@output) != "Process_variable_type"
		]
		colnames(deficit@output)[colnames(deficit@output) == "Process_variable_name"] <- "Fuel"
		deficit$Fuel <- gsub("Deficit$", "", deficit$Fuel)
		# Assume that the power company purchases any electricity deficit from the markets in order to satisfy demand
		result(deficit)[deficit$Fuel == "Electricity"] <- 0
		
		#excess <- EnergyNetworkOptim[
		#		grepl("Excess$", EnergyNetworkOptim$Process_variable_name), 
		#		colnames(EnergyNetworkOptim@output) != "Process_variable_type"
		#]
		#colnames(excess@output)[colnames(excess@output) == "Process_variable_name"] <- "Fuel"
		#excess$Fuel <- gsub("Excess$", "", excess$Fuel)
		
		#temp <- oapply(EnergyFlow, NULL, sum, c("Plant", "Burner"))
		
		#real <- oapply(EnergyFlow[EnergyFlow$Fuel %in% unique(excess$Fuel)], NULL, sum, c("Plant", "Burner"))
		#real <- real - excess
			
		real <- EnergyNetworkDemand - deficit
		
		# Combine 
		real$Burner <- "None"
		real$Plant <- "Domestic"

		heating <- EnergyFlowHeat[!EnergyFlowHeat$Fuel %in% unique(deficit$Fuel),]
		heating <- oapply(heating, NULL, sum, "Consumable")
		heating$Plant <- "Domestic"
		heating$Heating <- NULL
		
		EnergyFlow <- combine(0 - EnergyFlow, real, heating)
		EnergyFlow <- unkeep(EnergyFlow, sources = TRUE, prevresults = TRUE)
		#EnergyFlowTest <- oapply(EnergyFlow, c("Time", "Temperature", "Fuel"), sum)
		#ggplot(EnergyFlowTest@output, aes(x = Temperature, y = Result, group = Fuel, color = Fuel)) + geom_line() + facet_wrap( ~ Time)
		
		EnergyFlow <- EnergyFlow * temperdays * 3600 * 24
		EnergyFlow <- oapply(EnergyFlow, cols = c("Temperature"), FUN = sum)
		return(EnergyFlow)
	}
)

objects.store(EnergyConsumerDemandTotal, EnergyFlowHeat, EnergyFlowOther, EnergyNetworkDemand, EnergyNetworkOptim, fuelUse)
cat("Ovariables EnergyConsumerDemandTotal, EnergyFlowHeat, EnergyFlowOther, EnergyNetworkDemand, EnergyNetworkOptim and fuelUse stored.\n")
</rcode>

==== Original version ====

Stored objects below used by [[Energy balance in Kuopio]].

<rcode name="initiate" label="Initiate objects" store=1>
library(OpasnetUtils)
library(ggplot2)

solveMatrix <- function(equations, directinputtemp, N. = 0, verbose = FALSE, dbug = FALSE, ...) {
	if (N. == 0) N. <- openv$N
	N <- N.
	
	# Parse equations using regular expressions
	
	out <- gsub(" ", "", equations)
	out <- strsplit(out, "=")
	out <- lapply(out, strsplit, split = "+", fixed = TRUE)
	nterms <- rapply(out, length) # number of terms on each side of each equation
	signs <- rep(
			rep(c(1,-1), length(out)), # lhs - rhs = 0
			nterms
	)
	eqn <- factor(rep(rep(1:length(equations), each = 2), nterms)) # equation number
	out <- unlist(out)
	k <- rep(1, length(out))
	x <- out
	
	# Equation factors
	
	out <- strsplit(out, split = "*", fixed = TRUE)
	temp <- sapply(out, length) > 1
	
	for (i in (1:length(out))[temp]) {
		k[i] <- out[[i]][1]
		x[i] <- out[[i]][2]
	}
	
	# Constants
	
	temp <- substr(x, 1, 1) %in% 0:9 # is interpretable (probably)
	
	for (i in (1:length(out))[temp]) {
		k[i] <- x[i]
		x[i] <- "Constant"
	}
	
	# Direct input (substitutes for constants)
	
	if (nrow(directinputtemp) > 0) {
		l <- tapply(eqn, data.frame(eqn), length)
		di <- directinputtemp$energybalanceVars
		dires <- directinputtemp$directinputResult
		for (i in 1:length(di)) {
			temp <- eqn %in% names(l[l == 2]) & x %in% c("Constant", di[i])
			temp <- tapply(temp, eqn, sum)
			k[eqn %in% names(temp[temp == 2]) & x == "Constant"] <- dires[i]
		}
	}
	
	# Interpret possible distribution regular expressions (also converts text to numeric)
	
	out <- data.frame(eqn, x, signs, Result = k)
	out <- interpret(out, N = N)
	
	marg <- colnames(out)[colnames(out) %in% c("eqn", "x", "Iter")]
	
	out$k <- out$Result * out$signs
	
	# Separate constants and vars and make them into arrays
	
	temp <- out[out$x != "Constant", ]
	
	b <- out[out$x == "Constant", ]
	b <- tapply(b$k, b[marg[-2]], I)
	b[is.na(b)] <- 0
	
	temp <- tapply(temp$k, temp[marg], I)
	if ("Iter" %in% marg) {
		temp <- temp[,dimnames(temp)$x != "Constant",]
	} else {
		temp <- temp[,dimnames(temp)$x != "Constant"]
	}
	temp[is.na(temp)] <- 0
	
	if ("Iter" %in% marg) {
		out <- list()
		for (i in 1:N) {
			ret <- tryCatch(
				out[[i]] <- solve(temp[,,i], -b[,i]),
				error = function(...) return(NULL)
			)
			if (is.null(ret)) {
				print("Faulty matrix:")
				oprint(temp[,,i])
				stop(geterrmessage())
			}
		}
		out <- data.frame(
			energybalanceVars = names(unlist(out)), 
			Iter = rep(1:N, each = length(out[[1]])), 
			Result = unlist(out)
		)
	} else {
		ret <- tryCatch(
			out <- solve(temp, -b),
			error = function(...) return(NULL)
		)
		if (is.null(ret)) {
			print("Faulty matrix:")
			oprint(temp)
			stop(geterrmessage())
		}
		out <- data.frame(
			energybalanceVars = names(out),
			Result = out
		)
	}
	return(out)
}

energy.balance <- Ovariable( # NOTE! energy.balance requires that there exists object N that defines the number of iterations.
	name = "energy.balance",
	dependencies = data.frame(
		Name = c("balance", "nonlinearity", "directinput")
	),
	formula = function(...) {
		# Column Equation: change factor into character and demarginalise.
		balance@data$Equation <- as.character(balance@data$Equation)
		balance@marginal[colnames(balance@output) == "Equation"] <- FALSE
		
		# Take those indices that are not needed within solveMatrix and merge them to the balance from the directinput.
		directinputindex <- directinput@output[directinput@marginal]
		directinputindex <- directinputindex[!colnames(directinputindex) %in% c("Iter", "energybalanceVars")]
		
		indices <- unique(merge(balance@output[balance@marginal], directinputindex))
		
		# Look at each unique combination of index locations and solve the set of equations to each.
		
		out <- data.frame()
		for(k in 1:nrow(indices)) {
			
			equations <- merge(balance@output, indices[k , , drop = FALSE])
			
			directinputtemp <- merge(directinput@output[!is.na(result(directinput)),], indices[k, colnames(indices) != "Iter", drop = FALSE])
			directinputtemp <- directinputtemp[ , c("energybalanceVars", "directinputResult")]
			
			temp <- solveMatrix(as.character(equations[ , "Equation"]), directinputtemp, N. = N, ...)
			
			out <- rbind(out, merge(indices[k , , drop = FALSE], temp))
		}
		
		if(!(nonlinearity@output$critVar[1] == "" | is.na(nonlinearity@output$critVar[1]))) {
			
			# Go through the non-linear equations one at a time and find which values to use in which situation.
			
			for(i in 1:nrow(nonlinearity@output)) {
				
				critVar 	<- gsub(" ", "", as.character(nonlinearity@output$critVar[i]))
				critIndex 	<- as.character(nonlinearity@output$critIndex[i])
				rescol 		<- as.character(nonlinearity@output$rescol[i])
				critLocLow 	<- as.character(nonlinearity@output$critLocLow[i])
				critLocHigh	<- as.character(nonlinearity@output$critLocHigh[i])
				critValue 	<- result(nonlinearity)[i]
				
				marginal <- colnames(out) %in% c(colnames(balance@output)[balance@marginal], rescol, "Iter")
				
				choos <- out[out$energybalanceVars == critVar , marginal]
				choos <- choos[
					choos[critIndex] == critLocLow  & choos[rescol] <= critValue | 
						choos[critIndex] == critLocHigh & choos[rescol] >  critValue , 
					colnames(choos) != rescol # Remove result column 
				]
				
				out <- merge(out, choos)
			}
		}
		
		# Create the marginal: those from the parents and "energybalanceVars" but not nonlinear indices.
		outmarginal <- c(
			colnames(balance@output)[balance@marginal], 
			colnames(nonlinearity@output)[nonlinearity@marginal], 
			colnames(directinput@output)[directinput@marginal],
			"energybalanceVars"
		)
		outmarginal <- outmarginal[!outmarginal %in% nonlinearity@output$critIndex]
		outmarginal <- colnames(out) %in% outmarginal
		
		out <- new("ovariable", name = "energy.balance", output = out, marginal = outmarginal)
		
		return(out)
	}
)

objects.store(solveMatrix, energy.balance)

cat("Objects solveMatrix and energy.balance stored with code initiate.\n")

</rcode>

* [http://en.opasnet.org/en-opwiki/index.php?title=Energy_balance&oldid=30561 Model version] that was used to run results for ISEE2013.

How to give uncertain parameters?
* In equations, the content is interpreted only inside solveMatrix. Therefore, the typical approach where all unique index combinations are run one at a time does not work.
* There should be an update in parameter interpretation for terms with one entry only. It can no longer be based on as.numeric, if distributions (=text) is allowed.
** If it starts with [a-z.] it is a variable name.
** If it starts with [0-9<\\-] it is a parameter value.
* Instead of <nowiki>params[[i]] and [[vars]]</nowiki> vectors, a data.frame will be created with Result as the params column.
* The data.frame is then interpreted with N = N. If parameters are probabilistic, Iter column will appear.
* When all parameters have been interpreted, check if Iter exists.
* If Iter exists, make a for loop for all values of Iter.
** Create a matrix from the parameters and solve.
** Rbind the result to a data.frame with Iter.
* Return the output.

* Old code with an input table with columns Equation, Col, Result, Description: [http://en.opasnet.org/en-opwiki/index.php?title=Energy_balance&oldid=30515#Code]

==See also==

{{Helsinki energy decision 2015}}

* A new version of energy balance is using [https://stat.ethz.ch/R-manual/R-devel/library/stats/html/optim.html optim function]. See [[Energy balance in Helsinki]].
* [http://www.nzz.ch/meinung/kommentare/energiepolitik-bitte-wenden-1.18451262 Energiepolitik, bitte wenden!] Neue Zürcher Zeitung 27.12.2014 by Giorgio V. Müller
* [http://time.com/2981460/electric-cars-home-solar/ TIME: Electric Cars Will Change the Way You Power Your Home]. How the homes of the future will generate and store their own electricity, turning your house into a mini-power plant.
** [http://time.com/2865496/tesla-opens-patents/ TIME: Tesla opens patents to everyone]
* [[Energy supply in Europe]]
* [[media:Health impacts of energy production.ppt]] (a lecture that also contains explanation of an energy balance using matrices)
* [[Energy balance in Kuopio]] Describes the production and consumption of energy in Kuopio.
* [[Energy balance in Basel]] 
* [[Energy balance in Stuttgart]] Describes the production and consumption of energy in Stuttgart.
* [[Energy balance in Suzhou]] Describes the production and consumption of energy in Suzhou.
* [http://en.opasnet.org/en-opwiki/index.php?title=Energy_balance&oldid=25906 A previous method to calculate energy balances]. Includes also other pages:
** [[Energy transformations]] Describes the inputs and outputs of energy processes. Shows, which other things change when some input or output is changed.
** [[Market allocation factor]]
* [[:File:Energy supply in Europe.xls]]
* [[Climate change policies in Kuopio]] Indicates, which items change when a policy changes.
* [[:op_fi:Energiatase|Energiatase]]
* [[:op_fi:Kasvihuonekaasupäästöt/Kuopio|Kasvihuonekaasupäästöt/Kuopio]]
* [[:op_fi:tiedosto:Päätösanalyysia_ja_riskinhallintaa.ppt|Päätösanalyysia_ja_riskinhallintaa.ppt]]
* [[Urgenche]]
* http://www.energia.fi/sites/default/files/polttoaine-energian_maarittaminen_taselaskennan_avulla.pdf
* [http://www.ymparisto.fi/default.asp?contentid=408832&lan=fi&clan=fi Uusiutuvan energian riskit selvitetään]
* [http://www.dconnolly.net/research/planning/tools/mesap_planet.html Urgenche: Mesap Planet energy model]
* [[OpasnetUtils/Drafts]]

{{urgenche}}

==References==

<references/>

==Related files==