---
title: "Introduction to iotables"
author: "Daniel Antal, based on the work of Jorg Beutel"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Introduction to iotables}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setupknitr, include=FALSE, message=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```
```{r setup,  message=FALSE}
library(iotables)
library(dplyr, quietly = T)
library(tidyr, quietly = T)
```

This introduction shows the reproducible workflow of iotables with the examples of the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0). by Joerg Beutel (*Eurostat Manual*).  

This vignette uses `library(tidyverse)`, more particularly `dplyr` and `tidyr`, just like all analytical functions of `iotables`.  Even if you do not use `tidyverse`, this packages will be installed together with `iotables`. These functions are only used in the vignette to print the output, and they are not essential for the examples.

## Germany sample files

The `germany_1995` dataset is a simplified 6x6 sized SIOT taken from the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0) (page 481). It is brought to a long form similar to the Eurostat bulk files. The testthat infrastructure of the iotables package is checking data processing functions and analytical calculations against these published results.

The following data processing functions select and order the data from the Eurostat-type bulk file. Since the first version of this package, Eurostat moved all SIOT data products to the ESA2010 vocabulary, but the manual still follows the ESA95 vocabulary. The labels of the dataset were slightly changed to match the current metadata names. The changes are minor and self-evident, the comparison of the `germany_1995` dataset and the Manual should cause no misunderstandings.

```{r iotables}
germany_io <- iotable_get( labelling = "iotables" )
input_flow <- input_flow_get ( 
                  data_table = germany_io, 
                  households = FALSE)

de_output <- primary_input_get ( germany_io, "output" )
print (de_output[c(1:4)])
```

The `input_flow()` function selects the first quadrant, often called as input flow matrix, or inter-industry matrix, from the German input-output table. The `primary_input_get()` selects on of the primary inputs, in this case, the output from the table. 

## Direct effects

The input coefficient matrix shows what happens in the whole domestic economy when an industry is facing additional demand, and it increases production. In the Germany example, all results are rounded to 4 digits for easier comparison with the Eurostat manual.

The input coefficients for domestic intermediates are defined in the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0) on page 486. You can check the following results against Table 15.8 of the Eurostat manual. (Only the top-right corner of the resulting input coefficient matrix is printed for readability.) 

\deqn{sum_{i=1}^n X_i}

The `input_coefficient_matrix_create()` function relies on the following equation. The numbering of the equations is the numbering of the *Eurostat Manual*.

(9)  \eqn{a_{ij}} = \eqn{X_{ij} / x_{j}}  [recap: (43) is the same, and the same equation is (2) on page 484 with comparative results]

It checks the correct ordering of columns, and furthermore it fills up 0 values with 0.000001 to avoid division with zero.

```{r inputcoeff, echo=TRUE}
de_input_coeff <- input_coefficient_matrix_create( 
     data_table = germany_io, 
     digits = 4)

## which is equivalent to:
de_input_coeff <- coefficient_matrix_create( 
  data_table = germany_io, 
  total = "output", 
  return_part = "products", 
  households = FALSE,
  digits = 4)

print (de_input_coeff[1:3, 1:3])
```
These results are identical after similar rounding to the Table 15.6 of the Manual (on page 485.)

Similarly, the output coefficient matrix is defined in the following way:

(5)  \eqn{o_{ij}} = \eqn{x_{ij} / x_{i}} 

\eqn{o_{ij}} = output coefficient for domestic goods and services (i = 1, ..., 6; j = 1, ..., 6)
\eqn{x_{ij}= flow of commodity i to sector j
\eqn{x_{i} = output of sector i


```{r outputcoeff, echo=FALSE}
de_out <- output_coefficient_matrix_create ( 
 data_table = germany_io, 
 total = 'tfu',
 digits = 4)

# Rounding is slightly different in the Eurostat manual:
print(de_out[1:3, 1:3])
```

These results are identical after similar rounding to the Table 15.7 of the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0) on page 485. The diagonal values are the same in the input coefficient matrix and the output coefficient matrix. 

The Leontief matrix is derived from Leontief equation system.  

(19) \eqn{(I-A)x = y}

The Leontief matrix is defined as \eqn{(I-A)} and it is created with the `leontief_matrix_create()` function.

The Leontief inverse is \eqn{(I-A)^{-1}} and it is created with the `leontief_inverse_create()` function from the Leontief-matrix.

```{r leontief}
L_de <- leontief_matrix_create ( 
  technology_coefficients_matrix = de_input_coeff 
  )
I_de   <- leontief_inverse_create(de_input_coeff)
I_de_4 <- leontief_inverse_create(technology_coefficients_matrix=de_input_coeff,
                                  digits = 4)
print (I_de_4[,1:3])
```

*You can check the Leontief matrix against Table 15.9 on page 487 of the* Eurostat Manual, *and the Leontief inverse against Table 15.10 on page 488. The ordering of the industries is different in the manual.*

## Creating indicators 

### Creating technical indicators

Technical indicators assume constant returns to scale and fixed relationship of all inputs to each industry. With these conditions the technical input coefficients show how much input products, labour or capital is required to produce a unit of industry output.

(60) $a_{ij}$ = $z_{ij}$ / $x_j$  [technical input coefficients]

The helper function `primary_input_get()` selects a row from the SIOT and brings it to a conforming form. The `input_indicator_create()` creates the vector of technical input coefficients.

```{r employment_indicator}
de_emp <- primary_input_get(germany_io,
                            primary_input = "employment_domestic_total")

de_emp_indicator <- input_indicator_create( 
    data_table = germany_io,
    input_row  = "employment_domestic_total")

vector_transpose_longer(de_emp_indicator)
```

Often we want to analyze the effect of growing domestic demand on some natural units, such as employment or $CO_2$ emissions. The only difficulty is that we need data that is aggregated / disaggregated precisely with the same industry breakup as our SIOT table.

European employment statistics have greater detail than our tables, so employment statistics must be aggregated to conform the 60 (61, 62) columns of the SIOT.  There is a difference in the columns based on how national statistics offices treat imputed real estate income and household production, and trade margins. Czech SIOTs are smaller than most SIOTs because they do not have these columns and rows.

In another vignette we will show examples on how to work with these real-life data. For the sake of following the calculations, we are continuing with the simplified 1990 German data. 

### Creating income indicators

The input coefficients for value added are created with `input_indicator_create()`. 

```{r gva_indicator}
de_gva <- primary_input_get ( germany_io,
                              primary_input = "gva") 

de_gva_indicator  <- input_indicator_create( 
    data_table  = germany_io,
    input_row   = "gva")

vector_transpose_longer(de_gva_indicator)
```


This is equal to the equation on page 495 of the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0). The results above can be checked on the bottom of page 498.

(44) $w_{ij}$ = $W_{j}$ / $x_j$  [input coefficients for value added]

You can create a matrix of input indicators, or direct effects on (final) demand with `direct_supply_effects_create()`. The function by default creates input requirements for final demand.  With the code below it re-creates the Table 15.14 of the *Eurostat Manual*. 

```{r input_indicator}
direct_effects_de <- coefficient_matrix_create(
  data_table  = germany_io, 
  total       = 'output', 
  return_part = 'primary_inputs')

direct_effects_de[1:6,1:4]
```

*The 'total' row above is labelled as Domestic goods and services in the* Eurostat Manual. *The table can be found on page 498.*

## Multipliers

### Income multipliers

The SIOTs contain (with various breakups) three types of income:

* Employee wages, which is usually a proxy for all household income.

* Gross operating surplus, which is a form of corporate sector income.

* Taxes that are the income of government.

These together make gross value added (GVA).  If you are working with SIOTs that use basic prices, then GVA = GDP at producers' prices, or basic prices.

The GVA multiplier shows the additional gross value created in the economy if demand for the industry products is growing with unity. The wage multiplier (not shown here) shows the increase in household income.

The following equation is used to work with different income effects and multipliers:

(63) Z = B(I-A)^-1^ 

B = vector of input coefficients for wages or GVA or taxes.

Z = direct and indirect requirements for wages (or other income)

The indicator shows that manufacturing has the lowest, and other services has the highest gross value added component. This is hardly surprising, because manufacturing needs a lot of materials and imported components. When the demand for manufacturing in the domestic economy is growing by 1 unit, the gross value added is `r as.numeric(de_gva_indicator[3])`.

*You can check these values against the Table 15.16 of the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0) on page 501 (row 10).*

You can recreate the whole matrix, when the data data permits, with `input_multipliers_create()` as shown here. Alternatively, you can create your own custom multipliers with `multiplier_create()` as shown in the following example. 

```{r inputmultipliers}
input_reqr <- coefficient_matrix_create(
    data_table  = iotable_get(), 
    total       = 'output', 
    return_part = 'primary_inputs') 

multipliers <- input_multipliers_create(
  input_requirements = input_reqr,
  Im = I_de)

multipliers
```

*You can check these results against the Table 15.16 on page 501 of the* Eurostat Manual. *The label 'total' refers to domestic intermediaries. The ordering of the rows is different from the* Manual.

These multipliers are Type-I multipliers. The type-I GVA multiplier shows the total effect in the domestic economy. The initial extra demand creates new orders in the backward linking industries, offers new product to build on in the forward-linking industry and creates new corporate and employee income that can be spent. Type-II multipliers will be introduced in a forthcoming vignette [not yet available.]

### Employment multipliers

The E matrix contains the input coefficients for labor (created by `input_indicator_create()`). The following matrix equation defines the employment multipliers.

(64) Z = E(I-A)^-1^ 

The `multiplier_create()` function performs the matrix multiplication, after handling many exceptions and problems with real-life data, such as different missing columns and rows in the national variations of the standard European table. 

Please send a bug report on [Github](https://github.com/rOpenGov/iotables/issues) if you run into further real-life problems. 

```{r employment_multiplier}
de_emp_indicator <- input_indicator_create (
  data_table = germany_io, 
  input = 'employment_domestic_total') 

employment_multipliers <- multiplier_create ( 
  input_vector    = de_emp_indicator,
  Im              = I_de,
  multiplier_name = "employment_multiplier", 
  digits = 4 )

vector_transpose_longer(employment_multipliers,
                        values_to="employment_multipliers")
```

You can check against the [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0) page 501 that these values are correct and on page 501 that the highest employment multiplier is indeed $z_i$ = `r max (as.numeric(employment_multipliers), na.rm=TRUE)`, the employment multiplier of agriculture.

For working with real-life, current employment data, there is a helper function to retrieve and process Eurostat employment statistics to a SIOT-conforming vector `employment_get()`. This function will be explained in a separate vignette. 

### Output multipliers

`Output multipliers` and `forward linkages` are calculated with the help of output coefficients for product as defined on p486 and p495 of the the Eurostat Manual. The Eurostat Manual uses the definition of _output at basic prices_ to define output coefficients which is no longer part of SNA as of SNA2010.

(5) $b_{ij}$ = $X_{ij}$ / $x_i$  [also (45) output coefficients for products / intermediates].

$x_i$: output of sector i

```{r outputmult}
de_input_coeff <- input_coefficient_matrix_create( 
  data_table = iotable_get(), 
  digits = 4)

output_multipliers <- output_multiplier_create (input_coefficient_matrix = de_input_coeff)

vector_transpose_longer(output_multipliers, 
                        values_to = "output_multipliers")
```

*These multipliers can be checked against the Table 15.15 (The 8th, 'Total' row) on the page 500 of the* Eurostat Manual.

## Interindustrial linkage analysis

The `backward linkages`, i.e. demand side linkages, show how much incremental demand is created in the supplier sector when an industry is facing increased demand, produces more, and requires more inputs from suppliers.  

`Forward linkages` on the other hand show the effect of increased production, which gives either more or cheaper supplies for other industries that rely on the output of the given industry.

For example, when a new concert is put on stage, orders are filled for real estate, security services, catering, etc, which show in the backward linkages. The concert attracts visitors that creates new opportunities for the hotel industry in forward linkages. 

### Backward linkages 

```{r backward}
de_coeff <- input_coefficient_matrix_create(iotable_get(), digits = 4)
I_de     <- leontief_inverse_create (de_coeff)

vector_transpose_longer(backward_linkages(I_de),
                        values_to = "backward_linkage_strength")
```

*You can check the results against Table 15.19 on page 506 of the* Eurostat Manual.

Manufacturing has the highest backward linkages, and other services the least. An increased demand for manufacturing usually effects supplier industries. Service industry usually have a high labor input, and their main effect is increased spending of the wages earned in the services.

### Forward linkages 

Forward linkages show the strength of the new business opportunities when industry i starts to increase its production. Whereas backward linkages show the increased demand of the suppliers in industry i, forward linkages show the increased availability of inputs for other industries that rely on industry i as a supplier.

The forward linkages are defined as the sums of the rows in the Ghosh-inverse. The Ghosh-inverse is not explicitly named in the *Eurostat Manual*, but it is described in more detail in the United Nations' similar manual [Handbook on Supply and Use Tables and Input-Output Tables with Extensions and Applications](https://unstats.un.org/unsd/nationalaccount/docs/SUT_IOT_HB_Final_Cover.pdf) (see pp 636--638).

```{r ghoshinverse}
de_out <- output_coefficient_matrix_create(
  data_table = germany_io, 
  total = "final_demand", 
  digits = 4
  )

ghosh_inverse_create(de_out, digits = 4)[,1:4]
```

The Ghosh-inverse is \deqn{G = (I-B)^-1}

where B = the output coefficient matrix.

The forward linkages are the rowwise sums of the Ghosh-inverse


```{r forwardlinkages}
forward_linkages(output_coefficient_matrix = de_out)
```

*You can check the values of the forward linkages against the Table 15.20 on page 507 of* the Eurostat Manual.

## Environmental Impacts

At last, let's extend the input-output system with emissions data. For getting Eurostat's air pollution account data, use `airpol_get()`. We have included in the package the German emissions data from the *Eurostat Manual*.

```{r emissions}
data("germany_airpol")
emissions_de <- germany_airpol[, -3] %>%
  vector_transpose_wider(names_from = "iotables_col", 
                         values_from = "value")
```
```{r emissionsprint}
emissions_de
```

```{r outputbp}
output_bp <- output_get(germany_io)
```

The output coefficients are create from the emission matrix with 
\eqn{B_{ij}} = \eqn{Em_{ij} / output_{j}} 

```{r emmissioncoeffs}
emission_coeffs <- germany_io %>%
  supplementary_add(emissions_de) %>%
  input_indicator_create(input_row = as.character(emissions_de$airpol), digits = 4)
```
```{r emmissioncoeffsprint}
emission_coeffs[-1, 1:3]
```

And the multipliers (which include both the direct and indirect emissions of the industries) are created with 

\eqn{B(I-A)^{-1}}

You can create a single multiplier with `?multiplier_create`.

```{r CO2multiplier}
multiplier_create( 
  input_vector    = emission_coeffs[2,],
  Im              = I_de,
  multiplier_name = "CO2_multiplier", 
  digits = 4 )
```
To create a tidy table of indicators of using a loop:

```{r emissionmultipliers}
names(emission_coeffs)[1] <- names(I_de)[1]
emission_multipliers <- cbind (
  key_column_create(names(emission_coeffs)[1], 
                    gsub("_indicator", "_multiplier", unlist(emission_coeffs[-1,1]))), 
  do.call( rbind, 
           lapply ( 2:nrow(emission_coeffs), 
                    function(x) equation_solve(emission_coeffs[x, ], I_de) )
           ) %>% as.data.frame()
  )

emission_multipliers[, -1] <- round(emission_multipliers[, -1], 4)

emission_multipliers[1:3,1:4]
```

*You can check the results against the Table 15.13 of the* [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0)  *on page 494.*

Because the equation of the final demand is: 

`r paste0("final_demand = ", paste(names(germany_io)[9:13], collapse = " + "))`

we can calculate the final demand for the products of industries with creating the rowwise sums of the appropriate columns

```{r finaldemand}
final_demand <- rowSums(germany_io[1:6, 9:13])
```

And at last the emission content of the final demand is given by


\eqn{Z = B(I-A)^{-1}Y}

where the term 

\eqn{B(I-A)^{-1}}

is the definition of the emission multipliers.


```{r emissioncontent}
emission_content <- as.data.frame(
  round(as.matrix(emission_multipliers[1:3, -1]) %*% diag(final_demand), 0)
)
names(emission_content) <- names(emission_multipliers[,-1])

emission_content <- data.frame ( 
  iotables_row = gsub("_multiplier", "_content", emission_multipliers[1:3,1])
  ) %>%
  cbind(emission_content ) 

emission_content[,1:4]
```
*This final result can be found on the bottom of the page 494 of the* [Eurostat Manual of Supply, Use and Input-Output Tables](https://ec.europa.eu/eurostat/documents/3859598/5902113/KS-RA-07-013-EN.PDF/b0b3d71e-3930-4442-94be-70b36cea9b39?version=1.0).
