---
title: "Custom aggregation"
author: "Victor Granda (Sapfluxnet Team)"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Custom aggregation}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

`sapfluxnetr` package offers a very flexible but powerful system based on
`tidyverse` and `tibbletime` packages to aggregate and summarise the site/s data
in the form of the `sfn_metrics` functions. All the metrics family of functions
(`?metrics`) make use of `sfn_metrics` under the hood. If you want full control
to the statistics returned and aggregation periodswe recommend you to use this
main function. This vignette will show you how.

## Pre-fixed summarising functions

1. `daily_metrics`
1. `monthly_metrics`
1. `predawn_metrics`
1. `midday_metrics`
1. `nightly_metrics`
1. `daylight_metrics`

See each function help for a detailed description and examples of use.

## Custom summarising functions

`daily_metrics` and related functions return a complete set of metrics ready for
use, but if you want simpler metrics and/or avoid the computational burden of
returning all the pre-fixed metrics you can supply your own summarising functions
using the `funs` function belonging to the `dplyr` package in the `tidyverse`
suite:

```{r}
# libraries
library(sapfluxnetr)
library(dplyr)

### only mean and sd at a daily scale
# data
data('ARG_TRE', package = 'sapfluxnetr')

# summarising funs (built with funs function from dplyr package)
custom_funs <- funs(mean = mean(., na.rm = TRUE), std_dev = sd(., na.rm = TRUE))

# metrics
foo_simpler_metrics <- sfn_metrics(
  ARG_TRE,
  period = 'daily',
  .funs = custom_funs,
  solar = TRUE,
  interval = 'general'
)

foo_simpler_metrics[['sapf']]
```

You can also select if the "special interest" intervals (predawn, midday,
nighttime or daylight) are calculated or not. For example, if you are only
interested in the midday interval you can use:

```{r}
foo_simpler_metrics_midday <- sfn_metrics(
  ARG_TRE,
  period = 'daily',
  .funs = custom_funs,
  solar = TRUE,
  interval = 'midday', int_start = 11, int_end = 13
)

foo_simpler_metrics_midday[['sapf']]
```

> When supplying only one function names of variables are not changed to
  contain the metric name at the end, as the summary function returns the
  same columns as the original data

## Custom aggregation periods

`period` argument in `sfn_metrics` is passed to `collapse_index` function in the
`tibbletime` package, and so, it can use the same input:

  + "frequency period" format, where frequency is a number and period is an
    interval as character (i.e. "1 year" same as "yearly", "7 days")

```{r}
# weekly
foo_weekly <- sfn_metrics(
  ARG_TRE,
  period = '7 days',
  .funs = custom_funs,
  solar = TRUE,
  interval = 'general'
)

foo_weekly[['env']]
```

 + A vector of dates as period boundaries. This way you can build irregular
   or custom periods.

```{r}
# custom
foo_custom <- sfn_metrics(
  ARG_TRE,
  period = as.POSIXct(
    c('2009-11-17 00:00:00', '2009-11-22 14:00:00', '2009-11-30 23:59:00'),
                      tz = 'UTC'
  ),
  .funs = custom_funs,
  solar = TRUE,
  interval = 'general'
)
foo_custom[['sapf']]
```

## Extra parameters

`sfn_metrics` has a `...` parameter intended to supply additional parameters to
the internal functions used:

  1. `tibbletime::collapse_index` accepts the following extra arguments:
     
     - `start_date`  
        NULL by default in the sfn_metrics implementation
     - `side`  
        "start" by default in the sfn_metrics implementation
     - `clean`  
        TRUE by default in the sfn_metrics implementation
  
  1. `dplyr::summarise_all` accepts extra arguments intended to be applied to
     the summarising functions provided (to **all**, so they all must have the
     argument provided or an error will be raised). That's the reason because we
     recommend to use the `funs` function from `dplyr`, as the arguments are
     specified for the individual functions.

For example, if we want the TIMESTAMPs after aggregation to show the end of the
period instead the beggining (default) we can do the following:

```{r}
foo_simpler_metrics_end <- sfn_metrics(
  ARG_TRE,
  period = 'daily',
  .funs = custom_funs,
  solar = TRUE,
  interval = 'general',
  side = "end"
)

foo_simpler_metrics_end[['sapf']]
```

If it is compared with the `foo_simpler_metrics` calculated before, now the
period is identified in the TIMESTAMP by the ending of the period (daily in this
case).

## Temporary columns helpers

The internal aggregation process in `sfn_metrics` generates some transitory
columns which can be used in the summarising functions:

### `TIMESTAMP_coll`

When aggregating by the declared period (i.e. `"daily"`), the TIMESTAMP column
collapses to the period start/end value (meaning that all the TIMESTAMP values
for the same day becomes identical).  
This makes impossible to use any summarise functions that obtain the
time of the day at which one event happens (i.e. time of the day at which the
maximum sap flow occurs) because all TIMESTAMP values are identical.
For that kind of summarising functions, a transitory column called
`TIMESTAMP_coll` is created. So in this case we can create a function that
takes de variable values for the day, the TIMESTAMP_coll values for the day
and return the TIMESTAMP at which the max sap flow occurs and use it with
`sfn_metrics`:

```{r timestamp_coll}
max_time <- function(x, time) {
  
  # x: vector of values for a day
  # time: TIMESTAMP for the day 

  # if all the values in x are NAs (a daily summmarise of no measures day for
  # example) this will return a length 0 POSIXct vector, which will crash
  # dplyr summarise step. So, check if all NA and if true return NA as POSIXct
  if(all(is.na(x))) {
    return(as.POSIXct(NA, tz = attr(time, 'tz'), origin = lubridate::origin))
  } else {
    time[which.max(x)]
  }
}

custom_funs <- funs(max = max(., na.rm = TRUE), max_time(., TIMESTAMP_coll))

max_time_metrics <- sfn_metrics(
  ARG_TRE,
  period = 'daily',
  .funs = custom_funs,
  solar = TRUE,
  interval = 'general'
)

max_time_metrics[['sapf']]
```

