%\VignetteEngine{knitr::knitr}
%\VignetteIndexEntry{An intorduction to the stoRy package}
\documentclass[11pt,BCOR2mm,DIV14]{scrartcl}
\usepackage{booktabs}%For fancy tables
\usepackage[round]{natbib}% References
\usepackage{amsmath,amsfonts}
\usepackage{scrpage2}
%\usepackage{algorithm}
%\usepackage[utf8]{inputenc} %unicode support
\usepackage{color,hyperref}
\urlstyle{rm}

\usepackage{xcolor}
\hypersetup{
    colorlinks,
    linkcolor={red!50!black},
    citecolor={blue!50!black},
    urlcolor={blue!80!black}
}

\pagestyle{scrheadings}  
\setheadsepline{.4pt}
\ihead[]{An overview of the \cc{stoRy} package}
\chead[]{}
\ohead[]{}
\ifoot[]{}
\cfoot[]{}
\ofoot[\pagemark]{\pagemark}
\newcommand{\cc}{\texttt}

<<echo=FALSE, message=FALSE>>=
library(knitr)
library(stoRy)
options(replace.assign=FALSE)

opts_chunk$set(fig.path='knitr_figure_poweRlaw/graphics-', 
               cache.path='knitr_cache_poweRlaw/', 
               fig.align='center', 
               dev='pdf', fig.width=5, fig.height=5, 
               fig.show='hold', cache=FALSE, par=TRUE,
               out.width='0.4\\textwidth')
knit_hooks$set(crop=hook_pdfcrop)

knit_hooks$set(par=function(before, options, envir){
  if (before && options$fig.show!='none') {
    par(mar=c(3,4,2,1),cex.lab=.95,cex.axis=.9,
        mgp=c(3,.7,0),tcl=-.01, las=1)
  }}, crop=hook_pdfcrop)

set.seed(1)
palette(c(rgb(170,93,152, maxColorValue=255),
          rgb(103,143,57, maxColorValue=255),
          rgb(196,95,46, maxColorValue=255),
          rgb(79,134,165, maxColorValue=255),
          rgb(205,71,103, maxColorValue=255),
          rgb(203,77,202, maxColorValue=255),
          rgb(115,113,206, maxColorValue=255)))
@
% 
\begin{document}
\date{Last updated: \today}  
\title{The stoRy package: a general overview}
\author{Paul Sheridan and Mikael Onsj\"o}

\maketitle
\begin{abstract}
\noindent The \cc{stoRy} package implements 1) the hypergeometric test for over-representation of literary themes in a storyset (a list of stories) relative to a background list of stories, and 2) a recommendation system that takes a user-selected story as input and returns a ranked list of similar stories on the basis of shared themes. The package is currently implemented for the episodes of the Star Trek television franchhise series The Original Series (TOS), The Animated Series (TAS), The Next Generation (TNG), and Voyager (VOY).
\end{abstract}

\section{Installation}

The package is hosted on CRAN and can be installed by running the command
<<installation, eval=FALSE>>=
install.packages("stoRy")
@
\noindent Once installed, the package can be loaded by running the standard \cc{library} command
<<loading, eval=FALSE>>=
library("stoRy")
@

\section{Accessing documentation}

Each function in the package is documented. Run the command
<<eval=FALSE>>=
help(package="stoRy")
@
\noindent to see a brief overview of the package functions. The package vignette can be accessed by running
<<eval=FALSE>>=
vignette(package="stoRy")
@
\noindent Function help files can be accessed using the usual R command. For example, help on the method \verb$get_enriched_themes$ can be obtained with
<<tidy=FALSE, eval=FALSE>>=
?get_enriched_themes
@

\section{Example: View and edit a story's themes and metadata}

The \cc{stoRy} package contains themed Star Trek episodes with additional metadata for The Original Series (TOS), The Animated Series (TAS), The Next Generaton (TNG), and Voyager (VOY). A \verb$story$ object is created in order to view the themes (and metadata) for a particular episode of interest. For example, to view the data for the classic TOS episode ``Arena'' (story ID tos1x19), initialize a \verb$story$ object as follows
<<eval=FALSE>>=
story_id <- "tos1x19"
mystory <- story$new(story_id)
print(mystory)
@
\noindent Annotated themes are stored in the data frame object
<<eval=FALSE>>=
mystory$themes
@
\noindent Each theme is categorized as being either \emph{central} or \emph{peripheral} to the story; comments along with related characters, aliens, and things are also recorded. Annotated story settings and keywords can be accessed in a similar manner
<<eval=FALSE>>=
mystory$settings
mystory$keywords
@
\noindent Themes can be added and removed as desired. For example, run the following command to add the theme ``neo-luddist utopia'' as a central theme
<<eval=FALSE>>=
mystory$add_theme(theme = "neo-luddist utopia", level = "central")
@
\noindent The theme can be removed as follows
<<eval=FALSE>>=
mystory$remove_theme(theme = "neo-luddist utopia")
@
\noindent Settings and keywords can be added and removed in a similar manner
<<eval=FALSE>>=
mystory$add_setting(setting = "mountain")
mystory$remove_setting(setting = "mountain")
mystory$add_keyword(keyword = "Captain Kirk is climbing a mountain")
mystory$remove_keyword(keyword = "Captain Kirk is climbing a mountain")
@
\noindent There are $278$ themed episodes in total. The raw story IDs can be viewed by loading \verb$sysdata$ and running the command
<<eval=FALSE>>=
load("R/sysdata.rda")
sysdata$RESERVED_STORY_IDS
@
\noindent The following command sequence shows the first ten story IDs along with associated metadata
<<eval=FALSE>>=
load("R/sysdata.rda")
head(sysdata$story_metadata)
@

\section{Example: Exploring the themes}

\noindent It is possible to examine the individual themes contained in the \cc{stoRy} package collection. For example, to examine the theme ``utopia" run the following command sequence
<<eval=FALSE>>=
theme_name <- "utopia"
mytheme <- theme$new(theme_name)
mytheme$print()
@
\noindent The output includes the theme definition, sometimes an illustrative example, and its place in a theme hierarchy. In total, there are $1535$ different themes that are arranged into four hierarchies: the human condition, society, the pursuit of knowledge, and alternate reality. The theme ``utopia'' falls withing the society hierarchy. To view ``utpoia'' and its descendents in a tree format run the command
<<eval=FALSE>>=
print_tree(mytheme)
@
\noindent The themes from each of the four hierarchies can be viewed in the same manner. For example, for an overview of themes in the society hierarchy run the following command sequence  
<<eval=FALSE>>=
theme_name <- "society"
mytheme <- theme$new(theme_name)
print_tree(mytheme, pruneMethod = "dist", limit = 50)
@
\noindent Setting \verb$pruneMethod$ in this manner ensures that the upper levels in the hierarcy are displayed; the value of \verb$limit$ determines how many themes will be displayed.

\section{Example: Enriched themes in Klingon episodes}

This section is devoted to an example usage of the hypergeometric test for theme over-representation analysis. In the analysis, the hypergoemetric test is applied to identify over-represented themes in Klingon-centric episodes relative to the backgrounds TOS/TAS and TNG, respectively. First, read in the \verb$aliens.smt$ storysets file
<<eval=FALSE>>=
file <- system.file("storysets", "aliens.smt", package = "stoRy")
mystorysets <- storysets$new(file)
print(mystorysets)
@
\noindent The \verb$get_enriched_themes$ function performs the hypergeometric test to check whether each of the $2129$ individual themes is over-represented in a test sotryset relative to the background storyset. The \verb$get_enriched_themes$ function applies the test separately to each storyset in the \verb$mystorysets$ object relative to a background of all $446$ Star Trek television series episodes by default. To test the Klingon storyset against a background of all TOS and TAS series episodes run the command
<<eval=FALSE>>=
results <- get_enriched_themes(mystorysets, 
               test_storysets = "KLINGON",
               background_storyset = c("tos", "tas"))
@
\noindent The top twenty enriched themes can be viewed as follows
<<eval=FALSE>>=
results$KLINGON[1:20,]
@
\noindent In the data frame, $n$ is the size of the test storyset, $k$ is the number of stories in the test storyset featuring the associated theme, $N$ is the size of the background storyset, and $K$ is the number of stories in the background storyset featuring the associated theme. The P-value is calculated using the hypergeometric test.

Run the following command sequence to find over-represented themes in Klingon-centric episodes when the TNG series episodes are used as a background storyset
<<eval=FALSE>>=
results <- get_enriched_themes(mystorysets, 
               test_storysets = "KLINGON",
               background_storyset = "tng")
results$KLINGON[1:20,]
@

\section{Example: Enriched themes in the Star Trek TV series}

Here we demonstrate another example usage of the \verb$get_enriched_themes$ hypergeometric test for theme over-representation analysis. This time the hypergoemetric test is applied to identify over-represented themes in each series TOS, TAS, TNG, VOY relative to the background TOS/TAS/TNG/VOY. First, read in the \verb$series.smt$ storysets file
<<eval=FALSE>>=
file <- system.file("storysets", "series.smt", package = "stoRy")
mystorysets <- storysets$new(file)
print(mystorysets)
@
\noindent The \verb$get_enriched_themes$ function performs the calculation for each series relative to the background TOS/TAS/TNG/VOY by default
<<eval=FALSE>>=
results <- get_enriched_themes(mystorysets)
@
\noindent The results can be accessed as follows
<<eval=FALSE>>=
results$TOS[1:20,]
results$TAS[1:10,]
results$TNG[1:20,]
results$VOY[1:20,]
@

\section{Example: Finding episodes most similar to a selected episode}

The function \verb$get_similar_stories$ can be used to find episodes similar to a user-selected episode. Take finding episodes similar to the Voyager episode ``False Profits'' (\verb$voy3x05$) as an example. First, create a story object for the story in question
<<eval=FALSE>>=
story_id <- "voy3x05"
mystory <- story$new(story_id)
@
\noindent The following command evaluates episodes in the default background of TOS/TAS/TNG/VOY according to their similarity to \verb$mystory$. The default cosine similarity function can be changed to either the cosine tf-idf or soft cardinality similarity function (see the \verb$get_similar_stories$ help file for details).
<<eval=FALSE>>=
result <- get_similar_stories(mystory)
@
\noindent The top $10$ most similar episodes can be accessed as follows
<<eval=FALSE>>=
result[1:10,]
@

\end{document}