# Simulation Parameter Analysis R Toolkit ApplicatioN: 'spartan'

## Documentation for package ‘spartan’ version 3.0.2

- DESCRIPTION file.
- User guides, package vignettes and other documentation.

## Help Pages

A B C D E F G I K L M N O P R S T U V W

## -- A --

aa_getATestResults | Calculates the A-Test scores observed for all sets, for each sample size
aa_getATestResults_overTime | Get A-Test results for multiple simulation timepoints
aa_graphATestsForSampleSize | Produce a plot for each sample size, showing the A-Test scores for each set of that size
aa_graphSampleSizeSummary | Plots a comparison of the maximum A-Test score for each sample size
aa_sampleSizeSummary | Determines the median and maximum A-Test score observed for each sample size
aa_sampleSizeSummary_overTime | Determines median and maximum A-Test score for each sample size over time
aa_summariseReplicateRuns | Summarise results in set folder structure into one single CSV file
aa_summariseReplicateRuns_overTime | Calculate summary responses for consistency analysis simulations at multiple timepoints
add_parameter_value_to_file | Iterates through the parameters, adding their sampled value to the netlogo experiment file
analysenetwork_structures | Analyse each network structure provided as a potential NN structure
append_time_to_argument | Appends the time to an argument if processing timepoints
atest | Calculates the A-test score for two distributions
a_test_results | Analysed results from tutorial_consistency_set: a-test scores when sets compared

## -- B --

build_curve_results_from_r_object | When developing spartanDB, it became clear curve results may not be in separate files but in one R object. This takes an R object containing curve summaries and builds these into the format spartan requires to perform the analysis

## -- C --

calculate_atest_score | Calculate the A-Test score for a parameter set in comparison with baseline
calculate_fold_MSE | Calculate the mean squared error for this fold in k-fold cross validation
calculate_medians_for_all_measures | Calculate medians for all measures for a simulation parameter result
calculate_prccs_all_parameters | Calculate PRCC values for all parameter-measure pairs
calculate_prcc_for_all_measures | For all measures, calculate the prcc for each parameter
calculate_weights_for_ensemble_model | Internal function to calculate the weights for all emulators in the ensemble
check_argument_positive_int | Check that an argument that should be a positive integer has been specified correctly
check_boolean | Check that an argument that should be a boolean has been specified correctly
check_column_ranges | For aleatory analysis, checks the analysis start and end columns are sensible
check_confidence_interval | Check that a confidence interval is within a specified range
check_consistency_result_type | Check that the user has declared either a file name or an R object
check_double_value_in_range | Check that a double argument is within a specified range
check_filepath_exists | Check that the filepath of required data or output exists
check_file_exist | Checks for the existence of a file
check_file_exists | Check whether a file exists
check_function_dependent_paramvals | Call the correct paramvals check for the calling function, as netlogo & robustness differ
check_global_param_sampling_args | Checks the input values for global parameter sampling techniques
check_graph_output_type | Check the requested graph types are correct (PDF, PNG, TIFF, BMP)
check_input_args | Wrapper function called by all spartan methods to check input pre-execution
check_lengths_parameters_ranges | Check that the lengths of the parameters, minimum values, and maximum values, are equal
check_lhs_algorithm | Check that the chosen lhc sampling algorithm is either normal or optimal.
check_list_all_integers | Check that all objects of a list are integers
check_nested_filepaths | Check that result filepaths under the root directory exist
check_netlogo_parameters_and_values | Checks the netlogo parameters and values are formatted correctly
check_numeric_list_values | Check that two lists are numeric, and the values of one are less than the other
check_package_installed | Check that a required package has been installed
check_parameters_and_ranges | Pre-Check of the parameters and ranges specified for sampling parameter space
check_paramvals_length_equals_parameter_length | Where used in robustness analysis, check that the length of PARAMVALS equals
check_robustness_parameter_and_ranges_lengths | Where used, checks that PARAMETERS, PMIN, PMAX, PINC, and BASELINE are all the same length
check_robustness_paramvals_contains_baseline | Checks that the parameter values specified in PARAMVALS contain the BASELINE
check_robustness_range_contains_baseline | Checks that the range specified by PMIN and PMAX contains the BASELINE
check_robustness_range_or_values | For robustness, check whether using PMIN/PMAX/PINC entry or PARAMVALS
check_robustness_sampling_args | Pre-execution checks to perform before the spartan robustness samplng technique is executed. Checks all parameter input
check_text | Check that an argument that should be a text label has been specified correctly
check_text_list | Check that an arguments of a list that should be a text label has been specified correctly
close_and_write_netlogo_file | Close the current netlogo sample file and write out
compare_all_values_of_parameter_to_baseline | For one parameter, compare responses for all values with those at baseline
construct_result_filename | Appends the time to an eFAST argument, if processing multiple timepoints
createAndEvaluateFolds | Create and evaluate folds within k-fold cross validation
createtest_fold | Create test data fold for k-fold cross validation
createTrainingFold | Create training data fold for k-fold cross validation
create_abc_settings_object | Creates ensemble-specific parameters for ABC analysis
create_ensemble | Internal function to create the ensemble
create_neural_network | Create neural network emulator, using neuralnet package

## -- D --

dataset_precheck | Before partitioning data, removes any columns where the value is all equal, or all NA
determine_optimal_neural_network_structure | Determine the optimal hidden layer structure from those provided

## -- E --

efast_generate_medians_for_all_parameter_subsets | Generates summary file for stochastic simulations stored in multiple files
efast_generate_medians_for_all_parameter_subsets_overTime | Pre-process analysis settings if multiple timepoints are being considered
efast_generate_sample | Generates parameter sets for variance-based eFAST Sensitivity Analysis
efast_generate_sample_netlogo | Prepares Netlogo experiment files for a variance-based sensitivity analysis, using eFAST
efast_get_overall_medians | Calculates the summary stats for each parameter set (median of any replicates)
efast_get_overall_medians_overTime | Pre-process analysis settings if multiple timepoints are being considered
efast_graph_Results | Plot the parition of variance in a simulation response for each measure
efast_netlogo_get_overall_medians | Deprecated: Use 'efast_netlogo_get_overall_medians'
efast_netlogo_run_Analysis | Deprecated: Use 'efast_run_Analysis'
efast_process_netlogo_result | Analyses Netlogo simulation data for parameter sets generated for eFAST
efast_run_Analysis | Runs the eFAST Analysis for the pre-generated summary file
efast_run_Analysis_from_DB | Runs the eFAST Analysis for a set of results stored in a database
efast_run_Analysis_overTime | Pre-process analysis settings if multiple timepoints are being considered
emulated_lhc_values | Latin-hypercube value set use to demonstrate emulated sensitivity analysis
emulate_efast_sampled_parameters | Emulate simulations for a set of eFAST generated parameter values
emulate_lhc_sampled_parameters | Emulate simulations for a set of latin-hypercube generated parameter values
emulation_algorithm_settings | Initialise machine-learning algorithms settings for emulation creation
emulator_parameter_evolution | Evolve parameter sets that meet a desired ensemble outcome
emulator_predictions | Used to generate predictions from an emulator, normalising data if required
ensemble_abc_wrapper | Wrapper to allow EasyABC functions to run using Ensemble
execute_checks | Executes the list of check functions compiled for the calling function
exemplar_sim_output | Example of a dataset output from an agent-based simulation, used in package testing

## -- F --

format_efast_result_for_output | Joins the various results objects into an output ready format

## -- G --

generate_a_test_results_header | Generates the CSV file header for the A-Test results file
generate_a_test_score | Take the first set and compare it to a distribution from another set using the A-Test
generate_efast_parameter_sets | Use the eFAST approach to generate parameter sets
generate_emulators_and_ensemble | Generate a set of emulators and combine into an ensemble
generate_ensemble_from_existing_emulations | Generate an ensemble from previously created spartan emulation objects
generate_ensemble_training_set | Internal function used to combine test set predictions from emulators to form the ensemble training set
generate_headers_for_atest_file | Generates headers for the A-Test summary CSV and R Object
generate_list_of_checks | Defines which functions to call to check an input argument.
generate_medians_for_param_set | Generate the median responses for a set of parameter values
generate_parameter_table | Takes the value list and generates the sample that is output to csv file
generate_prcc_results_header | Generates the CSV file header for the prcc results file
generate_requested_emulations | Generate emulators for specified machine learning techniques with provided data
generate_sensitivity_indices | Generate eFAST Sensitivity Indices
generate_summary_stats_for_all_param_sets | Generate summary statistics for each value of all parameters in this analysis
get_argument_correct_case | Tries upper and lower case names for input arguments
get_correct_file_path_for_function | Gets the correct filepath for the column range input checker
get_file_and_object_argument_names | Gets the correct file and R object argument names for the input checker
get_max_and_median_atest_scores | Return the max and median A-Test score for all measures for a sample size
get_medians_for_size_subsets | For a given sample size, get the median results to summarise results for all sets
get_median_results_for_all_measures | For a model result, calculate the medians of the desired measures
graph_Posteriors_All_Parameters | Graph posterior distributions generated for all parameters, to PDF file
graph_sample_size_results | Graph the A-Test results for a sample size

## -- I --

import_model_result | Import a model result from either a CSV or XML file
initialise_netlogo_xml_file | Initialises the Netlogo setup file for this experiment

## -- K --

kfoldCrossValidation | Perform k-fold cross validation for assessing neural network structure performance

## -- L --

lhc_calculatePRCCForMultipleTimepoints | Calculates the PRCC for each parameter at each timepoint, storeing PRCC and P-Value in two different files to make the plot function easier
lhc_generateLHCSummary | Summarises simulation behaviour for each parameter set, by median of distribution of replicate runs
lhc_generateLHCSummary_overTime | Pre-process analysis settings if multiple timepoints are being considered
lhc_generatePRCoEffs | Generate Partial Rank Correlation Coefficients for parameter/response pairs
lhc_generatePRCoEffs_db_link | Generate Partial Rank Correlation Coefficients for parameter/response pairs for results in database
lhc_generatePRCoEffs_overTime | Pre-process analysis settings if multiple timepoints are being considered
lhc_generateTimepointFiles | Generates spartan-compatible timepoint files if simulation results over time are in one file
lhc_generate_lhc_sample | Generates sets of simulation parameters using latin-hypercube sampling
lhc_generate_lhc_sample_netlogo | Prepares Netlogo experiment files for a sampling-based sensitivity analysis, using latin-hypercube sampling
lhc_generate_netlogo_PRCoEffs | Deprecated. Use 'lhc_generatePRCoEffs' instead
lhc_graphMeasuresForParameterChange | Generates parameter/measure plot for each pairing in the analysis
lhc_graphMeasuresForParameterChange_from_db | Generates parameter/measure plot for each pairing in the analysis, from results stored in a database
lhc_graphMeasuresForParameterChange_overTime | Wrapper for graphing LHC results for multiple timepoints
lhc_netlogo_graphMeasuresForParameterChange | Deprecated. Use 'lhc_graphMeasuresForParameterChange' instead
lhc_plotCoEfficients | Plots the PRCC coefficients against each other for ease of comparison
lhc_polarplot | Creates a polar plot for each response, showing PRCC for each parameter
lhc_process_netlogo_result | Analyses Netlogo simulations generated for a latin-hypercube based sensitivity analysis
lhc_process_sample_run_subsets | Summarises results of runs for parameter sets generated by a latin-hypercube
lhc_process_sample_run_subsets_overTime | Pre-process analysis settings if multiple timepoints are being considered

## -- M --

make_graph_title | Make graph title, sub title, and file name
make_lhc_plot | Make the LHC output plot

## -- N --

normaliseATest | Normalises the A-Test such that it is above 0.5
normalise_dataset | Normalise a dataset such that all values are between 0 and 1
nsga2_set_user_params | Initialise analysis specific parameters for NSGA-2
num.decimals | Diagnostic function used to determine number of decimal places

## -- O --

oat_csv_result_file_analysis | Performs a robustness analysis for supplied simulation data, comparing simulation behaviour at different parameter values
oat_csv_result_file_analysis_from_DB | Performs a robustness analysis for simulation results stored in a database, comparing simulation behaviour at different parameter values
oat_csv_result_file_analysis_overTime | Pre-process analysis settings if multiple timepoints are being considered
oat_generate_netlogo_behaviour_space_XML | Creates a Netlogo compatible behaviour space experiment for robustness analysis
oat_graphATestsForSampleSize | Takes each parameter in turn and creates a plot showing A-Test score against parameter value.
oat_parameter_sampling | Create parameter samples for robustness (local) analysis
oat_plotResultDistribution | For stochastic simulations plots the distribution of results for each parameter value
oat_processParamSubsets | Summarises stochastic, repeated, simulations for all robustness parameter sets into a single file.
oat_processParamSubsets_overTime | Summarises stochastic, repeated, simulations for all robustness parameter sets into a single file, for multiple timepoints
oat_process_netlogo_result | Takes a Netlogo behaviour space file and performs a robustness analysis from that simulation data
output_ggplot_graph | Output a ggplot graph in the requested formats
output_param_sets_per_curve | Output the generated parameter sets for each curve

## -- P --

partition_dataset | Partition latin-hypercube summary file to training, testing, and validation
perform_aTest_for_all_sim_measures | Performs A-Test to compare all simulation measures
plotATestsFromTimepointFiles | Plots the A-Tests for all timepoints being examined
ploteFASTSiFromTimepointFiles | Plot the Si value for all parameters for multiple simulation timepoints
plotPRCCSFromTimepointFiles | Plots Graphs for Partial Rank Correlation Coefficients Over Time
plot_compare_sim_observed_to_model_prediction | Internal function used to create accuracy plots of the emulation against observed data
process_netlogo_parameter_range_info | Processes netlogo parameter information to generate names of those of interest to this analysis
process_parameter_value_if_exists | Process parameter value set if results exist
produce_accuracy_plots_all_measures | Internal function used to create accuracy plots of the emulation against observed data, for all measures
produce_accuracy_plots_single_measure | Internal function used to create accuracy plots of the emulation against observed data
produce_atest_score_summary | Generates A-Test score summary for all sample sizes
produce_summary_for_all_values_of_parameter | For one parameter, evaluate the results of all values that parameter can take

## -- R --

read_all_curve_results | Reads results from each curve into a multi-dimensional array
read_from_csv | To save retyping all options, function to read CSV data
read_model_result_file | Reads a model result file, either CSV or XML
read_simulation_results | Read in the simulation results either from a file, or R object The existance of these results was checked in pre-execution checks
retrieve_results_for_comparison_result_set | Get the first result set, to which all others are compared

## -- S --

sample_parameter_space | Generate the LHC design for the chosen algorithm
scale_lhc_sample | Scale the LHC design to be the range explored for each parameter
screen_nsga2_parameters | Screens NSGA-2 related parameters, guiding which to select for evolving parameter sets
selectSuitableStructure | Selects the most suitable neural network structure from the potentials made
set.nsga_sensitivity_params | Set parameters for NSGA-2 sensitivity analysis
sim_data_for_emulation | Set of parameter and response pairs for training an emulator of a simulation
summarise_lhc_sweep_responses | Processes an LHC sample, returning summary stats for all parameter sets
summarise_replicate_runs | Summarises replicate runs of a parameter set. Used by LHC and eFAST

## -- T --

tutorial_consistency_set | Example dataset showing the structure for consistency analysis data

## -- U --

updateErrorForStructure | Add the MSE for a newly examined structure to the list of those already seen
use_ensemble_to_generate_predictions | Predict simulation responses for a parameter set using an ensemble

## -- V --

visualise_data_distribution | Used to diagnose skew in a training dataset before use in emulation

## -- W --

weight_emulator_predictions_by_ensemble | Internal function to weight emulator predictions by that calculated for the ensemble
write_data_to_csv | Shortcut function for writing data to CSV file