Data Destinations
create-google-drive-folder
Creates a sub folder in a parent Google Drive. The parent folder should be shared to with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
datatable-to-dbf-file
Creates DBF files from a datatable.
datatable-to-sqlite-file
Writes a datatable to a SQLite DB file.
json-http-request
Sends a JSON encoded HTTP request.
submit-to-cog-uk-climb
Uploads metadata to COG-UK CLIMB via the Majora API See https://docs.covid19.climb.ac.uk/majora-api.
update-google-spreadsheet
Saves data to a Google Spreadsheet. The Spreadsheet should be shared to with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
upload-file-to-google-drive
Saves a file to a folder in Google Drive. The folder should be shared to with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
upload-files-to-google-drive
Saves a list of files to a folder in Google Drive. The folder should be shared to with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
upload-to-microreact
Creates Microreact projects.
whonet-epicollect-form
Creates an Epicollect5 form from a WHONET lab data.
whonet-file
Creates an Epicollect5 form from a WHONET lab configuration file.
Data Manipulation
calculate-time-difference
Calculates the time difference (in a specified unit of measurement) between two datatable columns.
group-and-slice-rows
Sums rows in specified columns
group-datatable-rows
Groups rows by specified columns and puts them in a collection.
sum-rows
Sums rows in specified columns
time-series-summation
Sums values in specified columns.
Data Sources
ariba-file
Imports a card summary file produced by ARIBA (https://github.com/sanger-pathogens/ariba).
dbf-file
Reads data from a dBASE (`.dbf`) file.
download-file
Downloads a file on the web.
dropbox-file
Imports a file from Dropbox.
epicollect-project
Extracts data from an Epicollect5 project.
figshare-file
Imports a file from Figshare.
ftp-file
Use the either the SSH File Transfer Protocol to File Transfer Protocol to import a file. **FTP server** When specifying the URL for connection to a FTP server use the syntax ``` ftp:://[username:password@]hostname.com/path/to/file.txt ``` username and password are optional. If you omit the `username:password@` then `anonymous` will be used for both username and password **SFTP server** When specifying the URL for connection to a SFTP server use the syntax ``` sftp:://username:password@hostname.com/path/to/file.txt ``` username and password are required.
google-drive-file
Imports a Google sheet and convert it to a datatable.
google-spreadsheet
Loads data from a Google Spreadsheet. Private sheets should be shared to with `data-flo@data-flo.iam.gserviceaccount.com`.
microreact-project
Extracts data from a Microreact project.
mysql-database
Connect to and query a MySQL database and convert the result to a datatable; SELECT statements only.
s3-file
Downloads a file from an S3 endpoint
spreadsheet-file
Imports a spreadsheet file and converts to a datatable. Supported file formats: `.xlsx`, `.xlsm`, `.xlsb`, `.xls`, `.xls`, `.xls`, `.xls`, `.ods`, `.fods`, `.csv`, `.txt`, `.sylk`, `.html`, `.dif`, `.dbf`, `.rtf`, `.prn`, and `.eth`.
sql-server-database
Connect to and query a SQL Server database and convert the result to a datatable; SELECT statements only.
update-epicollect-entries
Imports data to an Epicollect5 project.
whonet-lab-file
Loads data from a WHONET lab configuration file.
Data Tables
add-column
Adds a new column to an existing datatable.
change-column-case
Converts the case of text values of a datatable column.
columns-concatenation
Concatenate a columns values into a new column.
csv-file-to-datatable
Imports a CSV file and converts to a datatable.
csv-to-datatable
Converts a CSV string to a datatable.
datatable-columns
Converts a datatable to a list of said datatables columns.
datatable-to-csv
Converts a datatable to a CSV string.
datatable-to-csv-file
Converts a datatable to a CSV string.
discard-columns
Discards a list of columns from datatable.
extend-datatable
Add a column to a datatable.
fill-series
Fills empty values of a datatable column with series of values.
filter-blank-values
Finds rows in a datatable that match a search string or regular expression.
filter-columns
Removes columns whose name contains a string or matches regular expression from a datatable.
filter-rows
Finds rows in a datatable that match a search string or regular expression.
find-and-replace-column-values
Replaces values in source column with some or all matches of a pattern replaced by a replacement. The pattern can be a test or a regular expression. If pattern is a text, only the first occurrence will be replaced. Replacement values are added to target column.
format-date-column
Splits date values into new columns.
forward-geocoding
Forward geocode a place into latitude & longitude.
gather-rows
Gathers rows in wide format to long format. See http://www.milanor.net/blog/reshape-data-r-tidyr-vs-reshape2.
generate-uuid
Generates and adds UUID values to a datatable column.
join-datatables
Joins rows of two datatables that have the same value for a specified column.
map-columns
Renames an existing column in a datatable.
merge-datatables
Merges rows of two datatables into a new datatable.
remove-columns
Removes specified columns from a datatable.
rename-column
Renames an existing column in a datatable.
rename-columns
Renames an existing column in a datatable.
replace-column-values
Replaces values in source column with some or all matches of a pattern replaced by a replacement. The pattern can be a test or a regular expression. If pattern is a text, only the first occurrence will be replaced. Replacement values are added to target column.
replace-value
Finds and replaces a value in specified columns of a datatable. The pattern can be a test or a regular expression. If pattern is a text, only the first occurrence will be replaced.
reverse-geocoding
Forward geocode a place into latitude & longitude.
row-column-value
Returns the value of datatable column at a specified row.
select-columns
Selects a list of columns from datatable.
slice-datatable
Returns a portion of a datatable into a new datatable selected from `begin` to `end`.
split-column
Splits values in source column into a number of columns using a specified separator text to determine where to make each split.
split-datatable-rows
Finds rows in a datatable that match a search string or regular expression.
split-date-column
Splits date values into new columns.
spread-rows
Spreads rows in long format to wide format. See http://www.milanor.net/blog/reshape-data-r-tidyr-vs-reshape2.
Graph Manipulation
datatable-to-graph
Converts a datatable to a graph.
dot-to-graph
Parses a DOT string as a graph.
force-directed-layout
Applies forces to a graph to produce a layout based on node and edge relationships.
graph-to-dot
Converts a graph to a DOT string.
JSON Manipulation
add-collection-to-collection
Adds a collection to another collection.
add-datatable-to-collection
Adds a datatable to a collection.
add-list-to-collection
Adds a list to a collection.
add-number-to-collection
Adds a number to a collection.
add-text-to-collection
Adds a text to a collection.
create-collection
Creates an empty collection.
json-to-datatable
Creates a datatable from JSON text.
json-to-list
Creates a list from JSON text.
merge-collections
Merges the entries of two collections.
List Manipulation
append-to-list
Adds one element to the end of a list.
datatable-to-list
Creates a list from values of a datatable column.
filter-list
Finds values in a list that match a search string or regular expression.
find-value
Finds a value in a list that match a search string or regular expression.
list-element
Returns an element of list at a specified index.
list-to-datatable
Converts a list to a datatable.
list-to-files
Converts lists of text to a list of files.
merge-lists
Merges two lists into one.
prepend-to-list
Adds one element to the beginning of a list.
replace-text-in-list
Replaces each element in a list of text according to the given pattern, returning the new list
slice-list
Returns a portion of a list into a new list selected from begin to end.
sort-list
Sorts elements in a list.
split-list
Splits a list into two lists at the frist instance of a specified separator element.
unique-list-items
Returns unique list items
Map Manipulation
datatable-to-map
Creates a map from a datatable.
lookup-map-value
Looks up a value in a map.
Text Manipulation
compare-text
Searches for a pattern in a text using a regular expression.
concatenate-text
Concatenates text to the left and/or the right of the input, then returns the combined text as an output.
date-to-text
Formats a date value as a text.
file-to-text
Converts a file to text.
newick-leaf-labels
Gets the leaf labels from a Newick tree.
replace-text
Replaces values in a text with some or all matches of a pattern replaced by a replacement. The pattern can be a test or a regular expression. If pattern is a text, only the first occurrence will be replaced.
search-text
Searches for a pattern in a text using a regular expression.
split-text
Splits a text into a list of texts using a specified separator text to determine where to make each split.
text-to-file
Converts text a text to a file stream.
URL Manipulation
parse-url
Converts an URL to text.
text-to-url
Creates an URL from a input text.
url-builder
Creates an URL from a protocol, hostname, port, and pathname.