Data Destinations
create-google-drive-folder
Creates a sub folder in a parent folder in Google Drive. The parent folder should be shared with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
create-microreact-project
Creates a Microreact project.
datatable-to-dbf-file
Creates DBF files from a datatable.
datatable-to-sqlite-file
Exports a datatable to a SQLite DB file.
json-http-request
Sends a JSON encoded HTTP request.
send-email-message
Sends an email message via a SMTP server.
sqlite-database
Imports data from a SQLite file.
submit-to-cog-uk-climb
Uploads metadata to COG-UK CLIMB via the Majora API See https://docs.covid19.climb.ac.uk/majora-api.
update-google-spreadsheet
Exports data to an existing Google Sheets spreadsheet. Private spreadsheets should be shared with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
update-microreact-project
Updates a project on a Microreact server.
upload-file-to-google-drive
Saves a file to a folder in Google Drive. The folder should be shared to with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
upload-files-to-google-drive
Saves a list of files to a folder in Google Drive. The folder should be shared to with `data-flo@data-flo.iam.gserviceaccount.com` and given `can edit` permission.
upload-to-microreact
Creates Microreact projects.
whonet-epicollect-form
Creates an Epicollect5 form from a WHONET lab data.
whonet-file
Creates an Epicollect5 form from a WHONET lab configuration file.
Data Manipulation
calculate-time-difference
Calculates the time difference (in a specified unit of measurement) between two datatable columns. (reference column - value column)
group-and-slice-rows
Sums rows in specified columns
group-datatable-rows
Groups rows by specified columns and puts them in a collection.
sum-rows
Aggregates values in specified columns grouped by rows.
time-series-summation
Sums values in specified columns.
Data Sources
ariba-file
Imports a card summary file produced by ARIBA (https://github.com/sanger-pathogens/ariba).
dbf-file
Converts data from a dBASE (`.dbf`) file to a datatable.
download-file
Imports a file from the web.
dropbox-file
Imports a file from Dropbox.
epicollect-project
Imports data from an Epicollect5 project.
figshare-file
Imports a file from Figshare (https://figshare.com/).
ftp-file
Use the SSH File Transfer Protocol or File Transfer Protocol to import a file. **FTP server** When specifying the URL for connection to a FTP server use the syntax ``` ftp:://[username:password@]hostname.com/path/to/file.txt ``` username and password are optional. If you omit the `username:password@` then `anonymous` will be used for both username and password **SFTP server** When specifying the URL for connection to a SFTP server use the syntax ``` sftp:://username:password@hostname.com/path/to/file.txt ``` username and password are required.
google-drive-file
Imports a file from Google Drive
google-spreadsheet
Imports data from a Google Sheets spreadsheet. Private sheets should be shared with `data-flo@data-flo.iam.gserviceaccount.com`.
microreact-project
Imports the metadata table from a Microreact project.
mysql-database
Imports data from a MySQL database.
odbc-database
Imports data from an ODBC database.
oracle-database
Imports data from an Oracle Database and returns the results as a datatable.
postgress-database
Imports data from a PostgreSQL database.
read-remote-file
Imports a remote file from a remote network server.
s3-file
Imports a file from an S3 compatible endpoint,
smb-file
Imports a file from a shared network drive using standard SMB/CIFS protocol.
smb-files
Retrieves a list of files from a shared network drive using standard SMB/CIFS protocol
spreadsheet-file
Imports data from a spreadsheet file.
sql-server-database
Imports data from a Microsoft SQl Server database.
update-epicollect-entries
Exports data to an Epicollect5 project.
update-smb-file
Retrieves a file from a SMB/CIFS server.
whonet-lab-file
Loads data from a WHONET lab configuration file.
write-remote-file
Stores a file to a remote network server.
Data Tables
add-column
Adds a new column to an existing datatable.
add-noise
Adds random noise to specified latitude/longitude columns in a datatable.
change-column-case
Converts the case of text values in a datatable column.
columns-concatenation
Concatenate values from two or more columns, and/or adds a text prefix, and/or adds a text suffix into a new column.
compare-columns
Compares values in specified columns.
csv-file-to-datatable
Imports a CSV file and converts to a datatable.
csv-files-to-datatable
Imports a CSV file and converts to a datatable.
csv-to-datatable
Converts delimited text (e.g. comma-separated values) to a datatable.
datatable-columns
Creates a list of datatable column names.
datatable-to-csv
Converts a datatable to a CSV string.
datatable-to-csv-file
Exports a datatable to a CSV file.
discard-columns
Removes a list of columns from datatable.
extend-datatable
Add a column to a datatable.
fill-series
Fills empty values of a datatable column with series of values.
filter-blank-values
Filters blank rows in a datatable column.
filter-columns
Removes columns whose name contains a string or matches a regular expression from a datatable.
filter-rows
Finds rows in a datatable that match a search pattern.
filter-rows-numerically
Finds rows in a datatable that match a numeric comparison using the `>`, `>=`, `<` or `<=` operators.
find-and-replace-column-values
Replaces values in source column with some, or all matches of a text/pattern being replaced by a new text/pattern. The pattern can be a text or a regular expression. If the pattern is a text, only the first occurrence will be replaced. Replacement values are added to 'replace' column.
format-date-column
Formats date/time values within a datatable column.
forward-geocoding
Converts location names in a datatable column to latitude & longitude.
gather-rows
Reshapes rows from wide format to long format. See http://www.milanor.net/blog/reshape-data-r-tidyr-vs-reshape2.
generate-uuid
Generates and adds UUID values to a datatable column.
join-datatables
Joins two datatables based on a common column between them.
map-columns
Renames an existing column in a datatable.
merge-datatables
Appends rows of two datatables into a new datatable based on common column names.
remove-columns
Removes specified columns from a datatable.
remove-duplicate-rows
Removes duplicate rows from a datatable.
rename-column
Renames an existing column in a datatable.
rename-columns
Renames existing columns in a datatable.
replace-blank-values
Replaces blank values in a datatable column.
replace-column-values
Replaces values in a column with some or all matches of a pattern replaced by a new value. The pattern can be a text or a regular expression. If pattern is a text, only the first occurrence will be replaced. Replacement values can be added to a new column.
replace-value
Finds and replaces a value in specified columns of a datatable. The pattern can be a text or a regular expression. If pattern is a text, only the first occurrence will be replaced.
reverse-geocoding
Converts latitude & longitude in datatable columns to location names.
row-column-value
Returns the value of datatable column at a specified row.
select-columns
Selects a list of columns from a datatable in a specified order.
slice-datatable
Selects rows, based on position, from a datatable
sort-datatable
Sorts a datatable by one or more columns
split-column
Splits values in a column into a number of columns using a specified separator to determine where to make each split.
split-datatable-rows
Finds rows in a datatable that match a pattern. The pattern can be a text or a regular expression.
split-date-column
Splits date values into new columns.
spread-rows
Reshapes rows in long format to wide format.
Graph Manipulation
datatable-to-graph
Creates a graph structure from a datatable containing edge data.
dot-to-graph
Creates a graph structure from text in DOT language format
force-directed-layout
Applies a force-directed graph drawing algorithm (https://en.wikipedia.org/wiki/Force-directed_graph_drawing) on a specified graph.
graph-to-dot
Exports a graph as a text file in DOT format.
JSON Manipulation
add-collection-to-collection
Adds a collection to another collection.
add-datatable-to-collection
Adds a datatable to a collection.
add-list-to-collection
Adds a list to a collection.
add-number-to-collection
Adds a number to a collection.
add-text-to-collection
Adds a text to a collection.
create-collection
Creates an empty collection.
json-to-datatable
Creates a datatable from JSON text.
json-to-list
Creates a list from JSON text.
merge-collections
Merges the entries of two collections.
yaml-to-json
Creates JSON text from YAML text.
List Manipulation
append-to-list
Adds one or more text values to the end of a list.
datatable-to-list
Creates a list from values of a datatable column.
filter-list
Finds values in a list that match a pattern.
find-value
Finds a value in a list that matches a search pattern.
list-element
Returns an element of a list at a specified index.
list-to-datatable
Converts a list to a single-column datatable.
list-to-files
Converts a list of text to a list of files.
merge-lists
Appends two lists into one.
prepend-to-list
Adds one element to the beginning of a list.
replace-text-in-list
Replaces each value in a list of text based on a pattern. The pattern can be a text or a regular expression.
slice-list
Selects values, based on position, from a list
sort-list
Sorts elements in a list.
split-list
Splits a list into two lists at the first instance of a specified separator element.
unique-list-items
Removes duplicate values in a list.
Map Manipulation
add-text-to-map
Adds a text value to a map.
datatable-to-map
Creates a map from a datatable.
lookup-map-value
Finds a key in a map and returns the paired (associated) value.
Text Manipulation
compare-text
Searches for a pattern in a text using a regular expression.
concatenate-text
Concatenates two texts
date-to-text
Converts a date/time value to text in a specified format.
file-to-text
Converts a file to text.
newick-leaf-labels
Lists the leaf labels from a Newick tree.
replace-text
Replaces values in a text with some or all matches of a pattern replaced by a new value. The pattern can be a text or a regular expression. If pattern is a text, only the first occurrence will be replaced.
search-text
Searches for a pattern in a text using a regular expression.
split-text
Splits a text into a list of texts using a specified separator text to determine where to make each split.
text-template
Creates text using a Mustache template (https://mustache.github.io/).
text-to-file
Exports text as a file.
URL Manipulation
parse-url
Converts an URL to text.
text-to-url
Creates an URL from an input text.
url-builder
Creates an URL from a protocol, hostname, port, and pathname.