Run automatic quality control on a deployment

qc_deployment() will:

1. Read calibrated dissolved oxygen and conductivity data from one or more files. See import types below for format information.

2. Read metadata from a YAML or text file.

3. Combine and reformat the data.

4. Add automatic flags, and create columns for manual QC

5. Write the data and metadata.

6. If report = TRUE it will call make_deployment_report() to make and interactive HTML report with plots for QAQC.

Usage

qc_deployment(dir, report = TRUE)

Arguments

dir

The path to a deployment directory to run QC on. It should have a Calibration/ directory with calibrated DO and conductivity data associated Details files generated by HOBOware.

report

If TRUE (the default) an html report will be generated with plots to assist the QAQC process. Set to FALSE to skip generating the report. See make_deployment_report() to explicitly make the report as a separate step.

Value

A list with items:

d

The final tabular data as written to disk.

md

A nested list of the metadata.

Details

qc_deployment() calls import_calibrated_data() which currently supports three completely different import functions.

In all cases input files are looked for within a Calibration sub directory to the deployment directory (dir).

Import Types

Import Type 0 - Single CSV

The Single CSV import is a fallback in case updates to HOBOware or the loggers themselves break the other imports. It expects a single CSV file that contains data for both Salinity and DO and a YAML (.yml) file - described below.

To use this import:

1. Make sure in import_types.csv there's a default line: default,0 This means that if an import type isn't identified for a model listed in a placement that the Single CSV import will be used.

2. In placments.csv if you want to use the CSV import use a model name that is NOT in import_types.csv so the default CSV import is used. I recommend using the model name with -CSV appended; e.g. instead of MX801 use MX801-CSV.

3. In the deployment's Calibrated directory create an appropriate CSV and YAML file as described below.

CSV file

The CSV file would likely be created with external code or manually from calibrated output from HOBOWare.

In the CSV file the columns are resolved by name not order and the import will attempt to resolve several different column naming conventions. Any of the following should work:

• The canonical column names used by this package: "Date_Time", "Raw_DO", "Temp_DOLog", "DO", "DO_Pct_Sat", "High_Range", "Temp_CondLog", "Spec_Cond", "Salinity", "Depth", "Latitude", and "Longitude". The last three are optional, the rest are required.

• Columns names from the MX801 logger. For example by saving the first sheet as a CSV.

• Column names from the U24 and U26 loggers. However, when combining data from these two loggers into a single CSV the two temperature columns will have to be manually renamed to "Temp_CondLog", and "Temp_DOLog" as they are otherwise indistinguishable.

Please verify carefully that the various DO, Conductivity, and Salinity columns are properly identified. For example make sure the resulting "DO" column has the calibrated values and the "Raw_DO" column has the un-calibrated values.

YAML file

In the YAML file the following items are required:

• calibration_start: The date and time of the start of the deployment. For field calibrated sensors (U24, U26) this is also the calibration time. For factory calibrated sensors (MX801) this is NOT the calibration time.

• calibration_end: As in the above, the end of the deployment and/or calibrated window.

• timezone: The timezone as reported by the logger and/or calibration software. The output from HOBOware uses a GMT offset like "GMT-04:00", which is not a broadly supported timezone but is accepted here. The MX801 uses a timezone code "EST" which is also accepted here.

• do_device: Information on the DO sensor or logger with items:

  • product: The dissolved oxygen sensor e.g. "HOBO U26-001 Dissolved Oxygen", "U26-01", or "MX801".

  • serial_number: The device serial number.

• cond_device: List with information on the conductivity sensor with items:

  • product: The conductivity sensor e.g. "HOBO U24-002 Conductivity"

  • serial_number: Conductivity sensor serial number Additional items that appear in the metadata documentation are permitted and will be retained. Items that do not appear in that document will be ignored.

Import Type 1 - U24, U26

This input works with HOBOware calibrated CSV files from U24 and U26 loggers, and expects to find two CSV files and two details.txt files in the calibration directory. The Dissolved Oxygen files should have "Do_" somewhere in the names while the Conductivity files should have either "Cond_" or "Sal_ in the names.

Do not include "_TR_" or "_TR." in the file names as they are used to indicate Tide Rider location data.

Import Type 2 - MX801

This is the import type for the MX801 logger, it expects a .xlxs file with combined data from both loggers and on the first sheet, and a details.yml file in the calibrated directory. The YAML file will be created by users and should have this format:

calibration_start: 2025-01-02 15:50:02
calibration_end: 2025-01-04 13:00:02
timezone: EST
serial_number: 22145899

calibration_start and calibration_end define the start and end of the deployment window - even if no calibration took place. This is a legacy of the U24 and U26 loggers where the calibrated window and the deployment window were the same.

The path to an example file can be retrieved with: system.file("extdata/2025/BBC/2025-01-04/Calibrated/details.yml", package = "BuzzardsBay")

In placements.csv there should be two lines for the MX801 import. One each for "DO" and "Cond" both of which should indicate "MX801" as the model and have the same Serial Number.

To create a local copy of example data for the MX801 import run this in R:

example_paths <- setup_example_dir(site_filter = "BBC",
                                  year_filter = 2025)
deployment_dirs <- example_paths$deployments
print(deployment_dirs)

To run QC on example data:

qc_deployment(deployment_dirs[1])

Tide Rider Import

Tide Riders are mobile sensor platforms that actively control depth in the water column and move laterally with the tides.

Tide Rider data can be included with any of the above imports, although currently we only anticipate Tide Riders being equipped with the U24 and U26 loggers.

Each Tide Rider should be treated as a independent "Site" with a unique site ID that should be in the sites and placements tables. Tide riders do not need to be in the import_types table and the serial number for tide riders is not required in nor used from the placements table.

Tide rider data should be included along with the DO and Conductivity data in the calibrated directory for the "site". For example "2025/TRS102/2025-08-14/Calibrated" would hold both the logger and tide rider data for the TRS102 tide rider ("site") for the deployment ending on 2025-08-14. The Tide Rider data should be in a CSV file with"TR" in the name isolated from other characters with "_" and/or "." e.g. "data_TR.csv"; beginning the file with "TR" does not count it should not have "Cond_","DO_", or "Sal_" in the name as those indicate the calibrated CSV logger files.

The expected columns are listed below. Alternate names are allowed in the CSV but will be renamed on import to the standard name. Names used in the original example file are bold.

Standard Name	Alternates allowed in input
Date_Time	Time
Latitude	(none)
Longitude	(none)
Depth	Logger Depth, Logger Depth (m)
TR_Flags	Flags

The "TR_Flags" column is required for but not used.

Examples

if (FALSE) { # \dontrun{
  paths <- setup_example_dir()
  a <- qc_deployment(paths$deployment)
} # }