sda.api.load_data_pg#

Load timeseries data from a PostgreSQL database (Clever Cloud).

Each test is stored as a set of tables named timeseries_{tag}_{instrument}. load_data discovers all such tables for a given tag, queries the time column and every measurement column from each table, then merges them into a single wide DataFrame aligned on a rounded time axis (0.1 s precision).

Database credentials are read from ~/sda.json using the keys DB_USER, DB_PASSWORD, DB_HOST, DB_PORT, and DB_NAME.

Functions#

has_local_db_credentials()

Return True when all required DB_* keys are present in env vars or ~/sda.json.

load_commentary(tag[, start_time, end_time])

Load commentary entries for tag from the commentary table.

load_data(tag[, start_time, end_time])

Load all timeseries for a test and return a wide, time-aligned DataFrame.

Module Contents#

sda.api.load_data_pg.has_local_db_credentials()#

Return True when all required DB_* keys are present in env vars or ~/sda.json.

Environment variables take priority (used by the Docker/cloud container). Falls back to ~/sda.json for local developer setups.

Used by sda.api.load_data() to decide whether to connect directly to the database or fall back to the SDA cloud API.

sda.api.load_data_pg.load_commentary(tag, start_time=None, end_time=None)#

Load commentary entries for tag from the commentary table.

Returns a DataFrame with columns time (UTC-aware datetime) and commentary (str), ordered by time. If no rows match, an empty DataFrame with those two columns is returned.

Parameters:

  • tag (str) – P&ID code to filter on (pid_code column), e.g. T327.

  • start_time (str or datetime.datetime, optional) – Inclusive lower bound on the time column.

  • end_time (str or datetime.datetime, optional) – Inclusive upper bound on the time column.

Raises:

  • ValueError – If tag contains invalid characters or DB config is incomplete.

  • ConnectionError – If the database is unreachable or the query fails.

sda.api.load_data_pg.load_data(tag, start_time=None, end_time=None)#

Load all timeseries for a test and return a wide, time-aligned DataFrame.

The function discovers every table named timeseriesdata_{tag}_{instrument} in the database, queries all their columns for the requested time window, then merges them into a single DataFrame with one time column and one column per measurement.

Parameters:

  • tag (str) – Test name used to identify the tables (e.g. gif1_rev3). All tables named timeseries_{tag}_* are loaded and merged.

  • start_time (str or datetime.datetime, optional) – Start of the time range (inclusive). If omitted, no lower bound is applied.

  • end_time (str or datetime.datetime, optional) – End of the time range (inclusive). If omitted, no upper bound is applied.

Returns:

Wide DataFrame with a single time column (rounded to 0.1 s) followed by one column per measurement. Column names are derived from the last token(s) of the table name (e.g. temperature1 from timeseries_gif1_T377_tag_001_temperature1) and enriched with physical units when the measurement type is recognised (°C, m3/h, bar, NL/min).

Return type:

pandas.DataFrame

Raises:

  • ValueError – If tag contains invalid characters, no matching tables are found, or the database configuration in ~/sda.json is incomplete.

  • ConnectionError – If the database is unreachable or a query fails.

Examples

>>> import sda
>>> df = sda.load_data("T377")  # all data
>>> df = sda.load_data("T377", "2024-01-01", "2024-01-02")  # time-filtered
>>> df.columns.tolist()
['time', 'temperature1 (°C)', 'pressure (bar)', 'co2 (NL/min)', ...]