sda.dashboard.data.loader#

Data Loading Module.

Centralized data loading and caching for the dashboard to avoid loading the same data multiple times across different components.

Attributes#

DEFAULT_SCATTER_AXES_BY_TEST

data_manager

Classes#

DashboardDataManager

Manages data loading and caching for the dashboard.

Functions#

resolve_dashboard_test_inputs(test_inputs)

Resolve dashboard user inputs into concrete test/file identifiers.

Module Contents#

sda.dashboard.data.loader.DEFAULT_SCATTER_AXES_BY_TEST: dict[str, tuple[str, str]]#
sda.dashboard.data.loader.resolve_dashboard_test_inputs(test_inputs)#

Resolve dashboard user inputs into concrete test/file identifiers.

Folder inputs are recursively expanded to absolute Excel file paths. Direct Excel file paths are normalized to absolute paths. Canonical test IDs (e.g. T297) are returned unchanged.

class sda.dashboard.data.loader.DashboardDataManager#

Manages data loading and caching for the dashboard.

UNIFIED CACHING ARCHITECTURE:

  1. Raw Data Loading: - Individual tests: load_test_data() uses @lru_cache - Multiple tests: load_multiple_tests() uses cached individual test data

  2. UNIFIED Methods (handle both single tests and lists): - get_processed_data(test_input): Works with str or List[str] - get_column_names(test_input): Works with str or List[str] - get_filter_options(test_input): Works with str or List[str] - get_default_selections(test_input): Works with str or List[str] - get_all_available_columns(test_input): Works with str or List[str] - get_data_with_selected_columns(test_input, columns): Works with str or List[str]

  3. Benefits: - All methods use cached processed data - No duplicate file loading during dashboard startup - Simplified API with fewer methods to maintain - Consistent behavior across single and multiple test scenarios

verbose = False#
startup_test_name = None#
skip_invalid: bool = False#
source: str = 'local'#
groupby: str | None = None#
set_verbose(verbose)#

Set verbose mode for all data operations.

Parameters:

verbose (bool) – Whether to enable verbose output for data loading operations

set_skip_invalid(skip)#

Configure whether failing tests are skipped or raise an error.

Parameters:

skip (bool) – When True, tests that fail to load are skipped with a warning instead of aborting the whole multi-test load.

set_groupby(groupby)#

Set dashboard groupby column and invalidate processed cache if needed.

load_test_data(test_name)#

Load test data with caching.

Parameters:

test_name (str) – Name of the test to load

Returns:

Raw test data

Return type:

pd.DataFrame

load_multiple_tests(test_names)#

Load multiple tests and combine them using cached individual test data.

Parameters:

test_names (List[str]) – List of test names to load

Returns:

Combined test data

Return type:

pd.DataFrame

get_processed_data(test_input)#

Get processed and standardized data for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names to load

Returns:

Processed and standardized data

Return type:

pd.DataFrame

get_column_names(test_input)#

Get available column names for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

List of column names

Return type:

List[str]

get_filter_options(test_input)#

Get unique values for filters for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

Dictionary with filter options for each column

Return type:

Dict[str, List]

get_default_selections(test_input)#

Get default selections for dropdowns for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

Dictionary with default selections

Return type:

Dict[str, Optional[str]]

get_all_available_columns(test_input)#

Get all available column names for one or multiple tests using cached raw data (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

List of all column names available in the raw test data

Return type:

List[str]

get_data_with_selected_columns(test_input, selected_columns)#

Get data with only the selected columns for one or multiple tests (UNIFIED METHOD).

Parameters:

  • test_input (str or List[str]) – Single test name or list of test names

  • selected_columns (List[str]) – List of column names to include

Returns:

Data with only the selected columns

Return type:

pd.DataFrame

get_excel_file_path(test_name)#

Get the Excel file path for a test (same logic as used during data loading).

Parameters:

test_name (str) – Name of the test

Returns:

Path to the Excel file, or empty string if not found

Return type:

str

sda.dashboard.data.loader.data_manager#