sda.dashboard.data.loader

Data Loading Module.

---

Centralized data loading and caching for the dashboard to avoid loading the same data multiple times across different components.

Attributes

data_manager

Classes

DashboardDataManager

Manages data loading and caching for the dashboard.

Module Contents

class sda.dashboard.data.loader.DashboardDataManager

Manages data loading and caching for the dashboard.

UNIFIED CACHING ARCHITECTURE:

1. Raw Data Loading: - Individual tests: load_test_data() uses @lru_cache - Multiple tests: load_multiple_tests() uses cached individual test data

2. UNIFIED Methods (handle both single tests and lists): - get_processed_data(test_input): Works with str or List[str] - get_column_names(test_input): Works with str or List[str] - get_filter_options(test_input): Works with str or List[str] - get_default_selections(test_input): Works with str or List[str] - get_all_available_columns(test_input): Works with str or List[str] - get_data_with_selected_columns(test_input, columns): Works with str or List[str]

3. Benefits: - All methods use cached processed data - No duplicate file loading during dashboard startup - Simplified API with fewer methods to maintain - Consistent behavior across single and multiple test scenarios

verbose = False

startup_test_name = None

set_verbose(verbose)

Set verbose mode for all data operations.

Parameters:

verbose (bool) – Whether to enable verbose output for data loading operations

load_test_data(test_name)

Load test data with caching.

Parameters:

test_name (str) – Name of the test to load

Returns:

Raw test data

Return type:

pd.DataFrame

load_multiple_tests(test_names)

Load multiple tests and combine them using cached individual test data.

Parameters:

test_names (List[str]) – List of test names to load

Returns:

Combined test data

Return type:

pd.DataFrame

get_processed_data(test_input)

Get processed and standardized data for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names to load

Returns:

Processed and standardized data

Return type:

pd.DataFrame

get_column_names(test_input)

Get available column names for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

List of column names

Return type:

List[str]

get_filter_options(test_input)

Get unique values for filters for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

Dictionary with filter options for each column

Return type:

Dict[str, List]

get_default_selections(test_input)

Get default selections for dropdowns for one or multiple tests (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

Dictionary with default selections

Return type:

Dict[str, Optional[str]]

get_all_available_columns(test_input)

Get all available column names for one or multiple tests using cached raw data (UNIFIED METHOD).

Parameters:

test_input (str or List[str]) – Single test name or list of test names

Returns:

List of all column names available in the raw test data

Return type:

List[str]

get_data_with_selected_columns(test_input, selected_columns)

Get data with only the selected columns for one or multiple tests (UNIFIED METHOD).

Parameters:

• test_input (str or List[str]) – Single test name or list of test names

• selected_columns (List[str]) – List of column names to include

Returns:

Data with only the selected columns

Return type:

pd.DataFrame

get_excel_file_path(test_name)

Get the Excel file path for a test (same logic as used during data loading).

Parameters:

test_name (str) – Name of the test

Returns:

Path to the Excel file, or empty string if not found

Return type:

str

sda.dashboard.data.loader.data_manager