suboptimumg.loganalysis.intake#

PERDA log -> aligned/resampled pandas DataFrame, plus fast GPS-only loader.

suboptimumg.loganalysis.intake.perda_load_folder(folder, extra_vars=None, deduplicate=True, lat_var='pcm.vnav.posLla.latitude', lon_var='pcm.vnav.posLla.longitude', gps_fix_var='pcm.vnav.gpsFix', min_gps_fix=1.0)[source]#

Batch-load GPS data from all PER CSV logs using PERDA.

Uses PERDA’s Analyzer for CSV parsing instead of the custom lightweight parser in quick_load_gps().

Parameters:

  • folder (str or Path) – Directory containing .csv log files.

  • extra_vars (list[str] or None) – Full CAN variable names to extract alongside GPS.

  • deduplicate (bool) – Collapse consecutive duplicate GPS positions.

  • lat_var (str) – Full CAN variable names for latitude and longitude.

  • lon_var (str) – Full CAN variable names for latitude and longitude.

  • gps_fix_var (str) – CAN variable name for GPS fix status.

  • min_gps_fix (float) – Minimum GPS fix value to keep.

Returns:

Keyed by filename (not full path). Logs with no GPS lock are omitted.

Return type:

dict[str, pd.DataFrame]

suboptimumg.loganalysis.intake.perda_load_gps(path, lat_var='pcm.vnav.posLla.latitude', lon_var='pcm.vnav.posLla.longitude', gps_fix_var='pcm.vnav.gpsFix', min_gps_fix=1.0, extra_vars=None, deduplicate=True)[source]#

Load GPS data from a PER CSV log using PERDA’s Analyzer.

Parameters:

  • path (str or Path) – Path to a PER-format .csv log file.

  • lat_var (str) – Full CAN variable names for latitude and longitude.

  • lon_var (str) – Full CAN variable names for latitude and longitude.

  • gps_fix_var (str) – CAN variable name for GPS fix status.

  • min_gps_fix (float) – Minimum GPS fix value to keep (0 = no fix, 3 = 3D fix).

  • extra_vars (list[str] or None) – Additional CAN variables to extract alongside GPS.

  • deduplicate (bool) – Collapse consecutive duplicate GPS positions to remove ZOH-inflated samples (GPS updates at ~4 Hz but is logged at CAN rate ~100 Hz).

Returns:

DataFrame indexed by time_s with latitude, longitude columns (plus extras). Returns None if the file has no usable GPS data.

Return type:

pd.DataFrame or None

suboptimumg.loganalysis.intake.perda_to_dataframe(analyzer, var_names, resample_method='linear', resample_freq_hz=None, reference_var=None, deduplicate_vars=None)[source]#

Convert selected PERDA variables into a single aligned DataFrame.

Only loads the explicitly requested variables — never touches the full set of 200-300+ variables in a log.

Parameters:

  • analyzer (Analyzer) – A loaded PERDA Analyzer instance.

  • var_names (list[str]) – CAN names of the variables to extract.

  • resample_method (str or dict[str, str]) – Interpolation method for aligning to a common time grid. "linear" (default), "zoh" (zero-order hold / persist until update), "nearest", or "cubic". Pass a dict mapping variable names to methods to mix strategies.

  • resample_freq_hz (float or None) – If given, resample to a uniform grid at this frequency (Hz). Otherwise the union of all source timestamps is used.

  • reference_var (str or None) – Left-join all variables to this variable’s timestamp grid. If None and resample_freq_hz is also None, the variable with the most samples is chosen automatically.

  • deduplicate_vars (list[str] or None) – Variable names whose consecutive duplicate (ZOH) values should be collapsed before resampling. Useful for signals like GPS position that are logged at a high CAN rate but only update at a lower real rate (e.g. 100 Hz CAN / 4 Hz real). Deduplication keeps the first sample of each constant run so that resampling interpolates smoothly between real updates.

Returns:

Index: time_s (float64 seconds, zero-based from log start). One column per variable.

Return type:

pd.DataFrame

suboptimumg.loganalysis.intake.quick_load_folder(folder, lat_pattern='posLla.latitude', lon_pattern='posLla.longitude', extra_patterns=None, parallel=False, max_workers=None)[source]#

Batch-load GPS previews for all PER CSV logs in a folder.

Parameters:

  • folder (str or Path) – Directory containing .csv log files.

  • lat_pattern (str) – Passed to quick_load_gps().

  • lon_pattern (str) – Passed to quick_load_gps().

  • extra_patterns (list[str] or None) – Passed to quick_load_gps().

  • parallel (bool) – If True, load files in parallel using ProcessPoolExecutor.

  • max_workers (int or None) – Max worker processes when parallel=True. Defaults to os.cpu_count() - 2 (minimum 1).

Returns:

Keyed by filename (not full path). Logs with no GPS lock are omitted.

Return type:

dict[str, pd.DataFrame]

suboptimumg.loganalysis.intake.quick_load_gps(path, lat_pattern='posLla.latitude', lon_pattern='posLla.longitude', gps_fix_pattern='gpsFix', min_gps_fix=1.0, extra_patterns=None, _verbose=True)[source]#

Fast GPS extractor that parses a PER CSV without PERDA.

Reads only the header to discover variable IDs, then streams the data section keeping only lat/lon rows. Typically 5-10x faster than a full PERDA load for GPS preview purposes.

Parameters:

  • path (str or Path) – Path to a PER-format .csv log file.

  • lat_pattern (str) – Substrings matched (case-sensitive) against the cpp_name in the header to identify latitude and longitude variable IDs.

  • lon_pattern (str) – Substrings matched (case-sensitive) against the cpp_name in the header to identify latitude and longitude variable IDs.

  • extra_patterns (list[str] or None) – Additional variable patterns to extract alongside lat/lon (e.g. ["gpsFix", "posLla.altitude"]).

  • gps_fix_pattern (str)

  • min_gps_fix (float)

  • _verbose (bool)

Returns:

DataFrame indexed by time_s with latitude, longitude columns (plus any extras). Returns None if the file has no GPS lock (all lat/lon values are zero).

Return type:

pd.DataFrame or None