Missingness masks

Helpers to describe missingness, to convert a separately recorded availability mask into the NaN encoding the imputers expect, and to simulate missing views so that imputation strategies can be compared against a known ground truth.

Helpers to describe, build and simulate missing views.

Throughout polyview.imputation, missing data is encoded with NaN:

  • a missing entry is a single NaN cell inside a view,

  • a missing view is a sample whose whole row is NaN in that view, which is what happens when a modality (a sensor, a questionnaire, an imaging modality) was never acquired for that sample.

The functions below turn views into boolean masks, and back.

polyview.imputation.mask.complete_case_mask(views, how: str = 'all') → numpy.ndarray

Return the samples for which every view is observed.

Parameters:
Returns:

True for samples that have no missing view.

Return type:

ndarray of bool of shape (n_samples,)

Examples

>>> import numpy as np
>>> from polyview.imputation import complete_case_mask
>>> X1 = np.array([[1.0], [np.nan]])
>>> X2 = np.array([[2.0], [3.0]])
>>> complete_case_mask([X1, X2])
array([ True, False])
polyview.imputation.mask.mark_missing_views(views, mask) → List[numpy.ndarray]

Blank out views by setting whole rows to NaN.

This is the inverse of missing_view_mask(): it turns an explicit availability mask into the NaN encoding the imputers expect. Use it when missingness is recorded separately from the data itself.

Parameters:

  • views (list of array-like or MultiViewDataset)

  • mask (array-like of bool of shape (n_samples, n_views)) – True marks the view as missing for that sample.

Returns:

Copies of the views with the masked rows set to NaN.

Return type:

list of ndarray

Examples

>>> import numpy as np
>>> from polyview.imputation import mark_missing_views
>>> X1 = np.ones((2, 2))
>>> mark_missing_views([X1], np.array([[False], [True]]))[0]
array([[ 1.,  1.],
       [nan, nan]])
polyview.imputation.mask.missing_entry_mask(views) → List[numpy.ndarray]

Return the cell-level missingness mask of every view.

Parameters:

views (list of array-like or MultiViewDataset) – Views of shape (n_samples, n_features_i).

Returns:

One mask per view, same shape as the view, True where the entry is NaN.

Return type:

list of ndarray of bool

Examples

>>> import numpy as np
>>> from polyview.imputation import missing_entry_mask
>>> X = np.array([[1.0, np.nan], [3.0, 4.0]])
>>> missing_entry_mask([X])[0]
array([[False,  True],
       [False, False]])
polyview.imputation.mask.missing_rate(views, how: str = 'all') → numpy.ndarray

Return the fraction of samples for which each view is missing.

Parameters:
Returns:

Value in [0, 1] per view.

Return type:

ndarray of shape (n_views,)

Examples

>>> import numpy as np
>>> from polyview.imputation import missing_rate
>>> X1 = np.array([[1.0], [np.nan], [3.0], [np.nan]])
>>> X2 = np.array([[1.0], [2.0], [3.0], [4.0]])
>>> missing_rate([X1, X2])
array([0.5, 0. ])
polyview.imputation.mask.missing_view_mask(views, how: str = 'all') → numpy.ndarray

Return the sample-level missingness mask across views.

Parameters:

  • views (list of array-like or MultiViewDataset) – Views of shape (n_samples, n_features_i).

  • how ({"all", "any"}, default="all") – "all" marks view v as missing for sample i only when the whole row is NaN (a genuinely unobserved view). "any" marks it as soon as one entry is NaN (a partially corrupted view).

Returns:

True where the view is missing for that sample.

Return type:

ndarray of bool of shape (n_samples, n_views)

Examples

>>> import numpy as np
>>> from polyview.imputation import missing_view_mask
>>> X1 = np.array([[1.0, 2.0], [np.nan, np.nan]])
>>> X2 = np.array([[np.nan, 5.0], [6.0, 7.0]])
>>> missing_view_mask([X1, X2])
array([[False, False],
       [ True, False]])
>>> missing_view_mask([X1, X2], how="any")
array([[False,  True],
       [ True, False]])
polyview.imputation.mask.simulate_missing_views(views, missing_rate: float | Sequence[float] = 0.2, random_state: int | None = None, keep_one: bool = True) → Tuple[List[numpy.ndarray], numpy.ndarray]

Remove views completely at random, for benchmarking imputers.

Each (sample, view) pair is dropped independently with probability missing_rate (missing-completely-at-random). The returned mask lets you score an imputer against the ground truth you started from.

Parameters:

  • views (list of array-like or MultiViewDataset)

  • missing_rate (float or sequence of float, default=0.2) – Probability that a view is missing, shared across views or given per view. Must lie in [0, 1].

  • random_state (int or None, default=None) – Seed for reproducibility.

  • keep_one (bool, default=True) – If True, samples that lost every view get one view restored at random. A sample with no observed view carries no information, so no imputer can do better than the population mean for it.

Returns:

  • views_out (list of ndarray) – Copies of the views with the dropped rows set to NaN.

  • mask (ndarray of bool of shape (n_samples, n_views)) – True where a view was dropped.

Examples

>>> import numpy as np
>>> from polyview.imputation import simulate_missing_views
>>> views = [np.random.rand(100, 4), np.random.rand(100, 6)]
>>> incomplete, mask = simulate_missing_views(views, 0.3, random_state=0)
>>> mask.shape
(100, 2)
>>> bool(np.all(mask.sum(axis=1) < 2))  # keep_one leaves one view
True