API Reference: Data

masterful.data.DataParams

class masterful.data.DataParams(num_classes=None, task=None, image_shape=None, image_range=None, image_dtype=None, image_channels_last=True, label_dtype=None, label_shape=None, label_structure=None, label_sparse=None, label_bounding_box_format=None)

Parameters describing the datasets used during training.

These parameters describe both the structure of the dataset (image and label shapes for examples) as well as semantic structure of the labels (the bounding box format for example, or whether or not the labels are sparse or dense).

Parameters

  • num_classes (int) – The number of possible classes in the dataset.

  • task (masterful.enums.Task) – The task this dataset will be used for.

  • image_shape (Tuple) – The input shape of image data in the dataset, in the format (height, width, channels) if input_channels_last=True, otherwise (channels, height, width) if input_channels_last=False.

  • image_range (masterful.enums.ImageRange) – The range of pixels in the input image space that of the dataset.

  • image_dtype (tensorflow.python.framework.dtypes.DType) – The image data type in the dataset.

  • image_channels_last (bool) – The ordering of the dimensions in the inputs. input_channels_last=True corresponds to inputs with shape (height, width, channels) while input_channels_last=False corresponds to inputs with shape (channels, height, width). Defaults to True.

  • label_dtype (type) – The data type of the labels.

  • label_shape (Tuple) – The shape of the labels.

  • label_structure (masterful.enums.TensorStructure) – The tensor format of the label examples.

  • label_sparse (bool) – True if the labels are in sparse format, False for dense (one-hot) labels.

  • label_bounding_box_format (Optional[masterful.enums.BoundingBoxFormat]) – The format of bounding boxes in the label, if they exist.

Return type

None

masterful.data.learn_data_params

masterful.data.learn_data_params(*args, **kwargs)

Learns the DataParams for the given dataset.

Most parameters can be introspected from the dataset itself. Anything that cannot be introspected is passed into this function as an argument, or set on the DataParams after creation.

Parameters

  • dataset – A tf.data.Dataset instance to learn the parameters for.

  • image_range – The range of pixels in the input image space that of the dataset.

  • num_classes – The number of possible classes in the dataset.

  • sparse_labels – True if the labels are in sparse format, False for dense (one-hot) labels.

  • task – The task this dataset will be used for.

  • bounding_box_format – The format of bounding boxes in the label, if they exist.

Returns

A new instance of DataParams describing the passed in dataset.