hydra.utils.data_collator

Data collation utilities for models.

class hydra.utils.data_collator.DataCollatorForHydra(tokenizer: PreTrainedTokenizerBase, features: Features, padding: bool | str | PaddingStrategy = True, max_length: int | None = None, pad_to_multiple_of: int | None = None, label_pad_token_id: int = -100, return_tensors: str = 'pt')[source]

Bases: DataCollatorMixin

Handles padding for label sequences and matrices.

features: Features
label_pad_token_id: int = -100
max_length: int | None = None
numpy_call(batch)[source]

Not to be used.

Raises:

NotImplementedError – Always.

pad_to_multiple_of: int | None = None
padding: bool | str | PaddingStrategy = True
return_tensors: str = 'pt'
tokenizer: PreTrainedTokenizerBase
torch_call(batch)[source]

Prepare the batch for passing to a torch model.

Returns:

a padded batch.