hydra.training_and_evaluation

Implementation of the training and evaluation logic for Hydra parser.

class hydra.training_and_evaluation.ClassificationTask(name: str, dataset: DatasetDict, pretrained_path: str, segmentation: bool = False, lemmatisation: bool = False, tagging: bool = False, dependency: bool = False, graph: bool = False, hybrid: bool = False, named_entities: bool = False, root_label: str = 'ROOT', force_root_label: bool = False, morfeusz_dict_path: str | None = None, morfeusz_dict_name: str | None = None, nonterminal_features: list[str] | None = None, tree_weight: float = 1.0, batch_size: int = 16)[source]

Bases: object

A Hydra training configuration.

class hydra.training_and_evaluation.Trainer(task: ClassificationTask)[source]

Bases: object

Used for creating, training and evaluating instances of Hydra parser.

cross_validate(nfolds: int = 10, epochs: int = 10, patience: int = 4, lr: float = 1e-05, mst: str = 'mst', allow_masked: bool = False) → list[EvalResults][source]

Perform 10-fold cross validation.

Returns:

results of each fold run.

evaluate(parser: Hydra, mst: str = 'mst', allow_masked: bool = False, dataset: DatasetDict | None = None) → EvalResults[source]

Perform evaluation of a trained Hydra model.

Parameters:
  • parser – the parser instance to be evaluated.

  • mst – dependency tree decoding algoriths to be used.

  • allow_masked – accept masked labels in gold data (but exclude them from metric computation).

  • dataset – source of gold instances. Evaluation will be run on ‘validation’ and any ‘test*’ parts. Training dataset will be used as default if this argument is not provided.

Returns:

evaluation results.

train(epochs: int = 10, patience: int = 4, lr: float = 1e-05, log_dir: str | None = None, model_dir: str | None = None, dataset: DatasetDict | None = None) → Hydra[source]

Create and train a parser.

Parameters:
  • epochs – number of epochs.

  • patience – how many epochs are allowed with no training progress. The best model across all epochs is then retrieved.

  • lr – learning rate.

  • log_dir – where to save training logs.

  • model_dir – where to save the final model.

  • dataset – training data. Must contain ‘train’ and ‘validation’ parts (used for training), with any number of ‘test*’ parts for later evaluation.

Returns:

a parser instance.