hydra.parser
The main class for the Hydra parser is defined here.
- class hydra.parser.Hydra(pretrained_path: str, model_type: str, model: MultiTargetTokenClassification, labels: dict[str, list[str]], segmentation: bool = False, lemmatisation: bool = False, tagging: bool = False, dependency: bool = False, graph: bool = False, hybrid: bool = False, nonterminal_features: list[str] | None = None, named_entities: bool = False, morfeusz_dict_path: str | None = 'polimorf', morfeusz_dict_name: str | None = None, pretrained_tokenizer: PreTrainedTokenizerBase | None = None, tokenizer_kwargs: dict[str, Any] | None = None)[source]
Bases:
objectA Hydra parser. Use of
train()recommended for new instance creation.- classmethod create(pretrained_path: str, model_type: str, features: Features, segmentation: bool = False, lemmatisation: bool = False, tagging: bool = False, dependency: bool = False, graph: bool = False, hybrid: bool = False, nonterminal_features: list[str] | None = None, named_entities: bool = False, morfeusz_dict_path: str | None = None, morfeusz_dict_name: str | None = 'sgjp', pretrained_tokenizer: PreTrainedTokenizerBase | None = None, tokenizer_kwargs: dict[str, Any] | None = None) Hydra[source]
Create a new Hydra instance.
- Parameters:
pretrained_path – pretrained language model to be fine-tuned.
model_type – pretrained language model type.
features – the features signature of the dataset to be used for training.
segmentation – enable segmentation module.
lemmatisation – enable lemmatisation module.
tagging – enable tagging module.
dependency – enable dependency parsing module.
graph – enable graph parsing module (extended/additional edges).
hybrid – enable hybrid syntactic parsing module.
nonterminal_features – nonterminal features to predict (experimental).
named_entities – enable NER module.
morfeusz_dict_path – path to directory containing Morfeusz dictionary (if not using the default one).
morfeusz_dict_name – name of the Morfeusz dictionary (if not using the default one).
pretrained_tokenizer – tokenizer for the language model.
tokenizer_kwargs – addditonal kwargs for the language model tokenizer.
- Returns:
a Hydra instance.
- classmethod load(path: str, morfeusz_dict_path: str | None = None) Hydra[source]
Load a trained Hydra model.
- Returns:
a Hydra instance.
- parse(sentences: list[str] | str, correct_lemmata: bool = False, return_jsons: bool = False, return_conllu: bool = False, return_trees: bool = False, return_labels: bool = False, return_logits: bool = False, root_label: str | None = None, force_root_label: bool = False, skip_long: bool = False, is_tokenized: bool = False, morph_postprocess: Callable[[str, str, str], tuple[str, str]] | None = None, mst: str = 'mst', return_times: bool = False, mem_limit: int | None = None, verbose: bool = False) list[HydraAnalysis] | tuple[list[HydraAnalysis], dict[str, int]][source]
Parse a single sentence or a list of sentences.
- Parameters:
sentences – sentence(s) to process.
correct_lemmata – apply lemmata correction using Morfeusz.
return_jsons – include full json outputs in the results. Only available for models trained with hybrid=True.
return_conllu – include ConLLu outputs in the results. Only available for models trained with dependency=True.
return_trees – include tree objects in the results. Only available for models trained with hybrid=True.
return_labels – include label lists in the results.
return_logits – include raw predicted logits in the results.
root_label – used by force_root_label.
force_root_label – enforce a specific root label in resulting trees.
skip_long – omit sentences with too many tokens (an error message will be logged in the sentences’ metadata).
is_tokenized – don’t perform segmentation (treat spaces as segment delimiters).
morph_postprocess – custom postprocessing function for tags and lemmata.
mst – dependency tree decoding algorithm to use:
MSTorEAGER.return_times – include parsing time info in the results.
mem_limit – memory limit (experimental).
verbose – include additional info in the results.
- Returns:
the parsing results.
- process_labels(labels: LabelDict, tokens: TokenList, correct_lemmata: bool = False, morph_postprocess: Callable[[str, str, str], tuple[str, str]] | None = None, mst: str = 'mst', allow_masked: bool = False) LabelDict[source]
Process model-predicted labels.
- Parameters:
- Returns:
labels that can be returned as parsing result or used to construct a tree.