fuzzy_search.pattern package

Submodules

fuzzy_search.pattern.fuzzy_patterns module

Regex pattern definitions for recognizing Dutch person names, place names and dates, and helper functions to combine these patterns with surrounding context strings.

fuzzy_search.pattern.fuzzy_patterns.context_before_pattern(name, pattern_definition, context_string, max_distance=10)

Build a pattern definition that matches the context string, then up to max_distance characters, then the given pattern.

Parameters:

• name – the base name of the pattern

• pattern_definition – the pattern definition dictionary to extend

• context_string – the (already escaped) context string to match before the pattern

• max_distance – the maximum number of characters allowed between context and pattern

Returns:

a new pattern definition dictionary

Return type:

dict

fuzzy_search.pattern.fuzzy_patterns.context_then_pattern(name, pattern_definition, context_string)

Build a pattern definition that matches the context string followed by the given pattern.

Parameters:

• name – the base name of the pattern

• pattern_definition – the pattern definition dictionary to extend

• context_string – the (already escaped) context string to match before the pattern

Returns:

a new pattern definition dictionary

Return type:

dict

fuzzy_search.pattern.fuzzy_patterns.escape_string(string)

Escape regex special characters in a string so it can be embedded literally in a pattern.

Parameters:

string – the string to escape

Returns:

the escaped string

Return type:

str

fuzzy_search.pattern.fuzzy_patterns.get_context_patterns(context_type: None | str = None) → dict

Return the dictionary of context pattern functions registered for a given context type.

Parameters:

context_type (Union[None, str]) – a context type key in context_pattern (defaults to “all”)

Returns:

a dict mapping pattern function name to pattern function

Return type:

dict

fuzzy_search.pattern.fuzzy_patterns.get_search_patterns(pattern_type=None)

Return registered pattern definitions as a dictionary, optionally filtered by pattern type.

Parameters:

pattern_type – if given, only return definitions of patterns with this type

Returns:

a dict mapping pattern name to pattern definition

Return type:

dict

fuzzy_search.pattern.fuzzy_patterns.list_context_pattern_types(context_type=None)

Return the names of context pattern functions registered for a given context type.

Parameters:

context_type – a context type key in context_pattern (defaults to “all”)

Returns:

a list of context pattern function names

Return type:

list

fuzzy_search.pattern.fuzzy_patterns.list_pattern_definitions(pattern_type=None)

Return registered pattern definitions, optionally filtered by pattern type.

Parameters:

pattern_type – if given, only return definitions of patterns with this type

Returns:

a list (if filtered) or dict (if not) of pattern definitions

fuzzy_search.pattern.fuzzy_patterns.list_pattern_names(name_only=True, pattern_type=None)

Return the names of registered pattern definitions, optionally filtered by pattern type.

Parameters:

• name_only – unused; kept for API compatibility

• pattern_type – if given, only return names of patterns with this type

Returns:

a list of pattern names

Return type:

list

fuzzy_search.pattern.fuzzy_patterns.make_search_context_patterns(context_string, pattern_names, context_patterns)

Generate context-specific search patterns by combining a context string with named pattern definitions using the given context pattern functions.

Parameters:

• context_string – the context string to embed in the generated patterns

• pattern_names – the names of the pattern definitions to combine with the context

• context_patterns – a dict mapping context pattern function name to function

Returns:

a list of new pattern definitions

Return type:

list

fuzzy_search.pattern.fuzzy_patterns.pattern_before_context(name, pattern_definition, context_string, max_distance=10)

Build a pattern definition that matches the given pattern, then up to max_distance characters, then the context string.

Parameters:

• name – the base name of the pattern

• pattern_definition – the pattern definition dictionary to extend

• context_string – the (already escaped) context string to match after the pattern

• max_distance – the maximum number of characters allowed between pattern and context

Returns:

a new pattern definition dictionary

Return type:

dict

fuzzy_search.pattern.fuzzy_patterns.pattern_comma_then_context(name, pattern_definition, context_string)

Build a pattern definition that matches the given pattern followed by a comma and the context string.

Parameters:

• name – the base name of the pattern

• pattern_definition – the pattern definition dictionary to extend

• context_string – the (already escaped) context string to match after the pattern

Returns:

a new pattern definition dictionary

Return type:

dict

fuzzy_search.pattern.fuzzy_template module

FuzzyTemplate and its element classes, which combine labeled phrases from a PhraseModel into ordered or unordered, single or grouped, optionally required elements that together describe a structured fuzzy-matchable template (e.g. a name followed by a date).

class fuzzy_search.pattern.fuzzy_template.FuzzyTemplate(phrase_model: PhraseModel, template_json: List[str] | List[dict] | Dict[str, str | dict], ignore_unknown: bool = False, ordered: bool = False)

Bases: object

__init__(phrase_model: PhraseModel, template_json: List[str] | List[dict] | Dict[str, str | dict], ignore_unknown: bool = False, ordered: bool = False)

A fuzzy search template to register phrases from a phrase model as elements of the template. The template can be used in combination with a fuzzy template searcher to find a given set of phrases (the template) within a certain range of text. The order, cardinality of elements and the distance between them can be specified. Elements can be grouped into ordered or unordered blocks of elements. Element groups can be hierarchical.

Parameters:

• phrase_model (PhraseModel) – a PhraseModel object with phrases that correspond to the element label in the template.

• template_json (Union[List[str], List[dict], Dict[str, Union[str, dict]]]) – a dictionary of template groups or elements to be registered as part of the template

• ignore_unknown (bool) – whether to ignore elements with labels that are not in the phrase model

get_element(element_label: str) → None | FuzzyTemplateLabelElement | FuzzyTemplateGroupElement

Return the element corresponding to a given label.

Parameters:

element_label (str) – a fuzzy element label

Returns:

the element corresponding to the label or None if label is unknown

Return type:

Union[FuzzyTemplateElement]

get_elements_by_cardinality(cardinality: str = 'single') → List[FuzzyTemplateLabelElement]

Return all template elements with a given cardinality.

Parameters:

cardinality (str) – a cardinality type (‘single’ or ‘multi’)

Returns:

the list of labels of elements with a given cardinality

Return type:

List[str]

get_label_phrases(label: str) → List[Phrase]

Return a list of phrases that have a given label.

Parameters:

label (str) – a phrase label for phrases in the registered phrase_model

Returns:

a list of phrases from the registered phrase model that have a given phrase

Return type:

List[Phrase]

get_labels_by_cardinality(cardinality: str = 'single') → List[str]

Return the labels of all template elements with a given cardinality.

Parameters:

cardinality (str) – a cardinality type (‘single’ or ‘multi’)

Returns:

the list of labels of elements with a given cardinality

Return type:

List[str]

get_required_elements() → List[FuzzyTemplateLabelElement]

Return all required elements in the template.

Returns:

the list of labels of required elements

Return type:

List[FuzzyTemplateElement]

get_required_labels() → List[str]

Return the labels of all required elements in the template.

Returns:

the list of labels of required elements

Return type:

List[str]

has_group(group: str) → bool

Check if the template has group elements with a given group name.

Parameters:

group (str) – a fuzzy element group

Returns:

whether the group corresponds to any registered element(s)

Return type:

bool

has_label(label: str | List[str]) → bool

Check if the template has label elements with a given label or list of label (any or all).

Parameters:

label (Union[str, List[str]]) – a fuzzy element label

Returns:

whether the label corresponds to any registered element(s)

Return type:

bool

parse_group_element(group_info: Dict[str, any]) → FuzzyTemplateGroupElement

Parse a group element dictionary/JSON object into a fuzzy template group element.

Parameters:

group_info (dict) – a dictionary containing the properties of the template group element

Returns:

a fuzzy template group element

Return type:

FuzzyTemplateGroupElement

parse_label_element(label_info: str | Dict[str, any]) → FuzzyTemplateLabelElement | None

Parse a label element dictionary/JSON object into a fuzzy template label element.

Parameters:

label_info (dict) – a dictionary containing the properties of the template label element

Returns:

a fuzzy template label element, or None if the label is not used in the phrase model

Return type:

FuzzyTemplateLabelElement

register_template(template_json: List[str] | List[dict] | Dict[str, str | dict]) → None

Register a list of elements as a fuzzy template. Each element contains a label that corresponds to at least one phrase in the registered phrase model.

Parameters:

template_json (Union[List[str], List[dict], Dict[str, Union[str, dict]]]) – a dictionary of template groups or elements to be registered as part of the template

class fuzzy_search.pattern.fuzzy_template.FuzzyTemplateElement(label: None | str | List[str], element_type: str, required: bool)

Bases: object

Base class for elements of a fuzzy template: either a single labeled element or a group of elements.

__init__(label: None | str | List[str], element_type: str, required: bool)

Create a FuzzyTemplateElement.

Parameters:

• label (Union[None, str, List[str]]) – the label of the element, which can be a single string or a list of strings

• element_type (str) – the type of element, either ‘label’ or ‘group’

• required (bool) – whether or not the element must match for the template to match

class fuzzy_search.pattern.fuzzy_template.FuzzyTemplateGroupElement(elements: List[FuzzyTemplateElement], label: str | None = None, ordered: bool = True, required: bool = False)

Bases: FuzzyTemplateElement

A group of fuzzy template elements (labels and/or nested groups), which can be ordered or unordered, and becomes required if any of its sub-elements is required.

__init__(elements: List[FuzzyTemplateElement], label: str | None = None, ordered: bool = True, required: bool = False)

Create a FuzzyTemplateGroupElement from a list of sub-elements.

Parameters:

• elements (List[FuzzyTemplateElement]) – the sub-elements (label or group elements) that make up this group

• label (Union[str, None]) – an optional label for the group itself

• ordered (bool) – whether the sub-elements must occur in order for the group to match

• required (bool) – whether the group must match for the template to match

class fuzzy_search.pattern.fuzzy_template.FuzzyTemplateLabelElement(label: str, required: bool = False, cardinality: str = 'single', next_label: None | str | List[str] = None, next_distance_max: None | int = None, variable: bool = False)

Bases: FuzzyTemplateElement

A single labeled element of a fuzzy template, corresponding to phrases in a phrase model that share this label.

__init__(label: str, required: bool = False, cardinality: str = 'single', next_label: None | str | List[str] = None, next_distance_max: None | int = None, variable: bool = False)

A FuzzyTemplate element with properties to define its role in the template it is part of. The ‘label’ property is the only required property. All elements must have either a single string or list of strings as label.

Parameters:

• label (Union[str, List[str]]) – the label of the element, which can be a single string or a list of strings

• required (bool) – whether or not the element must match for the template to match

• cardinality (str) – whether the element can occur only once or multiple times in a template match. The default value is ‘multi’

• next_label (Union[str, List[str]]) – what the label of the next template element should be. This can be a list of labels to allow different types of element to come next

• next_distance_max (int) – the maximum distance allowed between this element and the next element in the template

• variable (bool) – flag to indicate the element has no phrases but has variable text (default is False)

fuzzy_search.pattern.fuzzy_template.generate_group_from_json(element_info: dict, group_elements: List[FuzzyTemplateElement]) → FuzzyTemplateGroupElement

Generate a FuzzyTemplateGroupElement from a element json dictionary and a list of group elements.

Parameters:

• element_info (dict) – a dictionary containing the properties of the template group element

• group_elements (List[FuzzyTemplateElement]) – a list of fuzzy template elements that are part of the group element

Returns:

a fuzzy template group element

Return type:

FuzzyTemplateGroupElement

fuzzy_search.pattern.fuzzy_template.generate_label_from_json(label: str, element_info: dict) → FuzzyTemplateLabelElement

Generate a FuzzyTemplateLabelElement from a label and an element json dictionary.

Parameters:

• label (str) – the label string for the label element

• element_info (dict) – a dictionary containing the properties of the template label element

Returns:

a fuzzy template label element

Return type:

FuzzyTemplateLabelElement

fuzzy_search.pattern.fuzzy_template.validate_element_properties(label: str, required: bool = False, cardinality: str = 'multi', next_label: None | str | List[str] = None, next_distance_max: None | int = None, variable: bool = False) → None

Validate the properties of a FuzzyTemplate element.

Parameters:

• label (Union[str, List[str]]) – the label of the element, which can be a single string or a list of strings

• required (bool) – whether or not the element must match for the template to match

• cardinality (str) – whether the element can occur only once (default) or multiple times in a template match.

• next_label (Union[str, List[str]]) – what the label of the next element should be. Use a list of labels for multiple options.

• next_distance_max (int) – the maximum distance allowed between this element and the next element in the template

• variable (bool) – flag to indicate the element has no phrases but has variable text (default is False)

Module contents

Regex-based pattern definitions and fuzzy templates for matching structured text such as names and dates, and for combining multiple phrase labels into ordered templates.