spark.core.utils

spark.core.utils#

Classes#

`InheritanceFlags`	Support for integer-based Flags
`InheritanceLeaf`	Leaf object for the InheritanceTree data structure.
`InheritanceTree`	Tree-like data structure to manage the inheritance status of variables in the Spark Graph Editor.

Functions#

`normalize_str`(s)	Converts any string into a consistent lowercase_snake_case format.
`to_human_readable`(s[, capitalize_all])	Converts a string from various programming cases into a human-readable format.
`get_einsum_labels`(num_dims[, offset])	Generates labels for a generalized dot product using Einstein notation.
`get_axes_einsum_labels`(axes[, ignore_repeated])	Generates labels for a generalized dot product using Einstein notation.
`get_einsum_dot_string`(x, y[, ignore_one_dims, side])	Generates labels for a generalized dot product using Einstein notation.
`get_einsum_dot_red_string`(x, y[, ignore_one_dims, side])	Generates labels for a generalized dot reduction product using Einstein notation.
`get_einsum_dot_exp_string`(x, y[, ignore_one_dims, side])	Generates labels for a generalized dot expansion product using Einstein notation.
`validate_shape`(obj)	Verifies that the object is broadcastable to a valid shape (tuple of integers).
`validate_list_shape`(obj)	Verifies that the object is broadcastable to a valid list ofshape (a list of tuple of integers).
`merge_shape_list`(shape_list)	Merges a list of shapes into a single shape.
`is_shape`(obj)	Checks if the obj is broadcastable to a shape.
`is_list_shape`(obj)	Checks if the obj is broadcastable to a shape.
`is_dict_of`(obj, value_cls[, key_cls])	Check if an object instance is of 'dict[key_cls, value_cls]'.
`is_list_of`(obj, cls)	Check if an object instance is of 'list[cls]'.
`is_dtype`(obj)	Check if an object is a 'DTypeLike'.
`is_float`(obj)	Check if an object is a 'DTypeLike'.
`ascii_tree`(text)	Build an ASCII tree from indentation-based text.

Module Contents#

spark.core.utils.normalize_str(s)[source]#

Converts any string into a consistent lowercase_snake_case format.

Parameters:: s (str) – str, string to normalize
Returns:: str, normalized string
Return type:: str

spark.core.utils.to_human_readable(s, capitalize_all=False)[source]#

Converts a string from various programming cases into a human-readable format.

Input:: s: str, string to normalize
Output:: str, human readable string

Parameters:

s (str)
capitalize_all (bool)

Return type:

str

spark.core.utils.get_einsum_labels(num_dims, offset=0)[source]#

Generates labels for a generalized dot product using Einstein notation.

Parameters:

num_dims (int) – int, number of dimensions (labels) to generate
offset (int) – int, initial dimension (label) offset

Returns:

str, a string with num_dims different labels, skipping the first offset characters

Return type:

str

spark.core.utils.get_axes_einsum_labels(axes, ignore_repeated=False)[source]#

Generates labels for a generalized dot product using Einstein notation.

Parameters:

axes (tuple[int, Ellipsis]) – tuple[int, …], requested dimensions (labels) to generate
ignore_repeated (bool)

Returns:

str, a string with num_dims different labels, skipping the first offset characters

Return type:

str

spark.core.utils.get_einsum_dot_string(x, y, ignore_one_dims=True, side='right')[source]#

Generates labels for a generalized dot product using Einstein notation.: right: (c,d)•(a,b,c,d)=(a,b) - cd,abcd->ab | (a,b,c,d)•(c,d)=(a,b) - abcd,cd->ab left: (a,b)•(a,b,c,d)=(c,d) - ab,abcd->cd | (a,b,c,d)•(c,d)=(c,d) - abcd,ab->cd

Parameters:

x (tuple[int, Ellipsis]) – tuple[int, …], shape for the first variable of the dot product
y (tuple[int, Ellipsis]) – tuple[int, …], shape for the second variable of the dot product
ignore_one_dims (bool) – bool, ignore one dimensions when computing the labels (squeeze shapes), default: True
side (str) – str, side of the dot product, default: “right”

Returns:

str, a string representing the dot product operation

Return type:

str

spark.core.utils.get_einsum_dot_red_string(x, y, ignore_one_dims=True, side='right')[source]#

Generates labels for a generalized dot reduction product using Einstein notation.: right: (a,b)•(a,b,c,d)=(a,b) - ab,abcd->ab | (a,b,c,d)•(a,b)=(a,b) - abcd,ab->ab left: (c,d)•(a,b,c,d)=(c,d) - cd,abcd->cd | (a,b,c,d)•(c,d)=(c,d) - abcd,ab->ab

Parameters:

x (tuple[int, Ellipsis]) – tuple[int, …], shape for the first variable of the dot product
y (tuple[int, Ellipsis]) – tuple[int, …], shape for the second variable of the dot product
ignore_one_dims (bool) – bool, ignore one dimensions when computing the labels (squeeze shapes), default: True
side (str) – str, side of the reduction-dot product, default: “right”

Returns:

str, a string representing the dot product operation

Return type:

str

spark.core.utils.get_einsum_dot_exp_string(x, y, ignore_one_dims=False, side='right')[source]#

Generates labels for a generalized dot expansion product using Einstein notation.: right: (a,b)•(a,b,c,d)=(a,b,c,d) - ab,abcd->abcd | (a,b,c,d)•(a,b)=(a,b,c,d) - abcd,ab->abcd left: (c,d)•(a,b,c,d)=(a,b,c,d) - cd,abcd->abcd | (c,d)•(a,b,c,d)=(a,b,c,d) - abcd,cd->abcd none: (a,b)•(c,d)=(a,b,c,d) - ab,cd->abcd | (a)•(b,c,d)=(a,b,c,d) - a,bcde->abcde

Parameters:

x (tuple[int, Ellipsis]) – tuple[int, …], shape for the first variable of the dot product
y (tuple[int, Ellipsis]) – tuple[int, …], shape for the second variable of the dot product
ignore_one_dims (bool) – bool, ignore one dimensions when computing the labels (squeeze shapes), default: True
side (str) – str, side of the expansion-dot, default: “right”

Returns:

str, a string representing the dot product operation

Return type:

str

spark.core.utils.validate_shape(obj)[source]#

Verifies that the object is broadcastable to a valid shape (tuple of integers). Returns the shape.

Parameters:: obj (Any) – tp.Any: the instance to validate
Returns:: list[tuple[int, …]], the shape
Return type:: tuple[int, Ellipsis]

spark.core.utils.validate_list_shape(obj)[source]#

Verifies that the object is broadcastable to a valid list ofshape (a list of tuple of integers). Returns the list of shapes.

Parameters:: obj (Any) – tp.Any: the instance to validate
Returns:: list[tuple[int, …]], the list of shapes
Return type:: list[tuple[int, Ellipsis]]

spark.core.utils.merge_shape_list(shape_list)[source]#

Merges a list of shapes into a single shape.

Parameters:: shape_list (list[tuple[int, Ellipsis]]) – list[tuple[int, …]]: the list of shapes
Returns:: tuple[int, …], the merged shape
Return type:: tuple[int, Ellipsis]

spark.core.utils.is_shape(obj)[source]#

Checks if the obj is broadcastable to a shape.

Parameters:: obj (Any) – tp.Any: the instance to check.
Returns:: bool, True if the object is broadcastable to a shape, False otherwise.
Return type:: bool

spark.core.utils.is_list_shape(obj)[source]#

Checks if the obj is broadcastable to a shape.

Parameters:: obj (Any) – tp.Any: the instance to check.
Returns:: bool, True if the object is broadcastable to a list of shapes, False otherwise.
Return type:: bool

spark.core.utils.is_dict_of(obj, value_cls, key_cls=str)[source]#

Check if an object instance is of ‘dict[key_cls, value_cls]’.

Parameters:

obj (Any) – tp.Any: the instance to check.
key_cls (type[Any]) – type[tp.Any], the class to compare keys against.
value_cls (type[Any]) – type[tp.Any], the class to compare values against.

Returns:

bool, True if the object is an instance of ‘dict[key_cls, value_cls]’, False otherwise.

Return type:

bool

spark.core.utils.is_list_of(obj, cls)[source]#

Check if an object instance is of ‘list[cls]’.

Parameters:

obj (Any) – tp.Any, the instance to check.
cls (type[Any]) – type[tp.Any], the class to compare values against.

Returns:

bool, True if the object is an instance of ‘list[cls]’, False otherwise.

Return type:

bool

spark.core.utils.is_dtype(obj)[source]#

Check if an object is a ‘DTypeLike’.

Parameters:: obj (tp.Any) – The instance to check.
Returns:: bool, True if the object is a ‘DTypeLike’, False otherwise.
Return type:: bool

spark.core.utils.is_float(obj)[source]#

Check if an object is a ‘DTypeLike’.

Parameters:: obj (tp.Any) – The instance to check.
Returns:: bool, True if the object is a ‘DTypeLike’, False otherwise.
Return type:: bool

spark.core.utils.ascii_tree(text)[source]#

Build an ASCII tree from indentation-based text. Each level is inferred from leading spaces.

Parameters:: text (str)
Return type:: str

class spark.core.utils.InheritanceFlags[source]#

Bases: enum.IntFlag

Support for integer-based Flags

Initialize self. See help(type(self)) for accurate signature.

CAN_INHERIT = 8[source]#

IS_INHERITING = 4[source]#

CAN_RECEIVE = 2[source]#

IS_RECEIVING = 1[source]#

class spark.core.utils.InheritanceLeaf[source]#

Leaf object for the InheritanceTree data structure.

name: str[source]#

type_string: str[source]#

inheritance_childs: list[list[str]][source]#

flags: InheritanceFlags = 0[source]#

break_inheritance: bool = False[source]#

parent: InheritanceTree = None[source]#

__post_init__()[source]#

Return type:: None

__repr__()[source]#

Return type:: str

to_dict()[source]#

Return type:: dict

classmethod from_dict(d)[source]#

Parameters:: d (dict)
Return type:: InheritanceLeaf

can_inherit()[source]#

Cheks the leaf node can inherit.

Return type:: bool

is_inheriting()[source]#

Cheks the leaf node is inheriting.

Return type:: bool

can_receive()[source]#

Cheks the leaf node can receive.

Return type:: bool

is_receiving()[source]#

Cheks the leaf node is receiving.

Return type:: bool

property path: list[str][source]#

Returns the path of the leaf node.

Return type:: list[str]

class spark.core.utils.InheritanceTree(path=[])[source]#

Tree-like data structure to manage the inheritance status of variables in the Spark Graph Editor.

This data structure is used to link variables with the same names and types for simultaneous updates within the GUI.

Parameters:: path (list[str])

__repr__()[source]#

Return type:: str

add_leaf(path, type_string='', inheritance_childs=[], flags=0, break_inheritance=False, **kwargs)[source]#

Adds a new leaf to the tree.

Input:: path: list[str], path to the new leaf node, with the last entry the name of the leaf type_string: str, string representation of the types this variable manages inheritance_childs: list[list[str]]=[], list of children that can inherit from this variable (Note: do not set by hand) flags: InheritanceFlags, 4-bit flags that represent inheritance possibilities (Note: do not set by hand) break_inheritance: bool, boolean flag to disconnect this variable from the inheritance dynamics

Parameters:

path (list[str])
type_string (str)
inheritance_childs (list[list[str]])
flags (InheritanceFlags)
break_inheritance (bool)

Return type:

None

add_branch(path)[source]#

Adds a new branch to the tree.

Input:: path: list[str], path to the new branch, with the last entry the name of the branch

Parameters:: path (list[str])
Return type:: None

validate(inheriting_labels={})[source]#

Validates the flags and the inheritance childs of the tree.

Parameters:: inheriting_labels (dict)
Return type:: None

get_leaf(path)[source]#

Returns the status of the leaf node.

Input:: path: list[str], path to the leaf node, with the last entry the name of the leaf

Returns:: InheritanceLeaf, returns the leaf node instance.
Parameters:: path (list[str])
Return type:: InheritanceLeaf

get_subtree(path)[source]#

Returns a subtree of the leaf node.

Input:: path: list[str], path to the subtree node, with the last entry the name of the branch

Returns:: InheritanceTree, returns the branch node instance.
Parameters:: path (list[str])
Return type:: InheritanceTree

to_dict()[source]#

InheritanceTree dict serializer.

Return type:: dict

classmethod from_dict(d, path=[])[source]#

InheritanceTree dict deserializer.

Parameters:

d (dict)
path (list[str])

Return type:

InheritanceTree

property path: list[str][source]#

Returns the path of the branch node.

Return type:: list[str]