flytekit.extend.TypeEngine

class flytekit.extend.TypeEngine[source]

Core Extensible TypeEngine of Flytekit. This should be used to extend the capabilities of FlyteKits type system. Users can implement their own TypeTransformers and register them with the TypeEngine. This will allow special handling of user objects

Methods

async classmethod async_to_literal(ctx, python_val, python_type, expected)[source]

Converts a python value of a given type and expected LiteralType into a resolved Literal value.

Parameters:
Return type:

Literal

async classmethod async_to_python_value(ctx, lv, expected_python_type)[source]
Parameters:
Return type:

Any

classmethod calculate_hash(python_val, python_type)[source]
Parameters:

  • python_val (Any)

  • python_type (Type[T])

Return type:

str | None

classmethod dict_to_literal_map(ctx, d, type_hints=None)[source]
Parameters:
Return type:

LiteralMap

async classmethod dict_to_literal_map_pb(ctx, d, type_hints=None)[source]
Parameters:
Return type:

LiteralMap | None

classmethod get_available_transformers()[source]

Returns all python types for which transformers are available

Return type:

KeysView[Type]

classmethod get_transformer(python_type)[source]

The TypeEngine hierarchy for flyteKit. This method looks up and selects the type transformer. The algorithm is as follows

d = dictionary of registered transformers, where is a python type v = lookup type

Step 1:

If the type is annotated with a TypeTransformer instance, use that.

Step 2:

find a transformer that matches v exactly

Step 3:

find a transformer that matches the generic type of v. e.g List[int], Dict[str, int] etc

Step 4:

Walk the inheritance hierarchy of v and find a transformer that matches the first base class. This is potentially non-deterministic - will depend on the registration pattern.

Special case:

If v inherits from Enum, use the Enum transformer even if Enum is not the first base class.

TODO lets make this deterministic by using an ordered dict

Step 5:

if v is of type data class, use the dataclass transformer

Step 6:

Pickle transformer is used

Parameters:

python_type (Type)

Return type:

TypeTransformer[T]

classmethod guess_python_type(flyte_type)[source]

Transforms a flyte-specific LiteralType to a regular python value.

Parameters:

flyte_type (LiteralType)

Return type:

type

classmethod guess_python_types(flyte_variable_dict)[source]

Transforms a dictionary of flyte-specific Variable objects to a dictionary of regular python values.

Parameters:

flyte_variable_dict (Dict[str, Variable])

Return type:

Dict[str, type]

classmethod lazy_import_transformers()[source]

Only load the transformers if needed.

classmethod literal_map_to_kwargs(ctx, lm, python_types=None, literal_types=None)[source]
Parameters:
Return type:

Dict[str, Any]

classmethod named_tuple_to_variable_map(t)[source]

Converts a python-native NamedTuple to a flyte-specific VariableMap of named literals.

Parameters:

t (NamedTuple)

Return type:

VariableMap

classmethod register(transformer, additional_types=None)[source]

This should be used for all types that respond with the right type annotation when you use type(…) function

Parameters:
classmethod register_additional_type(transformer, additional_type, override=False)[source]
Parameters:
classmethod register_restricted_type(name, type)[source]
Parameters:
classmethod to_html(ctx, python_val, expected_python_type)[source]
Parameters:
Return type:

str

classmethod to_literal(ctx, python_val, python_type, expected)[source]

The current dance is because we are allowing users to call from an async function, this synchronous to_literal function, and allowing this to_literal function, to then invoke yet another async functionl, namely an async transformer.

Parameters:
Return type:

Literal

classmethod to_literal_checks(python_val, python_type, expected)[source]
Parameters:
classmethod to_literal_type(python_type)[source]

Converts a python type into a flyte specific LiteralType

Parameters:

python_type (Type[T])

Return type:

LiteralType

classmethod to_python_value(ctx, lv, expected_python_type)[source]

Converts a Literal value with an expected python type into a python value.

Parameters:
Return type:

Any

async classmethod unwrap_offloaded_literal(ctx, lv)[source]
Parameters:
Return type:

Literal

Attributes

has_lazy_import = False
lazy_import_lock = <unlocked _thread.lock object>