retico_core package

Submodules

Abstract Module

This module defines the abstract classes used by the incremental modules. The AbstractModules defines some methods that handle the general tasks of a module (like the handling of Queues). The IncrementalQueue defines the basic functionality of an incremental queue like appending an IU to the queue and letting modules subscribe to the Queue. The Incremental Unit provides the basic data structure to exchange information between modules.

class retico_core.abstract.AbstractConsumingModule(queue_class=<class 'retico_core.abstract.IncrementalQueue'>, meta_data={}, **kwargs)

Bases: AbstractModule

An abstract consuming module that is able to incrementally process data.

The consuming module consumes IUs but does not return any data.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

subscribe(module, q=None)

Subscribe a module to the queue.

It returns a queue where the IUs for that module are placed. The queue is not shared with other modules. By default this method creates a new queue, but it may use an alternative queue given in parameter ‘q’.

Parameters:

module (AbstractModule) – The module that wants to subscribe to the output of the module.
q (IncrementalQueue) – A optional queue that is used. If q is None, the a new queue will be used

class retico_core.abstract.AbstractModule(queue_class=<class 'retico_core.abstract.IncrementalQueue'>, meta_data={}, **kwargs)

Bases: object

An abstract module that is able to incrementally process data.

EVENT_PROCESS_IU = 'process_iu'

EVENT_PROCESS_UPDATE_MESSAGE = 'process_update_message'

EVENT_START = 'start'

EVENT_STOP = 'stop'

EVENT_SUBSCRIBE = 'subscribe'

QUEUE_TIMEOUT = 0.01: Timeout in seconds for the incremental queues as not to block processing.

add_left_buffer(left_buffer)

Add a new left buffer for the module.

This method stops the execution of the module pipeline if it is running.

Parameters:: left_buffer (IncrementalQueue) – The left buffer to add to the module.

add_right_buffer(right_buffer)

Add a new right buffer for the module.

This method stops the execution of the module pipeline if it is running.

Parameters:: right_buffer (IncrementalQueue) – The right buffer to add to the module.

append(update_message)

Append an update message to all queues.

If update_message is None or there are no IUs in the update message, the method returns without doing anything.

Parameters:: update_message (UpdateMessage) – The update message that should be added to all output queues. May be None.

commit(iu)

Sets an IU as committed from the list of the current_input or current_output, depending on where it is found.

Parameters:: iu (IncrementalUnit) – The incremental unit to set as committed.

create_iu(grounded_in=None)

Creates a new Incremental Unit that contains the information of the creator (the current module), the previous IU that was created in this module and the iu that it is based on.

Do not discard (as in not using) any IU that was created by this method, because it will alreade have been introduced into the chain of IUs of this module!

Parameters:: grounded_in (IncrementalUnit) – The incremental unit that the new unit is based on. May be None.
Returns:: A new incremental unit with correct pointer to unit it is grounded in and to the previous IU that was generated by this module.
Return type:: IncrementalUnit

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

event_call(event_name, data={})

Calls all callback functions that are subscribed to the given event name with some data attached to it. The data is optional but should stay consistent with each call of the same event.

If * is passed as the event name, no callback function is called.

Event name should be a unique identifier to the event. “*” is not allowed as an event name.

Parameters:

event_name (str) – The name of the event (not “*”)
data (dict) – Optionally some data that is relevant to the event.

event_subscribe(event_name, callback)

Subscribe a callback to an event with the given name. If tge event name is “*”, then the callback will be called after every event.

The callback function is given three arguments: the module that triggered the event (AbstractModule), the name of the event (str) and a dict (dict) that may contain data relevant to the event.

Parameters:

event_name (str) – The name of the event to subscribe to
callback (function) – A function that is called once the event occurs

get_init_arguments()

Returns the arguments of the init function to create the current instance of the Module.

Returns:: A dictionary containing all the necessary arguments to create the current instance of the module.
Return type:: dict

input_committed()

Checks whether all IUs in the input are committed.

Returns:: True when all IUs in the current_input is committed, False otherwise.
Return type:: bool

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

is_valid_input_iu(iu)

Return whether the given IU is a valid input IU.

Valid is defined by the list given by the input_ius function. The given IU must be one of the types defined in that list or be a subclass of it.

Parameters:: iu (IncrementalUnit) – The IU to be checked.
Raises:: TypeError – When the given object is not of type IncrementalUnit.
Returns:: Whether the given iu is a valid one for this module.
Return type:: bool

latest_iu()

Provides reading access to the latest incremental unit that was produced by this module.

Thus, the information received by this method might be out of date or completely wrong (in case where a not yet initialized IU is returned). The iu returned should not be modified in any way, because it could still be processed by a module.

Returns:: The latest IU that was produced by the module.
Return type:: (IncrementalUnit)

left_buffers()

Returns the list of left buffers of the module.

Returns:: The left buffers of the module.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

prepare_run()

A method that is executed just before the module is being run.

While this method may seem similar to setup, it is called immediately before the run routine. This method may be used in producing modules to initialize the generation of output IUs. Other than the setup method, this method makes sure that other modules in the network are also already setup.

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

remove()

Removes all connections to all modules.

This methods removes all queues from the left buffer and right buffer. The queues are also removed from the buffers of the connected modules. This method can be used to remove a module completely from a network.

This method stops the execution of the module.

remove_from_lb(module)

Removes the connection to a module from the left buffers.

This method removes all queues between this module and the given module from the left buffer of this module and the right buffer of the given module. This method stops the execution of the module.

Parameters:: module – A module that this module is subscribed to

remove_from_rb(module)

Removes the connection to a module from the right buffers.

This method removes all queues between this module and the given module from the right buffer of this module and the left buffer of the given module. This method stops the execution of the module.

Parameters:: module – A module that is subscribed to this module

remove_left_buffer(left_buffer)

Remove a left buffer from the module.

This method stops the execution of the module pipeline if it is running.

Parameters:: left_buffer (IncrementalQueue) – The left buffer to remove from the module.

remove_right_buffer(right_buffer)

Remove a right buffer from the module.

This method stops the execution of the module pipeline if it is running.

Parameters:: right_buffer (IncrementalQueue) – The right buffer to remove from the module.

revoke(iu, remove_revoked=True)

Revokes an IU form the list of the current_input or current_output, depending on in which list it is found.

Parameters:

iu (IncrmentalUnit) – The incremental unit to revoke.
remove_revoked (bool) – Whether the revoked incremental unit should be deleted from the current_input or current_output list or if only the revoked flag should be set.

right_buffers()

Return the right buffers of the module.

Note that the returned list is only a shallow copy. Modifying the list does not alter the internal state of the module (but modifying the queues in that list does).

Returns:: A list of the right buffers, each queue corresponding to an input of another module.
Return type:: list

run(run_setup=True)

Run the processing pipeline of this module in a new thread. The thread can be stopped by calling the stop() method.

Parameters:

run_setup (bool) – Whether or not the setup method should be executed
started. (before the thread is)

setup()

This method is called before the module is run. This method can be used to set up the pipeline needed for processing the IUs.

However, after the setup method is called, the module may not immediately be run. For code that should be executed immediately before a module is run use the prepare_run method.

shutdown(): This method is called before the module is stopped. This method can be used to tear down the pipeline needed for processing the IUs.

stop(clear_buffer=True): Stops the execution of the processing pipeline of this module at the next possible point in time. This may be after the next incoming IU is processed.

subscribe(module, q=None)

Subscribe a module to the queue.

It returns a queue where the IUs for that module are placed. The queue is not shared with other modules. By default this method creates a new queue, but it may use an alternative queue given in parameter ‘q’.

Parameters:

module (AbstractModule) – The module that wants to subscribe to the output of the module.
q (IncrementalQueue) – A optional queue that is used. If q is None, the a new queue will be used

class retico_core.abstract.AbstractProducingModule(queue_class=<class 'retico_core.abstract.IncrementalQueue'>, **kwargs)

Bases: AbstractModule

An abstract producing module that is able to incrementally process data.

The producing module has no input queue and thus does not wait for any input. The producing module is called continously and may return new output when it becomes available.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

class retico_core.abstract.AbstractTriggerModule(queue_class=<class 'retico_core.abstract.IncrementalQueue'>, **kwargs)

Bases: AbstractProducingModule

An abstract trigger module that produces an update message once a trigger method is called. Unless the module is triggered no updates are produced

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

trigger(data={}, update_type=UpdateType.ADD)

The trigger method that should produce an update message and append it to the right buffer

Parameters:

data (dict) – A dictionary with data that can be used for the trigger
update_type (UpdateType) – The update type that the IU should have. Default is UpdateType.ADD

class retico_core.abstract.IncrementalQueue(provider, consumer, maxsize=0)

Bases: Queue

An abstract incremental queue.

A module may subscribe to a queue of another module. Every time a new incremental unit (IU) is produced, the IU is put into a special queue for every subscriber to the incremental queue. Every unit gets its own queue and may process the items at different speeds.

provider

The module that provides IUs for this queue.

Type:: AbstractModule

consumer

The module that consumes IUs for this queue.

Type:: AbstractModule

maxsize

The maximum size of the queue, where 0 does not restrict the size.

Type:: int

remove(): Removes the queue from the consumer and the producer.

class retico_core.abstract.IncrementalUnit(creator=None, iuid=None, previous_iu=None, grounded_in=None, payload=None, **kwargs)

Bases: object

An abstract incremental unit.

The IU may be used for ASR, NLU, DM, TT, TTS, … It can be redefined to fit the needs of the different module (and module-types) but should always provide these functionalities.

The meta_data may be used when an incremental module is having additional information because it is working in a simulated environemnt. This data can be used by later modules to keep the simulation going.

creator

The module that created this IU

Type:: AbstractModule

previous_iu

A link to the IU created before the current one.

Type:: IncrementalUnit

grounded_in

A link to the IU this IU is based on.

Type:: IncrementalUnit

created_at

The UNIX timestamp of the moment the IU is created.

Type:: float

meta_data

Meta data that offers optional meta information. This field can be used to add information that is not available for all uses of the specific incremental unit.

Type:: dict

MAX_DEPTH = 50: Maximum depth of the previous_iu and grounded_in connections.

age()

Returns the age of the IU in seconds.

Returns:: The age of the IU in seconds
Return type:: float

from_zmq(zmq_data): reconstitues an IU from a formatted zeromq string

is_processed_by(module)

Return True if the IU is processed by the given module.

If the given object is a module that has not processed this IU or is not a module it returns False.

Parameters:: module (AbstractModule) – The module to test whether or not it has processed the IU
Returns:: Whether or not the module has processed the IU.
Return type:: bool

older_than(s)

Return whether the IU is older than s seconds.

Parameters:: s (float) – The time in seconds to check against.
Returns:: Whether or not the age of the IU exceeds s seconds.
Return type:: bool

processed_list()

Return a list of all modules that have already processed this IU.

The returned list is a copy of the list held by the IU.

Returns:: A list of all modules that have alread processed this IU.
Return type:: list

set_processed(module)

Add the module to the list of modules that have already processed this IU.

Parameters:: module (AbstractModule) – The module that has processed this IU.

to_zmq(update_type): returns a formatted string that can be sent across zeromq

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.abstract.UpdateMessage

Bases: object

A class that encapsulates multiple incremental units and their update type. The update types can be any of the ones defined in the enum UpdateType

add_iu(iu, update_type, strict_update_type=True)

Adds an incremental unit to the update message with the given update type.

If a single UpdateType-IncrementalUnit-Tuple raises a TypeError or a ValueError, none of the units will be added to the update message.

Parameters:

iu (IncrementalUnit) – The incremental unit to be added to the update message
update_type (UpdateType) – The type of the update the should be associated with the incremental unit.
strict_update_type (bool) – Whether the update type should be checked and converted to type UpdateType. If the given argument is not of type UpdateType or a str that can be converted to UpdateType, a ValueError is raised.

Raises:

TypeError – When the given incremental unit is not of type IncrementalUnit or if the update_type does not correspond to an update type.
ValueError – When the given udpate type is not a valid update type or the given argument cannot be converted to an UpdateType. Only applies if the strict_update_type flag is set.

add_ius(iu_list, strict_update_type=True)

Adds a list of incremental units and according update types to the update message.

Parameters:

iu_list (list) – A list containing tuples of update types and incremental
format (units in the)
strict_update_type (bool) – Whether the update type should be checked and converted to type UpdateType. If the given argument is not of type UpdateType or a str that can be converted to UpdateType, a ValueError is raised.

Raises:

TypeError – When the given incremental unit is not of type IncrementalUnit or if the update_type does not correspond to an update type.
ValueError – When the given udpate type is not a valid update type or the given argument cannot be converted to an UpdateType. Only applies if the strict_update_type flag is set.

classmethod from_iu(iu, update_type)

Initializes the update message with an initial pair of incremental unit and update type.

Parameters:

iu (IncrementalUnit) – The first incremental unit of the update message
update_type (UpdateType) – The update type of the incremental unit.

classmethod from_iu_list(self, iu_list)

Initializes the update message with a list of tuples containing the update type and incremental units in the format (IncrementalUnit, UpdateType)

Parameters:: iu_list (list) – A list of IncrementalUnit-UpdateType-tuples in the format (IncrementalUnit, UpdateType) that will be added to the update message.

has_valid_ius(iu_classes)

Checks whether the IUs in this update message are all of the type provided in the ius argument.

Parameters:

iu_classes (list or class) – A list of incremental unit classes or a single incremental unit class that should be checked against.

Returns:

Returns true if all the incremental unit in the udpate message are: instances of the iu_classes given in the parameter.

Return type:

bool

incremental_units(): A generator that iterates of all the incremental units of the update message, ignoring the update types.

set_processed(module)

Sets all the incremental units of the update message as processed by the module that is given

Parameters:

module (IncrementalModule) – The module that has processed the incremental
message. (units of this update)

update_types(): A generator that iterates of all the update types of the update message, ignoring the incemental units

class retico_core.abstract.UpdateType(value)

Bases: Enum

The update type enum that defines all the types with which the incremental units can be transmitted. Per default, the UpdateMessge class checks that the update type is one of the types listed in this enum. However, the strict type checking can be disabled and any update type may be used.

ADD = 'add'

COMMIT = 'commit'

REVOKE = 'revoke'

UPDATE = 'update'

Audio Module

This module defines basic incremental units and incremental modules to handle audio input (via a standard microphone) and output.

class retico_core.audio.AudioDispatcherModule(target_frame_length=0.02, rate=44100, sample_width=2, speed=1.0, continuous=True, silence=None, interrupt=True, **kwargs)

Bases: AbstractModule

An Audio module that takes a raw audio stream of arbitrary size and outputs AudioIUs with a specific chunk size at the rate it would be produced if the audio was being played.

This could be espacially useful when an agents’ TTS module produces an utterance, but this utterance should not be transmitted as a whole but in an incremental way.

target_frame_length

The size of each output IU in seconds.

Type:: float

target_chunk_size

The size of each output IU in samples.

Type:: int

silence

A bytes array containing [target_chunk_size] samples of silence that is dispatched when [continuous] is True and no input IU is dispatched.

Type:: bytes

continuous

Whether or not the dispatching should be continuous. If True, AudioIUs with “silence” will be disptached if no input IUs are being dispatched. If False, no IUs will be produced during silence.

Type:: bool

rate

The sample rate of the outout and the input IU.

Type:: int

sample_width

The sample with of the output and input IU.

Type:: int

speed

The speed of the dispatching. 1.0 means realtime.

Type:: float

dispatching_mutex

The mutex if an input IU is currently being dispatched.

Type:: threading.Lock

audio_buffer

The current audio buffer containing the output IUs that are currently dispatched.

Type:: list

run_loop

Whether or not the dispatching loop is running.

Type:: bool

interrupt

Whether or not incoming IUs interrupt the old dispatching

Type:: bool

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

is_dispatching()

Return whether or not the audio dispatcher is dispatching a Speech IU.

Returns:: Whether or not speech is currently dispatched
Return type:: bool

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

prepare_run()

A method that is executed just before the module is being run.

While this method may seem similar to setup, it is called immediately before the run routine. This method may be used in producing modules to initialize the generation of output IUs. Other than the setup method, this method makes sure that other modules in the network are also already setup.

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

set_dispatching(value)

Set the dispatching value of this module in a thread safe way.

Parameters:: value (bool) – The new value of the dispatching flag.

shutdown(): This method is called before the module is stopped. This method can be used to tear down the pipeline needed for processing the IUs.

class retico_core.audio.AudioIU(creator=None, iuid=0, previous_iu=None, grounded_in=None, rate=None, nframes=None, sample_width=None, raw_audio=None, **kwargs)

Bases: IncrementalUnit

An audio incremental unit that receives raw audio data from a source.

The audio contained should be monaural.

creator

The module that created this IU

Type:: AbstractModule

previous_iu

A link to the IU created before the current one.

Type:: IncrementalUnit

grounded_in

A link to the IU this IU is based on.

Type:: IncrementalUnit

created_at

The UNIX timestamp of the moment the IU is created.

Type:: float

raw_audio

The raw audio of this IU

Type:: bytes[]

rate

The frame rate of this IU

Type:: int

nframes

The number of frames of this IU

Type:: int

sample_width

The bytes per sample of this IU

Type:: int

audio_length()

Return the length of the audio IU in seconds.

Returns:: Length of the audio in this IU in seconds.
Return type:: float

set_audio(raw_audio, nframes, rate, sample_width): Sets the audio content of the IU.

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.audio.AudioRecorderModule(filename, rate=44100, sample_width=2, **kwargs)

Bases: AbstractConsumingModule

A Module that consumes AudioIUs and saves them as a PCM wave file to disk.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

setup()

This method is called before the module is run. This method can be used to set up the pipeline needed for processing the IUs.

However, after the setup method is called, the module may not immediately be run. For code that should be executed immediately before a module is run use the prepare_run method.

shutdown(): This method is called before the module is stopped. This method can be used to tear down the pipeline needed for processing the IUs.

retico_core.audio.CHANNELS = 1: Number of channels. For now, this is hard coded MONO. If there is interest to do stereo or audio with even more channels, it has to be integrated into the modules.

class retico_core.audio.DispatchedAudioIU(**kwargs)

Bases: AudioIU

A type of audio incremental unit that is dispatched by an AudioDispatcherModule. It has the information of the percentual completion of the dispatched audio. This may be useful for a dialog manager that wants to track the status of the current dispatched audio.

set_dispatching(completion, is_dispatching)

Set the completion percentage and the is_dispatching flag.

Parameters:

completion (float) – The degree of completion of the current utterance.
is_dispatching (bool) – Whether or not the dispatcher is currently dispatching

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.audio.MicrophoneModule(frame_length=0.02, rate=16000, sample_width=2, **kwargs)

Bases: AbstractProducingModule

A module that produces IUs containing audio signals that are captured by a microphone.

callback(in_data, frame_count, time_info, status)

The callback function that gets called by pyaudio.

Parameters:

in_data (bytes[]) – The raw audio that is coming in from the microphone
frame_count (int) – The number of frames that are stored in in_data

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

prepare_run()

A method that is executed just before the module is being run.

While this method may seem similar to setup, it is called immediately before the run routine. This method may be used in producing modules to initialize the generation of output IUs. Other than the setup method, this method makes sure that other modules in the network are also already setup.

process_update(_)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

setup(): Set up the microphone for recording.

shutdown(): Close the audio stream.

class retico_core.audio.SpeakerModule(rate=44100, sample_width=2, use_speaker='both', device_index=None, **kwargs)

Bases: AbstractConsumingModule

A module that consumes AudioIUs of arbitrary size and outputs them to the speakers of the machine. When a new IU is incoming, the module blocks as long as the current IU is being played.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

setup(): Set up the speaker for outputting audio

shutdown(): Close the audio stream.

class retico_core.audio.SpeechIU(**kwargs)

Bases: AudioIU

A type of audio incremental unit that contains a larger amount of audio information and the information if the audio should be dispatched or not.

This IU can be processed by an AudioDispatcherModule which converts this type of IU to AudioIU.

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.audio.StreamingSpeakerModule(frame_length=0.02, rate=44100, sample_width=2, **kwargs)

Bases: AbstractConsumingModule

A module that consumes Audio IUs and outputs them to the speaker of the machine. The audio output is streamed and thus the Audio IUs have to have exactly [chunk_size] samples.

callback(in_data, frame_count, time_info, status): The callback function that gets called by pyaudio.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

prepare_run()

A method that is executed just before the module is being run.

While this method may seem similar to setup, it is called immediately before the run routine. This method may be used in producing modules to initialize the generation of output IUs. Other than the setup method, this method makes sure that other modules in the network are also already setup.

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

setup(): Set up the speaker for speaking…?

shutdown(): Close the audio stream.

retico_core.audio.TIMEOUT = 0.01: Timeout in seconds used for the StreamingSpeakerModule.

retico_core.audio.show_audio_devices(): Shows all availbale audio input and output devices using pyAudio.

Debug Module

This file contains general debug modules that can be used to output information from any incremental unit.

class retico_core.debug.CallbackModule(callback, **kwargs)

Bases: AbstractConsumingModule

A debug module that returns the incoming update messages into a callback function.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

class retico_core.debug.DebugModule(print_payload_only=False)

Bases: AbstractConsumingModule

A debug module that prints the IUs that are coming in.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

class retico_core.debug.TextPrinterModule(**kwargs)

Bases: AbstractConsumingModule

A debug module that prints the incoming text and updates it as text IUs are arriving. Once an IU is committed, the next incoming text is printed in a new line.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

Dialogue Module

This module contains basic dialogue functionality, like a DialogueActIU for a dialogue manager.

class retico_core.dialogue.DialogueActIU(creator=None, iuid=0, previous_iu=None, grounded_in=None, payload=None, act=None, concepts=None, **kwargs)

Bases: IncrementalUnit

A Dialog Act Incremental Unit.

This IU represents a Dialogue Act together with concepts and their values. In this implementation only a single act can be expressed with a single IU.

act

A representation of the current act as a string.

Type:: string

concepts

A dictionary of names of concepts being mapped on to their actual values.

Type:: dict

set_act(act, concepts=None, confidence=1.0)

Set the act and concept of the IU.

Old acts or concepts will be overwritten.

Parameters:

act (string) – The act of the IU as a string.
concepts (dict) – A dictionary containing the new concepts.
confidence (float) – Confidence of the act prediction

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.dialogue.DialogueActRecorderModule(filename, separator='\t', **kwargs)

Bases: AbstractConsumingModule

A module that writes dispatched dialogue acts to file.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

prepare_run()

A method that is executed just before the module is being run.

While this method may seem similar to setup, it is called immediately before the run routine. This method may be used in producing modules to initialize the generation of output IUs. Other than the setup method, this method makes sure that other modules in the network are also already setup.

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

setup()

This method is called before the module is run. This method can be used to set up the pipeline needed for processing the IUs.

However, after the setup method is called, the module may not immediately be run. For code that should be executed immediately before a module is run use the prepare_run method.

shutdown(): This method is called before the module is stopped. This method can be used to tear down the pipeline needed for processing the IUs.

class retico_core.dialogue.DialogueActTriggerModule(dispatch=True, **kwargs)

Bases: AbstractTriggerModule

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

trigger(data={}, update_type=UpdateType.ADD)

The trigger method that should produce an update message and append it to the right buffer

Parameters:

data (dict) – A dictionary with data that can be used for the trigger
update_type (UpdateType) – The update type that the IU should have. Default is UpdateType.ADD

class retico_core.dialogue.DispatchableActIU(dispatch=False, **kwargs)

Bases: DialogueActIU

A Dialogue Act Incremental Unit that can has the information if it should be dispatched once it has been transformed into speech.

dispatch

Whether the speech resulting from this IU should be dispatched or not.

Type:: bool

class retico_core.dialogue.EndOfTurnIU(**kwargs)

Bases: IncrementalUnit

An incremental unit used for prediction of the end of the turn. This information may be used by a dialogue management module to plan next turns and enabling realistic turn taking.

set_eot(probability=0.0, is_speaking=False)

Set the end-of-turn probability and a flag if the interlocutor is currently speaking (VAD).

Parameters:

probability (float) – The probability that the turn is ending.
is_speaking (bool) – Whether or not the interlocutor is speaking.

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.dialogue.GenericDictIU(creator=None, iuid=0, previous_iu=None, grounded_in=None, payload=None, **kwargs)

Bases: IncrementalUnit

A Generic IU type that holds dictionaries

dictionaries can be turned into JSON for interop
dictionary can be added to a dialogue state

set_payload(payload)

Set the dictionary payload

Parameters:: payload (dict)

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

Network Module

A Module that allows for saving and loading networks from and to file as well as starting and stopping them based on only a single module.

retico_core.network.discover(module)

Discovers all modules and connections from a single a list of modules.

The network is automatically discovered by traversing all left and right buffers of the modules given. If the argment module is only a single module, the network that is constructed consist only of modules and connections reachable by that module. A segmented network needs a module of each part of the network.

The function returns a touple containing a list of module and a list of connections between the modules. The connections are touples containing the providing module of the connection as the first element and the receiving module as the second element.

Parameters:

module (AbstractModule or list) – A module of the network or a list of multiple modules of the network

Returns:

A list of modules in the first return value and: a list of connections in the second return value.

Return type:

list, list

retico_core.network.load(filename: str)

Loads a network from file and returns a list of modules in that network.

The connections between the module have been set according to the file.

Parameters:

filename (str) – The path to the .rtc file containing a network.

Returns:

A list of Modules that are connected and ready to be run: and a list of connections between those modules.

Return type:

(list, list)

retico_core.network.load_and_execute(filename)

Loads a network from file and runs it.

The network is loaded via the load-Method. Before running the network, it is setup.

The networks runs until some input on the console is given

Parameters:: filename (str) – The path to the .rtc file containing a network.

retico_core.network.run(module)

Properly prepares and runs a network based on one module or a list of modules.

The network is automatically discovered so that only one module of the network has to be given to this function for the whole network to be executed. The function first calls the setup function of each module in the network and then runs all modules.

Parameters:: module (Abstract Module or list) – A module of the network or a list of multiple module of the network

retico_core.network.save(module, filename)

Saves a network to file given a module or a list of modules.

The network is automatically detected by traversing all left and right buffers of the modules given. If the argument module is only a single module, the network that is being saved consists only of the module reachable from this module. If a network should be saved that is splitted into multiple parts, at least one module of each split has to be included into the module-list.

Parameters:

module (AbstractModule or list) – A module of the network or a list of multiple modules of the network.
filename (str) – The path to where the network should be stored. This excludes the file-ending .rtc that will be automatically added by this function.

retico_core.network.stop(module)

Properlystops a network based on one module or a list of modules.

The network is automatically discovered so that only one module of the network has to be given to this function for the whole network to be stopped.

Parameters:: module (Abstract Module or list) – A module of the network or a list of multiple module of the network

Text Module

This file defines general incremental units and incremental modules that deal with text. This may be a transcription that is generated from an ASR module, a text IU containing words to be synthesized by a TTS module or other general purpose text.

class retico_core.text.EndOfUtteranceModule(**kwargs)

Bases: AbstractModule

A module that looks for the “final” flag of a SpeechRecognitionIU and forwards an EndOfTurnIU when the SpeechRecognition detected that the utterance is finished.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

class retico_core.text.GeneratedTextIU(dispatch=False, **kwargs)

Bases: TextIU

An IU that contains generated text.

This includes information about whether the text should be dispatched once it has been transformed into speech.

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.text.IncrementalizeASRModule(threshold=0.8, **kwargs)

Bases: AbstractModule

A module that takes the output of a non-incremental ASR module, where each IU contains the full text of the speech recognition and produces increments based on the difference to the last output.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

class retico_core.text.SpeechRecognitionIU(creator, iuid=0, previous_iu=None, grounded_in=None, payload=None)

Bases: TextIU

An IU that contains information about recognized speech.

get_text()

Return the text contained in the IU.

Returns:: The text contained in the IU.
Return type:: str

set_asr_results(predictions, text, stability, confidence, final)

Set the asr results for the SpeechRecognitionIU.

Parameters:

predictions (list) – A list of predictions. This will also set the payload. The last prediction in this list should be the latest and best prediction.
text (str) – The text of the latest prediction
stability (float) – The stability of the latest prediction
confidence (float) – The confidence in the latest prediction
final (boolean) – Whether the prediction is final

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.text.TextDispatcherModule(dispatch_final=True, **kwargs)

Bases: AbstractModule

A Moduel that turns SpeechRecognitionIUs or TextIUs into GeneratedTextIUs that have the dispatch-flag set.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

class retico_core.text.TextIU(creator=None, iuid=None, previous_iu=None, grounded_in=None, payload=None, **kwargs)

Bases: IncrementalUnit

An IU that contains text.

get_text()

Return the text contained in the IU.

Returns:: The text contained in the IU.
Return type:: str

set_text(text)

Sets the text contained in the IU.

Parameters:: text (str) – The new text of the IU

property text

Return the text contained in the IU.

Returns:: The text contained in the IU.
Return type:: str

static type()

Return the type of the IU in a human-readable format.

Returns:: The type of the IU in a human-readable format.
Return type:: str

class retico_core.text.TextRecorderModule(filename, separator='\t', **kwargs)

Bases: AbstractConsumingModule

A module that writes the received text into a file.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static input_ius()

Return the list of IU classes that may be processed by this module.

If an IU is passed to the module that is not in this list or a subclass of this list, an error is thrown when trying to process that IU.

Returns:: A list of classes that this module is able to process.
Return type:: list

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

process_update(update_message)

Processes the update message given and returns a new update message that can be appended to the output queues.

Note that the incremental units in the update message that is returned should be created by the create_iu method so that they have correct references to the previous incremental units generated by this module and the IUs that they are based on.

It is important that the process_update method processes the update messages in a timely manner (in regards to the production of update messages from the preceding module) so that the incremental queues do not overflow.

Parameters:: update_message (UpdateMessage) – The update message that should be processed by the module.
Returns:: An update message that is produced by this module based on the incremental units that were given. May be None.
Return type:: UpdateMessage

setup()

This method is called before the module is run. This method can be used to set up the pipeline needed for processing the IUs.

However, after the setup method is called, the module may not immediately be run. For code that should be executed immediately before a module is run use the prepare_run method.

shutdown(): This method is called before the module is stopped. This method can be used to tear down the pipeline needed for processing the IUs.

class retico_core.text.TextTriggerModule(dispatch=True, **kwargs)

Bases: AbstractTriggerModule

A trigger module that creates a TextIU once its trigger function is called.

static description()

Return the human-readable description of the module.

Returns:: A string containing the description of the module
Return type:: str

static name()

Return the human-readable name of the module.

Returns:: A string containing the name of the module
Return type:: str

static output_iu()

Return the class of IU that this module is producing.

Returns:: The class of IU this module is producing.
Return type:: class

trigger(data={}, update_type=UpdateType.ADD)

The trigger method that should produce an update message and append it to the right buffer

Parameters:

data (dict) – A dictionary with data that can be used for the trigger
update_type (UpdateType) – The update type that the IU should have. Default is UpdateType.ADD

retico_core.text.get_text_increment(module, new_text)

Compares the full text given by the asr with the IUs that are already produced (current_output) and returns only the increment from the last update. It revokes all previously produced IUs that do not match.

For example, if the current_output of the module consists of the follwing IUs:: [The] [quick] [bright]
and the new_text is:: “The quick brown fox”
The function would return a new UpdateMessage: UpdateMessage([bright], REVOKE)
and the follwing new tokens:: [“brown”, “fox”]

This function does not create UPDATE messages (e.g., updating [bright] to [brown] in the example above), but rather REVOKEs all unfitting IUs.

Parameters:

module (IncrementalModule) – The incremental module that produces TextIU s
new_text (str) – The full new text that should be incrementalized

Returns:

Returns an update message containing the IUs that were: revoked (with the REVOKE update type) and a list of new tokens (strings) for which new IUs should be created.

Return type:

UpdateMessage, list