RAI.metrics

Submodules

RAI.metrics.metric module

class RAI.metrics.metric.Metric(name, config)[source]

Bases: object

Metric class loads in information about a Metric as part of a Metric Group. Metrics are automatically created by Metric Groups.

load_config(config)[source]

loads the config details consisting of tags, has_range, range, explanation, type and display_name

Parameters:: config – Config details
Returns:: None

RAI.metrics.metric_group module

class RAI.metrics.metric_group.MetricGroup(ai_system)[source]

Bases: ABC

MetricGroups are a group of related metrics. This class loads in information about a metric group from its .json file. This class then creates associated metrics for the group, provides compatibility checking, run computes. Metric Groups are created by MetricManagers.

abstract compute(data)[source]

compute_batch(data)[source]

config = None

create_metrics(metrics_config)[source]

Create the metric and assign name and tags to it

Param:: metrics_config
Returns:: None

export_metric_values()[source]

Returns the metric with the name and its corresponding value

Param:: None
Returns:: dict

finalize_batch_compute()[source]

get_metric_values()[source]

Returns the metric with the name and its corresponding value

Param:: None
Returns:: dict

classmethod is_compatible(ai_system)[source]

Checks if the group is compatible with the provided AiSystem

Param:: ai_system
Returns:: Compatible object

load_config(config)[source]

Fetch the metric data from config

Param:: config
Returns:: Boolean

name = ''

reset()[source]

Reset the status

Param:: None
Returns:: None

update(data)[source]

RAI.metrics.metric_manager module

class RAI.metrics.metric_manager.MetricManager(ai_system)[source]

Bases: object

MetricManager is used to create and Manage various MetricGroups which are compatible with the AISystem. MetricManager is created by the AISystem, and will load in all available MetricGroups compatible with the AISystem. MetricManager also provides functions to run computes for all metric groups, get metadata about metric groups, and get metric values.

compute(data_dict) → dict[source]

Perform computation on metric objects and returns the value as a metric group in dict format

Parameters:: data_dict – Accepts the data dict metric object
Returns:: returns the value as a metric group

get_metadata() → dict[source]

Return the metric group metadata information

Parameters:: self – None
Returns:: dict-Metadata

get_metric_info_flat() → dict[source]

Returns the metric info

Parameters:: self – None
Returns:: Returns the metric info data in dict

initialize(user_config: dict | None = None, metric_groups: List[str] | None = None, max_complexity: str = 'linear')[source]

Find all compatible metric groups and Remove metrics with missing dependencies and Check for circular dependencies

Parameters:

user_config(dict) – user config data
metric_groups – metric groups data as a list
max_complexity – default linear

Returns:

None

iterator_compute(data_dict, preds: dict) → dict[source]

Accepts data_dict and preds as a input and returns the metric objects from a batch of metric group

Parameters:

data_dict – Accepts the data dict metric object
preds – prediction value from the detection

Returns:

returns the metric objects from a batch of metric group

reset_measurements() → None[source]

Reset all the certificate, metric, sample and time_stamp values

Parameters:: self – None
Returns:: None

search(query: str) → dict[source]

Searches all metrics.Queries based on Metric Name, Metric Group Name, Category, and Tags

Parameters:: query – query(str) group data information as input
Returns:: Returns the search results based on the query

standardize_user_config(user_config: dict)[source]

Accepts user config values and make in standard group

Parameters:: user_config(dict) – user config data
Returns:: None

RAI.metrics.metric_registry module

Registers Metric Classes on creation. All valid metric groups can then be found in the registry dictionary.

RAI.metrics.metric_registry.register_class(class_name, class_object)[source]