evaluation.main package

Submodules

evaluation.main.ccs4dt_api_client_calls module

class evaluation.main.ccs4dt_api_client_calls.APIClient

Bases: object

This class represents the API client to call the CCS4DT API.

API_get_all_input_batches_call(location_id)

CCS4DT GET API call for all input batches of a specific location

Parameters

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • location_id (integer) – location id for which the input batches should be retrieved

Returns

Returns API response

Return type

dict(json)

API_get_all_locations_call()

CCS4DT GET API call for all locations

Parameters

API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

Returns

Returns API response

Return type

dict(json)

API_get_input_batch_by_id_call(location_id, input_batch_id)

CCS4DT GET API call for input batch by id

Parameters

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • location_id (integer) – location id for which the input batch was posted

  • input_batch_id (integer) – input batch id for which the results should be returned

Returns

Returns API response

Return type

dict(json)

API_get_location_by_id_call(location_id)

CCS4DT GET API call for location by id

Parameters

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • location_id (integer) – location id for which the input batch was posted

Returns

Returns API response

Return type

dict(json)

API_get_output_batch_call(location_id, batch_id)

CCS4DT GET API call for all output batch by id

Parameters

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • location_id (integer) – location id for which the input batch was posted

  • batch_id (integer) – input batch id for which the results should be returned

Returns

Returns API response

Return type

dict(json)

API_post_input_batch_call(payload, location_id)

CCS4DT POST API call for input batch

Parameters

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • payload (dict(json)) – Sensor measurements json payload

  • location_id (integer) – location id for which the measurements were taken

Returns

Returns API response and reponse parameters (id, status, location_id)

Return type

dict(json), string, string, string

API_post_new_location_call(payload)

CCS4DT POST API call for new location

Parameters

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • payload (dict(json)) – Location json payload

Returns

Returns API response and reponse parameters (location id, name)

Return type

dict(json), string, string

convert_sensor_measurements_to_api_conform_payload(additional_file_generation=False)

Take existing sensor measurement dataframe and convert it to API confirm input batch payload

Parameters

  • dataframe (pandas dataframe) – Dataframe containing the sensor measurements

  • additional_file_generation (boolean) – Flag that states if the json payload should also be stored as a file in the generated_files directory

Returns

Returns measurement JSON payload for CCS4DT API

Return type

dict(json)

end_to_end_API_test(sensors, api_endpoint_path, measurement_points=250, return_simulation_output=True, print_progress=True, store_measurement_data_in_json_file=True, debugger_files=False, identifier_randomization_method='random', identifier_type='mac-address', transform_to_3D_data=False, synthetic_data_generation=True, measurement_file_path='/')

Wrapper function that combines all necessary API calls to run an end to end API test

Parameters

  • location (integer) – Location object for which API request should be run

  • sensors (List of Sensor) – List of sensors that should be simulated / included in the API request

  • API_endpoint_path (string) – CCS4DT endpoint path (includes port if necessary)

  • measurement_points (integer) – Amount of measurement points that should be taken into consideration for synthetic data generation

  • return_simulation_output (boolean) – Flag that indicates if measurement data (i.e. synthetic sensor measurements) should be included in function return, default = True

  • print_progress (boolean) – Flag that indicates if progress should be printed to terminal, default = True

  • store_measurement_data_in_json_file (boolean) – Flag that indicates if input_batch should be stored as persistent .json in generated_files folder, default = True

  • debugger_files (boolean) – Flag that indicates if intermediate results (, output_batch) should be stored as persistent .json in generated_files folder, default = False

  • identifier_randomization_type (string) – For synthetic data generation: Specifies the randomization approach for object identifiers, default: ‘none’

  • identifier_type (string) – For synthetic data generation: Specifies type of identifier that should be used if randomization takes place, default: mac-address

  • transform_to_3D_data (boolean) – For synthetic data generation: Specifies if data should either consider the true position coordinate as 0 (if == False) or set the z-coordinate of the true positions as ‘height’ (if == True), default is True

  • synthetic_data_generation (boolean) – Flag that identifies if synthetic data needs to be generated (True) or if real-world data file can be taken (False), default = True

  • measurement_file_path (string) – Filepath to real-world sensor measurement file (only relevant if synthetic_data_generation == False)

Returns

Returns output batch (and optionally measurement)

Return type

dict(json), dict(json)

class evaluation.main.ccs4dt_api_client_calls.PredictionEvaluator(api_output, measurement_data)

Bases: object

This class represents the evaluator for a API request / synthetic measurement combination

Parameters

  • api_output (dict) – Output batch of the CCS4DT API

  • measurement_data (dict) – Synthetic sensor measurement data

add_object_identifier_mapping_to_measurement_dataframe()

Function that adds object identifier mapping to measurement data frame

Returns

Returns dataframe that includes object identifier mapping

Return type

pandas dataframe

add_prediction_data_to_merged_identifier_dataframe(debugger_files=False)

Function that adds prediction data (object ids, coordinates) to the original measurement dataframe which was enriched with identifiers of objects, true individuals and predictions

Parameters

debugger_files (boolean) – Flag that indicates if dataframes should be persisted in separate files for debugging purposes, default = False

Returns

Returns dataframe of true data, sensor measurements and predictions

Return type

pandas dataframe

calculate_object_matching_accuracy(clustering_evaluation_method='bcubed_fscore', debugger_files=True)

Function that calculates object matching accuracy score for the requested evaluation method

Parameters

  • clustering_evaluation_method (string) – Clustering evaluation method, default = bcubed_fscore

  • debugger_files (boolean) – Flag that indicates if results should be persisted in separate files for debugging purposes, default = True

Returns

Returns object matching accuracy score for the evaluation method

Return type

float

calculate_prediction_accuracy(accuracy_estimation_method='euclidean-distance', debugger_files=False, output_include_dataframe=False, clear_mot_threshold=1000000, output_precision_euclidean_distance=2, remove_upsamples=False)

Function that calculates position prediction score for the requested evaluation method

Parameters

  • accuracy_estimation_method (string) – Position rpediction evaluation method, default = euclidean-distance

  • debugger_files (boolean) – Flag that indicates if results should be persisted in separate files for debugging purposes, default = False

  • output_include_dataframe (boolean) – Flag that indicates if function should return prediction dataframe too, default = False

  • clear_mot_threshold (integer) – For CLEAR-MOT metrics: Squared cut-off value until which a point should be considered “correctly identifier”, default = 1000000

  • output_precision_euclidean_distance (integer) – For naive metric: Precision of metrics (i.e. decimal digits), default = 2

  • remove_upsamples (boolean) – Relevant for CLEAR-MOT metrics: Flag that indicates if upsampled datapoints from API response should be removed, default = False

Returns

Returns position prediction accuracy (and optionally the dataframe containing the predictions)

Return type

float or list of floats, pandas dataframe

filter_false_positives_introduced_due_to_upsampling()

Function that removes upsampled results from API response (i.e. predictions for each timestamp where there were no reported sensor measurements)

Returns

Returns dataframe of predictions excluding umsampled values

Return type

pandas dataframe

generate_prediction_dataframe(add_matching_initial_identifiers=False)

Function that generates a prediction dataframe

Parameters

add_matching_initial_identifiers (boolean) – Flag that indicates if initial (true) object mappings should be added too, default = False

Returns

Returns dataframe of predictions

Return type

pandas dataframe

get_object_identifier_mappings()

Getter function for object identifer mappings (i.e. mappings between predicted individuals and object ids from sensor measurements) results

Returns

Returns object identifier mapping

Return type

dict

get_prediction_results()

Getter function for (position) prediction results

Returns

Returns predicted positions

Return type

dict

evaluation.main.ccs4dt_api_client_calls.plot_position_accuracy_distribution(dataframe, analysis_dimension='total', bin_size=5)

Function that generates plots of error distribution for naive prediction accuracy

Parameters

  • dataframe (pandas dataframe) – Dataframe that contains position prediction errors and sensor data

  • analysis_dimension (string) – dimension in which data should be analyzed (e.g. total, per sensor, per sensor type), default = total

  • bin_size (integer) – bin size of distribution in cm, default = 5

Returns

None return (but generation of .html plots in generated_files folder)

Return type

None

evaluation.main.ccs4dt_api_client_calls.print_CLEAR_MOT_explanation()

evaluation.main.tempCodeRunnerFile module

Module contents