neuraxle.logging.logging

Module-level documentation for neuraxle.logging.logging. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Neuraxle’s Logging module

This module contains the Logging class, which is used to log information about the execution of a pipeline. It is used by the classes inheriting from BaseStep to log information. It is also modified by the AutoML class and its Trainer and various Trial repositories-related classes to log info in various folders.

Functions

register_log_producer_for_main_logger_thread_to_consume(…)

Classes

NeuraxleLogger(name[, level])
ParallelLoggingConsumerThread(logging_queue)	This class is used to receive logging messages sent from worker processes that are running in different PROCESSES (not threads).

class neuraxle.logging.logging._FilterSTDErr(name='')

Bases: logging.Filter

filter(record)

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

class neuraxle.logging.logging.NeuraxleLogger(name, level=0)

Bases: logging.Logger

static from_identifier(identifier: str) → neuraxle.logging.logging.NeuraxleLogger

Returns a logger from an identifier. :type identifier: str :param identifier: The identifier of the logger :return: The logger

with_string_io(identifier: str) → neuraxle.logging.logging.NeuraxleLogger

Returns a logger from an identifier. :type identifier: str :param identifier: The identifier of the logger :return: The logger

with_file_handler(file_path: str) → neuraxle.logging.logging.NeuraxleLogger

Returns a logger from an identifier. :type file_path: str :param identifier: The identifier of the logger :return: The logger

without_file_handler() → neuraxle.logging.logging.NeuraxleLogger

Returns a logger from an identifier. :param identifier: The identifier of the logger :return: The logger

read_log_file() → List[str]

with_std_handlers() → neuraxle.logging.logging.NeuraxleLogger

get_scoped_string_history() → str

get_root_string_history() → str

get_short_scoped_logs() → List[str]

get_short_root_logs() → List[str]

print_root_string_history() → None

_add_stream_handler(handler_name: str, stream: IO, level: Optional[int] = None, _filter: Optional[logging.Filter] = None) → neuraxle.logging.logging.NeuraxleLogger

_add_partial_handler(handler_name: str, handler: logging.Handler, level: Optional[int] = None, _filter: Optional[logging.Filter] = None) → neuraxle.logging.logging.NeuraxleLogger

static shorten_log_lines_prefixes(logs)

class neuraxle.logging.logging.ParallelLoggingConsumerThread(logging_queue: multiprocessing.context.BaseContext.Queue = None)

Bases: object

This class is used to receive logging messages sent from worker processes that are running in different PROCESSES (not threads).

The logging will work when using threads. However, when using processes (multiprocessing.Process) multiprocessing.Process , the present class is needed in the main process to receive the logging messages from the worker processes properly in the main console output.

__init__(logging_queue: multiprocessing.context.BaseContext.Queue = None)

Initialize self. See help(type(self)) for accurate signature.

start()

Start the thread to read the logs from the queue.

static logger_consumer_thread_func(queue: multiprocessing.context.BaseContext.Queue)

This function is destined to be run in a separate thread. Receiving a None signal in the queue means it’s time to finish and to break the while True. This None signal is as sent from within the self. join() () method. The Queue normally consumes logging.LogRecord items.

join(timeout: float = None)

Send the None signal for the thread to finish, and join on the thread to finish it. Note that once this method is called, the thread will finish and the threading.Thread object will be destroyed. Therefore, this method should be called only once and after the producers of parallel logging messages finished their job.

Timeout is in seconds.

neuraxle.logging.logging.register_log_producer_for_main_logger_thread_to_consume(logging_queue: multiprocessing.context.BaseContext.Queue)