core.internals.runner¶
Submodule of khiops.core.internals
Classes implementing Khiops Python’ backend runners
Functions¶
|
|
Returns the status of a local or remote directory |
|
Detect Linux distribution name |
|
Returns the current KhiopsRunner of the module |
|
Sets the current KhiopsRunner of the module |
Classes¶
Implementation of a local Khiops runner |
|
Abstract Khiops Python runner to be re-implemented |
|
Deprecated |
|
Deprecated |
- class khiops.core.internals.runner.KhiopsLocalRunner¶
Bases:
KhiopsRunner
Implementation of a local Khiops runner
Requires either:
This package installed through Conda and run from a Conda environment, or
the Khiops desktop app installed on the local machine
Environment variables taken into account by the runner:
part of the Khiops API:
KHIOPS_PROC_NUMBER
: number of processes launched by KhiopsKHIOPS_MEMORY_LIMIT
: memory limit of the Khiops executables in megabytes; ignored if set above the system memory limitKHIOPS_TMP_DIR
: path to Khiops’ temporary directory
other environment variables:
KHIOPS_HOME
: Windows only path to the Khiops installation directoryKHIOPS_SAMPLES_DIR
: path to the Khiops sample datasets directory
advanced configuration variables. Most of the time the user does not need modify them:
KHIOPS_MPI_COMMAND_ARGS
: arguments to thempiexec
commandKHIOPS_MPIEXEC_PATH
: path to thempiexec
commandKHIOPS_MPI_LIB
: Linux and MacOS only path to the MPI library; added to the beginning ofLD_LIBRARY_PATH
Samples directory settings
Default values for the
samples_dir
:The value of the
KHIOPS_SAMPLES_DIR
environment variableOtherwise:
Windows:
%PUBLIC%\khiops_data\samples%
if it exists and is a directory%USERPROFILE%\khiops_data\samples%
otherwise
Linux and macOS:
$HOME/khiops_data/samples
- property khiops_bin_dir¶
str: Path of the directory containing Khiops’ binaries
Default values:
conda installation:
$CONDA_PREFIX/bin
system-wide installations :
Windows:
%KHIOPS_HOME%\bin
%KhiopsHome%\bin
(deprecated)
Linux:
/usr/bin
Mac OS:
/usr/local/bin
- raw_run(tool_name, command_line_args=None, use_mpi=True, trace=False)¶
Execute a Khiops tool with given command line arguments
- Parameters:
- tool_name{“khiops”, “khiops_coclustering”}
Name of the tool to execute.
- command_line_argslist of str, optional
Command line arguments of Khiops.
- use_mpibool, optional
Whether to execute the application with MPI
- tracebool, default False
If
True
print the trace of the process.
Examples
>>> raw_run("khiops", ["-b", "-i" , "scenario._kh"])
is equivalent to execute in a shell:
> khiops -b -i scenario._kh
- class khiops.core.internals.runner.KhiopsRunner¶
Bases:
ABC
Abstract Khiops Python runner to be re-implemented
- create_temp_dir(prefix)¶
Creates a unique directory in the runner’s root temporary directory
- Parameters:
- prefixstr
Prefix for the directory’s name.
- Returns:
- str
A unique directory path within the root temporary directory. The directory is created only in the case of a local filesystem.
- create_temp_file(prefix, suffix)¶
Creates a unique temporary file in the runner’s root temporary directory
Note
For remote filesystems no actual file is created, just a (highly probable) unique path is returned.
- Parameters:
- prefixstr
Prefix for the file’s name.
- suffixstr
Suffix for the file’s name.
- Returns:
- str
A unique path within the root temporary directory. The file is created only in the case of a local filesystem.
- property khiops_temp_dir¶
str: Temporary directory for Khiops executions
If equal to
""
it uses the system’s temporary directory.- Raises:
TypeError
If set to a non str object.
- property khiops_version¶
KhiopsVersion
: The version of the Khiops backend of this runner
- property max_cores¶
int: Maximum number of cores for Khiops executions
If set to 0 it uses the system’s default.
- Raises:
TypeError
If it is set to a non int object.
ValueError
If it is set to a negative int.
- property max_memory_mb¶
int: Maximum amount of memory (in MB) for Khiops executions
If set to 0 it uses the maximum available in the system.
- Raises:
TypeError
If it is set to a non int object.
ValueError
If it is set to a negative int.
- print_status()¶
Prints the status of the runner to stdout
- property root_temp_dir¶
str: The runner’s temporary directory
The temporary scenarios/templates and dictionary files created by khiops-python are stored here.
- Default value:
Windows:
%TEMP%\khiops\python
Linux:
$TMP/khiops/python
When set to a local path it tries to create the specified directory if it doesn’t exist.
- Raises:
KhiopsEnvironmentError
If set to a local path: if it is a file or if it does not have
+rwx
permissions.
- run(task, task_args, command_line_options=None, trace=False, general_options=None, stdout_file_path='', stderr_file_path='', force_ansi_scenario=False, **kwargs)¶
Runs a Khiops Task
- Parameters:
- task
KhiopsTask
Khiops task to be run.
- task_argsdict
Arguments for the task.
- command_line_options
CommandLineOptions
, optional Command line options for all tasks. If not set the default values are used. See the
CommandLineOptions
for more information.- tracebool, default
False
If True prints the command line executed of the process and does not delete any temporary files created.
- general_options
GeneralOptions
, optional Advanced: General options for all tasks. If not set then the runner’s values are used. Unless you know what are you doing, prefer setting these options with the runner’s accessors. See the
GeneralOptions
class for more information.- stdout_file_pathstr, default “”
Advanced Path to a file where the Khiops process writes its stdout stream. Normally Khiops should not write to this stream but MPI, filesystems plugins or debug versions may do it. The stream is captured with a UTF-8 encoding and replacing encoding errors. If equal to “” then it writes no file.
- stderr_file_pathstr, default “”
Advanced Path to a file where the Khiops process writes its stderr stream. Normally Khiops should not write to this stream but MPI, filesystems plugins or debug versions may do it. The stream is captured with a UTF-8 encoding and replacing encoding errors. If equal to “” then it writes no file.
- force_ansi_scenariobool, default
False
Advanced: If True the internal scenario generated by Khiops will force characters such as accentuated ones to be decoded with the UTF8->ANSI khiops transformation.
- task
- Raises:
ValueError
Unknown keyword argument
Files or executable not found
Errors in the execution of the Khiops tool
TypeError
Invalid type of a keyword argument
When the search/replace pairs are not strings
- property samples_dir¶
str: Location of the Khiops’ sample datasets directory. May be an URL/URI
- class khiops.core.internals.runner.PyKhiopsLocalRunner¶
Bases:
KhiopsLocalRunner
Deprecated
See
KhiopsLocalRunner
.
- class khiops.core.internals.runner.PyKhiopsRunner¶
Bases:
KhiopsRunner
Deprecated
See
KhiopsRunner
.
- khiops.core.internals.runner.get_dir_status(a_dir)¶
Returns the status of a local or remote directory
Against a local directory a real check is performed. A remote directory is detected but not checked.
- khiops.core.internals.runner.get_linux_distribution_name()¶
Detect Linux distribution name
Parses the
NAME
variable defined in the/etc/os-release
or/usr/lib/os-release
files and converts it to lowercase.- Returns:
- str
Name of the Linux distribution, converted to lowecase
- Raises:
- OSError
If neither
/etc/os-release
nor/usr/lib/os-release
are found
- khiops.core.internals.runner.get_runner()¶
Returns the current KhiopsRunner of the module
- Returns:
KhiopsRunner
The current Khiops Python runner of the module.
- khiops.core.internals.runner.set_runner(runner)¶
Sets the current KhiopsRunner of the module