pyiron.base.project package

Submodules

pyiron.base.project.generic module

class pyiron.base.project.generic.Project(path='', user=None, sql_query=None)[source]

Bases: pyiron.base.project.path.ProjectPath

The project is the central class in pyiron, all other objects can be created from the project object.

Parameters:
  • path (GenericPath, str) – path of the project defined by GenericPath, absolute or relative (with respect to current working directory) path
  • user (str) – current pyiron user
  • sql_query (str) – SQL query to only select a subset of the existing jobs within the current project
.. attribute:: root_path

the pyiron user directory, defined in the .pyiron configuration

.. attribute:: project_path

the relative path of the current project / folder starting from the root path of the pyiron user directory

.. attribute:: path

the absolute path of the current project / folder

.. attribute:: base_name

the name of the current project / folder

.. attribute:: history

previously opened projects / folders

.. attribute:: parent_group

parent project - one level above the current project

.. attribute:: user

current unix/linux/windows user who is running pyiron

.. attribute:: sql_query

an SQL query to limit the jobs within the project to a subset which matches the SQL query.

.. attribute:: db

connection to the SQL database

.. attribute:: job_type
Job Type object with all the available job types: [‘ExampleJob’, ‘SerialMaster’, ‘ParallelMaster’, ‘ScriptJob’,
‘ListMaster’]
.. attribute:: view_mode

If viewer_mode is enable pyiron has read only access to the database.

copy()[source]

Copy the project object - copying just the Python object but maintaining the same pyiron path

Returns:copy of the project object
Return type:Project
copy_to(destination)[source]

Copy the project object to a different pyiron path - including the content of the project (all jobs).

Parameters:destination (Project) – project path to copy the project content to
Returns:pointing to the new project path
Return type:Project
create_from_job(job_old, new_job_name)[source]

Create a new job from an existing pyiron job

Parameters:
  • job_old (GenericJob) – Job to copy
  • new_job_name (str) – New job name
Returns:

New job with the new job name.

Return type:

GenericJob

create_group(group)[source]

Create a new subproject/ group/ folder

Parameters:group (str) – name of the new project
Returns:New subproject
Return type:Project
static create_hdf(path, job_name)[source]

Create an ProjectHDFio object to store project related information - for example aggregated data

Parameters:
  • path (str) – absolute path
  • job_name (str) – name of the HDF5 container
Returns:

HDF5 object

Return type:

ProjectHDFio

create_job(job_type, job_name)[source]

Create one of the following jobs: - ‘ExampleJob’: example job just generating random number - ‘SerialMaster’: series of jobs run in serial - ‘ParallelMaster’: series of jobs run in parallel - ‘ScriptJob’: Python script or jupyter notebook job container - ‘ListMaster’: list of jobs

Parameters:
  • job_type (str) – job type can be [‘ExampleJob’, ‘SerialMaster’, ‘ParallelMaster’, ‘ScriptJob’, ‘ListMaster’]
  • job_name (str) – name of the job
Returns:

job object depending on the job_type selected

Return type:

GenericJob

get_child_ids(job_specifier, project=None)[source]

Get the childs for a specific job

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • project (Project) – Project the job is located in - optional
Returns:

list of child IDs

Return type:

list

get_db_columns()[source]

Get column names

Returns:
list of column names like:
[‘id’, ‘parentid’, ‘masterid’, ‘projectpath’, ‘project’, ‘job’, ‘subjob’, ‘chemicalformula’, ‘status’, ‘hamilton’, ‘hamversion’, ‘username’, ‘computer’, ‘timestart’, ‘timestop’, ‘totalcputime’]
Return type:list
get_job_id(job_specifier)[source]

get the job_id for job named job_name in the local project path from database

Parameters:job_specifier (str, int) – name of the job or job ID
Returns:job ID of the job
Return type:int
get_job_ids(recursive=True)[source]

Return the job IDs matching a specific query

Parameters:recursive (bool) – search subprojects [True/False]
Returns:a list of job IDs
Return type:list
get_job_status(job_specifier, project=None)[source]

Get the status of a particular job

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • project (Project) – Project the job is located in - optional
Returns:

job status can be one of the following [‘initialized’, ‘appended’, ‘created’, ‘submitted’, ‘running’,

’aborted’, ‘collect’, ‘suspended’, ‘refresh’, ‘busy’, ‘finished’]

Return type:

str

get_job_working_directory(job_specifier, project=None)[source]

Get the working directory of a particular job

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • project (Project) – Project the job is located in - optional
Returns:

working directory as absolute path

Return type:

str

get_jobs(recursive=True, columns=['id', 'project'])[source]

Internal function to return the jobs as dictionary rather than a pandas.Dataframe

Parameters:
  • recursive (bool) – search subprojects [True/False]
  • columns (list) – by default only the columns [‘id’, ‘project’] are selected, but the user can select a subset of [‘id’, ‘status’, ‘chemicalformula’, ‘job’, ‘subjob’, ‘project’, ‘projectpath’, ‘timestart’, ‘timestop’, ‘totalcputime’, ‘computer’, ‘hamilton’, ‘hamversion’, ‘parentid’, ‘masterid’]
Returns:

columns are used as keys and point to a list of the corresponding values

Return type:

dict

get_project_size()[source]

Get the size of the project in MegaByte.

Returns:project size
Return type:float
groups()[source]

Filter project by groups

Returns:a project which is filtered by groups
Return type:Project
inspect(job_specifier)[source]

Inspect an existing pyiron object - most commonly a job - from the database

Parameters:job_specifier (str, int) – name of the job or job ID
Returns:Access to the HDF5 object - not a GenericJob object - use load() instead.
Return type:JobCore
items()[source]

All items in the current project - this includes jobs, sub projects/ groups/ folders and any kind of files

Returns:items in the project
Return type:list
iter_groups()[source]

Iterate over the groups within the current project

Returns:Yield of sub projects/ groups/ folders
Return type:yield
iter_jobs(path=None, recursive=True, convert_to_object=True)[source]

Iterate over the jobs within the current project and it is sub projects

Parameters:
  • path (str) – HDF5 path inside each job object
  • recursive (bool) – search subprojects [True/False] - True by default
  • convert_to_object (bool) – load the full GenericJob object (default) or just the HDF5 / JobCore object
Returns:

Yield of GenericJob or JobCore

Return type:

yield

iter_output(recursive=True)[source]

Iterate over the output of jobs within the current project and it is sub projects

Parameters:recursive (bool) – search subprojects [True/False] - True by default
Returns:Yield of GenericJob or JobCore
Return type:yield
job_table(recursive=True, columns=['job', 'project', 'chemicalformula'], all_columns=True, sort_by='id', element_lst=None)[source]

Access the job_table

Parameters:
  • recursive (bool) – search subprojects [True/False] - default=True
  • columns (list) – by default only the columns [‘job’, ‘project’, ‘chemicalformula’] are selected, but the user can select a subset of [‘id’, ‘status’, ‘chemicalformula’, ‘job’, ‘subjob’, ‘project’, ‘projectpath’, ‘timestart’, ‘timestop’, ‘totalcputime’, ‘computer’, ‘hamilton’, ‘hamversion’, ‘parentid’, ‘masterid’]
  • all_columns (bool) – Select all columns - this overwrites the columns option.
  • sort_by (str) – Sort by a specific column
  • element_lst (list) – list of elements required in the chemical formular - by default None
Returns:

Return the result as a pandas.Dataframe object

Return type:

pandas.Dataframe

keys()[source]

List of file-, folder- and objectnames

Returns:list of the names of project directories and project nodes
Return type:list
list_all()[source]

Combination of list_groups(), list_nodes() and list_files() all in one dictionary with the corresponding keys: - ‘groups’: Subprojects/ -folder/ -groups. - ‘nodes’: Jobs or pyiron objects - ‘files’: Files inside a project which do not belong to any pyiron object

Returns:dictionary with all items in the project
Return type:dict
list_dirs(skip_hdf5=True)[source]

List directories inside the project

Parameters:skip_hdf5 (bool) – Skip directories which belong to a pyiron object/ pyiron job - default=True
Returns:list of directory names
Return type:list
list_files(extension=None)[source]

List files inside the project

Parameters:extension (str) – filter by a specific extension
Returns:list of file names
Return type:list
list_groups()[source]

List directories inside the project

Returns:list of directory names
Return type:list
list_nodes(recursive=False)[source]

List nodes/ jobs/ pyiron objects inside the project

Parameters:recursive (bool) – search subprojects [True/False] - default=False
Returns:columns are used as keys and point to a list of the corresponding values
Return type:dict
load(job_specifier, convert_to_object=True)[source]

Load an existing pyiron object - most commonly a job - from the database

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • convert_to_object (bool) – convert the object to an pyiron object or only access the HDF5 file - default=True accessing only the HDF5 file is about an order of magnitude faster, but only provides limited functionality. Compare the GenericJob object to JobCore object.
Returns:

Either the full GenericJob object or just a reduced JobCore object

Return type:

GenericJob, JobCore

load_from_jobpath(job_id=None, db_entry=None, convert_to_object=True)[source]

Internal function to load an existing job either based on the job ID or based on the database entry dictionary.

Parameters:
  • job_id (int) – Job ID - optional, but either the job_id or the db_entry is required.
  • db_entry (dict) – database entry dictionary - optional, but either the job_id or the db_entry is required.
  • convert_to_object (bool) – convert the object to an pyiron object or only access the HDF5 file - default=True accessing only the HDF5 file is about an order of magnitude faster, but only provides limited functionality. Compare the GenericJob object to JobCore object.
Returns:

Either the full GenericJob object or just a reduced JobCore object

Return type:

GenericJob, JobCore

move_to(destination)[source]

Similar to the copy_to() function move the project object to a different pyiron path - including the content of the project (all jobs).

Parameters:destination (Project) – project path to move the project content to
Returns:pointing to the new project path
Return type:Project
name

The name of the current project folder

Returns:name of the current project folder
Return type:str
nodes()[source]

Filter project by nodes

Returns:a project which is filtered by nodes
Return type:Project
parent_group

Get the parent group of the current project

Returns:parent project
Return type:Project
queue_delete_job(item)[source]

Delete a job from the queuing system

Parameters:item (int, GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Output from the queuing system as string - optimized for the Sun grid engine
Return type:str
static queue_enable_reservation(item)[source]

Enable a reservation for a particular job within the queuing system

Parameters:item (int, GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Output from the queuing system as string - optimized for the Sun grid engine
Return type:str
static queue_is_empty()[source]

Check if the queue table is currently empty - no more jobs to wait for.

Returns:True if the table is empty, else False - optimized for the Sun grid engine
Return type:bool
static queue_job_info(item)[source]

Short reporting for a particular job - using the qstat command of the sun grid engine.

Parameters:item (int, GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Short report returned from the queuing system - optimized for the Sun grid engine
Return type:pandas.DataFrame
static queue_report(item)[source]

Detailed reporting for a particular job - using the qacct command of the sun grid engine.

Parameters:item (int, GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Detailed report returned from the queuing system - optimized for the Sun grid engine
Return type:pandas.DataFrame
queue_table(project_only=True, recursive=True)[source]

Display the queuing system table as pandas.Dataframe

Parameters:
  • project_only (bool) – Query only for jobs within the current project - True by default
  • recursive (bool) – Include jobs from sub projects
Returns:

Output from the queuing system - optimized for the Sun grid engine

Return type:

pandas.DataFrame

queue_table_global()[source]

Display the queuing system table as pandas.Dataframe

Parameters:
  • project_only (bool) – Query only for jobs within the current project - True by default
  • recursive (bool) – Include jobs from sub projects
Returns:

Output from the queuing system - optimized for the Sun grid engine

Return type:

pandas.DataFrame

refresh_job_status_based_on_job_id(job_id, que_mode=True)[source]

Internal function to check if a job is still listed ‘running’ in the job_table while it is no longer listed in the queuing system. In this case update the entry in the job_table to ‘aborted’.

Parameters:
  • job_id (int) – job ID
  • que_mode (bool) – [True/False] - default=True
refresh_job_status_based_on_queue_status(job_specifier, status='running')[source]

Check if the job is still listed as running, while it is no longer listed in the queue.

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • status (str) – Currently only the jobstatus of ‘running’ jobs can be refreshed - default=’running’
remove(enable=False, enforce=False)[source]

Delete all the whole project including all jobs in the project and its subprojects

Parameters:
  • enforce (bool) – [True/False] delete jobs even though they are used in other projects - default=False
  • enable (bool) – [True/False] enable this command.
remove_file(file_name)[source]

Remove a file (same as unlink()) - copied from os.remove()

If dir_fd is not None, it should be a file descriptor open to a directory,
and path should be relative; path will then be relative to that directory.
dir_fd may not be implemented on your platform.
If it is unavailable, using it will raise a NotImplementedError.
Parameters:file_name (str) – name of the file
remove_job(job_specifier, _unprotect=False)[source]

Remove a single job from the project based on its job_specifier - see also remove_jobs()

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • _unprotect (bool) – [True/False] delete the job without validating the dependencies to other jobs - default=False
remove_jobs(recursive=False)[source]

Remove all jobs in the current project and in all subprojects if recursive=True is selected - see also remove_job()

Parameters:recursive (bool) – [True/False] delete all jobs in all subprojects - default=False
set_job_status(job_specifier, status, project=None)[source]

Set the status of a particular job

Parameters:
  • job_specifier (str) – name of the job or job ID
  • status (str) – job status can be one of the following [‘initialized’, ‘appended’, ‘created’, ‘submitted’, ‘running’, ‘aborted’, ‘collect’, ‘suspended’, ‘refresh’, ‘busy’, ‘finished’]
static set_logging_level(level, channel=None)[source]

Set level for logger

Parameters:
  • level (str) – ‘DEBUG, INFO, WARN’
  • channel (int) – 0: file_log, 1: stream, None: both
switch_to_central_database()[source]

Switch from viewer mode to user mode - if viewer_mode is enable pyiron has read only access to the database.

switch_to_local_database(file_name='pyiron.db', cwd=None)[source]
switch_to_user_mode()[source]

Switch from viewer mode to user mode - if viewer_mode is enable pyiron has read only access to the database.

switch_to_viewer_mode()[source]

Switch from user mode to viewer mode - if viewer_mode is enable pyiron has read only access to the database.

values()[source]

All items in the current project - this includes jobs, sub projects/ groups/ folders and any kind of files

Returns:items in the project
Return type:list
view_mode

Get viewer_mode - if viewer_mode is enable pyiron has read only access to the database.

Returns:returns TRUE when viewer_mode is enabled
Return type:bool
static wait_for_job(job, interval_in_s=5, max_iterations=100)[source]

Sleep until the job is finished but maximum interval_in_s * max_iterations seconds.

Parameters:
  • job (GenericJob) – Job to wait for
  • interval_in_s (int) – interval when the job status is queried from the database - default 5 sec.
  • max_iterations (int) – maximum number of iterations - default 100

pyiron.base.project.gui module

class pyiron.base.project.gui.ProjectGUI(project)[source]

Bases: object

Gui to quickly browse through projects and objects Note: requires “%matplotlib notebook” at the beginning of your notebook (to refresh plots)

connect_widgets()[source]

Returns:

create_widgets()[source]

Returns:

static dict_to_str(my_dict)[source]
Parameters:my_dict

Returns:

display()[source]

Returns:

get_rel_path(path)[source]
Parameters:path

Returns:

move_up()[source]

Returns:

on_value_change(change)[source]
Parameters:change

Returns:

plot_array(val)[source]
Parameters:val

Returns:

plot_to_html()[source]
refresh_view()[source]

Returns:

update_project(name)[source]
Parameters:name

Returns:

pyiron.base.project.gui.is_h5_dir(name)[source]
pyiron.base.project.gui.is_h5_file(name)[source]

pyiron.base.project.path module

class pyiron.base.project.path.GenericPath(root_path, project_path)[source]

Bases: object

Basic class to represent a project path in PyIron. A path consists of two parts, the root part which defines directory path where the project repository is located (top_level_path) and the project part which defines the relative path from the root_path to the project.

This class is meant for storing and accessing a path, not for moving around which is done by the ProjectPath class.

Parameters:
  • root_path (str) – absolute path name of the repository
  • project_path (str) – relative path to the specific project
.. attribute:: root_path

the pyiron user directory, defined in the .pyiron configuration

.. attribute:: project_path

the relative path of the current project / folder starting from the root path of the pyiron user directory

.. attribute:: path

the absolute path of the current project / folder

.. attribute:: base_name

the name of the current project / folder

Author: Jan Janssen

base_name

The name of the current project folder

Returns:name of the current project folder
Return type:str
copy()[source]

Copy the GenericPath object

Returns:independent GenericPath object pointing to the same project folder
Return type:GenericPath
path

The absolute path to of the current object.

Returns:current project path
Return type:str
project_path

the relative path of the current project / folder starting from the root path of the pyiron user directory

Returns:relative path of the current project / folder
Return type:str
root_path

the pyiron user directory, defined in the .pyiron configuration

Returns:pyiron user directory of the current project
Return type:str
class pyiron.base.project.path.ProjectPath(path)[source]

Bases: pyiron.base.project.path.GenericPath

close()[source]

return to the path before the last open if no history exists nothing happens

copy()[source]

copy the path without the history, i.e., to going back with close is not possible

Returns:
Return type:ProjectPath
history

The history of the previously opened paths

Returns:list of previously opened relative paths
Return type:list
listdir()[source]

equivalent to os.listdir list all files and directories in this path

Returns:list of folders and files in the current project path
Return type:list
open(rel_path, history=True)[source]

if rel_path exist set the project path to this directory if not create it and go there

Parameters:
  • rel_path (str) – path relative to the current project path
  • history (bool) – By default pyiron stores a history of previously opened paths
Returns:

New ProjectPath object pointing to the relative path

Return type:

ProjectPath

removedirs(project_name=None)[source]

equivalent to os.removedirs -> remove empty dirs

Parameters:project_name (str) – relative path to the project folder to be deleted
walk()[source]

equivalent to os.listdir list all files and directories in this path

Returns:Directory tree generator.
Return type:Generator

pyiron.base.project.readmode module

This class is currently not used !

class pyiron.base.project.readmode.ReadMode(mode='object')[source]

Bases: object

inspect
mode
object

pyiron.base.project.store module

class pyiron.base.project.store.ProjectStore(project, job_name)[source]

Bases: pyiron.base.job.generic.GenericJob

The ProjectStore object, is derived from the GenericJob class and allows the user to store aggregated information in an HDF5 file associated with the corresponding project. To the user the ProjectStore object behaves like a dictionary.

Parameters:
  • project – Project object (defines path where job will be created and stored)
  • job_name – name of the job (must be unique within this project path)
.. attribute:: key

keys of the ProjectStore object (like a dictionary)

.. attribute:: items

items of the ProjectStore object (like a dictionary)

.. attribute:: values

values of the ProjectStore object (like a dictionary)

.. attribute:: time_created

date when the ProjecStore object was created

.. attribute:: time_modified

date when the ProjecStore object was modified

collect_logfiles()[source]

Implement required function template - even though it is not required for this job type.

collect_output()[source]

Implement required function template - even though it is not required for this job type.

from_hdf(hdf=None, group_name=None)[source]

Restore object from hdf5 format

Parameters:
  • hdf – Optional hdf5 file, otherwise self is used.
  • group_name (str) – Optional hdf5 group in the hdf5 file.
items

a set-like object providing a view on ProjectStore’s dictionary items

keys

a set-like object providing a view on ProjectStore’s dictionary keys

run_if_lib()[source]

Internal function to handle jobs with Python based executables

Returns:Database ID of the ProjectStore object
Return type:int
time_created

Return the date when the ProjectStore object was created

Returns:the date when the ProjectStore object was created
Return type:DateTime
time_modified

Return the date when the ProjectStore object was modified

Returns:the date when the ProjectStore object was modified
Return type:DateTime
values

an object providing a view on ProjectStore’s dictionary values

write_input()[source]

Implement required function template - even though it is not required for this job type.

Module contents