pyiron.base.generic package

Submodules

pyiron.base.generic.hdfio module

class pyiron.base.generic.hdfio.FileHDFio(file_name, h5_path='/', mode='a')[source]

Bases: object

Class that provides all info to access a h5 file. This class is based on h5io.py, which allows to get and put a large variety of jobs to/from h5

Parameters:
  • file_name (str) – absolute path of the HDF5 file
  • h5_path (str) – absolute path inside the h5 path - starting from the root group
  • mode (str) – mode : {‘a’, ‘w’, ‘r’, ‘r+’}, default ‘a’ See HDFStore docstring or tables.open_file for info about modes
file_name
absolute path to the HDF5 file
h5_path
path inside the HDF5 file - also stored as absolute path
history
previously opened groups / folders
file_exists
boolean if the HDF5 was already written
base_name
name of the HDF5 file but without any file extension
file_path
directory where the HDF5 file is located
is_root
boolean if the HDF5 object is located at the root level of the HDF5 file
is_open
boolean if the HDF5 file is currently opened - if an active file handler exists
is_empty
boolean if the HDF5 file is empty
base_name

Name of the HDF5 file - but without the file extension .h5

Returns:file name without the file extension
Return type:str
close()[source]

Close the current HDF5 path and return to the path before the last open

copy()[source]

Copy the Python object which links to the HDF5 file - in contrast to copy_to() which copies the content of the HDF5 file to a new location.

Returns:New FileHDFio object pointing to the same HDF5 file
Return type:FileHDFio
copy_to(destination, file_name=None, maintain_name=True)[source]

Copy the content of the HDF5 file to a new location

Parameters:
  • destination (FileHDFio) – FileHDFio object pointing to the new location
  • file_name (str) – name of the new HDF5 file - optional
  • maintain_name (bool) – by default the names of the HDF5 groups are maintained
Returns:

FileHDFio object pointing to a file which now contains the same content as file of the current

FileHDFio object.

Return type:

FileHDFio

create_group(name)[source]

Create an HDF5 group - similar to a folder in the filesystem - the HDF5 groups allow the users to structure their data.

Parameters:name (str) – name of the HDF5 group
Returns:FileHDFio object pointing to the new group
Return type:FileHDFio
file_exists

Check if the HDF5 file exists already

Returns:[True/False]
Return type:bool
file_name

Get the file name of the HDF5 file

Returns:absolute path to the HDF5 file
Return type:str
file_path

Path where the HDF5 file is located - posixpath.dirname()

Returns:HDF5 file location
Return type:str
get(key)[source]

Internal wrapper function for __getitem__() - self[name]

Parameters:key (str, slice) – path to the data or key of the data object
Returns:data or data object
Return type:dict, list, float, int
get_from_table(path, name)[source]

Get a specific value from a pandas.Dataframe

Parameters:
  • path (str) – relative path to the data object
  • name (str) – parameter key
Returns:

the value associated to the specific parameter key

Return type:

dict, list, float, int

get_pandas(name)[source]

Load a dictionary from the HDF5 file and display the dictionary as pandas Dataframe

Parameters:name (str) – HDF5 node name
Returns:The dictionary is returned as pandas.Dataframe object
Return type:pandas.Dataframe
groups()[source]

Filter HDF5 file by groups

Returns:an HDF5 file which is filtered by groups
Return type:FileHDFio
h5_path

Get the path in the HDF5 file starting from the root group - meaning this path starts with ‘/’

Returns:HDF5 path
Return type:str
is_empty

Check if the HDF5 file is empty

Returns:[True/False]
Return type:bool
is_root

Check if the current h5_path is pointing to the HDF5 root group.

Returns:[True/False]
Return type:bool
items()[source]

List all keys and values as items of all groups and nodes of the HDF5 file

Returns:list of sets (key, value)
Return type:list
keys()[source]

List all groups and nodes of the HDF5 file - where groups are equivalent to directories and nodes to files.

Returns:all groups and nodes
Return type:list
list_all()[source]

List all groups and nodes of the HDF5 file - where groups are equivalent to directories and nodes to files.

Returns:{‘groups’: [list of groups], ‘nodes’: [list of nodes]}
Return type:dict
list_dirs()[source]

equivalent to os.listdirs (consider groups as equivalent to dirs)

Returns:list of groups in pytables for the path self.h5_path
Return type:(list)
list_groups()[source]

equivalent to os.listdirs (consider groups as equivalent to dirs)

Returns:list of groups in pytables for the path self.h5_path
Return type:(list)
list_nodes()[source]

List all groups and nodes of the HDF5 file

Returns:list of nodes
Return type:list
listdirs()[source]

equivalent to os.listdirs (consider groups as equivalent to dirs)

Returns:list of groups in pytables for the path self.h5_path
Return type:(list)
nodes()[source]

Filter HDF5 file by nodes

Returns:an HDF5 file which is filtered by nodes
Return type:FileHDFio
open(h5_rel_path)[source]

Create an HDF5 group and enter this specific group. If the group exists in the HDF5 path only the h5_path is set correspondingly otherwise the group is created first.

Parameters:h5_rel_path (str) – relative path from the current HDF5 path - h5_path - to the new group
Returns:FileHDFio object pointing to the new group
Return type:FileHDFio
put(key, value)[source]

Store data inside the HDF5 file

Parameters:
  • key (str) – key to store the data
  • value (pandas.DataFrame, pandas.Series, dict, list, float, int) – basically any kind of data is supported
remove_file()[source]

Remove the HDF5 file with all the related content

remove_group()[source]

Remove an HDF5 group - if it exists. If the group does not exist no error message is raised.

show_hdf()[source]

Iterating over the HDF5 datastructure and generating a human readable graph.

values()[source]

List all values for all groups and nodes of the HDF5 file

Returns:list of all values
Return type:list
class pyiron.base.generic.hdfio.HDFStoreIO(path, mode=None, complevel=None, complib=None, fletcher32=False, **kwargs)[source]

Bases: pandas.io.pytables.HDFStore

dict-like IO interface for storing pandas objects in PyTables either Fixed or Table format. - copied from pandas.HDFStore

Parameters:
  • path (str) – File path to HDF5 file
  • mode (str) –

    {‘a’, ‘w’, ‘r’, ‘r+’}, default ‘a’ 'r'

    Read-only; no data can be modified.
    'w'
    Write; a new file is created (an existing file with the same name would be deleted).
    'a'
    Append; an existing file is opened for reading and writing, and if the file does not exist it is created.
    'r+'
    It is similar to 'a', but the file must already exist.
  • complevel (int) – 1-9, default 0 If a complib is specified compression will be applied where possible
  • complib (str) – {‘zlib’, ‘bzip2’, ‘lzo’, ‘blosc’, None}, default None If complevel is > 0 apply compression to objects written in the store wherever possible
  • fletcher32 (bool) – bool, default False If applying compression use the fletcher32 checksum
open(**kwargs)[source]

Open the file in the specified mode - copied from pandas.HDFStore.open()

Parameters:**kwargs – mode : {‘a’, ‘w’, ‘r’, ‘r+’}, default ‘a’ See HDFStore docstring or tables.open_file for info about modes
Returns:self - in contrast to the original implementation in pandas.
Return type:HDFStoreIO
class pyiron.base.generic.hdfio.ProjectHDFio(project, file_name, h5_path=None, mode=None)[source]

Bases: pyiron.base.generic.hdfio.FileHDFio

The ProjectHDFio class connects the FileHDFio and the Project class, it is derived from the FileHDFio class but in addition the a project object instance is located at self.project enabling direct access to the database and other project related functionality, some of which are mapped to the ProjectHDFio class as well.

Parameters:
  • project (Project) – pyiron Project the current HDF5 project is located in
  • file_name (str) – name of the HDF5 file - in contrast to the FileHDFio object where file_name represents the absolute path of the HDF5 file.
  • h5_path (str) – absolute path inside the h5 path - starting from the root group
  • mode (str) – mode : {‘a’, ‘w’, ‘r’, ‘r+’}, default ‘a’ See HDFStore docstring or tables.open_file for info about modes
.. attribute:: project

Project instance the ProjectHDFio object is located in

.. attribute:: root_path

the pyiron user directory, defined in the .pyiron configuration

.. attribute:: project_path

the relative path of the current project / folder starting from the root path of the pyiron user directory

.. attribute:: path

the absolute path of the current project / folder plus the absolute path in the HDF5 file as one path

.. attribute:: file_name

absolute path to the HDF5 file

.. attribute:: h5_path

path inside the HDF5 file - also stored as absolute path

.. attribute:: history

previously opened groups / folders

.. attribute:: file_exists

boolean if the HDF5 was already written

.. attribute:: base_name

name of the HDF5 file but without any file extension

.. attribute:: file_path

directory where the HDF5 file is located

.. attribute:: is_root

boolean if the HDF5 object is located at the root level of the HDF5 file

.. attribute:: is_open

boolean if the HDF5 file is currently opened - if an active file handler exists

.. attribute:: is_empty

boolean if the HDF5 file is empty

.. attribute:: user

current unix/linux/windows user who is running pyiron

.. attribute:: sql_query

an SQL query to limit the jobs within the project to a subset which matches the SQL query.

.. attribute:: db

connection to the SQL database

.. attribute:: working_directory

working directory of the job is executed in - outside the HDF5 file

base_name

The absolute path to of the current pyiron project - absolute path on the file system, not including the HDF5 path.

Returns:current project path
Return type:str
copy()[source]

Copy the ProjectHDFio object - copying just the Python object but maintaining the same pyiron path

Returns:copy of the ProjectHDFio object
Return type:ProjectHDFio
create_hdf(path, job_name)[source]

Create an ProjectHDFio object to store project related information - for testing aggregated data

Parameters:
  • path (str) – absolute path
  • job_name (str) – name of the HDF5 container
Returns:

HDF5 object

Return type:

ProjectHDFio

create_object(class_name, **qwargs)[source]

Internal function to create a pyiron object

Parameters:
  • class_name (str) – name of a pyiron class
  • **qwargs – object parameters
Returns:

defined by the pyiron class in class_name with the input from **qwargs

Return type:

pyiron object

create_working_directory()[source]

Create the working directory on the file system if it does not exist already.

db

Get connection to the SQL database

Returns:database conncetion
Return type:DatabaseAccess
get_job_id(job_specifier)[source]

get the job_id for job named job_name in the local project path from database

Parameters:job_specifier (str, int) – name of the job or job ID
Returns:job ID of the job
Return type:int
inspect(job_specifier)[source]

Inspect an existing pyiron object - most commonly a job - from the database

Parameters:job_specifier (str, int) – name of the job or job ID
Returns:Access to the HDF5 object - not a GenericJob object - use load() instead.
Return type:JobCore
load(job_specifier, convert_to_object=True)[source]

Load an existing pyiron object - most commonly a job - from the database

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • convert_to_object (bool) – convert the object to an pyiron object or only access the HDF5 file - default=True accessing only the HDF5 file is about an order of magnitude faster, but only provides limited functionality. Compare the GenericJob object to JobCore object.
Returns:

Either the full GenericJob object or just a reduced JobCore object

Return type:

GenericJob, JobCore

load_from_jobpath(job_id=None, db_entry=None, convert_to_object=True)[source]

Internal function to load an existing job either based on the job ID or based on the database entry dictionary.

Parameters:
  • job_id (int) – Job ID - optional, but either the job_id or the db_entry is required.
  • db_entry (dict) – database entry dictionary - optional, but either the job_id or the db_entry is required.
  • convert_to_object (bool) – convert the object to an pyiron object or only access the HDF5 file - default=True accessing only the HDF5 file is about an order of magnitude faster, but only provides limited functionality. Compare the GenericJob object to JobCore object.
Returns:

Either the full GenericJob object or just a reduced JobCore object

Return type:

GenericJob, JobCore

path

Absolute path of the HDF5 group starting from the system root - combination of the absolute system path plus the absolute path inside the HDF5 file starting from the root group.

Returns:absolute path
Return type:str
project

Get the project instance the ProjectHDFio object is located in

Returns:pyiron project
Return type:Project
project_path

the relative path of the current project / folder starting from the root path of the pyiron user directory

Returns:relative path of the current project / folder
Return type:str
remove_job(job_specifier, _unprotect=False)[source]

Remove a single job from the project based on its job_specifier - see also remove_jobs()

Parameters:
  • job_specifier (str, int) – name of the job or job ID
  • _unprotect (bool) – [True/False] delete the job without validating the dependencies to other jobs - default=False
root_path

the pyiron user directory, defined in the .pyiron configuration

Returns:pyiron user directory of the current project
Return type:str
sql_query

Get the SQL query for the project

Returns:SQL query
Return type:str
to_object(object_type=None, **qwargs)[source]

Load the full pyiron object from an HDF5 file

Parameters:
  • object_type – if the ‘TYPE’ node is not available in the HDF5 file a manual object type can be set - optional
  • **qwargs – optional parameters [‘job_name’, ‘project’] - to specify the location of the HDF5 path
Returns:

pyiron object

Return type:

GenericJob

user

Get current unix/linux/windows user who is running pyiron

Returns:username
Return type:str
working_directory

Get the working directory of the current ProjectHDFio object. The working directory equals the path but it is represented by the filesystem:

/absolute/path/to/the/file.h5/path/inside/the/hdf5/file
becomes:
/absolute/path/to/the/file_hdf5/path/inside/the/hdf5/file
Returns:absolute path to the working directory
Return type:str

pyiron.base.generic.parameters module

class pyiron.base.generic.parameters.GenericParameters(table_name=None, input_file_name=None, val_only=False, comment_char='#', separator_char=' ', end_value_char='')[source]

Bases: pyiron.base.generic.template.PyironObject

GenericParameters class defines the typical input file with a key value structure plus an additional column for comments. Convenience class to easily create, read, and modify input files

Parameters:
  • table_name (str) – name of the input file inside the HDF5 file - optional
  • input_file_name (str) – name of the input file (if None default parameters are used)
  • val_only (bool) – input format consists of value (comments) only
  • comment_char (str) – separator that characterizes comment (e.g. “#” for python)
  • separator_char (str) – separator that characterizes the split between key and value - default=’ ‘
  • end_value_char (str) – special character at the end of every line - default=’‘
.. attribute:: file_name

file name of the input file

.. attribute:: table_name

name of the input table inside the HDF5 file

.. attribute:: val_only

boolean option to switch from a key value list to an value only input file

.. attribute:: comment_char

separator that characterizes comment

.. attribute:: separator_char

separator that characterizes the split between key and value

.. attribute:: multi_word_separator

multi word separator to have multi word keys

.. attribute:: end_value_char

special character at the end of every line

.. attribute:: replace_char_dict

dictionary to replace certain character combinations

comment_char

Get the separator that characterizes comment

Returns:comment character
Return type:str
define_blocks(block_dict)[source]

Define a block section within the GenericParameters

Parameters:block_dict (dict) – dictionary to define the block
end_value_char

Get the special character at the end of every line

Returns:end of line character
Return type:str
file_name

Get the file name of the input file

Returns:file name
Return type:str
from_hdf(hdf, group_name=None)[source]

Restore the GenericParameters from an HDF5 file

Parameters:
  • hdf (ProjectHDFio) – HDF5 group object
  • group_name (str) – HDF5 subgroup name - optional
get(parameter_name, default_value=None)[source]

Get the value of a specific parameter from GenericParameters - if the parameter is not available return default_value if that is set.

Parameters:
  • parameter_name (str) – parameter key
  • default_value (str) – default value to return is the parameter is not set
Returns:

value of the parameter

Return type:

str

get_attribute(attribute_name)[source]

Get the value of a specific parameter from GenericParameters

Parameters:attribute_name (str) – parameter key
Returns:value of the parameter
Return type:str
get_pandas()[source]

Output the GenericParameters object as Pandas Dataframe for human readability.

Returns:Pandas Dataframe of the GenericParameters object
Return type:pandas.DataFrame
get_string_lst()[source]

Get list of strings from GenericParameters to write to input file

load_default()[source]

Load defaults resets the dataset in the background to be empty

load_string(input_str)[source]

Load a multi line string to overwrite the current parameter settings

Parameters:input_str (str) – multi line string
modify(separator=None, append_if_not_present=False, **modify_dict)[source]

Modify values for existing parameters. The command is called as modify(param1=val1, param2=val2, …)

Parameters:
  • separator (str) – needed if the parameter name contains special characters such as par: use then as input: modify(separator=”:”, par=val) - optional
  • append_if_not_present (bool) – do not raise an exception but append the parameter in practice use set(par=val) - default=False
  • **modify_dict (dict) – dictionary of parameter names and values
multi_word_separator

Get the multi word separator to have multi word keys

Returns:multi word separator
Return type:str
read_input(file_name, ignore_trigger=None)[source]

Read input file and store the data in GenericParameters - this overwrites the current parameter settings

Parameters:
  • file_name (str) – absolute path to the input file
  • ignore_trigger (str) – trigger for lines to be ignored
remove_keys(key_list)[source]

Remove a list of keys from the GenericParameters

Parameters:key_list (list) – list of keys to be removed
replace_char_dict

Get the dictionary to replace certain character combinations

Returns:character replace dictionary
Return type:dict
separator_char

Get the separator that characterizes the split between key and value

Returns:separator character
Return type:str
set(separator=None, **set_dict)[source]

Set the value of multiple parameters or create new parameter key, if they do not exist already.

Parameters:
  • separator (str) – separator string - optional
  • **set_dict (dict) – dictionary containing the parameter keys and their corresponding values to be set
set_value(line, val)[source]

Set the value of a parameter in a specific line

Parameters:
  • line (int) – line number - starting with 0
  • val (str/bytes) – value to be set
table_name

Get the name of the input table inside the HDF5 file

Returns:table name
Return type:str
to_hdf(hdf, group_name=None)[source]

Store the GenericParameters in an HDF5 file

Parameters:
  • hdf (ProjectHDFio) – HDF5 group object
  • group_name (str) – HDF5 subgroup name - optional
val_only

Get the boolean option to switch from a key value list to an value only input file

Returns:[True/False]
Return type:bool
write_file(file_name, cwd=None)[source]

Write GenericParameters to input file

Parameters:
  • file_name (str) – name of the file, either absolute (then cwd must be None) or relative
  • cwd (str) – path name (default: None)

pyiron.base.generic.template module

Template class to list the required properties and functions for every pyiron object.

class pyiron.base.generic.template.PyironObject[source]

Bases: object

Template class to list the required properties and functions for every pyiron object.

child_ids

If the pyiron object is a meta object which includes a series of objects these objects ids are listed as child ids.

Returns:list of child ids
Return type:list
copy()[source]

Copy the pyiron object - this copies only the link to the HDF5 file - not the content of the HDF5 file.

Returns:a copy of the pyiron object
Return type:PyironObject
copy_to(new_project=None)[source]

Copy the pyiron object including the HDF5 file to an new location

Parameters:new_project (ProjectHDFio) – new location
Returns:a copy of the pyiron object
Return type:PyironObject
from_hdf(hdf, group_name='group')[source]

Restore the PyironObject from an HDF5 file

Parameters:
  • hdf (ProjectHDFio) – HDF5 group object
  • group_name (str) – HDF5 subgroup name - optional
id

Every pyiron object should have the ability to be stored in the database

Returns:object id
Return type:int
inspect(job_specifier)[source]

Inspect a pyiron object from the database - the same like load(self, job_specifier, convert_to_object=False). The inspect mode provides quick access to the HDF5 file without loading the full object, which is especially helpful if only a specific section of the output is required.

Parameters:job_specifier (str, int) – identifier of the pyiron object - this needs to be unique in the project.
Returns:a reduces pyiron object like JobPath
Return type:PyironObject
load(job_specifier, convert_to_object=True)[source]

Load a pyiron object from the database

Parameters:
  • job_specifier (str, int) – identifier of the pyiron object - this needs to be unique in the project.
  • convert_to_object (bool) – [True/False] - it is faster to only load the HDF5 access but the pyiron object offers a whole more functionality.
Returns:

the pyiron object

Return type:

PyironObject

master_id

If the pyiron object belongs to a series of objects, series object is linked by the master id

Returns:master id
Return type:int
move_to(new_project)[source]

Move the pyiron object including the HDF5 file to an new location

Parameters:new_project (ProjectHDFio) – new location
parent_id

If the pyiron object belongs to a serial series of objects, the predecessor is linked by the parent id

Returns:parent id
Return type:int
remove()[source]

Remove the pyiron obect from the database and delete the HDF5 file

save()[source]

Store the pyiron object in the HDF5 file an create a corresponding database entry.

to_hdf(hdf, group_name='group')[source]

Store the PyironObject in an HDF5 file

Parameters:
  • hdf (ProjectHDFio) – HDF5 group object
  • group_name (str) – HDF5 subgroup name - optional

Module contents