pyiron.base.database package

Submodules

pyiron.base.database.generic module

class pyiron.base.database.generic.AutorestoredConnection(engine)[source]

Bases: object

close()[source]
execute(*args, **kwargs)[source]
class pyiron.base.database.generic.DatabaseAccess(connection_string, table_name)[source]

Bases: object

A core element of PyIron, which generally deals with accessing the database: getting, sending, changing some data to the db.

Parameters:
  • connection_string (str) – SQLalchemy connection string which specifies the database to connect to typical form: dialect+driver://username:password@host:port/database example: ‘postgresql://scott:tiger@cmcent56.mpie.de/mdb’
  • table_name (str) – database table name, a simple string like: ‘simulation’

Murat Han Celik

add_column(col_name, col_type)[source]

Add an additional column - required for modification on the database

Parameters:
  • col_name (str, list) – name of the new column, normal string like: ‘myColumn’
  • (str, list (col_type) – SQL type of the new column, SQL type like: ‘varchar(50)’

Returns:

add_item_dict(par_dict)[source]

Create a new database item

Parameters:par_dict (dict) –
Dictionary with the item values and column names as keys, like:
{‘chemicalformula’: ‘BO’,

’computer’: ‘localhost’, ‘hamilton’: ‘VAMPS’, ‘hamversion’: ‘1.1’, ‘job’: ‘testing’, ‘subjob’ : ‘SubJob’, ‘parentid’: 0L, ‘myCol’: ‘Blubbablub’, ‘project’: ‘database.testing’, ‘projectpath’: ‘/root/directory/tmp’, ‘status’: ‘KAAAA’, ‘timestart’: datetime(2016, 5, 2, 11, 31, 4, 253377), ‘timestop’: datetime(2016, 5, 2, 11, 31, 4, 371165), ‘totalcputime’: 0.117788, ‘username’: ‘Test’}

Returns:Database ID of the item created as an int, like: 3
Return type:int
change_column_type(col_name, col_type)[source]

Modify data type of an existing column - required for modification on the database

Parameters:
  • col_name (str, list) – name of the new column, normal string like: ‘myColumn’
  • (str, list (col_type) – SQL type of the new column, SQL type like: ‘varchar(50)’

Returns:

delete_item(item_id)[source]

Delete Item from database

Parameters:item_id (int) – Databse Item ID (Integer), like: 38

Returns:

get_item_by_id(item_id)[source]

Get item from database by searching for a specific item Id.

Parameters:item_id (int) – Databse Item ID (Integer), like: 38
Returns:
Dictionary where the key is the column name, like:
{‘chemicalformula’: u’BO’,
’computer’: u’localhost’, ‘hamilton’: u’VAMPS’, ‘hamversion’: u‘1.1’, ‘id’: 1, ‘job’: u’testing’, ‘masterid’: None, ‘parentid’: 0, ‘project’: u’database.testing’, ‘projectpath’: u’/root/directory/tmp’, ‘status’: u’KAAAA’, ‘subjob’: u’SubJob’, ‘timestart’: datetime.datetime(2016, 5, 2, 11, 31, 4, 253377), ‘timestop’: datetime.datetime(2016, 5, 2, 11, 31, 4, 371165), ‘totalcputime’: 0.117788, ‘username’: u’Test’}
Return type:dict
get_items_dict(item_dict, return_all_columns=True)[source]
Parameters:
  • item_dict (dict) –

    a dict type, which has a certain syntax for this function: a normal dict like {‘hamilton’: ‘VAMPE’, ‘hamversion’: ‘1.1’} has similarities with a simple query like

    select * from table_name where hamilton = ‘VAMPE AND hamversion = ‘1.1’

    as seen it puts an AND for every key, value combination in the dict and searches for it.

    another syntax is for an OR statement, simply: {‘hamilton’: [‘VAMPE’, ‘LAMMPS’]}, the query would be:

    select * from table_name where hamilton = ‘VAMPE’ OR hamilton = ‘LAMMPS’
    and lastly for a LIKE statement, simply: {‘project’: ‘database.%’}, the query would be
    select * from table_name where project LIKE ‘database.%’

    that means you can simply add the syntax for a like statement like ‘%’ and it will automatically operate a like-search

    of course you can also use a more complex select method, with everything in use:
    {‘hamilton’: [‘VAMPE’, ‘LAMMPS’],
    ’project’: ‘databse%’, ‘hamversion’: ‘1.1’}
    select * from table_name where (hamilton = ‘VAMPE’ Or hamilton = ‘LAMMPS’) AND
    (project LIKE ‘database%’) AND hamversion = ‘1.1’
  • return_all_columns (bool) – return all columns or only the ‘id’ - still the format stays the same.
Returns:

the function returns a list of dicts like get_items_sql, but it does not format datetime:
[{‘chemicalformula’: u’Ni108’,

’computer’: u’mapc157’, ‘hamilton’: u’LAMMPS’, ‘hamversion’: u‘1.1’, ‘id’: 24, ‘job’: u’DOF_1_0’, ‘parentid’: 21L, ‘project’: u’lammps.phonons.Ni_fcc’, ‘projectpath’: u’D:/PyIron/PyIron_data/projects’, ‘status’: u’finished’, ‘timestart’: datetime.datetime(2016, 6, 24, 10, 17, 3, 140000), ‘timestop’: datetime.datetime(2016, 6, 24, 10, 17, 3, 173000), ‘totalcputime’: 0.033, ‘username’: u’test’},

{‘chemicalformula’: u’Ni108’,

’computer’: u’mapc157’, ‘hamilton’: u’LAMMPS’, ‘hamversion’: u‘1.1’, ‘id’: 21, ‘job’: u’ref’, ‘parentid’: 20L, ‘project’: u’lammps.phonons.Ni_fcc’, ‘projectpath’: u’D:/PyIron/PyIron_data/projects’, ‘status’: u’finished’, ‘timestart’: datetime.datetime(2016, 6, 24, 10, 17, 2, 429000), ‘timestop’: datetime.datetime(2016, 6, 24, 10, 17, 2, 463000), ‘totalcputime’: 0.034, ‘username’: u’test’},…….]

Return type:

list

get_items_sql(where_condition=None, sql_statement=None)[source]

Submit an SQL query to the database

Parameters:
  • where_condition (str) – SQL where query, query like: “project LIKE ‘lammps.phonons.Ni_fcc%’”
  • sql_statement (str) – general SQL query, normal SQL statement
Returns:

get a list of dictionaries, where each dictionary represents one item of the table like:
[{u’chemicalformula’: u’BO’,

u’computer’: u’localhost’, u’hamilton’: u’VAMPS’, u’hamversion’: u‘1.1’, u’id’: 1, u’job’: u’testing’, u’masterid’: None, u’parentid’: 0, u’project’: u’database.testing’, u’projectpath’: u’/TESTING’, u’status’: u’KAAAA’, u’subjob’: u’testJob’, u’timestart’: u‘2016-05-02 11:31:04.253377’, u’timestop’: u‘2016-05-02 11:31:04.371165’, u’totalcputime’: 0.117788, u’username’: u’User’},

{u’chemicalformula’: u’BO’,

u’computer’: u’localhost’, u’hamilton’: u’VAMPS’, u’hamversion’: u‘1.1’, u’id’: 2, u’job’: u’testing’, u’masterid’: 0, u’parentid’: 0, u’project’: u’database.testing’, u’projectpath’: u’/TESTING’, u’status’: u’KAAAA’, u’subjob’: u’testJob’, u’timestart’: u‘2016-05-02 11:31:04.253377’, u’timestop’: u‘2016-05-02 11:31:04.371165’, u’totalcputime’: 0.117788, u’username’: u’User’}…..]

Return type:

list

get_table_headings(table_name=None)[source]

Get column names

Parameters:table_name (str) – simple string of a table_name like: ‘jobs_username’
Returns:
list of column names like:
[‘id’, ‘parentid’, ‘masterid’, ‘projectpath’, ‘project’, ‘job’, ‘subjob’, ‘chemicalformula’, ‘status’, ‘hamilton’, ‘hamversion’, ‘username’, ‘computer’, ‘timestart’, ‘timestop’, ‘totalcputime’]
Return type:list
item_update(par_dict, item_id)[source]

Modify Item in database

Parameters:
  • par_dict (dict) –

    Dictionary of the parameters to be modified,, where the key is the column name. {‘job’ : ‘maximize’,

    ’subjob’ : ‘testing’, ……..}
  • item_id (int, list) – Database Item ID (Integer) - ‘38’ can also be [38]

Returns:

query_for_element(element)[source]
static regexp(expr, item)[source]

Regex function for SQLite :param expr: str, regex expression :param item: str, item which needs to be checked

Returns:

viewer_mode

Get viewer_mode - if viewer_mode is enable pyiron has read only access to the database.

Returns:returns TRUE when viewer_mode is enabled
Return type:bool

pyiron.base.database.jobtable module

pyiron.base.database.jobtable.get_child_ids(database, sql_query, user, project_path, job_specifier, status=None)[source]

Get the childs for a specific job

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • job_specifier (str) – name of the master job or the master jobs job ID
  • status (str) – filter childs which match a specific status - None by default
Returns:

list of child IDs

Return type:

list

pyiron.base.database.jobtable.get_db_columns(database)[source]

Get column names

Parameters:database (DatabaseAccess) – Database object
Returns:
list of column names like:
[‘id’, ‘parentid’, ‘masterid’, ‘projectpath’, ‘project’, ‘job’, ‘subjob’, ‘chemicalformula’, ‘status’, ‘hamilton’, ‘hamversion’, ‘username’, ‘computer’, ‘timestart’, ‘timestop’, ‘totalcputime’]
Return type:list
pyiron.base.database.jobtable.get_job_id(database, sql_query, user, project_path, job_specifier)[source]

get the job_id for job named job_name in the local project path from database

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • job_specifier (str) – name of the job or job ID
Returns:

job ID of the job

Return type:

int

pyiron.base.database.jobtable.get_job_ids(database, sql_query, user, project_path, recursive=True)[source]

Return the job IDs matching a specific query

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • recursive (bool) – search subprojects [True/False]
Returns:

a list of job IDs

Return type:

list

pyiron.base.database.jobtable.get_job_status(database, sql_query, user, project_path, job_specifier)[source]

Get the status of a particular job

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • job_specifier (str) – name of the job or job ID
Returns:

job status can be one of the following [‘initialized’, ‘appended’, ‘created’, ‘submitted’, ‘running’,

’aborted’, ‘collect’, ‘suspended’, ‘refresh’, ‘busy’, ‘finished’]

Return type:

str

pyiron.base.database.jobtable.get_job_working_directory(database, sql_query, user, project_path, job_specifier)[source]

Get the working directory of a particular job

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • job_specifier (str) – name of the job or job ID
Returns:

working directory as absolute path

Return type:

str

pyiron.base.database.jobtable.get_jobs(database, sql_query, user, project_path, recursive=True, columns=None)[source]

Internal function to return the jobs as dictionary rather than a pandas.Dataframe

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • recursive (bool) – search subprojects [True/False]
  • columns (list) – by default only the columns [‘id’, ‘project’] are selected, but the user can select a subset of [‘id’, ‘status’, ‘chemicalformula’, ‘job’, ‘subjob’, ‘project’, ‘projectpath’, ‘timestart’, ‘timestop’, ‘totalcputime’, ‘computer’, ‘hamilton’, ‘hamversion’, ‘parentid’, ‘masterid’]
Returns:

columns are used as keys and point to a list of the corresponding values

Return type:

dict

pyiron.base.database.jobtable.job_table(database, sql_query, user, project_path, recursive=True, columns=None, all_columns=False, sort_by='id', max_colwidth=200, element_lst=None)[source]

Access the job_table

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • recursive (bool) – search subprojects [True/False]
  • columns (list) – by default only the columns [‘job’, ‘project’, ‘chemicalformula’] are selected, but the user can select a subset of [‘id’, ‘status’, ‘chemicalformula’, ‘job’, ‘subjob’, ‘project’, ‘projectpath’, ‘timestart’, ‘timestop’, ‘totalcputime’, ‘computer’, ‘hamilton’, ‘hamversion’, ‘parentid’, ‘masterid’]
  • all_columns (bool) – Select all columns - this overwrites the columns option.
  • sort_by (str) – Sort by a specific column
  • max_colwidth (int) – set the column width
  • element_lst (list) – list of elements required in the chemical formular - by default None
Returns:

Return the result as a pandas.Dataframe object

Return type:

pandas.Dataframe

pyiron.base.database.jobtable.set_job_status(database, sql_query, user, project_path, job_specifier, status)[source]

Set the status of a particular job

Parameters:
  • database (DatabaseAccess) – Database object
  • sql_query (str) – SQL query to enter a more specific request
  • user (str) – username of the user whoes user space should be searched
  • project_path (str) – root_path - this is in contrast to the project_path in GenericPath
  • job_specifier (str) – name of the job or job ID
  • status (str) – job status can be one of the following [‘initialized’, ‘appended’, ‘created’, ‘submitted’, ‘running’, ‘aborted’, ‘collect’, ‘suspended’, ‘refresh’, ‘busy’, ‘finished’]

Module contents