pyiron.base.server package

Submodules

pyiron.base.server.generic module

class pyiron.base.server.generic.Server(host=None, queue=None, cores=1, run_mode='modal', new_hdf=True)[source]

Bases: pyiron.base.generic.template.PyironObject

Generic Server object to handle the execution environment for the job

Parameters:
  • host (str) – hostname of the local machine
  • queue (str) – queue name of the currently selected queue
  • cores (int) – number of cores
  • run_mode (str) – mode of the job [‘modal’, ‘non_modal’, ‘queue’, ‘manual’]
  • new_hdf (bool) – create a new HDF5 file [True/False] - default=True
.. attribute:: send_to_db

boolean option to decide which jobs should be store in the external/public database.

.. attribute:: structure_id

the structure ID to be linked to an external/public database.

.. attribute:: host

the hostname of the current system.

.. attribute:: queue

the que selected for a current simulation.

.. attribute:: cores

the number of cores selected for the current simulation.

.. attribute:: run_time

the run time in seconds selected for the current simulation.

.. attribute:: run_mode

the run mode of the job [‘modal’, ‘non_modal’, ‘queue’, ‘manual’]

.. attribute:: new_hdf

defines whether a subjob should be stored in the same HDF5 file or in a new one.

accept_crash
cores

The number of cores selected for the current simulation

Returns:number of cores
Return type:(int)
db_entry()[source]

connect all the info regarding the server into a single word that can be used e.g. as entry in a database

Returns:server info as single word
Return type:(str)
from_hdf(hdf, group_name=None)[source]

Recover Server object in HDF5 file

Parameters:
  • hdf – HDF5 object
  • group_name – node name in the HDF5 file
host

Get the hostname of the current system.

Returns:hostname
Return type:(str)
init_scheduler_run(working_dir, wait_for_prev_job=None, job_id=None)[source]

Setup the job scheduler to return the scheduler options which then can be used for the python subprocess.

Parameters:
  • working_dir (str) –
  • wait_for_prev_job (int) – job id to wait for
  • job_id (int) –
Returns:

list of que options, [True/ False] whether the job returns a job id or not.

Return type:

(list)

list_queues()[source]

List the available Job scheduler provided by the system.

Returns:(list)
new_hdf

New_hdf5 defines whether a subjob should be stored in the same HDF5 file or in a new one.

Returns:[True / False]
Return type:(bool)
queue

The que selected for a current simulation

Returns:schedulers_name
Return type:(str)
queue_id

Get the queue ID - the ID in the queuing system is most likely not the same as the job ID.

Returns:queue ID
Return type:int
run_mode

Get the run mode of the job

Returns:[‘modal’, ‘non_modal’, ‘queue’, ‘manual’]
Return type:(str)
run_time

The run time in seconds selected for the current simulation

Returns:run time in seconds
Return type:(int)
send_to_db

Get the boolean option to decide which jobs should be store in the external/public database

Returns:[True/False]
Return type:bool
structure_id

Get the structure ID to be linked to an external/public database

Returns:structure ID
Return type:int
to_hdf(hdf, group_name=None)[source]

Store Server object in HDF5 file

Parameters:
  • hdf – HDF5 object
  • group_name (str) – node name in the HDF5 file
view_queues()[source]

List the available Job scheduler provided by the system.

Returns:(pandas.DataFrame)

pyiron.base.server.queue module

class pyiron.base.server.queue.Queue(name, mini_cores, maxi_cores, divisor_list, run_time_limit)[source]

Bases: object

Configuration of an individual que

Parameters:
  • name (str) – name of the que
  • mini_cores (int) – minimum number of cores for an individual job
  • maxi_cores (int) – maximum number of cores for an individual job
  • divisor_list (int, list) – single int or list of int of cores the number of cores selected for an individual job should be divided by. With this we can force the user to always use full nodes by setting the divisor_list to 20 cores.
  • run_time_limit (int) – the time limit in seconds for an individual job
cores

The number of cores used for the current simulation, this validated that the number of cores selected agrees with the requirements of the individual que.

Returns:number of cores
Return type:int
divisors_for_que

Force the user to always use full nodes by setting the divisors to the number of cores per node, so the user can run a job either on one node or two nodes but not on one node and a single settings from a second node, which is insufficient in most cases.

Returns:list of divisors
Return type:list
maximum_number_of_cores

Get maximum number of cores for an individual job

Returns:minimum number of cores
Return type:int
minimum_number_of_cores

Get minimum number of cores for an individual job

Returns:minimum number of cores
Return type:int
run_time

The run time used for the current simulation, this validated that the run time selected agrees with the requirements of the individual que.

Returns:run time
Return type:int
run_time_limit

The run time limit defined by the que.

Returns:run time limit
Return type:int

pyiron.base.server.queuestatus module

pyiron.base.server.queuestatus.queue_delete_job(item)[source]

Delete a job from the queuing system

Parameters:item (int, pyiron.base.job.generic.GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Output from the queuing system as string - optimized for the Sun grid engine
Return type:str
pyiron.base.server.queuestatus.queue_enable_reservation(item)[source]

Enable a reservation for a particular job within the queuing system

Parameters:item (int, pyiron.base.job.generic.GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Output from the queuing system as string - optimized for the Sun grid engine
Return type:str
pyiron.base.server.queuestatus.queue_id_table(requested_id=None)[source]

Display the queuing system table as dictionary

Parameters:requested_id (int) – query for a specific job_id - optional
Returns:Dictionary with the output from the queuing system - optimized for the Sun grid engine
Return type:dict
pyiron.base.server.queuestatus.queue_info_by_job_id(job_id)[source]

Display the queuing system info of job by qstat | grep shell command as dictionary

Parameters:requested_id (int) – query for a specific job_id
Returns:Dictionary with the output from the queuing system - optimized for the Sun grid engine
Return type:dict
pyiron.base.server.queuestatus.queue_is_empty()[source]

Check if the queue table is currently empty - no more jobs to wait for.

Returns:True if the table is empty, else False - optimized for the Sun grid engine
Return type:bool
pyiron.base.server.queuestatus.queue_job_info(item)[source]

Short reporting for a particular job - using the qstat command of the sun grid engine.

Parameters:item (int, pyiron.base.job.generic.GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Short report returned from the queuing system - optimized for the Sun grid engine
Return type:pandas.DataFrame
pyiron.base.server.queuestatus.queue_report(item)[source]

Detailed reporting for a particular job - using the qacct command of the sun grid engine.

Parameters:item (int, pyiron.base.job.generic.GenericJob) – Provide either the job_ID or the full hamiltonian
Returns:Detailed report returned from the queuing system - optimized for the Sun grid engine
Return type:pandas.DataFrame
pyiron.base.server.queuestatus.queue_table(job_ids=[], project_only=True)[source]

Display the queuing system table as pandas.Dataframe

Parameters:
  • job_ids (list) – check for a specific list of job IDs - empty list by default
  • project_only (bool) – Query only for jobs within the current project - True by default
Returns:

Output from the queuing system - optimized for the Sun grid engine

Return type:

pandas.DataFrame

pyiron.base.server.queuestatus.wait_for_job(job, interval_in_s=5, max_iterations=100)[source]

Sleep until the job is finished but maximum interval_in_s * max_iterations seconds.

Parameters:
  • job (pyiron.base.job.generic.GenericJob) – Job to wait for
  • interval_in_s (int) – interval when the job status is queried from the database - default 5 sec.
  • max_iterations (int) – maximum number of iterations - default 100

pyiron.base.server.runmode module

Runmode class defines the different modes a pyiron job can be executed in

class pyiron.base.server.runmode.Runmode(mode='modal')[source]

Bases: str

Run mode describes how the job is going to be executed: - modal: the interactive run mode - non_modal: sending the job to the background on the same machine - queue: submit the job to the queuing system - manual: let the user manually execute the job - thread: internal job mode, which is selected when the master job is send to the queue. - interactive: the interactive run mode :param mode: [‘modal’, ‘non_modal’, ‘queue’, ‘manual’, ‘thread’, ‘interactive’] :type mode: str

mode

Get the run_mode of the job :returns: [‘modal’, ‘non_modal’, ‘queue’, ‘manual’, ‘thread’, ‘interactive’] :rtype: str

pyiron.base.server.shelloption module

ShellOption class as part of the queueing system interface

class pyiron.base.server.shelloption.ShellOption(name, key=None, value=None, value_prefix=None, active=True)[source]

Bases: object

The ShellOption allows to match the parameters of the queing system with the requirements of pyiron.

Parameters:
  • name (str) – name of the ShellOption
  • key (str) – command line option for the specific queing system - (optional)
  • value (str) – value the command line option of the specific queing system should be set to - (optional)
  • value_prefix (str) – a prefix for the command line option for the specific queing system - (optional)
  • active (bool) – if a tag is not supported by a certain queing system set it to False - [True/False]
active

Status of the ShellOption, active ?

Returns:[True/False]
Return type:bool
key

Command line option for the specific queing system

Returns:
Return type:str
key_only

Verify if the Shelloption is active and only contains a key but no value.

Returns:[True/False]
Return type:bool
output_as_list()[source]

Output the shell option as list, so it can be used in combination with a python sub process.

Returns:list of strings
Return type:list
value

Value of the command line option for the specific queing system

Returns:
Return type:str
value_only

Verify if the Shelloption is active and only contains a value but no key.

Returns:[True/False]
Return type:bool
value_prefix

Value prefix of the command line option for the specific queing system

Returns:
Return type:str

Module contents