| Home > Hacking Invenio > BibSched Internals > BibTask |
This is an howto create a Bibliographic Task suitable to be used with bibsched.
For starting developing a new bibtask first take a look at the source code of bibtaskex.py.
A BibTask "foobar" is well done when:
Being a bibtask, your script should use bibtask.write_message each time it needs to inform the user about something.
It should use bibtask.task_set_option and bibtask.task_get_option in order to interact with the bibtask options (e.g. verbosity, runtime, sleeptime, priority...)
It should use bibtask.task_set_task_param and bibtask.task_get_task_param in order to interact with its specific command line options.
It should use bibtask.task_update_progress in order to update the progress information on the task that will be displayed in bibsched.
It should call bibtask.task_sleep_now_if_required in each part of the code where it's safe to sleep or stop, e.g. outside of atomic operations or transactions. If not sure, just not use this function and the task won't ever be stopped or put to sleep.
It should implement a function to be passed to task_init via the task_submit_elaborate_specific_parameter_fnc parameter to handle specific command line parameters of the foobar bibtask.
It should implement a function to be passed to task_init via the task_submit_check_options_fnc parameter to have a chance to check the correctness of command line options before the task is submitted to the queue or executed.
It should implement a main function to be passed to task_init via the task_run_fnc parameter as the core of the bibtask.
It can be scheduled directly via a Python script by using the task_low_level_submission API.
def task_init(
authorization_action="",
authorization_msg="",
description="",
help_specific_usage="",
version=__revision__,
specific_params=("", []),
task_stop_helper_fnc=None,
task_submit_elaborate_specific_parameter_fnc=None,
task_submit_check_options_fnc=None,
task_run_fnc=None):
""" Initialize a BibTask.
@param authorization_action is the name of the authorization action
connected with this task;
@param authorization_msg is the header printed when asking for an
authorization password;
@param description is the generic description printed in the usage page;
@param help_specific_usage is the specific parameter help
@param task_stop_fnc is a function that will be called
whenever the task is stopped
@param task_submit_elaborate_specific_parameter_fnc will be called passing
a key and a value, for parsing specific cli parameters. Must return True if
it has recognized the parameter. Must eventually update the options with
bibtask_set_option;
@param task_submit_check_options must check the validity of options (via
bibtask_get_option) once all the options where parsed;
@param task_run_fnc will be called as the main core function. Must return
False in case of errors.
"""
def task_set_option(key, value):
"""Set an value to key in the option dictionary of the task"""
def task_get_option(key, default=None):
"""Returns the value corresponding to key in the option dictionary of the task"""
def task_has_option(key):
"""Map the has_key query to _options"""
def task_get_task_param(key, default=None):
"""Returns the value corresponding to the particular task param"""
def task_set_task_param(key, value):
"""Set the value corresponding to the particular task param"""
def task_update_progress(msg): """Updates progress information in the BibSched task table."""
def write_message(msg, stream=sys.stdout, verbose=1):
"""Write message and flush output stream (may be sys.stdout or sys.stderr).
Useful for debugging stuff."""
def task_sleep_now_if_required(can_stop_too=False):
"""This function should be called during safe state of BibTask,
e.g. after flushing caches or outside of run_sql calls.
"""
def task_low_level_submission(name, user, *argv):
"""Let special lowlevel enqueuing of a task on the bibsche queue.
@param name: is the name of the bibtask. It must be a valid executable under
C{CFG_BINDIR}.
@type name: string
@param user: is a string that will appear as the "user" submitting the task.
Since task are submitted via API it make sense to set the
user to the name of the module/function that called
task_low_level_submission.
@type user: string
@param argv: are all the additional CLI parameters that would have been
passed on the CLI (one parameter per variable).
e.g.:
>>> task_low_level_submission('bibupload', 'admin', '-a', '/tmp/z.xml')
@type: strings
@return: the task identifier when the task is correctly enqueued.
@rtype: int
@note: use absolute paths in argv
"""
def get_modified_records_since(modification_date):
"""
Return the set of modified record since the given
modification_date.
@param modification_date: Return records modified after this date
@type modification_date datetime
"""