NAME

FEI_Watchdog

Watches the FEI server for new files for pipeline source entries.

SYNOPSIS

FEI_Watchdog <options>

OPTIONS

-COnfiguration <pathname>

The configuration file pathname. The default is $HiRISE_ROOT/Configuration/<pipeline>/<pipeline>.conf.

-Type <file type>

The FEI file type to monitor. The default file type is "hirise", which can be overridden using the /<pipeline>/File_Type configuration parameter.

-Selection <pattern>

The pattern (regex) used to select files from the server. The default is to select all available files dated after that last timestamp, or the startup time if there is no timestamp. The default can be overridden using the /<pipeline>/File_Selection configuration parameter. The timestamp is held in the $HiRISE_ROOT/Configuration/<pipeline>/<pipeline>_Watchdog.last_time file.

-After <timestamp>

Files modified after the specified date will be seen as new files to be registered in the pipeline. The <timestamp> must conform to the format specified in the Timestamp section, below. If no selection date is specified, the current time will be used by default unless a Timestamp file is found that contains a timestamp.

-Pipeline <pipeline>

The name of the pipeline to receive Source_Pathname entries for new files found in the FEI server file type catalog. The portion of the command name preceeding the first '_' character will be the default pipeline name unless a configuration file is specified that contains the /Pipeline configuration parameter.

The order of priority in determining the pipeline name is: command line option, configuration file paramter, and command name prefix.

-Catalog <catalog>

The name of the database catalog that contains the pipeline tables. If not specified the /<pipeline>/Catalog, /Conductor/Catalog, and /Catalog configuration parameters will be tried, in that order. The default is HiRISE.

-Notify [+]<address>[,...]

A list of email address to receive notification in case an unexpected error conditition forces a shutdown of the watchdog. If the list is preceeded by a '+' character it will supplement any list obtained from a Notify parameter in the configuration file. Otherwise the configuration file is not consulted. If this option is not used any list from the configuration file is used instead.

-Rate <seconds>

The polling time between checks of the FEI catalog for new files. The default is 300 seconds, which can be overridden using the /<pipeline>/Polling_Rate configuration parameter. The minimum is 10 seconds.

DESCRIPTION

The FEI_Watchdog will poll an FEI server looking for the names of new files to fetch as source entries into a Conductor pipleline. Files are selected to be fetched by both a filename selection pattern and by being dated after a timestamp. The watchdog will continue to poll the FEI server at a selectable rate until it is shutdown.

Environment

The $HIRISE_ROOT environment variable must be present. It provides the location of the Configuration, Logs and Data subdirectories, each of which must contain a subdirectory with the pipeline name (see the Configuration file exception, below); these are, respectively, the configuration, logging and data respository directories. An attempt will be made to create all necessary directories if they do not exist.

The $FEI_HOME environment variable must be present. It provides the location of the FEI client utilities. The fei5list utility is used to poll the FEI server for available files.

Files

The default location of the configuration directory and the location of the data and log directories are relative to the $HiRISE_ROOT value. This value may be provided by the environment variable or configuration parameter of the same name. The value from the configuration file takes precedence. However, if the configuration file is located under the directory named by the environment variable (if present), then the value of the environment variable is used instead.

Configuration

The configuration file pathname is $HiRISE_ROOT/Configuration/<pipeline>/<pipeline>.conf by default. If the configuration file pathname is specified as a command line option, its directory pathname (a relative name will have the current working directory prepended) will be used as the configuration directory for the lock and timestamp files.

The configuration pathname is passed to the Pipeline_Source utility to provide access to the database for entering filenames in the HiRISE.<pipeline>_Sources Source_Pathname field as the insertion of a new source record.

The configuration file may contain a <pipeline> Group with parameters corresponding to the OPTIONS to set the corresponding values. Note that if the Pipeline parameter is in specified it must be at the top level (not in the <pipeline> Group. The Pipeline parameter is only used when the configuration file pathname in specified as a command line option and the pipeline name is not specifed as a command line option; otherwise the configuration file pathname is based on the default or specified pipeline name which will not be changed by a configuration file parameter. In addition, the values of a Notify parameter will be taken as the list of addresses to notify, using the Notify utility, in case an unexpected error conditition forces a shutdown of the watchdog.

Timestamp

If the timestamp file - <pipeline>_Watchdog.last_time - is present in the configuration directory the timestamp value will be read from the first line of the file. The timestamp must be in host system localtime reference with the following format:

YYYY-MM-DDThh:mm:ss.fff

Where:

YYYY

Year number

MM

Month number (01-12)

DD

Day of the month (01-31)

hh

Hour of the day (00-23)

mm

Minute of the hour (00-59)

ss

Second of the minute (00-59)

ff

Fractional second (000-999)

Leading zeros are required if necessary to fill the field width. The time values may be truncated by dropping segments from the end; in this case the default for the omitted values will be 0.

The timestamp file may be "seeded" with an initial value or backdated to force retransfer of previously transfered files. WARNING: Never modify the timestamp file while the watchdog is running.

In addition, the file will be updated with the name of the last file transferred, on the same line followed by a space. This is provided for informational purposes only; it is not used in any way.

The timestamp file is updated with the date and name of the last file registered as a pipeline source, but only if the date of the file is later than the current timestamp; i.e. the timestamp value will only be increased by the watchdog.

Data

The data repository directory will be provided with a "Pending" subdirectory if it doesn't already exist. This will be where fetch files will be written. A fetch file has the same name as the file to be fetched from the FEI server by a pipeline procedure. Its contents are the transfer operation specifications (called a "using" file by FEI) for the fei5get utility that is expected to be used as a pipeline procedure to carry out the download of the file from the FEI server to the data respository. The contents are (all on one line):

<file type> <filename> output <data directory>

Log

The log file - <pipeline>_Watchdog.log - in the logging directory will be appended with startup, source file registration, error condtion, and shutdown messages. The startup messae is preceeded by a line containing all '-' characters. The shutdown messages is followed by a line containing all '=' characters. Error messages are preceeded by a line containing all '!' characters and always followed by a shutdown message.

Lock

The lock file - <pipeline>_Watchdog.lock - in the configuration directory will be written with the hostname and process ID of the watchdog. Note: This file must not be present when the watchdog starts.

The lock file is used to help prevent two Watchdog processes from being associated with the same pipeline and thus possibly creating redundant pipeline source entries (and other potential conflicts). However, when a watchdog is started with a command line specified configuration file pathanme the lock file is written to the directory of the configuration file, so another watchdog process may be started with a configuration file in different directory but associated with the same pipeline. There may be a good reason to do this (e.g. a different filename selection pattern is used), but it should be kept in mind that each watchdog should be kept on its own chain; they can maul each other if they are not kept apart.

Operation

Setup

On startup the watchdog will confirm the presence of all the required enviroment variables and files. A failure of the setup requirements will result in an error message being printed (to stderr) and and the process will exit. If the lock file is present its contents will be printed (to stderr) along with an error message and the process will exit. The watchdog writes its hostname and process ID in the lock file as notice that it is on duty.

WARNING: Only one instance of the watchdog should be run to prevent the possibility of the same file being inadvertently fetched more than once from the FEI server. The watchdog guards the entry to its pipeline, only allowing the correctly selected files to enter.

If the timestamp file is present the timestamp value it contains intializes the current timestamp selection value; otherwise this value is intialized with the start time of the process.

Note: During setup any error conditions result in a message to stderr and an exit with an error status. The log file is not used during this stage. Once setup is complete the log file is written with the startup message.

Polling Cycle

The polling cycle begins by getting the list of files from the FEI server (using fei5list) that are dated after the current timestamp value. The long form of the file list is obtained from the server so each filename will be accompanied by its registration date; this date is provided in localtime reference as determined by the host system. This list is then filtered against the filename selection pattern, if one was specified.

The resultant list is used to create fetch files in the Pending subdirectory of the data respository. Each fetch file is named the same as the file to be fetch from the FEI server and contains the fei5get operation specifications to be used in downloading the file from the FEI server to the data repository. If a fetch file of the same name already exists, it is overwritten. The pathname of each fetch file is entered as a pipeline source using the Pipeline_Source utility. A fetch file pipeline source entry always results in a new record being inserted in the table; records having the same fetch file are not replaced.

If the date of the FEI file for which a fetch file was registered in the pipeline is newer than the current timestamp value, the timestamp value is assigned the new date and the timestamp file is updated with the new value.

When all the new FEI files have been registered with new fetch files entries in the pipeline, the watchdog goes to sleep for the polling interval, after which the polling cycle begins again.

Shutdown

At any time during the polling cycle the watchdog may be sent a quit (QUIT), terminate (TERM), hangup (HUP), or interrupt (INT) signal which will cause the watchdog to gracefully shutdown that includes writing the shutdown message to the log file and deleting the lock file. If a shutdown signal is received while the watchdog is not sleeping, the shutdown is defered until the polling cycle has completed.

Failures

If the watchdog detects an error condition while obtaining the file list from the FEI server, writing a fetch file, entering the fetch file as a pipeline source, or updating the timestamp file it records an appropriate error message, including any message from the utility being employed, to both the log file and an error log file - <pipeline>_Watchdog.error - in the logging directory. If the configuration file contains a <pipeline> Group with a "Notify" parameter (found using the PPVL_report -query utility), a copy of the error log file is sent (using the Notify utility) to each address assigned to the parameter.

Note: If the fei5list utility produces any output that is not recognized as file list lines it is simply written to the log file as an unexpected message, but otherwise ignored.

Exit Status

0 - Success

On shutdown as a result of an external signal.

1 - Bad command line syntax

A command line syntax usage message will be provided.

2 - System error

A required environment variable is not present, a required directory could not be created, the configuration file does not exist, the log file is not write accessible, the timestamp file exists but can not be read and written, the lock file exists or can not be written, or a fetch file can not be written.

3 - FEI error

An error condition was detected with the fei5list utility.

4 - Pipeline error

The pipeline source table could not be updated with a new source entry.

5 - Timestamp error

The timestamp file could not be updated. Note: The error and/or log file should have the information that could not be written to the timestamp file, which should be updated before restarting the watchdog.

Author

Bradford Castalia, UA/PIRL

Copyright

Copyright (C) 2004-2020 Arizona Board of Regents on behalf of the Lunar and Planetary Laboratory at the University of Arizona.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Version

1.35 2020/04/28 17:36:11