HiRISE

HiRISE.HiCat.Utilities
Class Geospatial_BackFill

java.lang.Object
  extended by HiRISE.HiCat.Utilities.Geospatial_BackFill

public class Geospatial_BackFill
extends Object

Geospatial_BackFill This utility back fills the GIS tables.

This program requires specifying -Mode_Suggestions, -Mode_Products, or -Mode_External See the usage method for more information on the usage of this program. Also see the usage for a sample configuration file.

This program will create geospatial information for RDR Products, and Suggested Observations and store it in special GIS tables. It also can use classes specified in the configuration file to load external databases into a local database.

If either Suggestions or Products are specified then a server and a catalog must be specified for the RDR_Products and Suggested_Observations GIS tables. A source table and a destination GIS table may be specified for both, but there are defaults if none are specified on the command line or in the configuration file.

The valid operating modes are Prodcuts, Suggestions, and External. At least one operating mode needs to be specified, but any combination of modes can be specified.

The server and catalog for this program can be specified on the command line or in a separate configuration file. The configuration filename can be specified on the command line, otherwise the program will use the default name. The server is the group in the configuration file which will contain information for connecting to the database. However, the Catalog and the Table names should be specified in the config group.

The command line usage description provides more details on the application operation.

TODO: The Handle_RDR_Products() and Handle_External_Databases() methods should be refactored into instance methods like the Handle_Suggestions() method has been.

Version:
1.16
Author:
Sean Whitsitt - UA/PIRL, Rodney Heyd - UA/PIRL
See Also:
PIRL.Database, PIRL.Configuration, PIRL.PVL

Field Summary
static String CONFIG_GROUP
          The configuration group for this program
static String CONFIG_PARAM_CATALOG
          The default parameter for the HiRISE Catalog
static String CONFIG_PARAM_EXTERNAL_ARGUMENTS
          The default parameter for the list of string arguments that the constructor for the class of an external database loader will use.
static String CONFIG_PARAM_EXTERNAL_CATALOG
          The default parameter for the catalog that an external database will be loaded to
static String CONFIG_PARAM_EXTERNAL_LOADERS
          The default parameter for the list of classes for loading external databases
static String CONFIG_PARAM_EXTERNAL_SERVER
          The default parameter for the server that an external database will be loaded to
static String CONFIG_PARAM_EXTERNAL_TABLE
          The default parameter for the table that an external database will be loaded to
static String CONFIG_PARAM_MAX_URL_ENTRIES
          The default parameter for the maximum url entries
static String CONFIG_PARAM_RDR_PRODUCTS_GIS_TABLE
          The default parameter for the RDR_Products_GIS table
static String CONFIG_PARAM_RDR_PRODUCTS_TABLE
          The default parameter for the RDR_Products table
static String CONFIG_PARAM_SUGGESTED_OBSERVATIONS_GIS_TABLE
          The default parameter for the Suggested Observations GIS table
static String CONFIG_PARAM_SUGGESTED_OBSERVATIONS_TABLE
          The default parameter for the Suggested Observations table
static String DEFAULT_CONFIGURATION_FILENAME
          The default configuration filename
static int DEFAULT_MAX_URL_ENTRIES
          The default maximum number of entries to read from a URL
static char DEFAULT_PATH_DELIMITER
          The default path delimiter for the configuration
static String DEFAULT_RDR_PRODUCTS_GIS_TABLE
          The table name to store the geospatial information for red, color, ctx, and moc images
static String DEFAULT_RDR_PRODUCTS_TABLE
          The RDR_Products table name
static String DEFAULT_SUGGESTED_OBSERVATIONS_GIS_TABLE
          The Suggested_Observation_GIS table name
static String DEFAULT_SUGGESTED_OBSERVATIONS_TABLE
          The Suggested_Observation table name
static int EXIT_COMMAND_LINE_SYNTAX_ERROR
          Application exit status for a command line syntax error
static int EXIT_CONFIGURATION_ERROR
          Application exit status for a configuration error
static int EXIT_DATABASE_ERROR
          Application exit status for a database error
static int EXIT_DATABASE_PROBLEM
          Application exit status for a database problem.
static int EXIT_IO_ERROR
          Application exit status for a failed IO operation
static int EXIT_NO_MODE
          Application exit status for when no mode is specified.
static int EXIT_SUCCESS
          Application exit status for success.
static int EXIT_URL_ERROR
          Application exit status for a failed url
static String ID
          The Class identification with revision number.
static short MODE_EXTERNAL
          The flag for running the backfill for MOC
static short MODE_NONE
          The flag for doing nothing
static short MODE_PRODUCTS
          The flag for running the backfill for RDR products and CTX
static short MODE_SUGGESTIONS
          The flag for running the backfill for suggestions
static String NL
          System specific end of line character
 
Constructor Summary
Geospatial_BackFill()
           
 
Method Summary
static boolean Compare_Vectors(Vector<String> v, Vector<String> u)
          This method compares two Vectors of Strings with compareToIgnoreCase
static void main(String[] arguments)
          Run the Geospatial_BackFill application.
 void updateSuggestedObservations()
          This method ultimately results in GIS objects being inserted, updated, or deleted from the Suggested_Observations_GIS table if a suggestion has been added, updated, or marked deleted in the Suggested_Observations table.
static void Usage()
          Command line usage syntax.
static void usage(org.apache.commons.cli.Options options)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

ID

public static final String ID
The Class identification with revision number.

See Also:
Constant Field Values

NL

public static final String NL
System specific end of line character


DEFAULT_CONFIGURATION_FILENAME

public static final String DEFAULT_CONFIGURATION_FILENAME
The default configuration filename

See Also:
Constant Field Values

DEFAULT_PATH_DELIMITER

public static char DEFAULT_PATH_DELIMITER
The default path delimiter for the configuration


CONFIG_PARAM_CATALOG

public static final String CONFIG_PARAM_CATALOG
The default parameter for the HiRISE Catalog

See Also:
Constant Field Values

CONFIG_PARAM_MAX_URL_ENTRIES

public static final String CONFIG_PARAM_MAX_URL_ENTRIES
The default parameter for the maximum url entries

See Also:
Constant Field Values

CONFIG_PARAM_RDR_PRODUCTS_TABLE

public static final String CONFIG_PARAM_RDR_PRODUCTS_TABLE
The default parameter for the RDR_Products table

See Also:
Constant Field Values

CONFIG_PARAM_RDR_PRODUCTS_GIS_TABLE

public static final String CONFIG_PARAM_RDR_PRODUCTS_GIS_TABLE
The default parameter for the RDR_Products_GIS table

See Also:
Constant Field Values

CONFIG_PARAM_SUGGESTED_OBSERVATIONS_TABLE

public static final String CONFIG_PARAM_SUGGESTED_OBSERVATIONS_TABLE
The default parameter for the Suggested Observations table

See Also:
Constant Field Values

CONFIG_PARAM_SUGGESTED_OBSERVATIONS_GIS_TABLE

public static final String CONFIG_PARAM_SUGGESTED_OBSERVATIONS_GIS_TABLE
The default parameter for the Suggested Observations GIS table

See Also:
Constant Field Values

CONFIG_PARAM_EXTERNAL_LOADERS

public static final String CONFIG_PARAM_EXTERNAL_LOADERS
The default parameter for the list of classes for loading external databases

See Also:
Constant Field Values

CONFIG_PARAM_EXTERNAL_SERVER

public static final String CONFIG_PARAM_EXTERNAL_SERVER
The default parameter for the server that an external database will be loaded to

See Also:
Constant Field Values

CONFIG_PARAM_EXTERNAL_CATALOG

public static final String CONFIG_PARAM_EXTERNAL_CATALOG
The default parameter for the catalog that an external database will be loaded to

See Also:
Constant Field Values

CONFIG_PARAM_EXTERNAL_TABLE

public static final String CONFIG_PARAM_EXTERNAL_TABLE
The default parameter for the table that an external database will be loaded to

See Also:
Constant Field Values

CONFIG_PARAM_EXTERNAL_ARGUMENTS

public static final String CONFIG_PARAM_EXTERNAL_ARGUMENTS
The default parameter for the list of string arguments that the constructor for the class of an external database loader will use.

See Also:
Constant Field Values

CONFIG_GROUP

public static final String CONFIG_GROUP
The configuration group for this program

See Also:
Constant Field Values

DEFAULT_RDR_PRODUCTS_GIS_TABLE

public static final String DEFAULT_RDR_PRODUCTS_GIS_TABLE
The table name to store the geospatial information for red, color, ctx, and moc images

See Also:
Constant Field Values

DEFAULT_RDR_PRODUCTS_TABLE

public static final String DEFAULT_RDR_PRODUCTS_TABLE
The RDR_Products table name

See Also:
Constant Field Values

DEFAULT_SUGGESTED_OBSERVATIONS_GIS_TABLE

public static final String DEFAULT_SUGGESTED_OBSERVATIONS_GIS_TABLE
The Suggested_Observation_GIS table name

See Also:
Constant Field Values

DEFAULT_SUGGESTED_OBSERVATIONS_TABLE

public static final String DEFAULT_SUGGESTED_OBSERVATIONS_TABLE
The Suggested_Observation table name

See Also:
Constant Field Values

MODE_NONE

public static final short MODE_NONE
The flag for doing nothing

See Also:
Constant Field Values

MODE_SUGGESTIONS

public static final short MODE_SUGGESTIONS
The flag for running the backfill for suggestions

See Also:
Constant Field Values

MODE_PRODUCTS

public static final short MODE_PRODUCTS
The flag for running the backfill for RDR products and CTX

See Also:
Constant Field Values

MODE_EXTERNAL

public static final short MODE_EXTERNAL
The flag for running the backfill for MOC

See Also:
Constant Field Values

DEFAULT_MAX_URL_ENTRIES

public static final int DEFAULT_MAX_URL_ENTRIES
The default maximum number of entries to read from a URL

See Also:
Constant Field Values

EXIT_SUCCESS

public static final int EXIT_SUCCESS
Application exit status for success.

See Also:
Constant Field Values

EXIT_COMMAND_LINE_SYNTAX_ERROR

public static final int EXIT_COMMAND_LINE_SYNTAX_ERROR
Application exit status for a command line syntax error

See Also:
Constant Field Values

EXIT_CONFIGURATION_ERROR

public static final int EXIT_CONFIGURATION_ERROR
Application exit status for a configuration error

See Also:
Constant Field Values

EXIT_DATABASE_ERROR

public static final int EXIT_DATABASE_ERROR
Application exit status for a database error

See Also:
Constant Field Values

EXIT_DATABASE_PROBLEM

public static final int EXIT_DATABASE_PROBLEM
Application exit status for a database problem. This means a database transaction ended with an unexpected result.

See Also:
Constant Field Values

EXIT_NO_MODE

public static final int EXIT_NO_MODE
Application exit status for when no mode is specified.

See Also:
Constant Field Values

EXIT_URL_ERROR

public static final int EXIT_URL_ERROR
Application exit status for a failed url

See Also:
Constant Field Values

EXIT_IO_ERROR

public static final int EXIT_IO_ERROR
Application exit status for a failed IO operation

See Also:
Constant Field Values
Constructor Detail

Geospatial_BackFill

public Geospatial_BackFill()
                    throws Database_Exception
Throws:
Database_Exception
Method Detail

updateSuggestedObservations

public void updateSuggestedObservations()
                                 throws Exception
This method ultimately results in GIS objects being inserted, updated, or deleted from the Suggested_Observations_GIS table if a suggestion has been added, updated, or marked deleted in the Suggested_Observations table.

In addition, for new or updated suggestions, the center coordinates and the minimum and maximum longitude and latitude of the target area is recomputed and the Suggested_Observations table is updated with these values.

Throws:
Exception

Compare_Vectors

public static boolean Compare_Vectors(Vector<String> v,
                                      Vector<String> u)
This method compares two Vectors of Strings with compareToIgnoreCase

This method will only return true if the vectors are the same size and each String within the Vectors matches its cooresponding entry. null values will match with other null values only.

The Vectors should follow the format where the first entry is a String, the second entry is a String, and the third entry is the WKT representation of a geospatial Point (ie: "Point( )"), and the fourth entry is the WKT representation of a Polygon (ie: "Polygon(( , , ... ))").

Parameters:
v - One vector to compare.
u - One vector to compare.
Returns:
true if the vectors match false otherwise.

main

public static void main(String[] arguments)
Run the Geospatial_BackFill application.

Exit status values:

EXIT_SUCCESS
All operations succeeded.
EXIT_COMMAND_LINE_SYNTAX_ERROR
Operation failed while processing command line arguments.
EXIT_CONFIGURATION_ERROR
Operation failed while acting on the configuration file.
EXIT_DATABASE_ERROR
Operation failed while acting on the database.
EXIT_DATABASE_PROBLEM
Operation resulted in an unexpected result while acting on the database.
EXIT_NO_MODE
Neither Products or Suggestions mode were specified.
EXIT_URL_ERROR
Operation on a URL resulted in an error.
EXIT_IO_ERROR
An IO operation failed.

Parameters:
arguments - The command line arguments.

Usage

public static void Usage()
Command line usage syntax.
Usage:
    Geospatial_BackFill
        [<Options>]
            [-Mode_Suggestions]
            [-Mode_Products]
            [-Mode_External]
            [-MAximum_external_entries]
            [-COnfiguration] <filename>
            [-CAtalog] <catalog name>
            [-SErver] <server name>
            [-Table_Rdr_products] <table name>
            [-Table_Rdr_Gis_products] <table name>
            [-Table_Suggested_observations] <table name>
            [-Table_Suggested_Observations_Gis] <table name>
            [-Loader]<loader name> [<loader argument> ...]
            [-Verbose]
            [-Help]

Mode_Suggestions:

If this option is specified then the program will copy geospatial information for suggestions in the HiRISE Catalog. The center and footprint will be copied from information in the specified Suggested Observations table into the specified GIS desination table. These tables can be specified either on the command line or in the Configuration file. The defaults for these are: DEFAULT_SUGGESTED_OBSERVATIONS_TABLE and DEFAULT_SUGGESTED_OBSERVATIONS_GIS_TABLE.

At least one mode needs to be specified. Any combination of modes may be specified.

-SUggestions will also specify this option.

Mode_Products:

If this option is specified then the program will copy geospatial information for RDR Products in the Catalog. The center and footprint will be copied from information in the specified RDR Products table into the specified GIS desination table. These tables can be specified either on the command line or in the Configuration file. The defaults for these are: DEFAULT_RDR_PRODUCTS_TABLE and DEFAULT_RDR_PRODUCTS_GIS_TABLE.

At least one mode needs to be specified. Any combination of modes may be specified.

-Products will also specify this option.

Mode_External:

If this option is specified then the program will copy information for any external databases listed in the configuration file under the main group's external loaders group. The full class name of the loader should be specified for each external loader to use. Each external loader should have its own group outside of the main group. The name of each of these groups should be the same as the full class name in the external loaders group. Each of these groups must specify a server, catalog, and table for the destination information. Additionally a group containing String arguments for the loader's constructor may be added as well as a group containing a list of fields that will be required by the database, but these are optional.

At least one mode needs to be specified. Any combination of modes may be specified.

-External will also specify this option.

Maximum_External_Entries

This option specifies the maximum external entries that should be read in from an external database at a time. Specifying this overrides the value in the configuration file and the default value. If this is only specified in the configuration file then that overrides the default value. If this is specified no where then the default value will be used.

Configuration:

If the name of the Configuration file is not specified the default will be used. If the configuration file is not in the current working directory, it will be looked for in the user's home directory. The configuration file must contain the necessary information needed to identify and connect with the database server (as required by the Database constructor and its Connect method). These are typically the server "Host" name and the databaes "Type", "User" and "Password" access values.

Only one configuration file may be used.

Below is an example Configuration file with all possible entries included. However, be mindful that there may be more entries in the External_Loaders group that would require more groups for those entries. In the HiRISE.HiCat.Utilities.CTX_Database_Loader group the Arguments group are the arguments as Strings which will be passed to the constructor of the loader, the required fields group includes the fields which must be included in the database for the loaded database to validate correctly, the Excluded Fields group contains fields that are in the external database but not in the local database, the Altered_Fields group has fields that have a different name in the local database than the one in the external database (These are of the format " = " minus the quotes).
Entries that can be specified on the command line but have no default value are shown in blue.
Entries that can be specified on the command line and have a default value are shown in red.
Entries that are optional but can only be specified in the configuration file are shown in green.
Entries that are required only if their group is specified are shown in bold.
Groups below the Geospatial_BackFill group are dependent on the list in the External_Loaders group in the Geospatial_BackFill group.

   Server = pirldb
   Group = pirldb
      Type = MySQL
      Host = pirldb
      User = whitsitt
      Password = "#######"
   End_Group

   Group = Geospatial_BackFill
      Catalog = HiRISE_Sean
      Max_External_Entries = 50000
      RDR_Products_Table = "RDR_Products"
      RDR_Products_GIS_Table = "RDR_Products_GIS"
      Suggested_Observations_Table = "Suggested_Observations"
      Suggested_Observations_GIS_Table = "Suggested_Observations_GIS"
      Group = External_Loaders
         HiRISE.HiCat.Utilities.CTX_Database_Loader
      End_Group
   End_Group

   Group = HiRISE.HiCat.Utilities.CTX_Database_Loader
      Server = pirldb
      Catalog = HiRISE_Sean
      Table = CTX
      Group = Arguments
         "http://pds-imaging.jpl.nasa.gov/data/mro/mars_reconnaissance_orbiter/ctx/mrox_0730/index/cumindex.tab"
      End_Group
      Group = Required_Fields
         Product_ID
         Center
         Footprint
      End_Group
      Group = Excluded_Fields
         ORIGINAL_PRODUCT_ID
      End_Group
      Group = Altered_Fields
         LINES = IMAGE_LINES
      End_Group
   End_Group

Catalog:

A Catalog must be specified when using either the Mode_Suggestions or Mode_Products options. This catalog contains the tables that store the information for RDR Products and Suggestioned Observations that will be used to generate and store geospatial data.

This option may be specified in the configuration file with the default group as the parameter Catalog.

Using the command line option for the Catalog will override the value stored in the configuration file. Only one Catalog may be used.

Server:

The Configuration file may contain connection information for more than one database. The information for each database is organized by Server name, which may be specified. If no server name is provided, the Database will attempt to use the default (first) Server specified in the Configuration.

Only one server may be used. Using the command line option for the catalog will override the value stored in the configuration file.

Table_RDR_Products:

The table where the RDR Products are stored in the database. Information from here will be used to create the GIS information for RDR Products.

This option may be specified in the configuration file with the default group as the parameter RDR_Products_Table.

Using the command line option will override the value stored in the configuration file. Only one RDR_Products_Table may be used.

Table_RDR_Products_GIS:

The table where the RDR Products GIS information will be stored.

This option may be specified in the configuration file with the default group as the parameter RDR_Products_GIS_Table.

Using the command line option will override the value stored in the configuration file. Only one RDR_Products_GIS_Table name may be used.

Table_Suggested_Observations:

The table where the Suggested Observations are stored in the database. Information from here will be used to create the GIS information for Suggested Observations.

This option may be specified in the configuration file with the default group as the parameter Suggested_Observations_Table.

Using the command line option will override the value stored in the configuration file. Only one Suggested_Observations_Table may be used.

Table_Suggested_Observations_GIS:

The table where the Suggested_Observations GIS information will be stored.

This option may be specified in the configuration file with the default group as the parameter Suggested_Observations_GIS_Table.

Using the command line option will override the value stored in the configuration file. Only one Suggested_Observations_GIS_Table name may be used.

Loader:

The loaders for the database may be specified on the command line. Any arguments for the loaders should be specified after the loader name in the order that they would be placed into the constructor of that loader. The name of the loader should be the full name of the class, including all package names. Inputing a loader on the command line overrides the arguments in the configuration file, but the Server, Catalog, and Table for the loader must be in the configuration file. Also, any Excluded Fields, Altered Fields, or Required Fields for the database must also be in the configuration file (though these groups are optional).

Multiple loaders may be specified, but a new -Loader must be placed before each new loader on the command line. Loaders specified in the configuration file but not on the command line will also be used and the arguments for those loaders in the configuration file will be used.

Verbose:

Prints verbose information to the command line.

This prints the programs ID and all output from sub-programs.

Help:

The command line syntax usage is listed and the program exits. N.B.: If the -Help option occurs anywhere on the command line no other command line argument will be used. If the command line is empty the usage will be listed.

N.B.The usage is printed to stderr. This method always results in a System.exit with a status of EXIT_COMMAND_LINE_SYNTAX_ERROR.


usage

public static void usage(org.apache.commons.cli.Options options)

HiRISE

Copyright (C) Arizona Board of Regents on behalf of the Planetary Image Research Laboratory, Lunar and Planetary Laboratory at the University of Arizona