Rows (e.g., X-ray events) in column-based data (i.e., FITS Binary
tables and raw event files) can be filtered or selected based on a
flexible set of user-specified criteria that allows column values to
be compared with numeric values, header parameters, functions, or
other column values. Both image and tables can be filtered using
ASCII spatial region filters.
To filter tables and images, the filter specification is added to the
FITS filename using bracket notation:
foo.fits[pha==1&&pi==2&&circle(512,512,10)]
The filter specification comes after the extension and image sections and
may optionally be enclosed in its own set of brackets. Thus:
foo.fits[EVENTS,400:599,400:599,pha==1&&pi==2&&circle(512,512,10)]
foo.fits[EVENTS,400:599,400:599][pha==1&&pi==2&&circle(512,512,10)]
foo.fits[BKGD][400:599,400:599,circle(512,512,10)]
foo.fits[BKGD][400:599,400:599][circle(512,512,10)]
define filters on an image section of size 200x200 defined on
the FITS EVENTS (binary table) and BKGD (image) extensions. Order
is important here: extension name is followed by section, which
is followed by filters.
To implement both row and spatial filtering, funtools utilizes a
technique in which the filter specification is converted into a small
C program, which is compiled and linked automatically so that table
rows or image sections can be fed to this program and filter results
returned to the calling program. The power of the technique lies in
these considerations:
- The generated filter program is very small, containing approximately 200
lines of code, so that it compiles and links in about second or less on
most modern machines.
- The filter specification itself becomes a hard-wired part of this
program, so that filter checking is performed as part of compiled
code, not in the usual interpreted mode. This use of a compiled filter
results in a speed-up factor of 4-5 over previous techniques,
even after the program compilation time is added.
- All C syntax and C operators become valid parts of the filter syntax,
making available a much wider range of filter possibilities than
previously.
- It is easy to replace the standard filter program with
a user-specified filter program for more sophisticated applications.
Three variations on this scheme of dynamic filters are supported:
- If gcc was used to build the filter library and if gcc is
available to build the filter program, then a shared object is built
dynamically and loaded into the existing program. Dynamic loading
avoids the overhead of communicating with a separate filter program
and can speed up filter processing by a (surprisingly small) 20
percent.
- If shared objects are not used, then a separate process is built.
In this case, if the pre-compiled region object can be located,
region support can simply be linked into the separate process. (The
pre-compiled region object is built along with the filter library and
is installed in the same directory as the filter library.)
- Alternatively, if the pre-compiled region object cannot be
located, a separate process is built by compiling the region support
as part of the separate process. Because the region filtering code is
more than 1000 lines, this can add a second or two to the time
required to build the filter program.
These choices are explored automatically by the filter init routines.
Table Filtering
can be performed on columns in a FITS binary table file by comparing
the value of a column to other columns, header values, constant values
or function values. Table columns and image pixels also can be
filtered with geometric Spatial Region Filters.
Index to the Funtools Help Pages
Last updated: March 30, 2001