Documentation on: qfits.htx

The following programs are available, based on the qfits package (from ESO) (http://www.eso.org/sci/software/eclipse/qfits/):

dfits
display FITS file header information

dtfits
display FITS table

fitsmd5
Compute/update the DATAMD5 keyword/value

fitsort
ort FITS header information from a list of files

flipx
Flip x axis in a FITS image

frameq
Classify frames from a directory

hierarch28
header conversion from ESO to standard FITS

iofits
FITS data format conversion (pixel depth)

qextract
Extract an wriions from FITS files

replacekey
replace keywords in a FITS header


1  Program dfits

dfits - display FITS file header information  

SYNOPSIS

dfits [-x xtnum] <list>
dfits [-x xtnum] -  

DESCRIPTION

dfits displays FITS header informations on stdout. Header information can be found in the main header only (default), in extensions, or in both. See the -x option below. dfits accepts multi-file input. 'dfits -' expects single file data coming from stdin. dfits also accepts now 'HDR' files. These are identified from the fact that they contain a line feed in one of the 80 first characters.  

OPTIONS

-x xtnum
Specifies the extension to print out. Extensions are numbered starting from 1. If this option is not specified, only the main header is printed out. If this option specifies an extension that does not exist, nothing is printed out.
Specify 0 as extension number to get a print of the main header plus all extension headers.

Examples :
dfits *.fits
dfits *.fits | grep NAXIS3
gzip -d < star.fits.gz | dfits - | more
dfits -x 0 *.fits
dfits -x 3 *.fits  

SEE ALSO

fitsort can be combined with dfits output to sort out keyword values of a group of FITS files.  

FILES

Files shall all comply with FITS format


2  Program dtfits

dtfits - display FITS table  

SYNOPSIS

dtfits <table>  

DESCRIPTION

dtfits dumps the contents of a FITS table in an ASCII format, either into a user-specified file or on stdout. The output is formatted on a fixed number of columns to make it readable by human beings. Additional informations are printed out before the table values are dumped, these informations can be skipped by using the -d option.

Last, if you want to dump the table into an easily parsable format (for a piece of software), you might want to use the -s option which specifies a character to use as separator. All data fields will be printed out separated by this character only. This allows to use string parsers to cut down the output lines into tokens by looking for this separator. Fields (lines) will still be delimited by the end-of-line character. This option produces ASCII tables which are easy to parse for a piece of software but mostly unreadable to human beings.

Notice that dtfits only accepts one single FITS table in input.  

OPTIONS

-d
Skip information output about the table and column names. Outputs only the table values. Beware that if the FITS file contains several extensions, they will all appear one after another, separated only by two blank lines. In that case, it would be preferrable to keep the complete output and parse out the returned stream to differentiate which data come from where.
-s <char>
Use the character <char> as separator in output. This option is useful if you want to produce a table that should be parsed by a piece of software (see above description). The separator can only be a single non-null character. To avoid special characters being interpreted by the shell, it is recommended to provide this character always between simple or double quotes. Example:
dtfits -s '&' table.tfits

If you want to use a special character as separator, such as a tab, use ^V to insert your character, such as:
dtfits -s '^V<TAB>' table.tfits

which means: you type CTRL-V and then the tab key.  

SEE ALSO

dfits  

FILES

FITS tables are stored into extensions. If there are several tables in a file, they will all be displayed one after another in the same output stream.


3  Program fitsmd5

fitsmd5 - Compute/update the DATAMD5 keyword/value  

SYNOPSIS

fitsmd5 [-u] [-s] [-a] <FITS files...>  

DESCRIPTION

fitsmd5 computes the MD5 signature of all data sections in a FITS file, and prints out the results on stdout. This command can optionally update the main FITS header in modifying the value of the DATAMD5 key.

This command is useful to give a unique ID to a FITS file. The algorithm simply browses through all data sections in the input file and passes the data blocks to an MD5 hash function. The final result is a 128-bit signature that can be used to uniquely identify the file.

This approach is meant to provide a tool to tag FITS files with unique IDs, it is not meant to be used as a checksum for file integrity (the CKSUM key is the solution for that), although it could be used in that spirit. The main point is that only data sections are taken into account, leaving the possibility of changing the headers without affecting the data signature.

MD5 hashing is cryptographically strong, which means the probability of having two different FITS files getting the same ID is almost zero. It should be good enough to assign a unique ID to several tens of thousands of frames. Since there is still a tiny but non-zero possibility that two different files will get an identical key, this approach is not recommended to tag very large numbers of files (typically: millions of them). If you do have a large database of FITS files, using a timestamp is usually a better approach.

The MD5 signature is a good solution to tag a list of FITS files which might have originated from various sources on which the database maintainer has no control. Typically, calibration databases holding calibration frames for a given instrument, receive data from different actors who might not be in sync with unique file naming conventions. This command makes sure it is always possible to assign a unique ID to each frame.

Notice that if the input FITS file has no data section, the returned MD5 key will be non-zero (it is exactly d41d8cd98f00b204e9800998ecf8427e). This signature also offers the interesting property that if two files have exactly the same pixels (bit-wise comparisons) they will get the same ID, this is useful e.g. for regression tests.

If you want to produce files containing the DATAMD5 key in their main headers, you should use the qfits library, which always inserts this key. If you are working with other FITS-processing software, you should allocate an empty DATAMD5 placeholder and apply this command with the -u option to update the value.

Notice that this command can also compute the MD5 sum of a complete file, not just its data sections (see -a option). In this mode, the command is completely identical to the GNU md5sum command, which is used to compute checksums on files. Input files in that case need not be FITS, though they still need to be regular files.  

OPTIONS

-u
Try to update the DATAMD5 keyword in the main header if present.
-s
Silent mode: run without printing any message.
-a
Compute the MD5 sum on all bits in the file. In this mode, the command behaves like the GNU md5sum command, to be used e.g. as a checksum. This option excludes all others.
 

FILES

Input files to fitsmd5 shall comply with the FITS format, except when used with -a option.


4  Program fitsort

fitsort - sort FITS header information from a list of files  

SYNOPSIS

dfits <FITS files...> | fitsort <FITS keywords...>  

DESCRIPTION

fitsort extract keyword values from a set of FITS headers and outputs it in an ASCII table format, which is compatible with most data processing software packages. It shall only be used in combination with the dfits utility.

The ASCII output is shown in columns. Columns are aligned with blank characters and also separated by tabulations. Blank alignment allows human readers to visualize the output in a pretty format, tabulations are there for spreadsheet compatibility. If you want to load out fitsort output into any spreadsheet, specify that fields shall be separated by tabulations and entries separated by linefeeds.

Examples :
dfits *.fits | fitsort BITPIX NAXIS NAXIS1 NAXIS2

The output would look like:
FILE BITPIX NAXIS NAXIS1 NAXIS2
file0001.fits 16 2 128 128
file0002.fits 32 2 512 512
...

ESO specific features in the FITS header are also supported. To get values for 'HIERARCH ESO' keywords, just give the complete names within double quotes. e.g.

dfits *.fits | fitsort "HIERARCH ESO INS LENS"

Another way of giving HIERARCH ESO keywords is to use the short FITS notation, the above example can be given as:

dfits *.fits | fitsort INS.LENS

Example: to retrieve the DPR keywords from an ESO FITS header, you would use:

dfits *.fits | fitsort To be completed... DPR.CATG DPR.TYPE DPR.TECH

This second way of requesting HIERARCH ESO keywords is not only shorter to type, it also avoids typing quotes or double-quotes on the command-line, making it easier to script with fitsort.

Notice that the keywords you give on the command-line are case-insensitive. The above line is equivalent to:

dfits *.fits | fitsort dpr.catg dpr.type dpr.tech  

OPTIONS

-d
Do not print out the first output line. This option is useful to get only the query results, without the top line (giving all column names). This makes it easy to script fitsort from programs like awk or perl.
 

FILES

Input files to dfits shall all comply with FITS format. fitsort also supports HIERARCH ESO FITS format.  

SEE ALSO

dfits (1)


5  Program flipx

flipx - Flip x axis in a FITS image  

SYNOPSIS

flipx <input images>  

DESCRIPTION

This is an example use of the qfits library. This program expects a list of FITS file names in input. For each input file, it will flip the image contained in the main data section in the X direction, i.e. pixel (i,j) is swapped with pixel (lx-i, j).

The pixel loading mechanism is independent from endian-ness of the local host or FITS pixel type. This program offers a good overview of how to use qfits for pixel-level operations.  

OPTIONS

None.  

FILES

Input files shall all comply with FITS format.  

BUGS

Notice that this program does not support cubes or image extensions, but could be easily extended to support that case.


6  Program frameq

frameq - Classify frames from a directory  

SYNOPSIS

frameq <directory_name>  

DESCRIPTION

frameq is aimed at classifying lists of frames a bit like the DO and RBS do. The aim is to put frames produced by the same template execution together.

 

ALGORITHM

The classification is using the TPl.NEXP and TPL.EXPNO to identify the sequences of frames. The TPL.ID keyword is also used.  

OPTIONS

None  

FILES

Files shall all comply with FITS format and contain keyword like TPL keys.


7  Program hierarch28

hierarch28 - header conversion from ESO to standard FITS  

SYNOPSIS

hierarch28 [options] <FITS> [table]  

DESCRIPTION

hierarch28 will convert keyword names in a FITS header to new names, using a user-provided ASCII conversion table. It is especially aimed at removing no-standard FITS features, such as the HIERARCH ESO keyword names.

hierarch28 can also perform a translation to the IRAF convention on the following four keywords: 'RA', 'DEC', 'UT' and 'LST'. IRAF requires these keywords to contain the string representation of their values, e.g.
RA = ' 09:45:14.594'
DEC = '-33:47:09.420'
UT = ' 01:17:21.950'
LST = ' 08:19:59.688'

The ESO standard (see http://archive.eso.org/dicb) defines these keywords as floating point values with the units degrees for RA/DEC and elapsed seconds since midnight for UT/LST.

In order to have this translation performed, add
RA = RA
DEC = DEC
UT = UT
LST = LST
to the conversion table.  

OPTIONS

-g
This option is used to generate default translation tables.

 

FILES

hierarch28 expects a conversion table in input. Default name for this table is table.conv in the current working directory. Indicate another name for this file as last argument on the command-line. This ASCII file contains a list of keywords to replace, in the following format:





OLDKEYWORD1 = NEWKEYWORD1
OLDKEYWORD2 = NEWKEYWORD2
etc.

Input keywords are character strings, they may contain blanks. Example:

HIERARCH ESO DET DIT = DETDIT

One important restriction is that the new keyword name may not be longer than the initial one. The program will complain that it cannot achieve search and replace if this is the case.  

IMPORTANT

This programs achieves maximal speed to operate because it modifies the input file directly. Be aware that using hierarch28 on a file will modify its contents in an irreversible way!


8  Program iofits

iofits - FITS data format conversion (pixel depth)  

SYNOPSIS

iofits [options] <in_fits> <out_fits> <depth>  

DESCRIPTION

iofits converts FITS files of any pixel depth to another pixel depth (bits per pixel). Depth is given as in the FITS norm :

8 bits per pixel, unsigned.
16 bits per pixel, signed
32 bits per pixel, signed
-32 IEEE floating point
-64 IEEE double precision floating point
 

OPTIONS

-v
Verbose mode.
-M megs
Memory handling. To avoid saturate memory for huge cubes, this parameter specifies the maximum amount of memory to use before swapping data to disk. The amount is given in Megabytes. (1Meg = 1,048,576 Bytes) The process time is then considerably slowed down. Default value is 50 megs.
 

FILES

Input/output files shall all comply with FITS format.

The original FITS header of the input FITS file is conserved along, except for the following keywords: NAXIS, NAXISn, BITPIX, BSCALE, BZERO, which are related to the newly created file.

HISTORY keywords are appended to the FITS header to indicate the eclipse process modifications, together with the command line which was used to generate the file.  

SEE ALSO

average, filt, deadpix  

BUGS

In general, decreasing the resolution in number of bits per pixel almost always means loss of precision. No rescaling is done on the data, use this command with care!


9  Program qextract

qextract - Extract an wriions from FITS files  

SYNOPSIS

qextract <in> <extension>  

DESCRIPTION

qextract reads a FITS file with many extensions, and writes the desired extension in a new FITS file. qextract supports FITS images, ASCII and BINARY tables.

 

OPTIONS

none
 

SEE ALSO

dtfits  

FILES

Files shall all comply with FITS format. The produced file is named "ext#.xxx" where # is the number of the requested extension and xxx is fits if the extension is a FITS image and TFITS if the extension is a TFITS table.


10  Program replacekey

replacekey - replace keywords in a FITS header  

SYNOPSIS

replacekey [options] <in>  

DESCRIPTION

replacekey is aimed at replacing keywords in a FITS file's header by new keywords provided by the user on the command-line.

 

ALGORITHM

The user can provide a new_keyword, an old_keyword, a value and a comment. first searchs for old_keyword if provided. If it finds it, it replaces the associated card by the new one (new_keyword=value/comment). If old_keyword is either not found or not provided, associated card by the new one (new_keyword=value/comment). new_keyword has to be provided.

Examples: You want to change the value of VALUE from 32 to 16 in myfile.fits: % replacekey -k "VALUE" -v "16" myfile.fits

If you want to replace the card with the keyword OLD by "NEW = 3.14 / This is a usefull key" in myfile.fits: % replacekey -k "NEW" -v "3.14" -c "This is a usefull key" -K "OLD" myfile.fits

If your do not care about the HISTORY cards you have in the header, and want to replace the first one by VALUE=32, you would do:
% replacekey -k "VALUE" -v "32" -K "HISTORY" myfile.fits  

OPTIONS

-k or --key key
To provide the new keyword.
-K or --old key
To provide the old keyword.
-v or --val val
To provide the new value.
-c or --com com
To provide the new comment.
 

SEE ALSO

hierarch28 to convert HIERARCH ESO keywords to regular 8-char keywords.  

FILES

Files shall all comply with FITS format  

BUGS

Modifications are so far only possible in the main header, not in the extensions.