COPIPE Operation Instructions
Brian Pohl 11/6/97
Updated 08/12/02
The COPIPE reduction stage computes the camera rotator angles for all GONG network sites. CAMOFFSET uses these angles to align the image orientation. The COPIPE module itself is a script that calls several other modules. The script automates the reduction process, but a few things need to be set properly first. First the input data needs to be available. The user must then set the software parameters correctly and finally enter in drift scan information if necessary. This document covers these points in detail and conclude with instructions on operation itself.
COPIPE is very flexible, designed to run on any machine and data set. This document discusses the routine operation on hohokam, but these are by no means the required operating instructions. The reduction processes mentioned in this document should be considered examples.
1. Setting up the input data.
COPIPE uses two types of data to compute the final camera angles (the mrcoyYYMMDD files used by CAMOFFSET.) First and foremost are the input images. The second are the accumulated history files (mrhiyYYMMDD) produced by previous COPIPE sessions. These files contain the camera rotator equations for that run and are used as the initial guess for the next session. Finally, COPIPE interpolates and smoothes these files over time to produce the final output file.
1.1 Input Images
COPIPE can use either velocity, modulation or intensity images as long as all the input data is of the same type. Every tenth image taken at the 0 minute over the day (ie. bbvzi961101t2140, bbvzi961101t2150etc...) is needed as input. The latest version of READTAPE contains a parameter for extracting only data necessary for COPIPE. COPIPE will run on a complete set of images by selecting only every tenth, but staging the unused images wastes disk space. The images from all sites should be placed in the same directory. In routine operation, the images are transferred online by VMICAL operators to the directory /ana1/OFFSET on anasazi.
1.2 The hiy files
COPIPE generates the hiy files that contain the polynomial coefficients for each site and are used as an initial guess for the next data set. In routine operation, COPIPE is run over a seven day long data set using the hiy file for the first day as the initial guess and generating a new hiy file for the center date. For example, the input range is 961101 to 961105, mrhiy961101 is the initial guess and mrhiy961103 is the produced file. All hiy files are stored and written to a single directory set by a parameter in COPIPE. Once 15 days worth of hiy files are accumulated, COPIPE automatically calls a module called CAMINTERP to smooth and interpolate the coefficients and produce the final mrcoyYYMMDD for the center date. For this reason, the coy files produced from an input range trail the center date by about seven to eight days. In other words, when mrhiy961118 is created, mrcoy961110 is produced. Therefore, in addition to the initial guess file, the previous fifteen day's worth of hiy files are required. For routine operation, all hiy files produced are stored in the same directory and never deleted.
2. Setting up the software parameters
The following parameters are used to run COPIPE on hohokam.
dn> lpar copipe
input = "in.dat" File containing list of path names to images
start_date = "970608" Starting date YYMMDD?
end_date = "970614" Ending date YYMMDD?
(tsample = 10) Time sample spacing (minutes)?
(tbl_path = "/ana1/CAMOFFSET") Path to offset tables
(dev = "stdplot") Graphics output device
(data_type = "vzi") Image data type (vzi, izi, etc...)
(image_type = "fits.gz") Image type (imh, fits, etc...)
(pm_tbl = "/ana1/CAMOFFSET/pm_data") Full name & path of PM data table
(use_tc = no) Use TC data if available?
input: This is an ascii text file containing the path where the
input images are stored. The file "in.dat" listed above
simply contains /ana1/OFFSET. When staging the data on
your computer, simply create this file and have it list
all the directories that contain input data. Multiple
directories can be listed, but it is recommended that everything
be placed in the same directory.
start_date and end_date: Simply the input date range. Standard
processing uses a 7 calendar day range (three days before and
after the center date.) The next date range will begin
on the previous center date. For example, after this input
range ran, the next would be 970611 970617.
tsample: This indicates the time interval of the input images. Standard
processing uses a 10 minute interval, but that can be changed
for special tasks.
tbl_path: This parameter indicates the directory that contains the
hiy files and consequently where the coy files are
written. It is very important to set this to the right
directory.
dev: Output device for the qa plots, default is stdgraph.
data_type: Velocity images are normally used for input, but it also
supports modulation and intensity. All input must be the
same type.
image_type: Implemented for the fits kernel. All the input on anasazi
is in fits format and uses the fits kernel.
pm_tbl: This lists the full path to the table containing PM driftscan
results. This is discussed in detail in the next section.
use_tc: Added to support the Tucson test site. The most common problems in running COPIPE are associated with the parameters tbl_path and pm_tbl. If set improperly, the software terminates with a mysterious error. COPIPE passes all parameters to the various modules it calls. The user never needs to adjust the parameters in these modules.
3. Handling PM drift scans.
Drift scans are taken during routine preventive maintenance, about one site per month. Since these produce an "observed" table of polynomial coefficients, they are assumed to be correct and COPIPE sets the other sites to match it. Once the results from the drift scan have been obtained (these are posted on the gong_ops mailing list and in the service logs on mojave,) they need to be entered into a file. The one on hohokam is called /hoh0/CAMOFFSET/pm_data.
dn> tail /hoh0/CAMOFFSET/pm_data 1996/03/06 BB 4 -0.02727 -0.1527 0.03963 0.07591 1996/04/21 UD 2 -0.02860 0.046457 0.00000 0.00000 1996/05/28 CT 3 0.07990 0.3164 0.09956 0.00000 1996/06/28 ML 0 ***** Site coordinates updated ***** 1996/08/06 TD 0 ***** No Drift Scan Obtained ***** 1996/08/26 LE 3 0.02126 -0.00928 -0.05652 0.00000 1996/10/11 UD 3 0.01075 -0.02767 0.01745 0.00000 1996/11/16 ML 0 ***** No Drift Scan Obtained ***** 1997/02/05 BB 0 ***** No Drift Scan Obtained ***** 1997/03/12 CT 4 -0.00370 0.43320 0.01410 -0.13554 1997/04/23 LE 4 0.00070 0.08090 -0.01250 -0.10080 1997/05/30 UD 4 -0.04970 -0.05345 -0.03701 0.02036
The first column is the date the drift scan was performed, the second is the site. The third column contains the number of coefficients (the order of the polynomial.) Note that it isn't always 4 and an entry of 0 will ignore the rest of the line. You can enter in comments when a driftscan wasn't taken, there is no specific format for commenting on a 0 line. The remaining columns are the coefficients (0 term, 1st term ... 3rd term.) The table does need to be in chronological order, new entries must go on the last line.
4. Operation Instructions
Now that everything is set up, operation is fairly simple. Here's an example of a typical COPIPE operation session.
dn> {
>>> COPIPE in.dat 970416 970422
>>> COPIPE in.dat 970419 970422
>>> COPIPE in.dat 970424 970501
>>> COPIPE in.dat 970427 970503
>>> COPIPE in.dat 970430 970506
}Note the seven day date range and that the range skips a day. The reason for this is that the first run computes the initial guess for the second run and so forth. Also note the operator skipped forward 4 days on the third line. This is because a PM driftscan was taken and entered into the pm_table file on 970423. None of the date ranges should overlap with a pm driftscan date; simply skip it. Because the interpolation and smoothing subprocess requires fifteen continuous days, the effect of this gap halts the production of coy files until the range is reestablished. The operator will see several COPIPE runs pass without producing any coy files, and then it will suddenly produce about 10 when it has caught up.