The p3d logo. a general data-reduction tool for fiber-fed integral-field spectrographs
About p3d

Readme

Download

References

Documentation & Tutorials

Links

Screenshots

Authors & Thanks

Sourceforge project page

Category: main tool

Summary

PROp3d
This routine is a wrapper for the p3d main data-reduction graphical user interface (GUI).
PROp3d_cdmask
This routine creates a dispersion-mask image from a set of arc images that is then used to perform wavelength calibration of IFS data.
PROp3d_cexposure
Combines several spectrum-image files with extracted spectra covering the same region, either just taking the average or accounting for saturated regions in the longer exposures.
PROp3d_cflatf
Creates an image with extracted spectra for the subsequent flat fielding (normalization) of extracted science spectra.
PROp3d_cmbias
Creates a master-bias image from a set of raw bias images.
PROp3d_cobjex
Creates an image with extracted, and optionally wavelength calibrated and flat-fielded, science-object spectra from a set of input images.
PROp3d_cr
Cleans one or several raw spectrum images of fiber-fed Integral-Field Spectroscopy instruments of cosmic-ray hits.
PROp3d_ctrace
Creates a trace-mask image from a set of continuum images.
PROp3d_cvimos_combine
Combines four extracted VIMOS spectrum images of the four separate detectors to create one RSS file for the entire field.
PROp3d_darc
Corrects densely arranged data for differential atmospheric refraction by shifting them spatially in each wavelength bin.
PROp3d_fluxcal
This tool is used to flux calibrate extracted spectrum images using an already prepared flux-sensitivity function.
PROp3d_ifsfit
Fits any number of emission and absorption lines in integral-field spectroscopy data using a simple plain-text configuration file.
PROp3d_ifsfit_mosaic
Produces publication-ready two-dimensional plots in the form of (encapsulated) PostScript, PDF, or bitmap files from data fitted with p3d_ifsfit.
PROp3d_rss2cube
Converts an extracted image file from the RSS format to a data cube.
PROp3d_sv
The p3d interactive data cube and RSS-data spectrum viewer.

Copyright

p3d: a general data-reduction tool for fiber-fed IFSs

Copyright 2009-2017 Leibniz Institute for Astrophysics Potsdam (AIP)

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see <http://www.gnu.org/licenses>.

Additional permission under GNU GPL version 3 section 7

If you modify this Program, or any covered work, by linking or combining it with IDL (or a modified version of that library), containing parts covered by the terms of the IDL license, the licensors of this Program grant you additional permission to convey the resulting work.

Routine Documentation

p3d.pro, line 1822, last changed at 2017-04-13 by christersandin (revision 4547)

p3d, path=, /cwd, detector=, /flames, /vimos, /nvimos, /bvimos, /larr2k, /ppak2k, /larr4k, /ppak4k, /imacs_ccd1, /imacs_ccd2, /spiral, /fsami, /virus, /mitchell, /wvirus, /fvirus, /gmoss, /gmosn, /mpfs3k, /oldintegral, /integral, /mvimos, /mnvimos, /era2, /erasmus, /mrs1, /wifu, /hetlrs2, userparfile=, logfile=, /setuponly, loglevel=, colortable=, /invert, bottom=, cindex=, title=, parfile=, display_name=, font=, cmdline=, verbose=, /quiet, /debug, /help, _extra=

This routine is a wrapper for the p3d main data-reduction graphical user interface (GUI).

The following documentation describes how to use p3d, including all the components of the GUI.

Note: p3d can, in principle, be used with any fiber-fed IFS. However, the actual use is only possible if p3d is first configured for an instrument. The parameter list below contains a complete list of currrently configured instruments.

Note: p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

How to start and use p3d

The different ways to use p3d and its tools

p3d consists of a set of separate tools for reduction of data of fiber-fed IFSs. The main GUI of p3d provides a straightforward mean to use these tools without the need to configure each component individually, by setting all required options. The actual GUI code is contained in the routine p3d_tool_gui.pro; this routine is a wrapper that calls p3d_tool_gui.pro with the appropriate instrument-specific parameter file and title string, along with the additional options.

If there is a need to do so, many of the tools can be executed separately, outside the GUI, from either the IDL command prompt or from a (shell) terminal (optionally using the IDL Virtual Machine, see below). If you want to do this, you will have to write IDL scripts, shell scripts, or Python scripts, which provide each tool with both the necessary and the optional input parameters. The following tools can be setup in this way:

  • The master-bias image creation tool, "p3d_cmbias.pro".

  • The trace-mask image creation tool, "p3d_ctrace.pro".

  • The dispersion-mask image creation tool, "p3d_cdmask.pro".

  • The fiber-flat image creation tool, "p3d_cflatf.pro".

  • The science-object spectrum extraction tool, "p3d_cobjex.pro".

  • The sensitivity-function creation tool "p3d_fluxsens.pro". This tool is available in the spectrum viewer as well as from the command line or from the shell.

  • The flux-calibration tool "p3d_fluxcal.pro".

  • The VIMOS extracted images quadrant combination tool, "p3d_cvimos_combine.pro".

  • The spectrum viewer, "p3d_sv.pro".

  • The single image cosmic-ray hit cleaning tool, "p3d_cr.pro"; in the GUI, this tool is only available with the science-object extraction tool "p3d_cobjex.pro". From the command-line it is also available with "p3d_ctrace.pro", "p3d_cdmask.pro", and "p3d_cflatf.pro".

  • The differential atmospheric refraction correction tool, "p3d_darc.pro". This tool is only available from the command line or from the shell.

  • The RSS to cube conversion tool "p3d_rss2cube.pro"; this tool is only available from the command line or from the shell.

  • The exposures combination tool "p3d_cexposure.pro"; this tool is only available from the command line or from the shell.

  • The line-fitting tool "p3d_ifsfit" and the associated plotting tool "p3d_ifsfit_mosaic.pro"; these tools are only available from the command line or from the shell.

Using p3d with the IDL Virtual Machine or with an IDL license

As is described in the README file, p3d can be used either with an IDL license from the command prompt, or from the shell or Python with (or without) the IDL Virtual Machine (IDL VM):

  • No IDL license is required when using the IDL VM, although it is still necessary to either use the p3d-specific IDL distribution or install IDL on the computer where p3d is to be used (and this before starting p3d).

  • The functionality of all routines of p3d is the same in either approach. However, if you need to modify the code because of, for example, a bug or an additional feature you want to include, it is necessary to recompile p3d – and this step requires an IDL license. If you only need to change an instrument- or method-specific parameter, you should not edit the code, but instead edit your user-parameter file (see below).

  • A bug might – depending on its form and location in the code – terminate p3d when it is run using the IDL VM from a shell or Python. Therefore, if you use p3d with the IDL VM from the shell, make sure that the DEBUG option in the Options menu is inactive (this is always the case when the provided scripts are used).

  • There are no differences in the execution speed of either approach. Note, however, that the graphics of some of the GUIs have been found to run up to two times faster using IDL version >= 7.0, instead of the older version 6.2.

Detailed descriptions on how to setup p3d to be used in either approach are provided in the README file.

The graphical user interface of p3d

The different graphical tools of p3d look very similar for all instruments, although several GUIs are scaled to make full use of your screen. If you do not tell p3d which instrument to set it up for when you launch it, you will be presented with an instrument-selection GUI before the main control panel is shown. Both these GUIs use a fixed size.

The instrument-selection GUI

This GUI allows you to start p3d configured for the instrument that you select here. Only the fully configured instruments can be selected, by clicking on the respective row under the 'Quick instrument setup' tab. Only the fully configured instruments are available; the remaining ones can also be selected if p3d is started with the debug option. The options in the remaining two tabs ('Program and data paths' and 'Configuration files') allow you to select a data directory and choose the instrument-parameter file manually.

If you do not want to select the instrument each time you start p3d using the GUI, you can use the appropriate keyword on the command line. Check the list of keywords below or see the last column in the instrument-selection GUI

Note: Although you launch p3d using your own instrument-specific configuration files for an unsupported instruments, some parts of p3d will likely not work as intended, such as the wavelength calibration. The reason to this is that many instruments use their own unique approach of specifying important information – some (often minute) changes are required in p3d to make these instruments work.

Note: Creating figures and images with color can be a hassle, which outcome depends on if you are using indexed (256 – "device, decomposed=0") or decomposed (16777216 – "device, decomposed=1") colors. p3d handles each case transparently for you – it is unnecessary to change the decomposed state using the IDL DEVICE command. If you do not have a clue what this is all about, do not worry!

The main control panel, and related GUIs

From the top, the different regions in the main control panel GUI are: the menu bar; a region with six rows, one for each data-reduction task; an object spectra preview section with a big window of the currently selected object on the left-hand side, and one smaller window for each of nine buffers; a status row with a log-file indicator on the left-hand side, and a status field on the right-hand side.

Here is a more detailed description of the different fields and options:

  • The menu bar

    The menu bar looks the same on all platforms. Here follows a description of all the available (and unavailable) entries. The string in the last column shows the key-accelerator combination for those tasks where it is available. As an example of a key accelerator 'Ctrl+B' means that while you press down the 'Ctrl' key you also press down the 'B' key (thereafter both keys are released).

    The 'File' sub menu field

    • Create a new master-bias image (Ctrl+B)

    • Create a new trace-mask image (Ctrl+T)

    • Create a new dispersion-mask image (Ctrl+M)

    • Create a new fiber-flat image (Ctrl+F)

    • Extract new object data (Ctrl+O)

    • Flux calibrate extracted data (Ctrl+G)

    • Open an existing master-bias image (Ctrl+Shift+B)

    • Open an existing trace-mask image (Ctrl+Shift+T)

    • Open an existing dispersion-mask image (Ctrl+Shift+M)

    • Open an existing fiber-flat image (Ctrl+Shift+F)

    • Open an existing extracted object (Ctrl+Shift+O)

    • Open an existing flux-calibrated object image (Ctrl+Shift+G)

      The function of these options are described in the "data-reduction tasks" section below.

    • Restart p3d

      Restarts p3d in order to, for example, use another instrument configuration.

      Note: All loaded information disappears when p3d is restarted.

    • Exit (Ctrl+Q)

      Exits p3d.

    The 'View' sub menu field

    • Evaluate telescope focus series

      n/a

    • View a data log file

      Allows the selection of a log file from your observations that is opened in a separate GUI (using the IDL routine XDISPLAYFILE).

    • View the data reduction log file

      Opens a GUI showing the contents of the logfile of your reductions (using the IDL routine XDISPLAYFILE). By default, the file is shown as it is when this option is first selected. In order to update the GUI, use the next option.

      Note: You have to open a logfile before you can use this option [Options->Open a log file].

      Note: Huge files are not opened correctly using XDISPLAYFILE, so if you have such files then consider looking at them outside p3d, using, for example, a shell tool.

    • Set the log view timer [off]

      If you are viewing a logfile using the above option, this option allows to set the GUI-update timer to 5, 20, or 60 s.

    • Select the previous detector (Ctrl+R)

      Select a lower-index detector with instrument configurations that use several detectors.

    • Select the next detector (Ctrl+E)

      Select a higher-index detector with instrument configurations that use several detectors.

    • Select the color table

      Allows you to select the color table used to color all images (axes, labels, and some additional indicators are always drawn with a separate set of default colors).

      Note: The color table does not affect data in any way, but is only used to represent data in a revealing (and appealing) way – so choose whichever color table that makes you satisfied.

      Note: The cubehelix, and two Califa color tables, as well as the (default) Sauron color table are only available from the menu, while the color tables, which are available with the IDL routine LOADCT, can also be selected and modified interactively with XLOADCT.

    The 'Options' sub menu field

    • Using the raw data directory

    • Using a separate output directory

      p3d, by default, writes all output data to the raw-data directory. If you instead want all output data to be written to a separate directory, you can select that directory in the file selector GUI that pops up by selecting this option.

      Note: A faster option than this, is to start p3d from the directory where you want all the output data using the keyword /CWD.

    • Number of used threads: X

      The C routines of p3d normally use all available threads. Choose any of the numbers in the popup menu to use as many threads instead.

    • View the instrument parameter file (Ctrl+I)

      This option pops up a window showing the contents of the (currently used) instrument-specific parameter file.

    • Select a user-parameter file (Ctrl+U)

      Opens a file selector that allows you to pick a user parameter file that can be used to configure all data-reduction tasks. If you do not select an existing file, p3d will copy the master user-parameter file (from ${p3d_path}/data/master_userparfile.dat) to the specified directory, and then use that file.

    • View the user-parameter file (Ctrl+Y)

      This option pops up a window showing the contents of the selected user-parameter file.

      Note: This option is only available after a user-parameter file has been selected.

    • Edit the user-parameter file (Ctrl+A)

      This option pops up a window that allows you to both view and edit the selected user-parameter file.

      Note: This option is only available after a user-parameter file has been selected.

    • D.mask rebin data [dbin=1]

      This option is used when the dispersion-mask creation GUI (dGUI) is opened. The values that can be selected are:

      1.

      Data are not rebinned, this is the default.

      2.

      Data are rebinned once.

      3.

      Data are rebinned until they fit the screen.

      The meaning behind these options is the following. Your screen has a number of pixels on the horizontal axis that may be smaller than the number of pixels on the dispersion axis in your data. With the default setting (dbin=1) your data are kept as they are, and a horizontal scrollbar is added in the dGUI, if the monitor has fewer pixels than the data.

      If the second option is used (dbin=2), your data are rebinned/rescaled to use half as many pixels on the x-axis. This is normally enough to fit all data on the screen (although most likely not with data of the GMOS IFS). With the third option, your data are rescaled by a factor two until all data fit on your screen.

      A good reason to not bin data is that pixel positions become inaccurate when data are rescaled. The recommended option is therefore to not bin data – keep dbin=1!

    • Spectrum viewer – window size [xxxx, yyyy]

      The spectrum-viewer window is normally opened using the maximum number of pixels available on the screen; the number of pixels corresponds to the integer doublet [xxxx, yyyy]. A set of additional pre-defined window sizes is available in the popup menu. Choose any of the available sizes to use that one instead.

    • Spectrum viewer – view sky fibers [no]

      Some of the instruments, for example PPAK, FLAMES, and GMOS, make use of separate sky fibers. The location of these sky fibers are normally not seen in the spatial map of the spectrum viewer. Use this option to change the behavior and also show the sky fibers in the spatial map.

      Note: This could be a useful option to see if your sky fibers are in fact on the sky, or if they point at some bright object, in which case they should not be used to subtract the sky component in your analysis.

      Note: In most cases, it is unlikely that you want to view the sky fibers, because all elements in the spatial map will become smaller when the sky fibers are also shown in the same window.

    • Append to a log file (Ctrl+J)

      Select this option to append log output to an existing log file. When you select this option, a file selector GUI will ask you for an existing (plain-text) file.

      Note: This option is only available if no log file has been opened.

    • Open a log file (Ctrl+L)

      Select this option to open a new log file; a file-selector GUI will query you for a filename.

      Note: The log file will be deleted if it already exists.

      Note: This option is only available if no log file has been opened.

    • Close the log file (Ctrl+K)

      Select this option to close the log file.

      Note: IDL writes to the file when the I/O buffer is full, so unless you close the file you may not see the final operations. (This file is closed automatically when you exit p3d.)

      Note: This option is only available if a log file has been opened.

    • Log level [1]

      The default log level is 1, which writes the most important information to the log file. The second option (2) writes more information to the log file. The third option (3) writes all information to the log file (this should only be a valuable option when debugging a routine).

    • Verbose level [0]

      This option allows you to choose the amount of information written to the standard output (STDOUT). By default, this setting is 'no information'. The other options are the same as for the logging level.

      Note: This option is typically used when debugging some part of p3d, it is more wise to save the information to a log file.

    • Debug [0 {=no}]

      The normal behavior of p3d is to leave any routine when an error is encountered, and return to the main execution level, which allows you to redo your task or do something different. If you instead want p3d to stop when it finds an error, you should use this option.

      Note: Please, also use this option when reporting a bug you have found. It makes it much easier to track down a bug if you copy and paste the output on the console where p3d stopped in your bug report.

      Note: If you use the IDL VM, you should keep this option switched off, otherwise p3d will stop promptly if, and when, a bug is encountered.

    The 'Help' sub menu field

    • About

      Opens a window with some information about p3d.

    • Copying

      Opens a window showing the contents of the GNU license.

    Note: IDL, and therefore p3d, is based on the Motif widget library (http://www.opengroup.org/motif/), which requires key accelerators (on UNIX platforms) to be placed in menu entries. Most options are therefore available in the menus listed above. Although, key accelerators are defined for most options, your window manager (such as, for example, KDE or GNOME) may have reserved any particular configuration, and if this is the case the accelerator will not work.

  • The data-reduction tasks

    There are six rows in the data-reduction section of the GUI.

    The label in the first column of each row indicates the task; the six available tasks are: Master bias, Trace mask, Disp. mask, Fiber flat, Object, and Flux cal. The second column shows the filename of the currently selected data set. The third column indicates the currently selected buffer, the numbers are 0 – 8, for buffers 1 – 9. By clicking on the icons in the remaining columns, the following operations are performed:

    • Create a new dataset (empty sheet of paper icon)

      Click this icon to open a file selector where you can choose one, or three or more, raw-data images.

      Note: After the new dataset has been created, p3d will present a GUI where you should select one of the nine buffers where the output data are stored.

      Note: The images must be of the correct type for the desired function. p3d does check if the image size is correct, but does not know how to differ between, for example, an arc image and a bias image.

    • Open an existing data set

      Click this icon to open a file selector where you can choose one already reduced image.

      Note: As soon as the data set has been loaded, p3d will present a GUI where you should select one of the nine buffers where data are stored.

    • Switch between different, and already loaded, data sets

      Click this icon to select the active data set buffer.

      Note: For object data, it is also possible to select the active data set by clicking in the respective panel, of the GUI object-preview section.

      Note: This option is only available if two or more data sets were created or loaded. If only one data set was loaded, that is the active data set.

    • View the loaded data in the p3d image viewer (landscape icon)

      When you click this icon, the p3d image viewer will present you with a simple GUI showing the raw-data image as well as the reduced image in two separate tabs.

      Note: For this option to work properly, the necessary header keywords must be present in the reduced image. p3d adds those keywords automatically for each image type.

      Note: When viewing the trace-mask image, the traces are drawn on top of the spectra in the raw-data image (so the trace 'image' is not actually shown).

    • View the loaded data in the p3d spectrum viewer (diagram icon)

      When you click this icon, the p3d spectrum viewer will present you with a GUI showing the spectrum image as well as a spatial map, and a spectrum that is a combination of any number of selected spatial elements. This option is only available for reduced images that contain spectra, i.e., dispersion-mask images, fiber-flat images, and extracted object spectra images.

      When viewing the dispersion mask, what is actually shown is the extracted and wavelength-calibrated spectra (where the wavelength calibration is done on the fly using the dispersion-mask image).

      Note: If a dispersion-mask image is selected, the flat-field image will be wavelength calibrated before it is shown (this is otherwise normally only done when the flat-field image is applied to the object spectrum image).

    • View the loaded data with the image viewer ds9 (sun icon)

      When you click on this icon, ds9 is launched with the loaded data, as well as any error data, cosmic-ray mask, and saturated-pixels mask image in separate frames. This option is only available if ds9 is found in the system path when p3d is launched. ds9 can be downloaded from its project web site: http://hea-www.harvard.edu/RD/ds9/site/Home.html

    p3d stores the loaded data in nine separate buffers. It is always the selected buffer – indicated with the filename and the buffer number in columns 2 and 3 in the image above – that is used in the data-reduction steps. The buffer-selection GUI looks like the image on the right-hand side. Simply click in any of the nine fields, and then click 'OK' in the lower left corner. If you change your mind, and do not want to save the data set, click 'Cancel' in the lower-right corner instead.

    The first steps of the data-reduction procedure are meant to be performed in the following order:

    1.

    Create a master-bias image, unless the prescan and/or overscan regions are used instead.

    2.

    Create a trace-mask image, and optionally calculate line profiles for use in the optimal extraction.

    3.

    Create a dispersion-mask image; this step is optional.

    It is not possible to create a dispersion-mask image before a trace-mask image has been created or loaded into a buffer. It is, however, possible to load an already created dispersion-mask image at any time.

    4.

    Create a fiber-flat image; this step is optional.

    It is not possible to create a fiber-flat image before a trace-mask image has been created or loaded into a buffer. It is, however, possible to load an already created fiber-flat image at any time. When you select this option, a setup interface is shown that allows you to configure the flat-field extraction options (see below for a description of this interface).

    5.

    Extract object data.

    It is not possible to extract spectra in an object-spectrum image before a trace-mask image has been created or loaded into a buffer. When you select this option, a setup interface is shown that allows you to configure the spectrum extraction (see below for a description of this interface).

    Note: The wavelength-calibration step is optional, to use this functionality you should create or load a dispersion-mask image before starting this task.

    Note: Normalization of data using a fiber-flat image is optional. To use this functionality, you should create or load a fiber-flat image before starting this task.

    6.

    Flux calibrate data; this step is optional.

    To flux calibrate data, it is necessary to first reduce data of observations of a standard star. The extracted standard-star data are opened with the spectrum viewer to first subtract the sky, and then co-add all spectra that contain flux of the standard star.

    A single standard-star spectrum can be used to create a sensitivity function using the tool p3d_fluxsens or the sensitivity-function tab in the spectrum viewer. The sensitivity function is then used with the flux-calibration tool p3d_fluxcal to calibrate the extracted spectra.

    • The dispersion-mask configuration GUI

      The dispersion-mask configuration GUI is shown when selecting 'Create a dispersion-mask image'. Here is a description of the different components of this interface, the items are described from the top:

      Raw arc image

      Select the arc raw-data image that will be used to create a dispersion mask. If you select several files, they are at first combined into one image.

      Do not use an input dispersion-mask image

      The normal procedure is to create a dispersion-mask image from scratch, which can be quite tedious work if there are many images to create. Unselect this option and choose an already created dispersion-mask image, to shorten the procedure.

      Note: It is important that the selected dispersion-mask image comes from data that were taken with the exact same instrument configuration as the currently selected raw-data image. Otherwise, it might be very difficult to create a new dispersion mask.

      Use the default arc-line list file

      By default, p3d attempts to select a default line-list file, which is stored in the ${p3d_path}/data/tables/linelists/ directory. Unselect this option, to choose an already prepared line-list file yourself.

    • The fiber-flat configuration GUI

      The fiber-flat field GUI is shown when selecting 'Create a fiber-flat image'. Here is a description of the different components of this interface, the items are described from the top:

      Raw flat-field image

      Select the flat field raw-data image that will be used to normalize data. If you select several files, they are at first combined into one image.

      Use the FF raw data for calc. a transmission array

      Unselect this option to choose a file that contains a separately calculated fiber-to-fiber transmission array. Such files are created when a flat-field image is extracted, and are either stored in the TRANSMISSION extension, or in a separate file, which typically has the suffix '_transmission'.

      Use the FF raw data for finding spectrum traces

      By default, the fiber-flat data are themselves used to trace spectra, before the object data are normalized in the object-extraction task. By unselecting this option, you can choose an, already created, trace-mask image instead.

      Note: With twilight flat-field images, you would normally leave this option selected. You might want to unselect this option and choose an already created trace-mask image if you intend to use, for example, a continuum-lamp image to flat field your data; where the same continuum-lamp image was used to calculate the trace-mask image.

      Do not use a dispersion mask with the FF data

      By default, the extracted fiber-flat spectra are wavelength-calibrated using the dispersion-mask image that is selected in the p3d main GUI. If this option is unselected, you can choose a separate, and already created, dispersion-mask image.

      Note: This option is useful if you want to use a separate dispersion-mask image with, for example, your twilight-flat-field images.

    • The object spectrum extraction configuration GUI

      The object-extraction setup GUI is shown when selecting 'Extract object data'. Here is a description of the different components of this interface, the items are described from the top:

      Extract file

      Select the object data that will be extracted. If several files are selected, they are at first combined into one image.

      Subtract the master-bias image

      Select this option to subtract the master-bias image from the raw data. This option must not be selected if prescan and overscan regions are to be used instead when estimating the bias level.

      Note: This option is obligatory if p3d is to calculate an error estimate for the extracted spectra, and no prescan and overscan regions are used to estimate the bias level. In the latter case, the bias error is set to zero.

      Remove cosmic ray hits (when one file is selected)

      Select this option to clean images from cosmic-ray hits; this option has the same effect as setting the keyword CRCLEAN. If you want to use a different configuration in the cosmic ray cleaning than the default setup, it is necessary to use the options in the user-parameter file.

      Wavelength calibrate using the dispersion-mask image

      If you want to wavelength-calibrate your data, select this option.

      Note: The wavelength calibration is applied after the spectra are extracted, and before (the optional) normalization with the fiber-flat image.

      Prompt for wavelength-related properties

      p3d calculates and uses default values for the wavelength array of the data. You can apply your custom values either by entering them in the user-parameter file, or by entering the values in a (self-explanatory) GUI that is shown when selecting this option.

      Align the disp.mask with telluric lines

      Because of flexure in the instrument the data may be shifted in the dispersion direction in the time elapsed between an arc image and an object image. By selecting this option, p3d calculates a median offset of telluric lines that are fitted in the data, and this offset is then applied to the dispersion-mask image. There are three options here:

      • "none|user par.file", no alignment is performed, or the telluric line list specified in the user parameter is used.

      • "Lo-res", uses the low-resolution line list "telluric_lines_lores.dat" (part of the p3d distribution).

      • "Hi-res", uses the high-resolution line list "telluric_lines_hires.dat" (part of the p3d distribution).

      Normalize the resulting spectra using the flat field

      Select this option to normalize the extracted spectra with the selected fiber-flat image.

      Export resulting spectra (also) in the E3D-format

      Select this option to save the outcome in the E3d-format.

      Note: p3d always saves the outcome in the row-stacked-spectrum (RSS) format, which requires a fiber position-table file to reconstruct a spatial map. The E3d-file includes this table in the file header.

      Note: p3d does not read E3D-files.

    • The flux-calibration configuration GUI

      The flux-calibration GUI is shown when selecting 'Flux calibrate extracted data'. Here is a description of the different components of this interface; the items are described from the top:

      Sensitivity-function file

      Select a single sensitivity-function file. This file must be saved in the FITS format, as described in the tool p3d_sensfunc.

      Extinction-data file

      Select a single extinction-data file. See the routine p3d_misc_fluxcal_atmospheric_extinction.pro for details on the accepted format of this file.

      Attempt a qualified quess of the wavelength unit

      Use this option to manually set the wavelength unit used with extinction data provided in a text file; this option corresponds to the keyword WAVEUNIT in p3d_misc_fluxcal_atmospheric_extinction.pro.

      Export resulting spectra (also) in the E3D-format

      Select this option to save the outcome in the E3d-format.

      Note: p3d always saves the outcome in the row-stacked-spectrum (RSS) format, which requires a fiber position-table file to reconstruct a spatial map. The E3d-file includes this table in the file header.

      Note: p3d does not read E3D-files.

  • The object spectrum preview section

    Once an object is extracted – or an already extracted object image is loaded – the panels in this section of the GUI show a spatial map of the data. In a spatial map, each element represents a spatial element on the integral-field unit, and the color represents the intensity at the chosen wavelength. The spatial-map preview is only a preview of what can be seen in more detail in the spectrum viewer, but it allows a quick check of the outcome without starting the spectrum viewer.

    The large panel on the left-hand side shows a spatial map of the active/currently selected object. Specifically, the spatial map is shown for a pre-defined wavelength (bin). In the row of widgets below the panel, the leftmost text field shows the currently selected wavelength bin (in pixels; the default is the center pixel); the label immediately to the right shows the corresponding wavelength of that bin (in Ångström). The following two text fields indicate the minimum and the maximum values used when calculating the color of each spatial element. The rightmost widget allows you to select the range of values used to determine the minimum and the maximum values (using a histogram). The default of 95% means that in a histogram of the intensity value of all spatial elements, of all wavelength bins, the minimum value is set above 2.5% of the lowest value bins, and the maximum value is set below 2.5% of the highest value bins. All values in the text fields can be changed by typing in a value manually, and thereafter pressing return. (The fields are also updated when the respective widget loses the focus.) The wavelength bin can also be changed easily by using the slider widget on the left-hand side of the panel.

    Each panel on the right-hand side shows a spatial map of one of the nine buffers. Black panels indicate buffers where no data have been loaded. An additional way to switch between data sets - in addition to using the selector widget on the object-extraction row in the data-reduction section of the GUI – is to click in the panel that should become the active data set.

  • The status field

    The status line of the p3d main control panel shows you what p3d is doing, or can do. The text field on the right-hand side shows a descriptive text of the function of each button and field widget – whenever the mouse pointer is kept above the respective widget. The log-file field on the right-hand side is set to 'Log: yes' ('Log: no') when a (no) log file has been opened. If the mouse pointer is kept above this region, the text field on the right-hand side will show the name of the currently opened log file.

The image viewer

The image viewer of p3d is available to allow a quick view and comparison between raw data and the location of spectrum traces (for all tasks, but the master bias). When you click on the image viewer icon, you are presented with a window that is adjusted to fill your screen; unless the image of your particular detector is smaller than your screen.

Two images are shown in two separate tabs. The raw data are shown in the leftmost tab, and the same raw data with overplotted spectrum traces in the rightmost tab. For large images, scroll bars are available to the right and below the image to allow you to look at different parts of the image.

The widgets below the image(s) allow you to control the presentation of the image. The four fields are:

  • Alpha

    Drag the scrollbar from 0.0 – 1.0 to change the transparency of the overlaid traces. The alpha threshold value can also be entered in the adjacent text field; the default value is 0.5. The transparency calculations are slow, so be careful with dragging the scrollbar with large images.

  • Value range

    The colors are calculated using a minimum and a maximum value, applying a pre-defined color table. Enter the minimum and the maximum values in the text widgets, or select a value in the drop list. A percentage of 97% (this is the default value) means that in a histogram of the intensity value of all pixels, the minimum value is set at 1.5% above the lowest value bins, and the maximum value is set at 1.5% below the highest value bins.

  • X, Y

    The values in the label widget show the current position of the mouse pointer when it is dragged across the image. The values are counted from the bottom left corner of the image.

  • statistics

    Image statistics are shown for a 3x3 pixel box centered on the current position of the mouse pointer. From the left-hand side, the five values are: the maximum, the minimum, the mean, the median, and the standard deviation.

Note: With multi-block instruments (such as the 4kx4k CCD of PMAS), the different blocks may be read out using a different gain value. If this is the case, and the gain values differ significantly, it could be tricky to set the minimum and the maximum values so that the relevant data are seen easily in all parts (i.e. blocks) of the image.

Note: In several cases, the pre-scan and/or over-scan regions are not removed in the raw image shown in the image viewer. If this is the case, the traces are also shown in these regions, where they, of course, are unimportant.

Note: The image viewer determines which raw data file and trace mask to use from custom header keywords. These keywords are added by p3d when the images are created, so if you use an image that was not created by p3d then the image viewer will (unlikely) find the required information.

Note: The transparency calculations performed when overlaying the traces are slow, and they also need huge amounts of memory. For data of the GMOS IFS you will need >= 4GB of RAM, or your swap-disk will go warm for a while (2GB RAM and 2GB swap is the absolute minimum requirement).

The spectrum viewer

See the file p3d_sv.pro for a full description of all parts and functionality of the spectrum viewer.

Configuring p3d

p3d is designed to be a general data-reduction tool that works with all fiber-fed integral-field spectrographs. All parts of p3d are configured for the separate instruments using instrument-specific configuration files. There are two kinds of configuration files: the instrument-specific configuration files that should not be modified, and a user-parameter file that should be modified to meet any requirements. All these files are, and must be, available in plain-text format. All provided files are typed with 90 columns to show maximum verbosity.

Hint!

Please, read the relevant configuration files, the p3d A&A paper, and the p3d SPIE paper carefully to understand how p3d is configured for your instrument, and to see which parameters you can modify.

Hint!

If you want to print out a parameter file to have a closer look, you could – on *NIX platforms – create an nicely appearing (postscript) printout with the a2ps command. Here are two examples on how to use this command from the p3d directory:

a2ps --columns=2 -o tmp.ps -l 90 data/instruments/pmas/larr4k.prm

or

a2ps -R --columns=1 -o tmp.ps -l 120 \
  data/instruments/pmas/larr4k.prm

You would then print the file tmp.ps (using e.g. 'lp -c tmp.ps').

Note: While all instrument-specific default options are setup automatically, the user-specific reduction parameters are always read from the user-parameter file. Hence, to modify your data reduction, you must provide such a file; either by opening a file from the options menu in the GUI, or by specifying the name of an existing file on the command line (using the input parameter USERPARFILE). The user-parameter file can be edited using the GUI.

The instrument parameter file

These files are all stored in the instrument-specific directories (under ${p3d_path}/data/instruments/). The main configuration file is, for each instrument, the one with the ending '.prm'. All parameters are collected in groups of the same function. The seven parameter groups are:

1.

General instrument parameters, and setup files

2.

Bias subtraction

3.

Cosmic-ray hit cleaning-related parameters

4.

Tracing-related parameters

5.

Dispersion mask creation-related parameters

6.

Flat field related parameters

7.

Object extraction-related parameters

The meaning of each parameter is described in the file, and some parameters are described in more detail in the p3d papers. Most importantly, all parameters indicated with an asterisk ('*') can, if necessary, be modified from the user-parameter file (see below).

The user-parameter file

When you need to change the behavior of the data-reduction procedure of p3d, the correct way is to set tool-specific keywords, or to edit the user-parameter file. If you use the GUI, p3d will copy a master user-parameter file when a non-existent file is selected. The master user-parameter file is copied to the data-reduction directory from ${p3d_path}/data/master_userparfile.dat. The file can be edited from outside p3d, or it can be edited using the 'Edit user-parameter file' option in the menu.

As is the case with the instrument-specific parameter file, this file is divided into separate categories of parameters. The fourteen different categories are:

1.

Instrument-specific parameters

This entry is empty by default, but if you need to modify any of the parameters in the instrument-specific file, add the parameter in this section.

2.

General parameters

This section contains parameters that do not fit in the other sections. For example, here you can modify the behavior of:

  • The image-combination method.

  • Decide whether all data should be written to the same file (using extensions) or to separate files.

  • Configure the use of multiple threads, C routines, and printing page setup.

  • Gain and readout noise values (these values are sometimes incorrectly specified in the data headers).

  • File suffixes, in case you are unsatisfied with the default names.

  • If the output should be compressed (using gzip – this also works on Windows platforms!)

  • Should single images be cleaned of cosmic rays, if yes, set the parameter crclean (or use the keyword /CRCLEAN).

3.

Cleaning images of cosmic-ray hits

Information on all parameters related to the cleaning of images of both the L.A. Cosmic and the PyCosmic algorithms are set here. Related filename suffixes can also be changed.

4.

Bias subtraction

All aspects of masterbias and prescan or overscan bias images are configured in this section.

5.

Tracing

Most of the tracing configuration parameters are available in the instrument-parameter file. A few additional parameters are available in this section.

6.

Wavelength calibration

Here you can modify the behavior of, for example:

  • If the dispersion axis on the CCD was flipped, due to a rotation of the grating, set the "dflip" parameter to yes and redo all related reductions.

  • The wavelength array. A useful option if you want to use the same wavelength array with all data.

  • Whether telluric lines in the data should be used to re-align the dispersion mask.

  • Provide the name of one or several custom arc-line files. A useful option if you don't want to create a new line list every time you start the dispersion-mask tool.

7.

Flat fielding

Configure all aspects of fiber flat-field images here.

8.

Spectrum extraction – general parameters

Here you can modify the behavior of how the spectrum extraction is performed. Some of the parameters are:

  • The spectrum extraction method, tophat (default) or optimal.

  • Which optimal extraction method to use.

  • Should also the spectrum (cross-dispersion) position be fitted when calculating line profiles?

  • Some data seem to require to fit a scattered-light component.

9.

Tophat / tramline / aperture extraction

Configuration options for the simplest extraction method.

10.

Optimal spectrum extraction: modified Horne method (MOX)

This section allows you to modify the behavior of the spectrum extraction when it is performed using the modified Horne method. For more details regarding the options, see the p3d A&A paper.

11.

Optimal spectrum extraction: multi-profile deconvolution method (MPD)

This section allows you to modify the behavior of the spectrum extraction when it is performed using the multi-profile deconvolution method of Sharp & Birchall. For more details regarding the options, see the p3d A&A paper.

12.

Combination of reduced (and flux-calibrated) VIMOS data

VIMOS data of all 1600 fibers are split on four detectors. The p3d tool p3d_cvimos_combine combines the four data sets, which are reduced separately. This is a delicate procedure, with many related configuration parameters. All parameters, but the file suffixes, can be set using keywords to the above mentioned tool, but they can also be set using the keywords in this section.

13.

Correction of the data for diff. atmospheric refraction

Data of instruments using square-shaped spatial elements are relatively easily corrected for effects of differential atmospheric refraction. All configuration parameters, but the temperature-lapse rate and the filename suffix, can be configured using keywords to the tool p3d_darc. All options can be configured with the parameters in this section.

14.

Flux calibration and creation of a sensitivity function

All configuration options that can be changed when creating a sensitivity function or when flux calibrating data can be set using keywords to the tools p3d_fluxsens and p3d_fluxcal; filename suffixes are again an exception. All options can be configured with the parameters in this section.

15.

Extracted images combination

Extracted spectrum images can be combined with the tool p3d_cexposure. All configuration options that can be changed when combining extracted images can be set using keywords to the above mentioned tool, the filename suffix is an exception. All options can be configured with the parameters in this section.

16.

Spectrum Viewer

The options here all concern the chosen abscissa, which can be set to use nanometers, micrometers, or wavenumber instead of the default Angstrom.

Note: The user-parameter file is read anew every time a data-reduction function is launched. This means that if you make a change in the file you only have to run the task where you made a parameter change anew – there is no need to restart p3d.

Note: The (NASA astro-lib) routine READCOL will issue a warning if a user-parameter file is selected where all entries are commented out (i.e. all rows begin with a semi-colon). The warning will not affect the data-reduction process.

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside IDL check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below.

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Keyword parameters:
path- This input string defines the default directory to look for data inside (used by all file dialogs). The default value is: !p3d_path
cwdIf this keyword is set then all output files are saved in the current directory, instead of in the raw data directory (this is the default).
detectorThis scalar integer selects the detector to use, if the instrument is configured in a multi-detector configuration. For example, VIMOS has four detectors, but the Mitchell Spectrograph [VIRUS-P] is also configured for its both fiber bundles using the same instrument-parameter file. In these, and similar, cases it is possible to specify the starting detector/IFU by providing the DETECTOR keyword. The first detector/IFU is 0, the second 1, etc.
flames,With these keywords it is possible to selects the nstrument setup (LARR2K is the default). An nstrument-selection GUI is shown if none of these eywords is set.
userparfileThis scalar string defines a file that is used for user-specific parameters. If this keyword is set here then the corresponding file must already exist (in the specified path). If the file does not yet exist then copy the master user-parameter file (${p3d_path}/data/master_userparfile.dat) to your output directory and provide the new filename, including the path, in this parameter.
logfileA scalar string that specifies the name of a logfile. If a file already exists, with the name LOGFILE, it is overwritten. Use this keyword to save the log output of the various routines of p3d. Use the VERBOSE keyword instead to only print information on the command line or in the terminal.
loglevelA scalar integer, which determines the amount of information recorded in the log file LOGFILE. LOGLEVEL can be set to 1-3, where a higher number means that more information is written to the log file.
setuponlyXSet this keyword to exit this tool after setting up the IDL !path to include all parts of p3d; this could be useful when one only wants to use one or some of the subroutines of p3d, but not the tools themselves.
colortableA scalar string or integer that specifies which color table should be loaded. If COLORTABLE is a string, the string must contain two parts separated by a comma, without any whitespace between. The first part before the comma specifies the color-table number in the file name specified after the comma; the file must be formatted as described in the documentation for MODIFYCT. As an example, COLORTABLE='25,brewer.tbl'. Components in the ColorBrewer color-table file "brewer.tbl" found in the resource directory can be specified explicitly by preceeding the string with "CB"; for example, COLORTABLE='CB25'. If COLORTABLE is an integer, the integer must be given in the range -4, -3, -2, -1, 0, ..., up to the maximum number of available tables in IDL, plus the available tables in the ColorBrewer file (resource/) "brewer.tbl". Select -4, -3, -2, or -1 to load the cubehelix, 2 * Califa, and the Sauron color tables, the other values use the respective color map as defined by LOADCT. These are the permitted values:

-4

Loads the cubehelix color table.

-3

Loads the Califa project intensity color table, as defined September 2012.

-2

Loads the Califa project velocity field color table, as defined September 2012.

-1

Loads the Sauron color table (default).

0-74 (IDL version >= 8.3)

Loads the corresponding color table with LOADCT.

0-40 (IDL version < 8.3)

Loads the corresponding color table with LOADCT.

'CB0'-'CB85'

Loads the corresponding ColorBrewer color table; the color tables are defined in the file "resource/brewer.tbl"

75-159 (IDL version >= 8.3)

Loads the ColorBrewer color table corresponding to this number after 75 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

41-125 (IDL version < 8.3)

Loads the ColorBrewer color table corresponding to this number after 41 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

'x,ctfile'

Loads the x:th entry ('x' must be an integer) in the color-table file 'ctfile.

The default value is: -1
invertSet this keyword to invert the loaded color table.
bottomA scalar integer that specifies the lower index of the color table used to scale the color map (the upper index is the maximum possible; see p3d_misc_colortable).
cindexAn array of integers specifying which color indices to use for reserved colors. It doesn't make sense to use more than 4 elements in the CINDEX array. The indices should also be set as low as possible in order to allow the remaining indices to be used by the (scaled) COLORTABLE (see p3d_misc_colortable).
titleThis scalar string is used as the title of the GUI. Only use this parameter if you do not want to use any of the pre-configured instrument setups.
parfileThis scalar string can be set to the name of a file that contains instrument-specific parameters for use in p3d. Only set this parameter if none of the pre-configured instrument setups work.
display_nameSet this keyword equal to a string that specifies the name of the X Windows display on which the base should be displayed; this keyword has no effect on Microsoft Windows platforms.
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used when the tool is launched by the p3d server or using the IDL VM.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when p3d starts up.
debugXNo error handler is setup if this keyword is set. The default is otherwise to setup an error handler (using the routine CATCH), which makes each subroutine exit quietly in case a bug is encountered. Use this keyword when debugging p3d.
helpXSet this keyword to show this routine documentation, and then exit.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) From the IDL command line

It is possible to start p3d without using any input parameter; all parameters can be set interactively. However, it may be more convenient to give the options already on the command line.

Recommended options are: the instrument keyword, the keyword specifying if the current directory is the output directory (/cwd), and the name of a(n output) log file:

userparfile = './user_p3d.dat' ;; This file must exist if
                               ;; it is specified here.
p3d, /cwd, /mitchell, detector=0, userparfile=userparfile, $
    logfile='dred.log', loglevel=2, /verbose

Here are some additional examples of how to start the p3d GUI from the command line:

p3d, /debug    ;; Start p3d with the DEBUG option, to allow all
               ;;   configured instruments to be selected.
p3d, /ppak4k   ;; Start p3d for the PMAS/PPAK/4kx4k CCD instrument.
p3d, /mitchell ;; Start p3d for the Mitchell Spectrograph [VIRUS-P]
               ;;   instrument, which is setup for the newer fiber
               ;;   bundle 2.
p3d, /larr2k, /debug, /cwd, logfile='test.log', loglevel=1
               ;; Start p3d for the PMAS/LARR/2kx4k CCD, turn on
               ;;   debugging, write all output to the current
               ;;   directory, write log messages to a file called
               ;;   'test.log', and set the logging level to 1.

ii) As a stand-alone tool from a terminal, using the IDL VM

As in the previous case, it is possible to start p3d without using any input parameter; all parameters can be set interactively. However, it may be more convenient to give the options already on the command line.

Recommended options are: the instrument keyword, the keyword specifying if the current directory is the output directory (/cwd), and the name of a(n output) logfile.

The recommended approach is to use the provided shell script p3d_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, without using the p3d queue system:
userparfile = ./user_p3d.dat # This file must exist if it
                             # is specified here.
p3d_vm.sh --noqueue /cwd /mitchell detector=0 \
    userparfile=$userparfile logfile=dred.log /verbose

# Using the Python tool, using the p3d queue system:
userparfile = ./user_p3d.dat # This file must exist if it
                             # is specified here.
p3d_dispatch.py p3d /cwd /bvimos \
    userparfile=$userparfile logfile=dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_vm.sh userparfile=$userparfile" and -NOT- "p3d_vm.sh us=$userparfile".

Note: Simplify your reduction work by starting p3d from the reduction output directory.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from Python. Here is an example:

# Launch p3d in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
userparfile = 'user_p3d.dat'
commandline = '/mitchell detector=0 userparfile=' + userparfile +
  ' logfile=dred.log loglevel=2 /verbose nthreads=8'
p3d_dispatch('p3d', commandline, noqueue=True)

Once you've become accustomed to using p3d on a regular basis, it will be easier to start the tool with some extra keywords, so that you do not have to configure everything using the menus each time. Here are a few examples of useful startup calls, which can be used when running p3d with an IDL license. Most of these examples also work when p3d is used from the terminal (although some of the input parameters are unavailable in this case, see the list of parameters above for more details):

  • The header of all routines of p3d can be accessed using the help keyword. The main routines to setup the main control panel of p3d are 'p3d.pro' and 'p3d_tool_gui.pro':

    p3d, /help
    p3d_tool_gui, /help

  • Start p3d for the PPAK IFU of PMAS, for the new 4kx4k CCD. Additionally, instruct p3d to save all the outcome in the current directory, write all logging output to the file 'dred.log', and set the logging level to '1':

    p3d, /ppak4k, /cwd, logfile='dred.log', loglevel=1

  • As the previous example, but use the Mitchell Spectrograph [VIRUS-P] instrument instead, and also use the already existing user-parameter file 'user_p3d.dat' in the current directory:

    p3d, /mitchell, /cwd, logfile='dred.log', loglevel=1, $
         userparfile='user_p3d.dat'

  • When debugging p3d it is useful to use the debug option, as well as increasing the amount of textual output to the command line. Since the instrument is not selected it has to be chosen using the instrument-selection GUI (see above):

    p3d, logfile='dred.log', loglevel=2, verbose=3, $
         userparfile='user_p3d.dat', /debug

  • If, in your opinion, using the right color table is important for a visually appealing data-reduction session, then you can set it using the COLORTABLE keyword. The following command uses the green/white linear color table (8) instead of the default Sauron color table (-1):

    p3d, colortable=8

    A separate color-table file can beloaded by specifying COLORTABLE as a string with an index of the color table to use in the file name provided after a comma, e.g.:

    p3d, colortable='25,brewer.tbl'

    The color-table file must be formatted according to the description in the documentation of the tool MODIFYCT. For more details, see p3d_misc_colortable.

    Here is a list of all available color tables in IDL. The first four entries are custom p3d color tables; IDL version < 8.3 only include the entries 0-40:

    -4 :             cubehelix         36 :               Volcano
    -3 :      Califa intensity         37 :                 Waves
    -2 : Califa velocity field         38 :             Rainbow18
    -1 :                Sauron         39 :       Rainbow + white
     0 :            B-W LINEAR         40 :       Rainbow + black
     1 :            BLUE/WHITE         41 :             CB-Accent >=8.3
     2 :       GRN-RED-BLU-WHT         42 :              CB-Dark2 >=8.3
     3 :       RED TEMPERATURE         43 :             CB-Paired >=8.3
     4 : BLUE/GREEN/RED/YELLOW         44 :            CB-Pastel1 >=8.3
     5 :          STD GAMMA-II         45 :            CB-Pastel2 >=8.3
     6 :                 PRISM         46 :               CB-Set1 >=8.3
     7 :            RED-PURPLE         47 :               CB-Set2 >=8.3
     8 :    GREEN/WHITE LINEAR         48 :               CB-Set3 >=8.3
     9 :   GRN/WHT EXPONENTIAL         49 :              CB-Blues >=8.3
    10 :            GREEN-PINK         50 :               CB-BuGn >=8.3
    11 :              BLUE-RED         51 :               CB-BuPu >=8.3
    12 :              16 LEVEL         52 :               CB-GnBu >=8.3
    13 :               RAINBOW         53 :             CB-Greens >=8.3
    14 :                 STEPS         54 :              CB-Greys >=8.3
    15 :         STERN SPECIAL         55 :            CB-Oranges >=8.3
    16 :                  Haze         56 :               CB-OrRd >=8.3
    17 :   Blue - Pastel - Red         57 :               CB-PuBu >=8.3
    18 :               Pastels         58 :             CB-PuBuGn >=8.3
    19 :   Hue Sat Lightness 1         59 :               CB-PuRd >=8.3
    20 :   Hue Sat Lightness 2         60 :            CB-Purples >=8.3
    21 :       Hue Sat Value 1         61 :               CB-RdPu >=8.3
    22 :       Hue Sat Value 2         62 :               CB-Reds >=8.3
    23 :  Purple-Red + Stripes         63 :               CB-YlGn >=8.3
    24 :                 Beach         64 :             CB-YlGnBu >=8.3
    25 :             Mac Style         65 :             CB-YlOrBr >=8.3
    26 :                 Eos A         66 :               CB-BrBG >=8.3
    27 :                 Eos B         67 :               CB-PiYG >=8.3
    28 :             Hardcandy         68 :               CB-PRGn >=8.3
    29 :                Nature         69 :               CB-PuOr >=8.3
    30 :                 Ocean         70 :               CB-RdBu >=8.3
    31 :            Peppermint         71 :               CB-RdGy >=8.3
    32 :                Plasma         72 :             CB-RdYlBu >=8.3
    33 :              Blue-Red         73 :             CB-RdYlGn >=8.3
    34 :               Rainbow         74 :           CB-Spectral >=8.3
    35 :            Blue Waves

    These are the ColorBrewer color tables available in the file 'brewer.tbl':

    CB00 :         YlGn (Sequential)   CB43 : Pastel1 (Qualitative) - 9
    CB01 :       YlGnBu (Sequential)   CB44 :    Set1 (Qualitative) - 3
    CB02 :         GnBu (Sequential)   CB45 :    Set1 (Qualitative) - 4
    CB03 :         BuGn (Sequential)   CB46 :    Set1 (Qualitative) - 5
    CB04 :       PuBuGn (Sequential)   CB47 :    Set1 (Qualitative) - 6
    CB05 :         PuBu (Sequential)   CB48 :    Set1 (Qualitative) - 7
    CB06 :         BuPu (Sequential)   CB49 :    Set1 (Qualitative) - 8
    CB07 :         RdPu (Sequential)   CB50 :    Set1 (Qualitative) - 9
    CB08 :         PuRd (Sequential)   CB51 : Pastel2 (Qualitative) - 3
    CB09 :         OrRd (Sequential)   CB52 : Pastel2 (Qualitative) - 4
    CB10 :       YlOrRd (Sequential)   CB53 : Pastel2 (Qualitative) - 5
    CB11 :       YlOrBr (Sequential)   CB54 : Pastel2 (Qualitative) - 6
    CB12 :      Purples (Sequential)   CB55 : Pastel2 (Qualitative) - 7
    CB13 :        Blues (Sequential)   CB56 : Pastel2 (Qualitative) - 8
    CB14 :       Greens (Sequential)   CB57 :    Set2 (Qualitative) - 3
    CB15 :      Oranges (Sequential)   CB58 :    Set2 (Qualitative) - 4
    CB16 :         Reds (Sequential)   CB59 :    Set2 (Qualitative) - 5
    CB17 :        Greys (Sequential)   CB60 :    Set2 (Qualitative) - 6
    CB18 :           PuOr(Diverging)   CB61 :    Set2 (Qualitative) - 7
    CB19 :           BrBG(Diverging)   CB62 :    Set2 (Qualitative) - 8
    CB20 :           PRGn(Diverging)   CB63 :   Dark2 (Qualitative) - 3
    CB21 :           PiYG(Diverging)   CB64 :   Dark2 (Qualitative) - 4
    CB22 :           RdBu(Diverging)   CB65 :   Dark2 (Qualitative) - 5
    CB23 :           RdGy(Diverging)   CB66 :   Dark2 (Qualitative) - 6
    CB24 :         RdYlBu(Diverging)   CB67 :   Dark2 (Qualitative) - 7
    CB25 :       Spectral(Diverging)   CB68 :   Dark2 (Qualitative) - 8
    CB26 :         RdYlGn(Diverging)   CB69 :  Paired (Qualitative) - 3
    CB27 :    Set3 (Qualitative) - 3   CB70 :  Paired (Qualitative) - 4
    CB28 :    Set3 (Qualitative) - 4   CB71 :  Paired (Qualitative) - 5
    CB29 :    Set3 (Qualitative) - 5   CB72 :  Paired (Qualitative) - 6
    CB30 :    Set3 (Qualitative) - 6   CB73 :  Paired (Qualitative) - 7
    CB31 :    Set3 (Qualitative) - 7   CB74 :  Paired (Qualitative) - 8
    CB32 :    Set3 (Qualitative) - 8   CB75 :  Paired (Qualitative) - 9
    CB33 :    Set3 (Qualitative) - 9   CB76 : Paired (Qualitative) - 10
    CB34 :   Set3 (Qualitative) - 10   CB77 : Paired (Qualitative) - 11
    CB35 :   Set3 (Qualitative) - 11   CB78 : Paired (Qualitative) - 12
    CB36 :   Set3 (Qualitative) - 12   CB79 :  Accent (Qualitative) - 3
    CB37 : Pastel1 (Qualitative) - 3   CB80 :  Accent (Qualitative) - 4
    CB38 : Pastel1 (Qualitative) - 4   CB81 :  Accent (Qualitative) - 5
    CB39 : Pastel1 (Qualitative) - 5   CB82 :  Accent (Qualitative) - 6
    CB40 : Pastel1 (Qualitative) - 6   CB83 :  Accent (Qualitative) - 7
    CB41 : Pastel1 (Qualitative) - 7   CB84 :  Accent (Qualitative) - 8
    CB42 : Pastel1 (Qualitative) - 8

p3d_cdmask.pro, line 2181, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_cdmask, filename, parfile, saved, out, masterbias=, tracemask=, dispmaskin=, bpmask=, crmask=, /biaspx, /biaspy, /biasox, /biasoy, biasconstant=, /savebiassub, userparfile=, arclinefile=, ofilename=, opath=, opfx=, detector=, dbin=, track=, /exmonitor, /compress, logfile=, loglevel=, colortable=, /invert, /cinv, /satmask, crnthreads=, nthreads=, /allinone, /crclean, /nocrmask, sigclip=, objlim=, ratlim=, crfwhm=, gausskernelsize=, sigfrac=, growradius=, maxiter=, /imagemethod, /imageclean, dispmedian=, /writeall, /showcrgui, /nocrc, font=, cmdline=, /gui, eventwid=, stawid=, topwid=, logunit=, verbose=, /noobcheck, /quiet, error=, /debug, cdebug=, /help, _extra=

This routine creates a dispersion-mask image from a set of arc images that is then used to perform wavelength calibration of IFS data.

The selected arc images are at first combined, if required, to create a combined (master) arc image. The instrument-specific parameters and the information in the data header are used to determine a default arc-line list file, and calculate a first-guess wavelength range. The spectra of the (combined) arc image are extracted. Thereafter, the interactive graphical user interface (GUI; see below, and the routine p3d_wavecal_dispmask_gui.pro) is opened. The interactive procedure thereafter consists of matching entries in the arc-line list with features in the extracted arc spectra. More accurate pixel positions are calculated for each entry in the line list and spectrum, before a polynomial of pre-defined order is fitted to each spectrum separately. The fitting parameters are then saved for each spectrum separately – this is the dispersion-mask image.

Note: Once a dispersion-mask image is created, it is possible to use that image as a starting point when creating an image for another set of data that use the same instrument setup. To use this functionality, specify the already created dispersion-mask image with the DISPMASKIN keyword.

Please, after reading through the documentation below, also see the p3d A&A paper as well as the master user-parameter file and the p3d documentation WIKI, for help on how to setup the parameters to achieve the highest accuracy possible in the wavelength calibration.

This routine calls the routine p3d_cr, if requested to do so, which cleans single input images of cosmic-ray hits; see the keyword /CRCLEAN. The outcome of the cleaning procedure depends both on properties of the input image as well as the algorithm parameters. Different parameters might produce a different outcome. There is no guarantee whatsoever that the cleaning removes only cosmic-ray hits. Check the output images to build your own opinion regarding the quality of the outcome.

The output-data files – consisting of the extracted spectrum image, its error image, the bad-pixels mask image, the saturated-pixels mask image, and the data-quality mask image – are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE, or the user-parameter 'allinone', is unset. There is no real advantage to either approach; although, if separate files are used and an output file is renamed, it will not be found by other tools.

Here follows a description in full detail of the creation of the line list, the calculation of the wavelength range, and all parts of the GUI.

How p3d determines which arc-line list file to use

Before the dispersion mask GUI is opened, p3d has to determine which arc-line list file to use. The arc-line file is a plain-text file where the each line contains the known wavelength of an arc (emission) line. The following methods are used, ordered after priority:

1.

One or several line-list files are specified using either the ARCLINEFILE keyword (highest priority) or the 'arclinefile' parameter in the user-parameter file (lower priority). In the latter case, two or more files are specified by appending a suffix to the keyword; the file contents are then concatenated into one list.

This approach works with all instruments.

2.

Depending on the presence of observing-setup specific keywords in the data header, a default line-list file is selected from an instrument-specific list of files.

This approach is used with the VIMOS and the FLAMES instruments.

3.

Keywords in the data header indicate which calibration lamps were used in the observations. A default line-list file is then selected from an instrument-specific list of files.

This approach is used with PMAS, where the calibration lamps that are internal to the instrument are used.

4.

When p3d cannot determine which line list to use a line-list selection GUI is opened that allows the user to choose among a set of pre-defined line lists.

This approach is used with the following instruments: GMOS-S/N, INTEGRAL, MPFS, PMAS/PPAK (when external calibration lamps are used instead of the internal ones), SPIRAL, and VIRUS-P.

The default arc-line list file may contain more lines than can be identified in the image, or it may be the case that some of the lines are saturated. If this applies to the used data, the corresponding entries should be removed from the arc-line list before creating the dispersion mask. Such entries can be removed interactively while using this tool, and here is how this is done (this is also described below, in the GUI section on the 'Delete lines' function):

Press the 'Delete lines' button in the 'Line Mask Functions' tab of the main control panel. The (pixel) position of each entry in the line list is shown as a cross. When the mouse cursor is moved over the spectrum plot, the wavelength is shown of the entry in the arc-line list that lies the closest to the cursor. Press the left mouse button to delete the entry from the line list. Press the right button to finish the deletion procedure.

When done, the resulting arc-line list can be saved to a file by selecting the menu entry File->Save the current line list. The same arc-line list file can then be loaded into any subsequent wavelength-calibration session; either using the ARCLINEFILE keyword, the user-parameter file, or the menu option File->Read line list(s).

Note: In some cases, the difference between the fitted wavelength and the tabulated wavelength of any one line may always be larger than for the other lines. It is possible that the accuracy of that particular wavelength is low. If this should occur, remove the entry from the arc-line list, or correct the value, and then try anew. If you find it necessary to correct any line list, please notify us, so that we can apply the proper correction to the data files.

How p3d initially determines the wavelength range

Before the dispersion mask GUI is launched, p3d determines an initial wavelength range covered in the arc image. This initial wavelength range is used when determining the first-guess pixel position of each entry in the arc-line list. The determination of the wavelength range depends on the instrument:

FLAMES/ARGUS

This instrument is used with a fairly large number of fixed configurations. The initial wavelength range is specified in the file "flames_gratings.dat". The wavelength and the dispersion of the center pixel are determined from these tabularized values.

Note: This instrument seems to be the most difficult one to use with a constant dispersion, which is why the default wavelength ranges seem to work poorly. To create a dispersion-mask image with this instrument, it is important that all lines are identified using a prepared arc spectrum. A better -future- solution will likely include the dispersion equation found in the FLAMES manual.

GMOS-S/N

The file-header provides information on the center pixel wavelength, and which grating was used. The total wavelength range is listed for each grating in the file "gmos_gratings.dat", and this value is used to calculate the center pixel dispersion.

Note: The number of pixels on the dispersion axis is huge with this instrument. The default constant dispersion seems to be a less than ideal solution, which is why the position offset can be large between the red and the blue ends of the detector. It is, nevertheless, possible to create an accurate dispersion solution. Use an arc spectrum on a sheet next to the monitor to identify the individual lines across the entire spectrum.

INTEGRAL

The file header provides information on both the wavelength and the dispersion of the center pixel.

MPFS

The file header provides information on both the wavelength and the dispersion of the center pixel.

PMAS

Using input of the grating rotation angle and the number of lines per mm, the grating equation is used to calculate the wavelength and the dispersion of the center pixel. The initial guess mostly works very well with the older 2kx4k CCD, while it is necessary to adjust the dispersion solution manually using the newer 4kx4k CCD.

SPIRAL

Just like with PMAS input of the grating rotation angle and the number of lines per mm are used with the grating equation to calculate the wavelength and the dispersion of the center pixel.

VIMOS

This instrument is used with a small number of configurations. The initial wavelength range is specified in the file "vimos_gratings.dat". The listed ranges differ slightly from those specified in the VIMOS manual. The wavelength and the dispersion of the center pixel is determined from this initial value range. The deviation between actual line pixel positions and the initial solution seems to be fairly small, which is why it is easy to identify the arc lines and create a manual dispersion solution.

Note: So far the default wavelength range has been properly configured for the HR-Blue, the HR-Orange, and the HR-Red grisms - this has been done for each quadrant individually. The default values still have to be adjusted for the remaining grism, i.e., MR.

VIRUS-P

The grating angle, and therefore the wavelength range, is set manually in this instrument – there is no information in the data header that can be used to determine which range was used during the observations. Default values of the lower and upper wavelengths are read from the file "virus_gratings.dat"; the center wavelength and the constant dispersion are determined from the provided numbers.

Note: In case some other wavelength range was used than the default blue one (3620-~5910Å), copy the file "virus_gratings.dat" to the directory where the data are being reduced. Then edit the file to correspond to the used wavelength range, and set the parameter 'gratingsfile_dm' to the proper filename in the user-parameter file.

The graphical user interface of the dispersion-mask creation tool

After the spectra in the (combined) arc image have been extracted, the dispersion-mask creation GUI tool is launched. This tool consists of six separate regions: a menu bar, the spectrum image, a wavelength-range indicator field, a plot of the reference spectrum, a status-line field, and a control panel. Here is a more complete description of each of these regions. Key accelerators are, where available, justified to the right.

The menu bar

Here follows a description of all the available (and sometimes unavailable) entries in the menu bar.

The 'File' sub menu field

  • View fits header

    Opens a window that displays the fits-file header of the used arc image.

  • View fits header – second file

    Opens a window that displays the fits-file header of the used second arc image. This entry is only visible when two arc image files are used.

  • Read line-list file(s)

    Opens a file dialog that allows any number of arc-line list files to be selected; use the shift and the ctrl key modifiers to select multiple files.

  • Save the current line list

    Opens a file dialog where you can choose the name of a single file that will contain the current arc-line list. The output is a plain-text file where each line contains the wavelength of an arc line. The wavelength values [Å] will be increasing monotonically.

  • View the current line list

    Opens a window that contains a numbered list of the currently used arc lines. The third column contains the pixel position of each line. Lines, which should be placed left of the left-hand side boundary, get the pixel position 0.000. Likewise, lines, which should be placed right of the right-hand side boundary, get the pixel position -1.000. Some, or all, of either group of lines can be shifted into the pixel range using the 'shift mask' function. The shown line list is dynamically updated as emission lines are deleted using the 'Delete lines' function. Select the menu option File->Exit to quit the GUI.

  • Reset (Ctrl+R)

    n/a

  • Undo all changes (Ctrl+U)

    All parts of the GUI return to their state as they were when the GUI was first launched.

  • Exit (Ctrl+Q)

    Quits the dispersion-mask creation GUI, without saving any dispersion mask that might have been created.

The 'View' sub menu field

  • Color map; use 100% of the pixel values (F1)

  • Color map; use 99.5% of the pixel values (F2)

  • Color map; use 99% of the pixel values (F3)

  • Color map; use 98% of the pixel values (F4)

  • Color map; use 97% of the pixel values (F5)

  • Color map; use 95% of the pixel values (F6)

  • Color map; use 90% of the pixel values (F7)

  • Color map; use 80% of the pixel values (F8)

  • Color map; use 70% of the pixel values (F9)

    The function of these options are described in the spectrum-image section below.

  • Invert the colors

    Toggles the minimum and the maximum pixel values with the effect that the color table is "inverted"; for example, instead of black->white, the colors would become white->black.

  • Select the color table

    Allows the selection of another color table than the preset one, either using the (standard) interactive routine XLOADCT, or by choosing any of the pre-configured menu entries.

The 'Function' sub menu field

  • Match Curvature (Ctrl+1)

  • Shift mask (Ctrl+2)

  • Delete lines (Ctrl+3)

  • Match Centroids (Ctrl+4)

  • Fit Centroids (Ctrl+5)

  • Create Dispersion Mask (Ctrl+6)

  • Save the Mask and Exit (Ctrl+W)

    Each of these functions/actions are described in detail below.

The 'Options' sub menu field

  • Subtract the Background

  • Set the Reference Position

  • Change Dispersion

  • Reset Disp

    Each of these functions/actions are described in detail below.

The spectrum image

The top-most field shows an image with all extracted spectra stacked on top of each other. Each (horizontal) line represents a spectrum, where each spectrum is one pixel wide. The first spectrum in the file is shown at the bottom, and the last spectrum at the top. Shorter (longer) wavelengths are always shown on the left-hand (right-hand) side.

Depending on the user parameter 'dbin', the image is not rescaled (dbin=1, this is the default) or rescaled (dbin=2, 3) on the horizontal axis:

A horizontal and a vertical scrollbar are shown if the entire image does not fit on the screen; the image is initially centered horizontally when these scrollbars are present.

Before the extracted image is shown on the screen, it is rebinned to use half as many pixels on the dispersion axis.

Before the extracted image is shown on the screen, it is rebinned several times to use half as many pixels on the dispersion axis, until all pixels fit on the screen without using any scrollbar.

The (false) colors used to show the spectrum image are calculated by a linear scaling of a defined intensity range. The intensity range values can be set explicitly (see the view tab options in the control panel below) or by using a histogram function with a set of nine preset x-values: 100%, 99.5%, 99%, 98%, 97%, 95%, 90%, 80%, and 70%. The meaning of the values is as follows. In a histogram of all pixel values in the spectrum image, the lowest (100 – x) / 2 per cent, as well as the highest (100 – x) / 2 per cent, of all pixel values are discarded before setting the minimum and the maximum intensity values. The overall minimum and maximum values are used with the 100% setting. The color table is chosen using the COLORTABLE keyword; the default is the so-called Sauron color table.

The initial pixel position for each entry in the line-list file is shown as a vertical white line, which is plotted on top of the spectrum image. These white lines are initially always straight, in comparison to arc lines in the spectrum image, which are mostly curved, to some degree; the curvature reflects differences in the optical path across the surface of the integral-field unit.

A reference spectrum is shown in the reference spectrum plot. The reference spectrum and a reference pixel are defined as a spectrum located near to the center, and the center pixel. The reference pixel is shown with a red vertical bar, and the reference spectrum is shown with a red disk. The initial reference spectrum and the reference pixel can both be changed using the options under the "Reference spectrum and Reference pixel" tab. The reference spectrum should be chosen as a spectrum where, ideally, only arc lines are visible. Blue- or redshifted emission lines, originating in the object, can otherwise decrease the accuracy of the dispersion-mask calculations if they lie close to the arc lines.

The wavelength range indicator field

When the dispersion-mask tool is launched, the first-guess wavelength range is indicated with the lowest wavelength on the left-hand side and with the highest wavelength on the right-hand side of the narrow wavelength-range indicator field. The center wavelength is shown in the middle. When the spectrum-image field is shown with a horizontal scrollbar, an arrow is shown in front of (behind) the lowest (highest) wavelength value. The pixel position of the center value shifts along with the horizontal scrollbar; if the pixel position falls outside the shown range, the center wavelength is shown with a green color, and a green arrow is also shown to indicate that the center pixel is located to the left (right) of the lower (upper) boundary.

A plot of the reference spectrum

A plot of the reference spectrum is shown in the third field from the top; this is, mostly, the second largest field. The range of pixel values of this plot always agrees with the range of the spectrum image and the wavelength-range indicator field. The same range of intensity values used to calculate the colors of the spectrum image is used to scale this plot.

Each entry in the line-list file is shown as a white x symbol; the positions of the x:s agree with the positions of the straight lines in the spectrum image. The wavelength of each emission line is shown above the x when the left mouse button is clicked.

The currently selected line-list entry and its wavelength are shown with a red color when the 'Delete lines' function of the 'Line Mask Function' tab is active.

The status line

When the mouse cursor is positioned above an interaction field (i.e. a widget), such as, for example, a drawing area or a button, a descriptive text of the function of the field is shown on the status line. Some functions also provide dynamic instructions in this field.

The control panel

All functions and options to control the dispersion-mask creation procedure are collected under tabs of similar actions of the control panel. The six tabs are: Line mask functions, Dispersion, Check the current mask fit, Check the final fit, Viewing options, and Reference spectrum and reference pixel. Here follows a description of each tab and its components:

Line mask functions

This tab collects all the basic functions necessary to create a dispersion mask. The functions should be executed one by one, preferrably starting from the left-hand side. Not all functions are available when the tool starts up (the corresponding buttons are not sensitive), those ones become available as the previous functions provide the required information. Here follows a description of each of the seven functions:

  • Match curvature (Ctrl+1)

    There is, when the tool is first started, no information available about differences in the optical path of separate spectra. The initial pixel position of each emission line is thus independent of the spectrum. Extracted arc spectrum images, however, show that the optical path varies, with the result that the arc lines appear curved – as a function of spectrum number – instead of being straight lines. The 'match curvature' function uses the arc image to calculate a "curvature" for all entries in the arc-line list as a function of wavelength. The curvature is calculated by following the maximum position of an initially selected arc line through subsequent spectra along the dispersion axis.

    The function is activated by clicking with the left mouse button on the 'Match curvature' button. There are thereafter two options to proceed:

    Use one arc line

    Click with the left mouse button on top of a fairly bright, but not saturated emission line. A curvature is then calculated for all spectra and entries in the arc-line list, assuming a constant dispersion with wavelength.

    Use two arc lines

    Click with the right mouse button on top of at first one, and thereafter a second, fairly bright, emission line. A curvature is then calculated for all spectra and entries in the line list, assuming that the dispersion changes linearly with the wavelength. In order to increase the accuracy of the curvature calculation, it is both important to select sufficiently bright arc lines that are also not saturated, and to choose arc lines that are very well separated to properly estimate the change in dispersion. Try to choose one arc line near the blue edge and one arc line near the red edge of the image.

    Attention: This option is unavailable when using an input dispersion-mask image using the DISPMASKIN keyword.

  • Shift mask (Ctrl+2)

    The first-guess wavelength range is likely shifted on the dispersion axis relative to the positions of the arc lines in the input image – the shift is greater for some instruments than for others. The line mask, consisting of all entries in the arc-line list, can be shifted linearly on the dispersion axis using the 'Shift mask' function.

    The function is activated by clicking with the left mouse button on the 'Shift mask' button. The mouse cursor must thereafter be positioned in the spectrum-image field. Press the left mouse button and drag the line mask until satisfied. The function finishes once the left mouse button is released.

    It may be necessary to change the first-guess value of the dispersion (given in Å/pixel), this can be done with the options under the Dispersion tab. Use the 'Change disp.' function to change the constant dispersion value. Few instruments, however, show a constant, or even linear, dispersion across a larger wavelength range. In such cases, the best approach is to first shift the line mask to allow an identification of some known lines, and thereafter use the 'Manual disp.' function.

    Note: The 'Shift mask' function assumes a constant, or linear, dependence of the dispersion with wavelength, which is why it MUST NOT be used after the 'Manual disp.' function is used.

    Attention: This option is unavailable when using an input dispersion-mask image using the DISPMASKIN keyword.

  • Delete lines (Ctrl+3)

    In particular the default line lists contain entries of (faint) arc lines that are unlikely to be found in the extracted arc image. Such entries can be removed from the line list using the 'Delete lines' function. In case a tailored line-list file is used, which only contain entries that are present in the arc image, there is, of course, no need to use this function.

    The function is activated by clicking with the left mouse button on the 'Delete lines' button. Thereafter, the mouse cursor should be positioned in the spectrum-plot image. The arc-line entry that lies the closest to the mouse cursor is indicated in red color; its wavelength is also shown in the same red color. Remove that line by clicking on the left mouse button (note that the entry also is removed immediately in the line-list view, if it is shown). The function is active, and the same procedure is repeated, until the right mouse button is clicked.

  • Match centroids (Ctrl+4)

    At this point, the entries of the arc-line file should have been shifted, and the dispersion modified, if necessary. The x symbol of all available arc lines should lie close to the peak of the respective emission line in the arc image.

    The (non-interactive) function 'Match centroids' traverses all entries of the arc-line list and checks if there is an intensity peak in the arc image near the corresponding pixel position of each entry. A positive identification requires two conditions to be fulfilled:

    1.

    The pixel distance to the arc-image peak must be less than, or equal to, linewidth_dm pixels (see the instrument-specific parameter file).

    2.

    The maximum of the peak must lie above three times the noise level, which is estimated from data in the region near the peak.

    New arc-line positions, which are weighted with the peaks in the arc-image data, replace the input positions. The procedure is iterated five times to further increase the accuracy of the new positions. All entries where no peak is found are removed from the arc-line list.

    The function is used by clicking with the left mouse button on the 'Match centroids' button. After the function has finished an rms value, which relates the n new positions (pos_new) to the n input positions (pos_input), is shown in the spectrum-plot field. This rms value is calculated using the equation: rms = sqrt(sum((pos_new – pos_input)²) / n).

    Note: This function can be used repeatedly if you are not satisfied with the outcome; for example, if you decide that you want to remove another entry in the arc-line list after using either this function or the 'Fit centroids' function, you can delete the line, and thereafter press the 'Match centroids' button anew.

    Note: For more details, see the routine 'p3d_wavecal_match_maskwd.pro'.

    Attention: This option is unavailable when using an input dispersion-mask image using the DISPMASKIN keyword.

  • Fit centroids (Ctrl+5)

    In order to create an accurate dispersion mask, it is necessary that all the pixel positions of the used arc-line entries are accurate. The (non_interactive) 'Fit centroids' function calculates such accurate positions for each entry in the arc-line list. Two methods are available, using the parameter centermethod_dm (see the instrument-specific parameter file):

    1.

    The first approach is to first fit a "Gaussian" profile to the peak; the initial arc-line position is thereafter replaced with the center position of the fitted profile.

    2.

    The second approach is to replace the initial arc-line position with a "weighted" position; in this case, the arc-image spectrum is used as a weight. This approach is mostly less accurate than the Gaussian approach, but it has the advantage that it is much faster.

    The function is used by clicking with the left mouse button on the 'Fit centroids' button. The procedure can be repeated as many times as required, although the increase in accuracy is minute in subsequent calls. After the function has finished an rms value, which relates the n new positions (pos_new) to the n input positions (pos_input), is shown in the spectrum-plot field. This rms value is calculated using the equation: rms = sqrt(sum((pos_new – pos_input)²) / n).

    Note: This function can only be used AFTER the 'Match centroid' function has been executed.

    Note: For more details, see the routine 'p3d_wavecal_correct_maskpos.pro'.

  • Create dispersion mask (Ctrl+6)

    The dispersion-mask image is created as follows. A polynomial of order n_p is fitted to each spectrum individually, using the known wavelengths of the arc-line list entries as a function of their respective pixel position. The default polynomial order (n_p), which is shown immediately to the right of the control panel (where it can also be changed), is provided by the parameter polynomialorder_dm (see the instrument-specific parameter file).

    This (non-interactive) function is used by clicking with the left mouse button on the 'Create dispersion mask' button. After the dispersion mask has been created, the extracted image in the spectrum-image field is replaced with a wavelength corrected image. If the fit was successful, the initially curved arc lines should now be straight. One way to check the quality of the dispersion mask is to study the plots now available under the 'Check the final fit' tab.

    The fitting procedure is performed twice. In the first step, it is done for all spectra in the input image. A zero-weight is assigned to all arc-line entries, for each spectrum individually, where the input-to-fitted wavelength residual is larger than, or equal to, the pre-defined value residualcut_dm (see the instrument-specific parameter file). Only arc-line entries positively identified in at least half of the spectra during the first step are used when fitting the polynomial in the second fitting step.

    Note: This function can only be used AFTER the 'Fit centroids' function has been executed. None of the previous five functions can be used after this function has been called.

    Note: For more details and additional options, see the routine 'p3d_wavecal_fit_maskwd.pro'.

  • Save mask and exit (Ctrl+7)

    In order to save the disperson-mask image and thereafter close the graphical user interface, press the 'Save mask and exit' button.

    Note: This function can only be used AFTER the 'Create dispersion mask' function has been executed.

Dispersion

This tab collects all functions used to change the dispersion, i.e. the measure Å [Angstrom] per pixel. The functions can be executed in any order, as required, but if the 'Manual disp.' function is used, the use of any of the other functions afterwards will discard the just created non-linear solution.

  • Change disp.

    The constant dispersion value can be changed interactively using the 'Change disp.' function. To use this function, press the corresponding button and thereafter position the mouse cursor in the spectrum-image field, but not atop the (red) reference pixel.

    The wavelength of the associated pixel is changed by first clicking with the left mouse button. Thereafter, drag the mouse cursor away from the initial position. The mouse cursor must always be kept on the same side of the reference pixel as when the dragging was initiated. The function is finished when the left mouse button is released. The new value of the dispersion is determined as follows: let the initial separation between the mouse cursor and the reference pixel be x_init, and let the 'dragged' separation be x_dragged, then the new dispersion will be set to x_dragged / x_init.

    Note: When changing the dispersion using this function, a higher precision is achieved if the left mouse button is first pressed when the mouse cursor is not too close to the reference pixel. Thereafter, only move the cursor with small movements to avoid too drastic changes of the dispersion value. If necessary, use the 'Reset disp.' function to restore the initial dispersion value.

  • Reset disp.

    Click on the 'Reset disp.' button to reset all the dispersion-related values to the routine entry values. These three numbers include: the constant dispersion value, the relative change in dispersion between pixels (delta; this value allows for a linear function of the dispersion), and the wavelength of the center pixel.

  • Manual disp.

    Few instruments show a constant, or even linear, dispersion across the full wavelength range. With the manual dispersion function, it is possible to calculate a custom non-linear dispersion solution using all available arc lines.

    To use this interactive function, first click with the left mouse button on the 'Manual disp.' button. Thereafter position the mouse cursor in the spectrum-plot field. The next task is to create a list of new (modified) arc-line pixel positions. It is necessary to repeatedly use the following procedure when creating this new list:

    1.

    Click a first time, with the left mouse button, near the chosen entry in the arc-line list; the symbol of the entry that will be chosen is indicated with a red color.

    2.

    Click a second time, again with the left mouse button, at the (exact) position where the arc-line entry should be moved. The old and the new positions are then shown with a green arrow that indicates whether the change is towards a shorter or a longer wavelength.

    3.

    Should the last modified position be useless, it can be removed from the new list by pressing the middle mouse button. In fact, all entries in the new position list can be removed by using this function repeatedly.

    Once you have finished adding pixel positions to the new list, for all the arc-line list entries that are to be used, the procedure is finalized by clicking the right mouse button. Subsequently a rational spline is fitted to the new pixel positions; as wavelength[pixel]. This dispersion solution should – if the new pixel-position list was created carefully - be more accurate than the initial constant-dispersion solution. To make possible the creation of an accurate dispersion solution at this step, follow these guidelines, which will also subsequently allow the creation of an accurate dispersion mask:

    1.

    Make sure all arc lines are identified correctly. If you do not know which arc lines are present in your spectrum, study a printed arc spectrum of your wavelength range, which uses the same arc lamps.

    2.

    Cover the full wavelength range with arc lines. The spline function is unable to create an accurate dispersion solution for the lowest or highest pixel regions if no arc lines are used in these regions. If the range is large, as is the case if there are, for example, no entries at all across half of the CCD, the spline routine will likely fail completely.

    3.

    It is not necessary to specify exact pixel positions in the new list. More exact positions are calculated by the 'Fit centroids' function.

    4.

    Avoid abrupt changes in the dispersion across few pixels. If you cannot achieve an accurate dispersion mask, first check your arc-line spectrum again to see if you might have associated the wrong arc line with an entry in the arc-line list.

    Note: The dispersion solution created by the 'Manual disp.' function is not accurate enough to calculate a dispersion mask. The only purpose of this preparatory function is to produce a dispersion solution that allows p3d to match as many arc lines as possible in the input spectra with the entries in the arc-line list.

  • Disp.

    The editable field immediately to the right of this label shows the value of the constant dispersion; in [Å/pixel]. Enter a new value in the field number to replace the currently used value.

    Note: This value is not used after the 'Manual disp.' function has been executed.

  • Delta

    The editable field immediately to the right of this label shows the value of the differential, and linear, change of the dispersion; in [Å/pixel]. Enter a new number in the field to replace the currently used value. The dispersion is constant when this value is equal to zero, otherwise it is linear.

    Note: This value is not used after the 'Manual disp.' function has been executed.

  • Reference pixel wavelength

    The editable field immediately to the right of this label shows the value of the wavelength at the reference pixel; in [Å]. Enter a new value in the field to replace the currently used value.

    Note: This value is not used after the 'Manual disp.' function has been executed.

Check the current mask fit

This tab provides a mean to check both the quality of a (linear) polynomial fit to the current arc-line pixel positions (click on the 'Fit' button) and the residual of each arc-line entry of the preset-wavelength minus the fitted-wavelength. The check is only done for the reference spectrum.

The spectrum image (in the upper panel) is replaced with a plot of the wavelength as a function of pixels – using the full wavelength range (also in the spectrum-plot panel below). The fitted curve is drawn with a solid line between the individual arc-line entries, which are shown with the x symbol.

In the polynomial-fit mode, four different informational strings are shown in the upper left corner of the plot. The four fields are separated by two colons and show: the polynomial order, the output status code of the IDL routine POLY_FIT (0==successful completion), the chi-square value of the fit, and the root-mean-square (RMS) value of the fit. Only the RMS value is shown in the residual-mode plot.

Please note that, the status check of this tab does not show quantitatively what the quality of the dispersion-mask image will be. However, if it is executed after the 'Fit centroids' function, it provides a quick mean to check if the polynomial order is appropriate, and if all the arc-line entries provide a sufficient accuracy. Maybe some entry could be removed to improve the accuracy.

Note: This results checking mode is only available before a dispersion-mask image has been created using the 'Create dispersion mask' function.

Check the final fit

This tab provides a mean to check the quality of the (linear) polynomial fit to the created dispersion mask. In comparison to the current fit tab, all spectra can be checked here.

The spectrum image (in the upper panel) is replaced with a plot of the wavelength as a function of pixels – using the full wavelength range. The fitted curve is drawn with a solid line between the individual arc-line entries, which are shown with the x symbol. Some information about the fit is shown in the upper left corner of the plot. The information contains the spectrum number (these are 1-based, i.e., 1, 2, 3, ...; inside of a pair of brackets; note that any calibration spectra have been removed from the image prior to launching the dispersion-mask creation tool, which is why there might be an offset compared to the actual spectrum number in the image), the polynomial-fit order, and the chi-square number of the fit. If the shown spectrum is listed in the instrument-specific dead-fibers file, the text in the second column of that file is shown to indicate if the spectrum: belongs to a low-transmission fiber, belongs to a dead fiber, or is not used (unused; which would be the case with spectra falling outside the detector). Spectra of these latter types are often more difficult, and even impossible, to calibrate than the normal spectra – there is no cause for alarm if they cannot be well calibrated.

The plot of the reference spectrum (in the lower panel) is replaced with a plot of the the residuals for each arc-line entry (of the preset-wavelength minus the fitted-wavelength) as a function of pixels – also here using the full wavelength range. A horizontal dotted line indicates the zero level. Some information about the fit is shown in the upper left corner of this plot as well. The information contains the spectrum number (inside a pair of brackets; see the above description of the upper panel), and the maximum residual value, which is determined from all of the arc-line entries of the currently shown spectrum.

The currently shown spectrum can be selected using the scrollbar, which is shown in the control panel below the panels, or by typing a (valid) spectrum number in the field left of the scrollbar. The arrow button at the left (right) end of the scrollbar can be used to decrement (increment) the spectrum number by one. The upwards pointing arrow button on the right-hand side of the scrollbar selects the spectrum with the maximum residual; dead or unused spectra, as specified in the instrument-specific dead-fibers file are ignored when using this function.

Note: This results checking mode is only available after a dispersion-mask image has been created using the 'Create dispersion mask' function.

Viewing options

This tab provides a set of options to control the presentation of the spectrum image, the spectrum plot, and the fitting procedure. From the left-hand side to the right-hand side, the different options are:

If a continuum is present in the arc spectra, it can be subtracted by clicking the 'Subtract the Background' button; this button can only be used once.

The current minimum (maximum) value used to calculate the colors (using any pre-defined color table), and set the ranges of the spectrum plot, is shown in the left (right) editable field. The (false) colors used to show the spectrum image are calculated by a linear scaling of a defined intensity range. The intensity range values can be set explicitly by typing any (valid) values into the fields, or by using the droplist on the right-hand side of the editable fields. With the help of the droplist, a histogram function is used, with a set of nine preset x-values: 100%, 99.5%, 99%, 98%, 97%, 95%, 90%, 80%, and 70%. The meaning of the values is as follows. In a histogram of all pixel values in the spectrum image, the lowest (100 – x) / 2 per cent, as well as the highest (100 – x) / 2 per cent, of all pixel values are discarded before setting the minimum and the maximum intensity values. The overall minimum and maximum values are used with the 100% setting. The droplist values can also be set using the key accelerators F1 – F9 (see the Viewing options menu item).

The rightmost button determines whether the arc-line center position fits are shown or not. There are four options, which are cycled by clicking the button repeatedly:

Hide refe. spectrum fits

The default is to not show any center-position fit plots.

View refe. spectrum fits

View the center-position fit plot for the reference spectrum and each arc-line entry.

View fits of all spectra

Show a center-position fit plot for all spectra and each arc-line entry. This mode is slow, in particular if the data contain many spectra.

View all fits for line k

Show a center-position fit plot for all spectra of the arc-line entry k. The arc line is chosen by entering a value in the text field on the right-hand side of the button.

Reference spectrum and reference pixel

This tab provides a set of options to set the reference spectrum and the reference pixel (see the section above on the spectrum image for more details). From the left-hand side to the right-hand side the different options are:

The reference spectrum and the reference pixel can be selected interactively by clicking the left mouse button on the button (icon) with a yellow arrow. Thereafter, click (repeatedly) with the left mouse button on the required spectrum (row) and pixel (column). Finish the selection by clicking the right mouse button.

The selected reference spectrum and reference pixel are shown in the editable fields on the right-hand side of the arrow button. Any (valid) integer can be typed into the fields to choose another value.

Pol. order

The editable field on the right-hand side of the label shows the preset (linear) polynomial order. The preset value is read from the instrument-specific parameter file, using the keyword "polynomialorder_dm"; any positive integer is valid.

How to create a dispersion-mask image

There are no strict directions on how to create a dispersion mask using the tool of p3d. Here are, however, some guidelines on how you could proceed with your data.

1. Use an already created dispersion-mask image

Assume there is already a created dispersion-mask image for the same instrument configuration as the current data. If this is the case, this image can be used to create a good first-guess dispersion solution – using very few interactive steps.

At first, specify the starting-point disperson-mask image using the DISPMASKIN keyword. If the match between the already calculated line positions and the arc image look OK, proceed with the creation of the new dispersion mask; continue with step vi in the 'Fit the arc-line entries to the data'-subsection under section 4 below.

If the match does not appear to be OK, it might be necessary to create a new dispersion-mask image from scratch (using the following steps).

2. Determine exactly which arc lines are present in the data

p3d is in most cases able to make a qualified guess regarding which arc lamp(s) were used during the observations. In some cases, however, this is not possible. For example, the PPAK IFU of the PMAS IFS is (nowadays), as well as the VIRUS-P IFS, used with arc lamps placed in the dome; the data headers do not contain any information about the actually used lamps. Please see your notes, or make your own (qualified) guess on which arc-line lists to use.

When you know which arc lamps are present in your data, it is necessary to find out exactly which emission lines are present. The arc-line list files mostly contain the strong lines, as well as the very faint ones. Depending on the exposure time(s), the strong ones may be saturated – and should not be used then – and the faintest lines may not even be visible. Use available arc spectrum plots of separate arc lamps to identify the lines.

The VIMOS and FLAMES IFUs have pre-configured arc-line lists for all used instrument configurations. Use these line lists and save some work (plain-text versions of those arc-line files, which are used by p3d, are available in the directory p3d_path/data/tables/linelists).

3. Modify the minimum and the maximum intensities

Depending on the data, the default maximum and minimum intensities (histogram, 95%) might not reveal the faintest arc lines. Try to modify the minimum and maximum values by hand to find a range that works with the data. For example, click on the 'Viewing options' tab and try the minimum value 0.0 and the maximum value 5000.0.

It might be easier to identify the stronger lines with the help of an existing arc-spectrum plot if the 100% range (droplist) option is used.

4. Line mask functions – create the dispersion mask

The following functions, which are all found under the 'Line mask functions' tab, with the exception of the 'Manual disp.' function found under the 'Dispersion' tab, are now used to create the dispersion-mask image.

i) Match the curvature

The initial dispersion mask is the same for all spectra; i.e., all arc-line list entries are straight lines in the spectrum image. To account for optical paths that vary with different spectra, by tracing the curved arc lines, use this function on either one or two of the brighter arc lines.

ii) Shift the mask

The wavelength-to-pixel accuracy that can be achieved with the first-guess dispersion solution might be low – possibly because of instrument flexure. Use this function to shift the entire mask of arc-line entries towards lower or higher-value pixels, to achieve a better match.

In case the amplitude shifts show a non-constant variation from the blue side to the red side, you could start by shifting the arc-line mask to allow you to identify the arc lines, before you proceed to the next step.

iii) Create a dispersion solution manually

For most instruments, the dispersion varies between the blue end and the red end. If this is the case with the data, it is necessary to replace the initial constant (or linear) dispersion solution with a dispersion solution created manually. To do this, follow the instructions for the 'Manual disp' function, which is found under the 'Dispersion' tab in the control panel.

iv) Delete the saturated and faint arc-line entries

Most default arc-line list files, but those for the VIMOS and the FLAMES IFSs, are likely to include many more entries than can be seen in the data. Over-exposed lines should not be used since they are not symmetric. Create a more accurate dispersion-mask image by deleting all the superfluous entries; use the 'Delete lines' function.

It is a good idea to save the arc-line list once it is satisfactory. The next time any data are reduced, using the same instrument configuration, use the same arc-line file; either from within the dispersion-mask creation tool, or using the user-parameter file.

v) Match the arc-line entries with the data

In the next step, select the 'Match centroids' function to make p3d match all the entries of the arc-line list with the data in the extracted image. Arc-line entries not found in the data, are then deleted. The arc-line entries are also better centered on top of the data.

vi) Fit the arc-line entries to the data

With the 'Fit centroids' function, each entry in the arc-line mask is fitted accurately to the arc lines in the data.

Click on the long horizontal button under the 'Viewing options' tab to view all or some of the fits – this allows eventual problems to be spotted.

vii) Create the dispersion mask

Select the 'Create dispersion mask' function to fit a polynomial to each spectrum – where the wavelength as a function of pixel.

viii) Check the final fit

Use the 'Check the final fit' tab to interactively inspect the quality of the dispersion mask; thereby learn which arc-line entries are problematic, if any, and what the accuracy of the dispersion-mask image is.

ix) Save the mask and exit

The final step is to save the dispersion-mask image. After selecting this function the tools exits.

General information on how to learn more about this routine

The procedure of creating a dispersion mask is described in detail in the p3d A&A paper: Sandin et al. 2010, A&A, 515, 35

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside IDL check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Any plain text file can be used as an arc line-list file, where the wavelengths of (well) known emission lines are given in the first column (specified in [Å]ngström; columns are separated by white space). Default line-list files are available in the directory "!p3d_path/data/tables/linelists/". If you want to use your own arc line-list file, copy one of the default files to your data-reduction output directory and edit it, or create your own file. Edit the file(s) to suit your needs. To use the file in your reduction work you can select it from the dispersion-mask creation GUI, or by specifying it (or them) in your user-parameter file using the parameter "arclinefile".

NOAO provides spectrum plots of some arc lamps at the following address "http://www.noao.edu/kpno/specatlas/", use these plots, or similar ones for your arc lamp, to identify all the arc lines.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes one, or several, raw arc images as input. The images are combined (by default), but p3d can also calculate a dispersion mask for each file if the parameter combineimages_dm is set to "no" (in the user-parameter file). Prescan and overscan regions must not be removed in the input data, they are removed by p3d during the reduction. Input files of instruments using several readout ports must also not be combined in advance, p3d combines the images automatically.

Additionally a file with a list of known arc lines is needed; p3d will attempt to use a default line list if it can. If p3d cannot figure out which file to use by itself, it queries for a file. It is also possible to specify a file (or several files) in the user-parameter file.

The output – the dispersion-mask image – consists of a list of //n + 1// parameters for each spectrum. These are the //n + 1// parameters used to fit an //n//:th order polynomial to each spectrum. There are always the same, pre-defined, number of extracted spectra in the output file. The number of pixels on the dispersion axis is the input size (where the prescan and the overscan regions have been removed). The spectra are ordered as in the input image – this is the so-called row-stacked spectrum (RSS) format. To create a spatial map of the intensities at any wavelength, it is necessary to use a fiber position table. In addition to the standard fits-file header keywords, the extracted spectrum images contain additional p3d-specific keywords:

IMRAW

The name(s) of the raw input flat-field image file(s).

IMRA2

The name of the second raw input flat-field image file, if two input files were specified.

IMMBI

The name of the master-bias image file, if it is used.

IMTRC

The name of the trace-mask image file.

IMERR

The name of the flat-field image error file. Note! This keyword is only written if ALLINONE is unset.

IMONC

The name of the output bad-pixels mask image filename; this keyword is only added if either a bad-pixels mask image or a cosmic-ray mask image is specified, or calculated. Note! This keyword is only written if ALLINONE is unset.

IMSMK

The name of the saturated-pixels mask filename; this keyword is only added if the SATMASK is set. The saturated-pixels mask image in addition contains the keyword SATLIMIT, which specifies the pixel value that was used to define the saturation limit. Note! This keyword is only written if ALLINONE is unset.

IMTYPE

The p3d image type is set to 'p3d: extracted arc image'; the error-image type is 'p3d: extracted arc image – error'.

EXTNAME

Five extensions are used in the output-data file when ALLINONE is set:

  • The extracted spectrum image [DATA].

  • The extracted spectrum error image [ERROR].

  • The bad-pixels mask image [MASK].

  • The saturated-pixels mask image [SATMASK].

  • The data-quality mask image [QUALITY].

The dispersion-mask image header also contains a few additional p3d-specific keywords:

IMEXTR

The name of the file with the extracted spectra.

IMEXT2

The name of the second file with extracted spectra; this keyword is only set if two files were used.

IMORDER

An integer specifying which one of two files is used for lower wavelengths (0 or 1).

IMSPBIN

An integer specifying the lower pixel where the higher-wavelengths file, out of two input files, is used.

IMDAXIS

An integer specifying the dispersion axis, 1 or 2.

If several input files were specified, which were thereafter combined by p3d, before extracting the spectra, the combined image contains the following p3d-specific header keywords:

IMCMBxxx

Is set to the name(s) of the input image file(s).

P3DDFLIP

Is set to 1 if the input images were flipped on the dispersion axis.

NCOMBINE

The number of raw images used to calculate the combined image. If you use the min/max-filtered average method, the minimum and maximum values at each pixel will be discarded before calculating the average, and NCOMBINE will therefore be smaller than the number of input images.

Saved, and optionally already combined, bias-subtracted raw-data images contain only one additional header keyword:

IMTYPE

The p3d image type is set to 'p3d: bias-subtracted image'.

Input parameters:
filenameAn array of strings that specifies the names of the raw data files, which are used when creating the dispersion mask.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
masterbiasA scalar string with the name of a master-bias image file. This file is required if optimal extraction is used and none of the following four keywords is used: BIASPX, BIASPY, BIASOX, BIASOY. Note! This keyword must not be set if the prescan and the overscan regions should be used to estimate the bias level.
tracemaskA scalar string with the name of a trace-mask image file.
dispmaskinA scalar string with the name of a starting dispersion-mask image file. When this keyword is present, the corresponding file is used to create the first-guess wavelength solution; instead of matching all arc lines individually, the new dispersion-mask image can be created immediately (and faster) using this functionality.
bpmaskA scalar string with the name of a bad-pixels mask file; this could be, for example, a cosmic-ray mask file. Note! This mask is only used if optimal extraction is switched on in the user-parameter file.
crmaskA scalar string, or a two-element array, with the name of a cosmic-ray mask image file; this could be, for example a bad-pixels mask image file. Note! This mask is only used if optimal extraction is switched on in the user-parameter file. Note! CRMASK in not used if CRCLEAN is set.
biaspxSet this keyword to use the x-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis prescan region. The same array is used with all y-axis columns of bias subtracted data.
biaspySet this keyword to use the y-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis prescan region. The same array is used with all x-axis rows of bias subtracted data.
biasoxSet this keyword to use the x-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis overscan region. The same array is used with all y-axis columns of bias subtracted data.
biasoySet this keyword to use the y-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis overscan region. The same array is used with all x-axis rows of bias subtracted data.
biasconstantThis keyword can be used in two ways:

  • Set this keyword to use a constant value as a bias value; otherwise an array is used. All prescan and overscan regions specified using the keywords BIASPX, BIASPY, BIASOX, and BIASOY can be used when this keyword is set; otherwise, only one of them.

  • If this keyword is set to a double type array or to a comma-separated string with values greater than one (when called from the shell), these values are used to create a bias. The number of elements must be either the same as the number of blocks or extensions, or if two input images are specified, two times that.

savebiassubSet this keyword to write the bias-subtracted raw-data image before the spectrum extraction. The name of the output file is the same as the (combined) input file, with the added intermediate suffix '_subbias'. Also see the user parameter 'savebiassub'.
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'gratingsfile_dm'

To specify the instrument-specific gratings file.

'deadfibersfile'

To specify a dead-fibers file.

'lampsfile_dm'

To specify a lamps file.

'combineimages_dm'

To specify if input images are to be combined or not.

'compress'

Corresponds to COMPRESS.

'opfx'

Corresponds to OPFX.

'sfx'

To specify the data output file suffix.

'icsfx'

To specify the data output file intermediate suffix of combined files.

'dmsfx'

To specify the data output file intermediate suffix of the created dispersion-mask image.

'mosfx'

To specify the data output file intermediate suffix of the created bad pixels-mask image.

'mmsfx'

To specify the data output file intermediate suffix of the created saturated pixels-mask image.

'obsfx'

To specify the data output file intermediate suffix of the extracted spectrum image.

'exmonitor'

Corresponds to EXMONITOR.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'methimcombine_dm'

To specify the method used to combine input images.

'allinone'

Corresponds to ALLINONE.

'nthreads_cr'

Corresponds to CRNTHREADS.

'nthreads'

Corresponds to NTHREADS.

'writesatmask'

Corresponds to SATMASK.

'satlimit'

To define the limit of saturated values.

'noobcheck'

Corresponds to NOOBCHECK.

'crclean'

Corresponds to CRCLEAN.

'nocrmask'

To avoid using the cosmic-rays mask when images are cleaned of cosmic rays.

'biaspx'

Corresponds to BIASPX.

'biaspy'

Corresponds to BIASPY.

'biasox'

Corresponds to BIASOX.

'biasoy'

Corresponds to BIASOY.

'biasconstant'

Corresponds to BIASCONSTANT.

'savebiassub'

Corresponds to SAVEBIASSUB.

'methodextract_dm'

To setup the spectrum-extraction method.

'dbin'

To set the DBIN keyword of p3d_wavecal_dispmask_gui

'linewidth_dm'

To setup the linewidth parameter.

'refdist_dm'

To setup the refdist parameter.

'nrows_dm'

To setup the nrows parameter.

'residualcut_dm'

To setup the residualcut parameter.

'polynomialorder_dm'

To setup the polynomial order parameter.

'fwhm_tr'

To setup the fwhm parameter.

'centermethod_dm'

To setup the centermethod parameter.

'linelistlimit_dm'

To setup the linelistlimit parameter.

'vignettedfile_dm'

To setup the vignetted spectra file.

Additionally, the following user parameters are used by p3d_cr:

'laimagemethod'

Corresponds to IMAGEMETHOD.

'laimageclean'

Corresponds to IMAGECLEAN.

'ladispmedian'

Corresponds to DISPMEDIAN.

'crwriteall'

Corresponds to WRITEALL.

'nocrc'

Corresponds to NOCRC.

'sigclip'

Corresponds to SIGCLIP.

'objlim'

Corresponds to OBJLIM.

'ratlim'

Corresponds to RATLIM.

'crfwhm'

Corresponds to FWHM.

'crgausskernelsize'

Corresponds to GAUSSKERNELSIZE.

'fwhm_tr'

Used to set the width of the Gaussian profile when crfwhm is unavailable and IMAGEMETHOD is unset.

'sigfrac'

Corresponds to SIGFRAC.

'growradius'

Corresponds to GROWRADIUS.

'maxiter'

Corresponds to MAXITER.

'cleansfx'

Corresponds to CLEANSFX.

'masksfx'

Corresponds to MASKSFX.

'laplsfx'

Corresponds to LAPLSFX.

'poflsfx'

Corresponds to POFLSFX.

'siprsfx'

Corresponds to SIPRSFX.

'laprsfx'

Corresponds to LAPRSFX.

'finesfx'

Corresponds to FINESFX.

'wallsfx'

Corresponds to WALLSFX.

Note! If any of the corresponding keywords is specified, that value takes precedence.
arclinefileA scalar or one-dimensional array of strings with the names of plain-text arc-line list file(s); this keyword is passed on to the routine p3d_wavecal_set_linelist.pro.
linelistlimitA scalar decimal value that can be used to select arc lines brighter than the value defined by this parameter; LINELISTLIMIT > 0. This parameter is only considered if the arc-line list file(s) contain(s) at least two columns, where the arc-line intensity is set in the second column. Entries that do not have any intensity specified in the second column are always used. Also see the user parameter 'linelistlimit_dm'.
ofilenameXThis keyword returns the full name of the created dispersion-mask image file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
dbinThis parameter determines if the input data are re- binned on the dispersion axis (DBIN = 2 || 3), or if data are kept as they are (DBIN = 1). A good reason to rebin data are if all pixels should fit on the screen. The different values allowed are: DBIN = 1: Data are not rebinned. DBIN = 2: Data are rebinned by a factor 2. DBIN = 3: Data are rebinned by a factor 2 until all pixels fit on the screen. The default value is: 1
trackIf this keyword is set, the dispersion mask creation GUI is set up with a status line. The default value is: 1
exmonitorIf this keyword is set, a line-profiles viewer is shown after all fitting is done when using optimal extraction.
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is then appended to all output filenames. The default value is: 0
logfileThis keyword specifies the name of an optional log file when p3d_cdmask is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
colortableA scalar string or integer that specifies which color table should be loaded. If COLORTABLE is a string, the string must contain two parts separated by a comma, without any whitespace between. The first part before the comma specifies the color-table number in the file name specified after the comma; the file must be formatted as described in the documentation for MODIFYCT. As an example, COLORTABLE='25,brewer.tbl'. Components in the ColorBrewer color-table file "brewer.tbl" found in the resource directory can be specified explicitly by preceeding the string with "CB"; for example, COLORTABLE='CB25'. If COLORTABLE is an integer, the integer must be given in the range -4, -3, -2, -1, 0, ..., up to the maximum number of available tables in IDL, plus the available tables in the ColorBrewer file (resource/) "brewer.tbl". Select -4, -3, -2, or -1 to load the cubehelix, 2 * Califa, and the Sauron color tables, the other values use the respective color map as defined by LOADCT. These are the permitted values:

-4

Loads the cubehelix color table.

-3

Loads the Califa project intensity color table, as defined September 2012.

-2

Loads the Califa project velocity field color table, as defined September 2012.

-1

Loads the Sauron color table (default).

0-74 (IDL version >= 8.3)

Loads the corresponding color table with LOADCT.

0-40 (IDL version < 8.3)

Loads the corresponding color table with LOADCT.

'CB0'-'CB85'

Loads the corresponding ColorBrewer color table; the color tables are defined in the file "resource/brewer.tbl"

75-159 (IDL version >= 8.3)

Loads the ColorBrewer color table corresponding to this number after 75 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

41-125 (IDL version < 8.3)

Loads the ColorBrewer color table corresponding to this number after 41 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

'x,ctfile'

Loads the x:th entry ('x' must be an integer) in the color-table file 'ctfile.

The default value is: -1
invertSet this keyword to invert the loaded color table.
cinvSet this keyword to plot white lines on a black background instead of the default, which is to plot black lines on a white background.
bottomA scalar integer that specifies the bottom colormap index to use with the data. Note, 0 <= BOTTOM <= !d.table_size – 1.
cindexAn array of integers specifying the color indices used as reserved colors. CINDEX ideally has 7 elements, P3D_SV uses 7 reserved colors. If the number of elements is smaller, the upper range of reserved colors will use the largest index available (also see CINDV).
cindvAn array of 7 integers specifying which of the indices in CINDEX are used as the reserved colors: If CINDEX has 7 elements then: cindv = CINDEX[0:6] If CINDEX has 3 elements then: cindv = [CINDEX[0], CINDEX[1], CINDEX[2], CINDEX[2], CINDEX[2], CINDEX[2], CINDEX[2]]
ch_startThis scalar decimal value defines the color when COLORTABLE = -4. The default value is: 0.5
ch_rotsThis scalar decimal value defines the rotation in color when COLORTABLE = -4. The default value is: -1.5
ch_hueThis scalar decimal value defines the hue intensity scaling when COLORTABLE = -4. The default value is: 1.2
ch_gammaThis scalar decimal value defines the gamma correction factor when COLORTABLE = -4. The default value is: 1.0
satmaskSet this keyword to write saturated-pixels mask images to the disk. Such images indicate if any saturated pixels in the input images were used when calculating a spectrum image.
crnthreadsA scalar integer that specifies how many threads used when cleaning single input images of mic-ray hits. If this keyword is not set, READS is used instead. If nthreads was not set her, the default is to use the maximum number of ilable threads minus one, but the maximum number 16: (!cpu.hw_ncpu – 1) < 16. The default value is: max-1
nthreadsA scalar integer that specifies how many threads to use in the parallelized line profile calculations. The calculation speed scales nearly linearly with the number of threads. The default is to use the maximum number available, but no more than 16: max = !cpu.hw_ncpu < 16. The default value is: max
allinoneUnset this keyword to write the output data to separate files, instead of writing the extracted spectra to the first (DATA) extension, its error to the ERROR extension, the bad-pixels mask image to the MASK extension, the saturated-pixels mask image to the SATMASK extension, and the data-quality mask image to the QUALITY extension. The default value is: 1
crcleanSet this keyword to clean all uncombined input images of cosmic-ray hits before the extraction procedure is started; either a master-bias image or prescan and overscan regions must be used to provide the bias level to use this functionality.
nocrmask

Set this keyword to avoid using the cosmic-ray hit mask image, which is created when CRCLEAN is set, when extracting the spectra optimally. Normally it is recommended to keep this keyword unset; it is available to allow the user the ultimate choice.

The following keywords are used to configure the cosmic-ray cleaning procedure; they are only used if CRCLEAN is set; see p3d_cr and p3d_tool_crays_lapycosmic for more details:

sigclipA scalar decimal value that specifies the clipping sigma.
objlimA scalar decimal value that specifies the object limit value; this value is only used if IMAGEMETHOD is set.
ratlimA scalar decimal value that specifies the ratio limit value; this value is only used if IMAGEMETHOD is unset.
crfwhmA decimal scalar value, or a two-element array, that specifies the full width at half maximum of the gaussian kernel used to convolve the raw data; this keyword corresponds to FWHM in p3d_cr, and it is only used if IMAGEMETHOD is unset.
gausskernelsizeA scalar integer that specifies the size of the Gaussian kernel used when convolving the input image with a Gaussian kernel using the PyCosmic algorithm. This value is by default set to 2 * //q// + 1, where //q// is the maximum value of the //x// and the //y// pixel where a one-dimensional Gaussian reaches 15% of its maximum value (1.0); this condition makes use of the full-width-at-half-maximum value of FWHM. The value must be odd, and the maximum value is in any case 19. Be aware that convolution times increase strongly with the size of the Gaussian kernel.
sigfracA scalar decimal value that specifies a scale factor for pixels surrounding a cosmic-ray masked pixel; this parameter is only used if GROWRADIUS is non-zero. The default value is: 0.5
growradiusSet this keyword to grow all cosmic-ray hit pixels by a frame this many pixels wide; this is done to check if surrounding pixels are affected by cosmic-ray hits as well. The default value is 1, the values 0 and 2 are also accepted. Surrounding pixels are checked if they are cosmic-ray affected using the first criterion S' > SIGFRAC * SIGCLIP. The default value is: 1
maxiterA scalar integer value that specifies the maximum number of iterations used when identifying cosmic-ray hits in each image.
imagemethodSet this keyword to use the original algorithm of P. van Dokkum (L.A. Cosmic), which was developed for cosmic-ray detection in images. The default is otherwise to use the IFS-modified algorithm with IMAGEMETHOD unset.
imagecleanSet this keyword to use the original two-dimensional median filter when replacing the flux in masked pixels with that of the surrounding pixels. Otherwise an [11 pixel] one-dimensional median filter is used. The default value is: 1
dispmedianSet this keyword to a positive integer > 1 to subtract sampling flux along the dispersion axis. The value defines the width of the median window.
writeallSet this keyword to write all intermediate images in addition to the cleaned image and the cosmic-ray mask.
showcrguiSet this keyword to show a properties GUI of the cosmic-ray cleaning parameters before the algorithm is executed.
nocrcSet this keyword to force the use of the slower IDL-based cosmic-ray cleaning routine instead of the faster C-based routine.
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
eventwidXThis variable is set to the widget id of a button widget that will receive an event from p3d_wavecal_dispmask_gui once it finishes.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

noobcheckSet this keyword to avoid checking the OB header keyword; this only applies to the instruments that use it. To see if it applies to your instrument, check if the OBID parameter is defined in the instrument keywords file.
quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation and then exit.
Output parameters:
savedXAn integer scalar, set to 1 if a dispersion mask was created, otherwise it is set to 0.
outXUpon a successful execution, this variable contains the dispersion mask (data) when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to your input arc images, it is necessary to pass the name of the (provided) instrument-specific parameter file, a master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), a trace-mask image file, as well as any additional arguments to p3d:

files = ['arcfile_1.fits.gz', 'arcfile_2.fits.gz']
parfile = 'larr2k.prm'
opath = 'Output'
masterbias = opath + '/masterbias.fits.gz'
tracemask = opath + '/tracemask.fits.gz'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
detector = 0 ;; Required with multi-detector configurations.
p3d_cdmask, files, parfile, masterbias=masterbias, $
    tracemask=tracemask, userparfile=userparfile, opath=opath, $
    detector=detector, logfile=opath + 'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your input arc images, it is necessary to pass the name of the (provided) instrument-specific parameter file, a master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), a trace-mask image file, as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_cdmask_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
files = arcfile_1.fits.gz,arcfile_2.fits.gz
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
tracemask = ${opath}/tracemask.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_cdmask_vm.sh $files $parfile masterbias=$masterbias \
    tracemask=$tracemask userparfile=$userparfile opath=$opath \
    detector=$detector logfile=$opath/dred.log /verbose

# Using the Python tool, without using the p3d queue system:
files = arcfile_1.fits.gz,arcfile_2.fits.gz
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
tracemask = ${opath}/tracemask.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_dispatch.py --noqueue cdmask $files $parfile \
    masterbias=$masterbias tracemask=$tracemask \
    userparfile=$userparfile opath=$opath detector=$detector \
    logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cdmask_vm.sh tracemask=$tracemask" and -NOT- "p3d_cdmask_vm.sh tr=$tracemask".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the two output variable parameters OFILENAME, and OUT.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_cdmask in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'arcfile_1.fits.gz,arcfile_2.fits.gz'
parfile = 'larr2k.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
masterbias = opath + '/masterbias.fits.gz'
tracemask = opath + '/tracemask.fits.gz
commandline = files + ' ' + parfile + ' masterbias=' + masterbias +
  ' tracemask=' + tracemask ' userparfile=' + userparfile +
  ' opath=' + opath + ' detector=' + detector +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('cdmask', commandline, noqueue=True)

p3d_cexposure.pro, line 593, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_cexposure, filenames, parfile, out, dout, /saturated, badpixnlimit=, pixrange=, /compress, /savee3d, userparfile=, ofilename=, opath=, opfx=, detector=, postable=, deadfibersfile=, logfile=, loglevel=, /allinone, cmdline=, /gui, /subroutine, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, /help

Combines several spectrum-image files with extracted spectra covering the same region, either just taking the average or accounting for saturated regions in the longer exposures.

There are (at least) two reasons to take multiple exposures of the same region on the sky. The first reason is to achieve an increased signal-to-noise ratio. Several exposures are therefore taken, possibly using different exposure times, and these exposures are then combined. A second reason is that one desires complete measurements of an object that shows a large dynamic range; some parts of the object might be saturated at longer exposure times.

In the first case, shorter exposures could be combined already before spectra are extracted, but this is not recommended with longer exposures. With longer exposure times, the observing conditions are likely significantly different for the separate exposures. Important effects that affect the observations include instrument flexure and differential atmospheric refraction, which both vary with time. In the latter case, it is therefore recommended to combine the exposures after they have been extracted and corrected.

This routine handles both a straight combination of files, and accounting for saturated pixels in the raw data using different exposure times (see the keyword SATURATED). The result is, in both cases, a new merged data set.

The output data files – where a set consists of a combined spectrum image, a combined error image, a combined bad-pixels mask image, a combined saturated-pixels mask image, and a combined data-quality mask image – are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE or the user-parameter 'allinone' is unset. There is no real advantage to either approach; although, if separate files are used and the error file, the combined bad-pixels mask image file, or the combined saturated-pixels mask image, are renamed they will not be found by other tools.

Note: It is assumed that the exact same region was observed in all data files. Data are not checked to ensure this assumption.

The method of combining data – just adding them up

  • The input files are checked for consistency

    Here, the selected input files are first checked for consistency. In this case, it means that the images of all files must be of the same size, and they must all be wavelength calibrated with the same value on CRVAL and CDELT (to produce the exact same wavelength array).

    If the extracted-spectrum image error file (IMERR) is found, a merged error file is created along with the main data file.

  • Data are merged

    The separate images are added up, dividing with the number of exposures, weighting with the individual exposure time.

    Output bad-pixels mask images are used here if such files exist (these files are indicated with the keyword IMONC in the file header). Only pixels where the bad-pixels mask is smaller than or equal to a limiting number are used. The limiting number is set with the keyword BADPIXNLIMIT or the user parameter 'badpixnlimit'; its default value is 3, which means that three pixels across the cross-dispersion profile of the raw science image were masked as bad pixels or cosmic-ray affected pixels.

  • The output filename

    Based on the first input image file, a suffix is added at the end of the input filename.

The method of combining data – accounting for saturated spectra

  • Creation of a saturated-pixels mask image

    During the (object) spectrum extraction a saturated-pixels mask image is written to the disk along with the extracted spectra; if the SATMASK keyword is set. That mask indicates which spectrum elements, for all spectra and wavelength bins, included saturated pixels in the raw data.

  • The input files are checked for consistency

    Here the selected input files are first checked for consistency. In this case it means that the images of all files must be of the same size, they must all be wavelength calibrated with the same value on CRVAL and CDELT (to produce the exact same wavelength array), and the exposure time must be different in all files.

    The filename of the saturated-pixels mask image must be available in the IMSMK header keyword, and the corresponding file must also exist in the same directory as the extracted spectrum image file. Additionally, if the extracted-spectrum image error file (IMERR) is found, then a merged error file is created along with the main data file.

  • Data are merged

    The extracted spectrum image, along with its saturated-pixels mask image, that used the longest exposure time is selected as reference. The remaining images are sorted with decreasing exposure time.

    For each spectrum in the main image, all pixels p in the range p – PIXRANGE <= p <= p + PIXRANGE are masked to be replaced with the corresponding spectrum pixel value in the next shorter exposure. The pixels p are the previously masked pixels of the current spectrum in the saturated-pixels mask image. The value of the PIXRANGE parameter is changed using the keyword with the same name. Before the spectrum pixels of the shorter exposure are inserted in the main image they are multiplied with the scaling factor; exptime(reference) / exptime(shorter).

    If the image error files corresponding to the main data are found, they are used to create a modified error image file using the same masks.

  • The output filename

    Based on the input image file with the longest exposure time, adding a suffix at the end of the input filename.

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if you have not already installed p3d on your computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes two or more row-stacked spectrum (RSS) images as input; this is the default format of all extracted data of p3d.

Extracted data can, in addition to an RSS file, also be saved using the Euro3D format (use the keyword SAVEE3D or the user parameter 'savee3d'), and data of instruments using square-shaped spatial elements are easily converted to cube format using the tool p3d_rss2cube. In addition to the standard fits-file header keywords, the RSS image header contains additional p3d-specific keywords:

IMTYPE

The p3d image type is set to 'p3d: Merged exposures image'; the error image type is 'p3d: Merged exposures image – error'.

COMMENT

Multiple comments, which indicate values of the involved spectrum files, are added to the output data header.

An additional file is written when output bad-pixels mask images are used and SATURATED is unset; this file contains a final mask, which indicates pixels that did not contain any unmasked pixels at all:

IMTYPE

The p3d image type is set to 'p3d: Merged exposures image; mask'.

Input parameters:
filenamesAn array of strings that specifies the names of the extracted and calibrated spectrum-image files that will be combined.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
saturatedSet this keyword to use the second combination approach, where saturated pixels in the extracted data are accounted for by replacing data of longer exposures in affected regions with data of shorter exposures.
badpixnlimitOutput bad-pixels images contain a number for ach spectrum and wavelength bin. This number is he number of bad pixels (or cosmic-ray masked ixels) used during spectrum extraction. Only pectrum-image pixels that have a bad-pixels image ixel value less than or equal to the value in this arameter are considered in the image combination, hen SATURATED is unset. The resulting spectrum ntensity (error) is set to 0.0 (1.0) if the same ixel in all images are masked as bad pixels with value higher than BADPIXNLIMIT. Allowed values re 1 <= BADPIXNLIMIT <= 5. This keyword is only onsidered if output bad-pixels mask images are ound. The default value is: 3
pixrangeA scalar integer that specifies a range of pixels on the lower side and the upper side of the saturated pixel of each spectrum. All pixels in this range are replaced with the corresponding pixel value of the file with the next shorter exposure. The default value is: 30
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
savee3dThe output data are saved in the Euro3D (e3d) format in addition to the usual RSS format when this keyword is set.
postableAn optional scalar string with the name of a fiber positition-table file. This keyword is only required if the data were not reduced using p3d. In that case, the header keywords IMTYPE (and POSTAB) are undefined; p3d uses these keywords to determine which file to use. This keyword is only considered if SAVEE3D is set.
userparfileA scalar string with the name of a file with user- defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'compress'

corresponds to COMPRESS.

'allinone'

corresponds to ALLINONE.

'savee3d'

corresponds to SAVEE3D.

'opfx'

corresponds to OPFX.

'cexsfx'

To specify the data output file suffix.

'mmsfx'

To specify the data output file suffix.

'mosfx'

To specify the data output file suffix.

'badpixnlimit'

correponsds to BADPIXNLIMIT.

ofilenameXThis keyword returns the full name of the combined spectrum-image file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
deadfibersfileAn optional scalar string with the name of a file that lists dead, unused, and low-transmission fibers. The file must conform to the standard of the regular dead-fibers files of p3d. If this keyword is not set, the corresponding file is used as provided in the instrument-specific parameter file PARFILE.
logfileThis keyword specifies the name of an optional log file.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
allinoneUnset this keyword to write the output data to separate files, instead of writing the combined spectra to the first (DATA) extension, the error image to the ERROR extension, the combined bad-pixels mask image to the MASK extension, the combined saturated-pixels mask image to the SATMASK extension, and the combined data-quality mask image to the QUALITY extension. The default value is: 1
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine opening message will not be written to the log file.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution this variable contains the combined spectrum-image data when the routine exits.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) As a stand-alone tool from the IDL command line

In addition to the extracted and calibrated RSS images, it is necessary to pass the name of the (provided) instrument-specific parameter file as well as any additional arguments:

files = ['file1.fits.gz', 'file2.fits.gz']
parfile = 'nvimos_hr.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
p3d_cexposure, files, parfile, pixrange=40, $
    userparfile=userparfile, opath=opath, $
    logfile=opath + 'dred.log', /verbose

ii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your RSS images, it is necessary to pass the name of the (provided) instrument-specific parameter file as well as any additional arguments to p3d_cexposure. The recommended approach is to use the provided shell script p3d_cexposure_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
files = file1.fits.gz,file2.fits.gz
parfile = nvimos_hr.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_cexposure_vm.sh $files $parfile pixrange=40 \
    userparfile=$userparfile opath=$opath \
    logfile=$opath/dred.log /verbose

# Using the Python tool, without using the p3d queue system:
files = file_oextr_1.fits.gz,file_oextr_2.fits.gz
parfile = nvimos_hr.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_dispatch.py cexposure $files $parfile pixrange=40 \
    userparfile=$userparfile opath=$opath \
    logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cexposure_vm.sh userparfile=$userparfile" and -NOT- "p3d_cexposure_vm.sh user=$userparfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the three output variable parameters OFILENAME, OUT, and DOUT.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to ii), but is executed from the Python command line. Here is an example:

# Launch p3d_cexposure in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'file1.fits.gz,file2.fits.gz'
parfile = 'vimos_hr.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
commandline = files + ' ' + parfile + ' pixrange=40' +
  ' userparfile=' + userparfile + ' opath=' + opath +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('cexposure', commandline, noqueue=True)

p3d_cflatf.pro, line 1075, last changed at 2016-07-12 by christersandin (revision 4332)

p3d_cflatf, filename, parfile, out, masterbias=, tracemask=, dispmask=, bpmask=, crmask=, /biaspx, /biaspy, /biasox, /biasoy, biasconstant=, /savebiassub, transmission=, pcutlow=, pcuthigh=, userparfile=, ofilename=, opath=, opfx=, detector=, /exmonitor, /compress, /scattlightsubtract, detsepgrp=, detsepnum=, logfile=, loglevel=, /cinv, /satmask, crnthreads=, nthreads=, /allinone, /noobcheck, /crclean, /nocrmask, sigclip=, objlim=, ratlim=, crfwhm=, gausskernelsize=, sigfrac=, growradius=, maxiter=, /imagemethod, /imageclean, dispmedian=, /writeall, /showcrgui, /nocrc, cmdline=, /gui, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Creates an image with extracted spectra for the subsequent flat fielding (normalization) of extracted science spectra.

Flat-field spectra are used to normalize extracted science spectra for instrument differences in the optical path. The approach of p3d is to first extract the spectra in a flat-field (continuum) image through a call to this routine, and thereafter perform the normalization using these spectra when calling p3d_cobjex. The actual normalization is carried out by p3d_tool_flatfield (which is called from p3d_cobjex), after data have been wavelength calibrated.

All parts of the normalization procedure, including all related user-parameters, and also including the normalization performed first when calling p3d_cobjex, are described below. The best procedure depends on properties of available data and observing conditions. The default behavior also dependends on the instrument.

Please, after having read through the documentation below, also see the p3d A&A paper as well as the master user-parameter file and the p3d documentation WIKI, for help on how to setup the parameters to allow an accuracy as high as possible in the flat fielding.

The output-data files – consisting of the extracted spectrum image, the error image, the bad-pixels mask image, the saturated-pixels mask image, the data-quality mask image, and the fiber-to-fiber transmission array – are by default written to the same file in consecutive extensions. They are instead written to separate files if the keyword ALLINONE, or the user-parameter 'allinone', is unset. There is no real advantage to either approach; although, if separate files are used and the error file, the output bad-pixels mask image file, the saturated-pixels mask image file, or the fiber-to-fiber transmission array file are renamed, they will not be found by other tools.

This tool calls p3d_cr if requested to do so, which cleans single input images of cosmic-ray hits; see the keyword /CRCLEAN. The outcome of the cleaning procedure depends both on properties of the input image as well as the algorithm parameters. Different parameters might produce a different outcome. There is no guarantee whatsoever that the cleaning removes only cosmic-ray hits. Check the output images to build an own opinion regarding the quality of the outcome.

Data normalization; replace data with polynomial fits

The first task is to smooth data and replace them with a polynomial fit (see the user parameters 'smowidth_ff' and 'deg_polyfit_ff'). This smoothing is made to reduce the influence of errors in the flat-field data. These tasks can both be switched off by setting the respective parameter to zero.

The flat-field data smoothing is performed both by this tool, and when the final flat-field normalization is made through a call from the tool p3d_cobjex. However, when called from here, the smoothed data are only used to calculate a fiber-to-fiber transmission array.

This task is by default used by all instruments, with an exception for VIMOS. Data of the VIMOS instrument show wavelength-dependent fringes for several grisms and spectra, which could not be corrected for if flat-field data are first smoothed. Check the instrument parameter file for the default values on the two parameters mentioned above.

Data normalization; dividing all spectra with their mean

This task is used when flat-field data are normalized after a call to the tool p3d_cobjex; it is also activated by default, please see the user-parameter 'ff_normdisp' for more details.

To perform a wavelength-dependent normalization of flat-field data, all spectra are divided by the mean spectrum. Only spectra not listed in the dead-fibers file (if present), are used when calculating a mean spectrum.

Data normalization; making a fiber-to-fiber correction

This task is used when flat-field data are normalized after a call to the tool p3d_cobjex; it is deactivated by default since its main use is with twilight flat-field images. Please see the keyword NF2F and the user parameter 'ff_normf2f' for more details.

After all flat-field spectra have been replaced with fitted polynomials (if this functionality is used), and have been divided by a mean spectrum (if this functionality is used), p3d can normalize data anew using a normalized fiber-to-fiber transmission array. Such a transmission array is always calculated and saved (but not applied) when spectra are first extracted (through a call to p3d_cflatf). The name of the transmission-array file is written to the extracted flat-field image data header, using the keyword IMTRM. When the flat-field data are finally normalized in the subsequent call to p3d_cobjex, it is the transmission-array file recorded in the IMTRM keyword of the flat-field data header that is used.

Instead of calculating a new fiber-to-fiber transmission array, this tool allows the specification of a separate transmission-array file; using the input keyword TRANSMISSION, or the user parameter 'ff_trm_file'. Specify such a separate file when calling p3d_cflatf if the intention is to use the transmission file of extracted twilight flat-field image(s), and otherwise lamp-flat image(s). The same data are used for this purpose if the keyword IMTRM is unavailable in the extracted flat-field header.

Before the fiber-to-fiber transmission array is calculated, the lower PCUTLOW (user parameter 'ff_lowerpx') pixels and the upper PCUTHIGH (user parameter 'ff_upperpx') pixels are removed from the spectrum. These two parameters are used to exclude regions on the CCD where there is no signal. The transmission array could contain spurious values if this functionality is not used. The default values are 5% of the pixel range on the dispersion axis. As an example, with the HR-blue grism of the VIMOS instrument it is recommended to use PCUTLOW = 32.0.

Attention: The fiber-to-fiber transmission correction is mostly, and perhaps only, useful if the intention is to use a separate twilight flat-field image for the fiber-to-fiber transmission correction (please see the instructions in the next section on how to do this). It is in other cases recommended to use only a normalization by the flat-field mean spectrum, and keep NF2F unset.

Using both a twilight flat-field image and a lamp-flat image

Use the following approach if the intention is to use a twilight-flat image to make a fiber-to-fiber transmission correction and a lamp-flat image to normalize all spectra with the mean spectrum on the dispersion axis.

At first, use p3d_cflatf on the twilight flat-field images. Thereafter, use p3d_cflatf on the lamp-flat images using the NDISP keyword (or the 'ff_normdisp' user parameter; this is also the default), and remember to specify the transmission-array file from the twilight-flat flat-field creation with the TRANSMISSION keyword (or the user parameter 'ff_trm_file'). Finally, when calling the tool p3d_cobjex on the science images, set the NF2F keyword or the user parameter 'ff_normf2f' parameter.

Following these recommendations, the fiber-to-fiber transmission correction of the extracted science data is at first removed, before the provided fiber-to-fiber transmission correction is applied.

About the normalization procedure of previous revisions

In historical versions of p3d (release <= 1.1.1, revisions < 460), the normalization was made by dividing extracted flat-field spectra with the mean spectrum before the same spectra were replaced with fitted polynomials; spectra of dead and low-transmission fibers were included in the calculation of the mean spectrum. If you want to use this behavior, perhaps because you want to compare the different approaches, set the user parameter 'ff_originalnorm', which automatically disables the other normalization options.

General information on how to learn more about this routine

The normalization procedure is mentioned briefly in the p3d paper: Sandin et al. 2010, A&A, 515, 35. However, it has been significantly modified since the writing of the paper, which is why you are encouraged to learn about the current procedure by reading the documentation here (above).

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes one, or several, continuum-data raw images as input. The images are combined (by default), but p3d can also calculate a flat-field image for each file if the parameter combineimages_ff is set to "no" (in the user-parameter file). Prescan and overscan regions must not be removed in the input data, they are removed by p3d during the reduction. Input files of instruments using several readout ports must also not be combined in advance, p3d combines the images automatically.

Spectra in data are at first extracted. Thereafter, all elements <= 0.0 are masked, and a normalized transmission-correction array is calculated; with as many elements as there are spectra. Extracted flat-field spectra are then saved in an output file; the data normalization is made through a separate, and subsequent, call to the tool p3d_cobjex. The transmission-correction array is saved in a separate binary-table fits file, with one column and as many rows as there are spectra; p3d_cobjex alwayss uses the transmission-array file recorded in the IMTRM header keyword.

The output file always contains a pre-defined number of extracted spectra (with as many pixels on the cross-dispersion axis). The number of pixels on the dispersion axis is the input size (where the prescan and the overscan regions have been removed). The spectra are ordered as in the input image – this is the so-called row-stacked spectrum (RSS) format. To create a spatial map of the intensities at any wavelength, it is necessary to use a fiber position table. In addition to the standard fits-file header keywords, extracted spectrum images contain additional p3d-specific keywords:

IMRAW

The name(s) of the input flat-field image(s).

IMMBI

The name of the master-bias image file, if it is used.

IMTRC

The name of the trace-mask image file.

IMDMK

The name of the dispersion-mask image file, if it is used.

IMTRM

The name of the transmission-array file, if it is used.

IMERR

The name of the flat-field image error file. Note! This keyword is only written if ALLINONE is unset.

IMONC

The name of the output bad-pixels mask image file; this keyword is only added if either a bad-pixels mask image or a cosmic-ray mask image is specified, or calculated. Note! This keyword is only written if ALLINONE is unset.

IMSMK

The name of the saturated-pixels mask file; this keyword is only added if the SATMASK is set. The saturated-pixels mask image in addition contains the keyword SATLIMIT, which specifies the pixel value used to define the saturation limit. Note! This keyword is only written if ALLINONE is unset.

IMTYPE

The p3d image type is set to 'p3d: flat-field image'; the error image type is 'p3d: flat-field image – error'.

EXTNAME

Six extensions are used in the output-data file when ALLINONE is set:

  • The extracted spectrum image [DATA].

  • The extracted spectrum error image [ERROR].

  • The bad-pixels mask image [MASK].

  • The saturated-pixels mask image [SATMASK].

  • The data-quality mask image [QUALITY].

  • The fiber-to-fiber binary-table transmission array [TRANSMSN].

The fiber-to-fiber transmission-array header contains four additional p3d-specific keywords:

PCUTLOW

The value of PCUTLOW; this parameter is only stored if it is specified either as a keyword or using the user-parameter file.

PCUTHIGH

The value of PCUTHIGH; this parameter is only stored if it is specified either as a keyword or using the user-parameter file.

IMFFL

The name of the extracted flat-field image. Note! This keyword is only written if ALLINONE is unset.

IMTYPE

The p3d image type is set to 'p3d: transmission'.

The combined image contains the following three p3d-specific header keywords if several input files were specified that were thereafter combined by p3d, before extracting the spectra:

IMCMBxxx

These keywords are set to the names of the input image files.

P3DDFLIP

Is set to 1 if the input images were flipped on the dispersion axis.

NCOMBINE

The number of raw images used to calculate the combined image. If the min/max-filtered average method is used, the minimum and maximum values at each pixel will be discarded before calculating the average, and NCOMBINE will therefore be lower than the number of input images.

Saved, and optionally already combined, bias-subtracted raw-data images contain only one additional header keyword:

IMTYPE

The p3d image type is set to 'p3d: bias-subtracted image'.

Input parameters:
filenameAn array of strings that specifies the names of the raw flat-field data files used when creating the master flat-field image.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
masterbiasA scalar string with the name of a master-bias image file. This file is required if optimal extraction is used and none of the following four keywords is used: BIASPX, BIASPY, BIASOX, BIASOY. Note! This keyword must not be set if the prescan and the overscan regions should be used to estimate the bias level.
tracemaskA scalar string with the name of a trace-mask image file. A new trace-mask image is calculated using the (combined) input raw-data file if TRACEMASK is not present.
dispmaskA scalar string with the name of a dispersion-mask image file to use; by p3d_cobjex, when wavelength calibrating the flat-field image data.
bpmaskA scalar string with the name of a bad-pixels mask file; this could be, for example, a cosmic-ray mask file. Note! This mask is only used if optimal extraction is switched on in the user-parameter file.
crmaskA scalar string with the name of a cosmic-ray mask image file; this could be, for example, a bad-pixels mask image file. Note! This mask is only used if optimal extraction is switched on in the user-parameter file. Note! CRMASK in not used if CRCLEAN is set.
biaspxSet this keyword to use the x-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis prescan region. The same array is used with all y-axis columns of bias subtracted data.
biaspySet this keyword to use the y-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis prescan region. The same array is used with all x-axis rows of bias subtracted data.
biasoxSet this keyword to use the x-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis overscan region. The same array is used with all y-axis columns of bias subtracted data.
biasoySet this keyword to use the y-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis overscan region. The same array is used with all x-axis rows of bias subtracted data.
biasconstantThis keyword can be used in two ways:

  • Set this keyword to use a constant value as a bias value; otherwise an array is used. All prescan and overscan regions specified using the keywords BIASPX, BIASPY, BIASOX, and BIASOY can be used when this keyword is set; otherwise, only one of them.

  • If this keyword is set to a double type array or to a comma-separated string with values greater than one (when called from the shell), these values are used to create a bias. The number of elements must be the same as the number of blocks or extensions.

savebiassubSet this keyword to write the bias-subtracted raw-data image before the spectrum extraction. The name of the output file is the same as the (combined) input file, with the added intermediate suffix '_subbias'. Also see the user parameter 'savebiassub'.
transmissionA scalar string with the filename of a normalized fiber-to-fiber transmission array. The file is actually not used by this tool, but the filename is written to the header of the output flat-field image as the IMTRM keyword, and that keyword is later used by the tool p3d_cobjex to select the transmission-array file when NF2F is set. Note! A transmission file can also be specified in the user-parameter file using the parameter 'ff_trm_file'.
pcutlowA scalar floating-point value that specifies the extent of the total number of pixels on the dispersion axis removed from each spectrum at the lower (blue) end before calculating a fiber-to-fiber transmission array. This value is specified in per cent. Also see the user parameter 'ff_lowerpx'. The default value is: 5.0
pcuthighA scalar floating-point value that specifies the extent of the total number of pixels on the dispersion axis removed from each spectrum at the upper (red) end before calculating a fiber-to-fiber transmission array. This value is specified in per cent. Also see the user parameter 'ff_upperpx'. The default value is: 5.0
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'combineimages_ff'

To specify if input images are to be combined or not.

'compress'

Corresponds to COMPRESS.

'opfx'

Corresponds to OPFX.

'sfx'

To specify the data output file suffix.

'icsfx'

To specify the data output file intermediate suffix of combined files.

'ffsfx'

To specify the data output file intermediate suffix of the extracted spectrum image.

'mosfx'

To specify the data output file intermediate suffix of the created bad pixels-mask image.

'mmsfx'

To specify the data output file intermediate suffix of the created saturated pixels-mask image.

'trmsfx'

To specify the data output file intermediate suffix of the created fiber-to-fiber transmission binary-table file.

'scattlightsubtract'

Corresponds to SCATTLIGHTSUBTRACT.

'sls_writefile'

To write an image with the scattered-light component.

'methimcombine_ff'

To specify the method used to combine input images.

'allinone'

Corresponds to ALLINONE.

'nthreads_cr'

Corresponds to CRNTHREADS.

'nthreads'

Corresponds to NTHREADS.

'noobcheck'

Corresponds to NOOBCHECK.

'crclean'

Corresponds to CRCLEAN.

'savebiassub'

Corresponds to SAVEBIASSUB.

'biaspx'

Corresponds to BIASPX.

'biaspy'

Corresponds to BIASPY.

'biasox'

Corresponds to BIASOX.

'biasoy'

Corresponds to BIASOY.

'biasconstant'

Corresponds to BIASCONSTANT.

'nocrmask'

Corresponds to NOCRMASK.

'writesatmask'

Corresponds to SATMASK.

'satlimit'

To define the limit of saturated values.

'exmonitor'

Corresponds to EXMONITOR.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'smowidth_ff'

To smooth the flat-field image, see the master user-parameter file.

'deg_polyfit_ff'

To replace the flat-field image with a polynomial fit, see the master user-parameter file.

'deadfibersfile'

To specify a dead-fibers file.

'ff_originalnorm'

see the master user-parameter file.

'ff_normdisp'

see the master user-parameter file.

'ff_lowerpx'

see the master user-parameter file.

'ff_upperpx'

see the master user-parameter file.

'ff_trm_file'

Corresponds to TRANSMISSION.

'methodextract'

see p3d_misc_extract_method.

Additionally, the following parameters are used:

'laimagemethod'

Corresponds to IMAGEMETHOD.

'laimageclean'

Corresponds to IMAGECLEAN.

'ladispmedian'

Corresponds to DISPMEDIAN.

'crwriteall'

Corresponds to WRITEALL.

'nocrc'

Corresponds to NOCRC.

'sigclip'

Corresponds to SIGCLIP.

'objlim'

Corresponds to OBJLIM.

'ratlim'

Corresponds to RATLIM.

'crfwhm'

Corresponds to FWHM.

'crgausskernelsize'

Corresponds to GAUSSKERNELSIZE.

'fwhm_tr'

Used to set the width of the Gaussian profile when crfwhm is unavailable and IMAGEMETHOD is unset.

'sigfrac'

Corresponds to SIGFRAC.

'growradius'

Corresponds to GROWRADIUS.

'maxiter'

Corresponds to MAXITER.

'cleansfx'

Corresponds to CLEANSFX.

'masksfx'

Corresponds to MASKSFX.

'laplsfx'

Corresponds to LAPLSFX.

'poflsfx'

Corresponds to POFLSFX.

'siprsfx'

Corresponds to SIPRSFX.

'laprsfx'

Corresponds to LAPRSFX.

'finesfx'

Corresponds to FINESFX.

'wallsfx'

Corresponds to WALLSFX.

Note! If any of the corresponding keywords is specified, that value takes precedence.
ofilenameXThis keyword returns the full name of the created master flat-field image file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
exmonitorIf this keyword is set, a line-profiles viewer is shown after all fitting is done when using optimal extraction. The default value is: 1
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
logfileThis keyword specifies the name of an optional log file when p3d_cflatf is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
scattlightsubtractSet this keyword to first calculate and thereafter subtract a scattered-light image. Such an image is calculated by smoothing and interpolating the raw-data image, using regions on the CCD that lie far enough from nearby spectra; considering only the cross-dispersion axis. The scattered-light image is subtracted from the masterbias-subtracted raw image before (calculating the error image and) spectra are extracted. Note! This procedure only works with data where there is some "empty" space left near the edges, and preferrably also between spectrum traces on the CCD, on the cross-dispersion axis, which can be used to measure the extent of scattered light.
detsepgrpSee p3d_ctrace.
detsepnumSee p3d_ctrace.
cinvSet this keyword to plot white lines on a black background instead of the default, which is to plot black lines on a white background.
crnthreadsA scalar integer that specifies how many threads used when cleaning single input images of mic-ray hits. If this keyword is not set, READS is used instead. If nthreads was not set her, the default is to use the maximum number of ilable threads minus one, but the maximum number 16: (!cpu.hw_ncpu – 1) < 16. The default value is: max-1
nthreadsA scalar integer that specifies how many threads to use in the parallelized line profile calculations. The calculation speed scales nearly linearly with the number of threads. The default is to use the maximum number available, but no more than 16: max = !cpu.hw_ncpu < 16. The default value is: max
satmaskSet this keyword to write saturated-pixels mask images to the disk. Such images indicate if any saturated pixels in the input images were used when calculating a spectrum image.
allinoneUnset this keyword to write the output data to separate files, instead of writing the extracted spectra to the first (DATA) extension, its error to the ERROR extension, the bad-pixels mask image to the MASK extension, the saturated-pixels mask image to the SATMASK extension, the data-quality mask image to the QUALITY extension, and the fiber-to-fiber binary-table transmission array to the TRANSMSN extension. The default value is: 1
noobcheckSet this keyword to avoid checking the OB header keyword; this only applies to the instruments that use it. To see if it applies to your instrument, check if the OBID parameter is defined in the instrument keywords file.
crcleanSet this keyword to clean all uncombined input images of cosmic-ray hits before the extraction procedure is started; either a master-bias image or prescan and overscan regions must be used to provide the bias level to use this functionality.
nocrmask

Set this keyword to avoid using the cosmic-ray hit mask image, which is created when CRCLEAN is set, when extracting the spectra optimally. Normally, it is recommended to keep this keyword unset; it is available to allow the user the ultimate choice.

The following keywords are used to configure the cosmic-ray cleaning procedure; they are only used if CRCLEAN is set; see p3d_cr and p3d_tool_crays_lapycosmic for more details:

sigclipA scalar decimal value that specifies the clipping sigma.
objlimA scalar decimal value that specifies the object limit value; this value is only used if IMAGEMETHOD is set.
ratlimA scalar decimal value that specifies the ratio limit value; this value is only used if IMAGEMETHOD is unset.
crfwhmA decimal scalar value, or a two-element array, that specifies the full width at half maximum of the gaussian kernel used to convolve the raw data. This keyword corresponds to FWHM in p3d_cr. This value is only used if IMAGEMETHOD is unset.
gausskernelsizeA scalar integer that specifies the size of the Gaussian kernel used when convolving the input image with a Gaussian kernel using the PyCosmic algorithm. This value is by default set to 2 * q + 1, where q is the maximum value of the x and the y pixel where a one-dimensional Gaussian reaches 15% of its maximum value (1.0); this condition makes use of the full-width-at-half-maximum value of FWHM. The value must be odd, and the maximum value is in any case 19. Be aware that convolution times increase strongly with the size of the Gaussian kernel.
sigfracA scalar decimal value that specifies a scale factor for pixels surrounding a cosmic-ray masked pixel; this parameter is only used if GROWRADIUS is non-zero. The default value is: 0.5
growradiusSet this keyword to grow all cosmic-ray hit pixels by a frame that is this many pixels wide; this is done to check if the surrounding pixels are affected by cosmic-ray hits as well. The default value is 1, the values 0 and 2 are also accepted. Surrounding pixels are checked if they are cosmic-ray affected using the first criterion S' > SIGFRAC * SIGCLIP. The default value is: 1
maxiterA scalar integer value that specifies the maximum number of iterations used when identifying cosmic-ray hits in each image.
imagemethodSet this keyword to use the original algorithm of P. van Dokkum (L.A. Cosmic), which was developed for cosmic-ray detection in images. The default is otherwise to use the IFS-modified algorithm with IMAGEMETHOD unset.
imagecleanSet this keyword to use the original two-dimensional median filter when replacing the flux in masked pixels with that of the surrounding pixels. Otherwise an [11 pixel] one-dimensional median filter is used. The default value is: 1
dispmedianSet this keyword to a positive integer > 1 to subtract sampling flux along the dispersion axis. The value defines the width of the median window.
writeallSet this keyword to write all intermediate images in addition to the cleaned image and the cosmic-ray mask.
showcrguiSet this keyword to show a properties GUI of the cosmic-ray cleaning parameters before the algorithm is executed.
nocrcSet this keyword to force the use of the slower IDL-based cosmic-ray cleaning routine instead of the faster C-based routine.
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution this variable contains the flat-field (data) when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to your input flat-field images, it is necessary to pass the name of the (provided) instrument-specific parameter file, a master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), a trace-mask image file, and a dispersion-mask image file (for use when object data are normalized), as well as any additional arguments to p3d:

files = ['flatfile_1.fits.gz', 'flatfile_2.fits.gz']
parfile = 'larr2k.prm'
opath = 'Output'
masterbias = opath + '/masterbias.fits.gz'
tracemask  = opath + '/tracemask.fits.gz'
transmission = opath + '/transmission.fits.gz'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
detector = 0 ;; Required with multi-detector configurations.
p3d_cflatf, files, parfile, masterbias=masterbias, $
    tracemask=tracemask, transmission=transmission, $
    userparfile=userparfile, opath=opath, detector=detector, $
    logfile=opath + 'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your input flat-field images, it is necessary to pass the name of the (provided) instrument-specific parameter file, a master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), a trace-mask image file, and a dispersion-mask image file, as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_cflatf_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
tracemask  = ${opath}/tracemask.fits.gz
transmission = ${opath}/transmission.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_cflatf_vm.sh \
    flatfile_1.fits.gz,flatfile_2.fits.gz,flatfile_3.fits.gz \
    $parfile masterbias=$masterbias tracemask=$tracemask \
    transmission=$transmission userparfile=$userparfile \
    opath=$opath detector=$detector logfile=$opath/dred.log /verbose

# Using the Python tool, without using the p3d queue system:
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
tracemask  = ${opath}/tracemask.fits.gz
transmission = ${opath}/transmission.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_dispatch.py --noqueue cflatf \
    flatfile_1.fits.gz,flatfile_2.fits.gz,flatfile_3.fits.gz \
    $parfile masterbias=$masterbias tracemask=$tracemask \
    transmission=$transmission userparfile=$userparfile \
    opath=$opath detector=$detector logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cflatf_vm.sh tracemask=$tracemask" and -NOT- "p3d_cflatf_vm.sh tr=$tracemask".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the two output variable parameters OFILENAME, and OUT.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_cflatf in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'flatfile_1.fits.gz,flatfile_2.fits.gz,flatfile_3.fits.gz'
parfile = 'larr2k.prm'
opath = 'Output'
masterbias = opath + '/masterbias.fits.gz'
tracemask  = opath + '/tracemask.fits.gz'
transmission = opath + '/transmission.fits.gz'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
commandline = files + ' ' + parfile + ' masterbias=' + masterbias +
  ' tracemask=' + tracemask + ' transmission=' + transmission +
  ' userparfile=' + userparfile + ' opath=' + opath +
  ' detector=' + detector +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('cflatf', commandline, noqueue=True)

p3d_cmbias.pro, line 440, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_cmbias, filename, parfile, out, /nodialog, userparfile=, ofilename=, opath=, opfx=, detector=, /compress, nthreads=, logfile=, loglevel=, cmdline=, /gui, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Creates a master-bias image from a set of raw bias images.

To reduce CCD image data properly, it is necessary to subtract the bias, or the zero-level value, of each pixel. The subtraction can be done by using information in the prescan and the overscan regions, or using an entire image. The latter approach is handled by this tool. The former approach is used through the bias?? keywords and user parameters in the five tools: p3d_cr, p3d_ctrace, p3d_cdmask, p3d_cflatf, and p3d_cobjex.

The purpose of this tool is to combine several raw bias images into one master-bias image, where the error in each pixel is smaller than in a single image. The image combination method can be selected between the available options using the "methodimcombine" parameter in the user-parameter file (see the master user-parameter file, and also p3d_misc_imcombine.pro).

While the recommended approach is to select, and use, a large number of bias images, it is possible to select only one file; in this case the only difference between the input bias image and the output master-bias image are a few added header keywords. While it is also not recommended, p3d offers the possibility to smooth a single image (using first an average and thereafter a median box car smoothing with a kernel of 5x5 pixels). All information of individual pixels is lost if this approach is used.

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this tool, are found in the directory "!p3d_path/data/instruments/*/" (outside IDL check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this tool are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes one, or several, raw bias images as input. The images are combined by p3d. Prescan and overscan regions must not be removed in the input data, they are removed by p3d during the reduction (when using the other routines). Input files of instruments using several readout ports must also not be combined in advance, p3d combines the images automatically.

In addition to the standard fits-file header keywords the master-bias image contains additional p3d-specific keywords:

IMCMBxxx

The name(s) of the raw input bias image file(s).

P3DDFLIP

Is set to 1 if the input raw images were flipped on the dispersion axis.

NCOMBINE

The number of raw images used to calculate the combined image. If the min/max-filtered average method is used, the minimum and maximum values at each pixel will be discarded before calculating the average, and NCOMBINE will therefore be smaller than the number of input images.

Input parameters:
filenameAn array of strings that specifies the names of the raw bias image files, which are used when creating the master-bias image.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
nodialogThe file dialog that allows a single file to be smoothed is not shown when this keyword is set. Smoothing is always avoided when this keyword is set. It is possible to use the user parameter 'biasnodialog' instead. The default value is: 1
userparfileA scalar string with the name of a file with user- defined parameters. The following parameters are read here from this file:

'biasnodialog'

corresponds to NODIALOG.

'methodimcombine'

see p3d_misc_imcombine.pro.

'detsec'

see p3d_misc_detsec.pro.

ofilenameXThis keyword returns the full name of the written master-bias image file.
opathA scalar string that specifies the path, where the output image is saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is then appended to all output filenames. The default value is: 0
nthreadsA scalar integer that specifies how many threads to use in the parallelized line-profile calculations. The calculation speed scales nearly linearly with the number of threads. The default is to use the maximum number available, but no more than 16: max = (!cpu.hw_ncpu < 16L). The default value is: max
logfileThis keyword specifies the name of an optional log file when p3d_cmbias is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the more output, the higher the value). The default value is: 1
cmdlineXThis keyword is used when the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXError messages are displayed using DIALOG_MESSAGE instead of MESSAGE if this keyword is set, using this widget id as DIALOG_PARENT.
logunitXMessages are saved to the file pointed to by this logical file unit if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all; this is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines; this may be a useful mode when debugging the code.

quietSet this keyword to avoid printing the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup when this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation and then exit.
Output parameters:
outXUpon a successful execution, this variable contains the master-bias image when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to the bias images, it is necessary to pass the name of the (provided) instrument-specific parameter file as well as any additional arguments to p3d:

files = ['biasfile_1.fits.gz', 'biasfile_2.fits.gz']
parfile = 'larr2k.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
detector = 0 ;; Required with multi-detector configurations.
p3d_cmbias, files, parfile, userparfile=userparfile, opath=opath, $
    detector=detector, logfile=opath + 'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to the bias images, it is necessary to pass the name of the (provided) instrument-specific parameter file as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_cmbias_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, without using the p3d queue system:
parfile = larr2k.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_cmbias_vm.sh -s biasfile_1.fits.gz,biasfile_2.fits.gz,... \
    $parfile userparfile=$userparfile opath=$opath \
    detector=$detector logfile=$opath/dred.log /verbose

# Using the Python tool, using the p3d queue system:
parfile = larr2k.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_dispatch.py cmbias biasfile_1.fits.gz,biasfile_2.fits.gz,... \
    $parfile userparfile=$userparfile opath=$opath \
    detector=$detector logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cmbias_vm.sh userparfile=$userparfile" and -NOT- "p3d_cmbias_vm.sh user=$userparfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the two output variable parameters OFILENAME, and OUT.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_cmbias in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
bias = 'biasfile_1.fits.gz,biasfile_2.fits.gz,biasfile_3.fits.gz'
parfile = 'larr2k.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
commandline = bias + ' ' + parfile + ' userparfile=' + userparfile +
  ' opath=' + opath + ' nthreads=4' + ' detector=' + detector +
  ' logfile=dred.log loglevel=2 /verbose'
p3d_dispatch('cmbias', commandline, noqueue=True)

p3d_cobjex.pro, line 1430, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_cobjex, filename, parfile, out, bswout, group=, masterbias=, tracemask=, dispmask=, flatfield=, bpmask=, crmask=, /biaspx, /biaspy, /biasox, /biasoy, biasconstant=, /savebiassub, waveprompt=, /drizzle, resample_startpx=, /originalerrors, skyalign=, /oneskyoffset, maxskyoffset=, /savee3d, /sbsw, userparfile=, /exmonitor, /compress, ofilename=, bsw_ofilename=, opath=, opfx=, detector=, logfile=, loglevel=, /cinv, recenterval=, recenterlimval=, /nf2f, pcutlow=, pcuthigh=, /scattlightsubtract, /savefinalflat, /satmask, crnthreads=, nthreads=, /allinone, /crclean, /nocrmask, sigclip=, objlim=, ratlim=, crfwhm=, gausskernelsize=, sigfrac=, growradius=, maxiter=, /imagemethod, /imageclean, dispmedian=, /writeall, /showcrgui, /nocrc, /noobcheck, font=, cmdline=, /gui, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Creates an image with extracted, and optionally wavelength calibrated and flat-fielded, science-object spectra from a set of input images.

The purpose of this tool is to extract science-object spectra. This extraction can be made in several different ways, optionally correcting for several different instrument-specific issues. The input images are at first combined, and the spectra are thereafter extracted using the tophat extraction method, or alternatively using an optimal extraction method. A trace-mask image is always required to find and trace the spectra; such an image is created with the tool p3d_ctrace. If optimal extraction is to be performed, it is also necessary to provide a line-profiles image; which is also created by the tool p3d_ctrace. After the spectra are extracted, they are (optionally) wavelength calibrated using a dispersion mask; which was created using p3d_cdmask. The final procedure is to normalize the spectra with an extracted flat-field image; which was created using p3d_cflatf.

The output-data files – consisting of the extracted spectrum image, the extracted error image, the bad-pixels mask image, the saturated-pixels mask image, the data-quality mask image, the normalized flat-field image, and the calculated scattered-light image – are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE, or the user-parameter 'allinone', is unset. There is no real advantage to either approach; although, if separate files are used and the error file, the output bad-pixels mask image file, or the saturated-pixels mask image file are renamed, they will not be found by other tools.

The following sections describe the individual steps in more detail.

This tool calls p3d_cr if requested to do so, which cleans single input images of cosmic-ray hits; see the keyword /CRCLEAN. The outcome of the cleaning procedure depends both on properties of the input image as well as the algorithm parameters. Different parameters might produce a different outcome. There is no guarantee whatsoever that the cleaning removes only cosmic-ray hits. Check the output images to build your an opinion regarding the quality of the outcome.

Spectrum extraction

The spectrum-extraction method is selected using the 'methodextract' parameter in the user-parameter file; available options are 'tophat' and 'optimal'. The method of the optimal extraction is selected with the user parameter 'optexmethod'. A simple setup only requires these two parameters to be properly considered. Also, if you want to study the outcome, ensure that the keyword EXMONITOR is set (alternatively set the user parameter 'exmonitor'). With most instruments, there are, however, additional issues that might be necessary to consider, see the following two subsections.

Instrument flexure

Many instruments are affected by flexure, meaning that spectra move around on the detector, both during the exposure and in the time between science exposures and calibration exposures. The shift is often smaller than one pixel, but may still influence the outcome significantly (see, e.g., Sandin et al. 2010). Furthermore, the shift occurs both on the dispersion axis and on the cross-dispersion (spatial) axis. p3d allows a correction for shifts that occur between calibration and the science exposures, before spectra are extracted.

For shifts that occur on the cross-dispersion axis, see the keywords below related to, a so-called, recentering (RECENTERVAL and RECENTERLIMVAL) and the recentering section in the master user-parameter file. Also see the p3d documentation WIKI for a more thorough discussion with examples.

For shifts that occur on the dispersion axis, see the SKYALIGN, ONESKYOFFSET, and MAXSKYOFFSET keywords below, and also the sky-align section in the master user-parameter file. Also see the p3d documentation WIKI for a more thorough discussion with examples.

As an example, set the SKYALIGN keyword to either the name of a plain-text file listing the wavelengths of (accurately determined) telluric emission lines, or to an array of corresponding wavelengths. All telluric lines are fitted for all (working) spectra. The wavelength arrays of the dispersion mask are thereafter shifted with a weighted offset of the respective line fit from the pre-defined rest wavelength, see the routine p3d_wavecal_dispersion_correction for more details.

Scattered light on the detector

Some instruments are more affected than others by scattered light on the detector; this scattered light is not the same as scattered light from the primary and secondary mirrors or atmospheric scattered light. The spectrograph scattered light can be estimated and removed before spectra are extracted – if there is enough "empty" space between the spectra on the detector. To activate the scattered-light subtraction, see the SCATTLIGHTSUBTRACT keyword below as well as the corresponding section in the master user-parameter file. Also see the p3d documentation WIKI for a more thorough discussion with examples.

About the wavelength-calibration procedure

The wavelength calibration is performed using a dispersion-mask image provided with the DISPMASK keyword. For each spectrum, the n + 1 parameters of the dispersion-mask file are used to calculate an n:th order polynomial (wavelength as function of pixel). All spectra are thereafter interpolated to use the same starting wavelength (crval) and pitch (cdelt). The default behavior is to use the minimum number of pixels that contains data for all spectra (npix). The wavelength l for pixel i is then: l = crval + cdelt * (i – crpix), where crpix is always set to 1 in p3d. Set the keywords CRVAL, CDELT, and NPIX to use values other than the default values, see below. Alternatively, set the corresponding lower-case strings in the master user-parameter file ('crval', 'cdelt', and 'npix'). The values can also be set interactively using the WAVEPROMPT keyword.

For multi-detector instruments, such as VIMOS, you most likely want to use the same wavelength array for all detectors. Default grism-dependent values are provided in the four instrument-parameter files of VIMOS.

For instruments where the first (x) axis is the dispersion axis, the following header keywords are used: crval1, cdelt1, crpix1, and ctype1. The header keywords will instead be: crval2, cdelt2, crpix2, and ctype2 if the second axis (y) is the dispersion axis. The keyword crpix is not queried; it is set to 1 to ensure that it is not used. The same keywords on the cross-dispersion axis are all removed from the header if they are defined; the following World-Coordinate-System keywords are also removed if they are available in the input data header: cunit, cd1_1, cd1_2, cd2_1, and cd2_2.

p3d performs an additional wavelength-related calibration if the correction due to flexure on the dispersion axis is used, see the "Instrument flexure" subsection above.

About the flat-fielding procedure

The normalization is performed using an extracted flat-field image provided with the FLATFIELD keyword. A normalized flat-field is created in two steps: at first spectra are extracted using the tool p3d_cflatf, thereafter the extracted flat-field image is normalized by this tool, just before it is used to normalize the extracted science spectra.

After the flat-field image has been normalized, it is wavelength calibrated using the same dispersion mask as the science-object data. This behavior can be changed if a separate dispersion mask was selected when running the tool p3d_cflatf.

Please see p3d_cflatf for all details on how to setup the flat-field procedure. That tool also describes how to use both a lamp flat-field and a twilight flat-field image.

Note: To achieve a consistent flat-fielding procedure, ensure that you use the same parameters – i.e. user-parameter file – when running p3d_cflatf and p3d_cobjex.

General information on how to learn more about this routine

The spectrum extraction procedure is described in excruciating detail in the p3d paper: Sandin et al. (2010), A&A, 515, 35. The flexure-correction procedures as well as the scattered-light subtraction are described in the SPIE paper Sandin et al. (2012), SPIE, 8451, 84510F, and also see the p3d documentation WIKI (see below).

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this tool are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This tool takes one or several raw science-object images as input. The images are combined (by default), but p3d can also extract spectra for each file if the parameter 'combineimages_ob' is set to "no" (in the user-parameter file). Prescan and overscan regions must not be removed in the input data, they are removed by p3d during the reduction. Input files of instruments using several readout ports must also not be combined in advance, p3d combines the images automatically.

The output file always contains the pre-defined number of extracted spectra (with as many pixels on the cross-dispersion axis), which are optionally wavelength calibrated and normalized with a flat-field image. The number of pixels on the dispersion axis is the input size (where the prescan and the overscan regions have been removed). The spectra are ordered as in the input image – this is the so-called row-stacked spectrum (RSS) format. To create a spatial map of the intensities at any wavelength, using this RSS file, it is necessary to make use of a fiber position-table file, as the tool p3d_sv does. Please, use the fiber position-table file defined in the respective instrument-specific parameter file. Extracted data can also be saved using the Euro3D format (use the keyword SAVEE3D or the user parameter 'savee3d'), and data of instruments using square-shaped spatial elements are easily converted to cube format using the tool p3d_rss2cube. In addition to the standard fits-file header keywords, extracted spectrum images contain additional p3d-specific keywords:

IMRAW

The name(s) of the input science-object image file(s).

IMMBI

The name of the master-bias image file, if it is used.

IMTRC

The name of the trace-mask image file.

IMDMK

The name of the dispersion-mask image file, if it is used.

IMFFL

The name of the flat-field image file, if it is used.

IMERR

The name of the science-object image error file. Note! This keyword is only written if ALLINONE is unset.

IMONC

The name of the output bad-pixels mask image file; this keyword is only added if either a bad-pixels mask image or a cosmic-ray mask image is specified, or calculated. Note! This keyword is only written if ALLINONE is unset.

IMSMK

The name of the saturated-pixels mask file; this keyword is only added if the SATMASK is set. The saturated-pixels mask image in addition contains the keyword SATLIMIT, which specifies the pixel value used to define the saturation limit. Note! This keyword is only written if ALLINONE is unset.

IMTYPE

The p3d image type is set to 'p3d: extracted science-object image'; the error image type is 'p3d: extracted object image – error'.

EXTNAME

Seven extensions are used in the output-data file when ALLINONE is set:

  • The extracted spectrum image [DATA].

  • The extracted spectrum error image [ERROR].

  • The bad-pixels mask image [MASK].

  • The saturated-pixels mask image [SATMASK].

  • The data-quality mask image [QUALITY].

  • The normalized flat-field image [NORMFLAT].

  • The calculated scattered-light image [SCATTLIG].

The following two keywords are set if a fiber-flat field image is used and the NF2F keyword is set to normalize the fiber-flat field fiber-to-fiber:

PCUTLOW

The value of PCUTLOW.

PCUTHIGH

The value of PCUTHIGH.

The combined image contains the following three p3d-specific header keywords if several input files were specified that were thereafter combined by p3d, before extracting the spectra:

IMCMBxxx

These keywords are set to the names of the input image files.

P3DDFLIP

Is set to 1 if the input images were flipped on the dispersion axis.

NCOMBINE

The number of raw images used to calculate the combined image. If the min/max-filtered average method is used, the minimum and maximum values at each pixel will be discarded before calculating the average, and NCOMBINE will therefore be lower than the number of input images.

Saved, and optionally already combined, bias-subtracted raw-data images contain only one additional header keyword:

IMTYPE

The p3d image type is set to 'p3d: bias-subtracted image'.

Input parameters:
filenameAn array of strings that specifies the names of the raw-data image files used when creating the extracted science object file.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
groupAn optional integer array with as many elements as the FILENAME argument. The elements of FILENAME with the same group number are combined if this keyword is set. Note! This keyword can only be used when p3d_cobjex is called as a stand-alone tool, i.e., it is inaccessible using the p3d GUI.
masterbiasA scalar string with the name of a master-bias image file. This file is required if optimal extraction is used and none of the following four keywords is used: BIASPX, BIASPY, BIASOX, BIASOY. Note! This keyword must not be set if the prescan and the overscan regions should be used to
tracemaskA scalar string with the name of a trace-mask image file.
dispmaskA scalar string with the name of a dispersion-mask image file. Note! The flat-field image header must contain the entry IMDMK if it is to be calibrated with a separate dispersion-mask image. The filename stored in IMDMK is then used when applying the dispersion correction to the fiber-flat image.
flatfieldA scalar string with the name of an extracted flat-field image.
bpmaskA scalar string with the name of a bad-pixels mask file; this could be, for example, a cosmic-ray mask file. Note! This mask is only used if optimal extraction is switched on in the user-parameter file.
crmaskA scalar string with the name of a cosmic-ray mask image file; this could be, for example, a bad-pixels mask image file. Note! This mask is only used if optimal extraction is switched on in the user-parameter file. Note! CRMASK in not used if CRCLEAN is set.
biaspxSet this keyword to use the x-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis prescan region. The same array is used with all y-axis columns of bias subtracted data.
biaspySet this keyword to use the y-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis prescan region. The same array is used with all x-axis rows of bias subtracted data.
biasoxSet this keyword to use the x-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis overscan region. The same array is used with all y-axis columns of bias subtracted data.
biasoySet this keyword to use the y-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis overscan region. The same array is used with all x-axis rows of bias subtracted data.
biasconstantThis keyword can be used in two ways:

  • Set this keyword to use a constant value as a bias value; otherwise an array is used. All prescan and overscan regions specified using the keywords BIASPX, BIASPY, BIASOX, and BIASOY can be used when this keyword is set; otherwise, only one of them.

  • If this keyword is set to a double type array or to a comma-separated string with values greater than one (when called from the shell), these values are used to create a bias. The number of elements must be the same as the number of blocks or extensions.

savebiassubSet this keyword to write the bias-subtracted raw-data image before spectrum extraction. The name of the output file is the same as the (combined) input file, with the added intermediate suffix '_subbias'. Also see the user parameter 'savebiassub'.
wavepromptThis keyword determines if the the user should be queried for changing CRVAL, CDELT, and NPIX using an interactive interface (they can all be set with the user-parameter file). The default value is: 0
drizzleUnset this keyword to use a linear interpolation instead of the default one-dimensional drizzling algorithm when resampling data from pixels to wavelengths (p3d_wavecal_dispersion_correction). The default value is: 1
resample_startpxA scalar value that specifies the starting pixel in the calculation of an output wavelength array when resampling pixel values to wavelength values. The keyword can be set to 'first', 'middle' [default], or 'last', to use of the first, the middle, or the last output pixel, respectively. Alternatively, the keyword can be set to an integer within the same range. Negative values, and values higher than the maximum number of pixels, short of one, are truncated. The default value is: 'middle'
originalerrorsSet this keyword to use the original, less correct, version of the error calculation applied during the resampling (with the linear interpolation scheme).
skyalignA scalar string that contains the name of a line-list file with sky-emission lines, or an array of floating-point values with the wavelengths of sky-emission lines (the unit is Angstrom [Å]); this keyword is used when re-aligning the wavelength array in the dispersion-mask image due to effects of flexure (p3d_wavecal_dispersion_correction).
oneskyoffsetSet this keyword to use one median offset value on all wavelength arrays in the dispersion mask. Otherwise, each wavelength array in the dispersion mask is offset individually. Additionally, when this parameter is unset, the offset at each wavelength bin is weighted with the wavelength extent of the pixel relative to the corresponding extent at the selected sky-emission line (since uncalibrated wavelength arrays generally deviate from a linear function).
maxskyoffsetA scalar floating-point value that specifies the imum allowed offset of sky-emission lines, from wavelength provided in the dispersion-mask ge; the unit is Angstrom [Å]. See _wavecal_dispersion_correction for more details. The default value is: 2.0
savee3dThe output data are saved in the Euro3D (e3d) format in addition to the usual RSS format when this keyword is set.
sbsw(n/a) This keyword returns the beam switch setting; sbsw=1: beam switch data == true.
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'combineimages_ob'

To specify if input images are to be combined or not.

'compress'

Corresponds to COMPRESS.

'savee3d'

Corresponds to SAVEE3D.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'opfx'

Corresponds to OPFX.

'sfx'

To specify the data output file suffix.

'icsfx'

To specify the data output file intermediate suffix of combined files.

'obsfx'

To specify the data output file intermediate suffix of the extracted spectrum image.

'mosfx'

To specify the data output file intermediate suffix of the created bad pixels-mask image.

'mmsfx'

To specify the data output file intermediate suffix of the created saturated pixels-mask image.

'scattlightsubtract'

Corresponds to SCATTLIGHTSUBTRACT.

'sls_writefile'

To write an image with the scattered-light component.

'methimcombine_ob'

To specify the method that is used to combine input images.

'allinone'

Corresponds to ALLINONE.

'writesatmask'

Corresponds to SATMASK.

'ff_save_final_image'

Corresponds to SAVEFINALFLAT.

'satlimit'

To define the limit of saturated values.

'nthreads_cr'

Corresponds to CRNTHREADS.

'nthreads'

Corresponds to NTHREADS.

'drizzle'

Corresponds to DRIZZLE.

'resample_startpx'

Corresponds to RESAMPLE_STARTPX.

'originalerrors'

Corresponds to ORIGINALERRORS.

'deadfibersfile'

To specify a dead-fibers file.

'crclean'

Corresponds to CRCLEAN.

'savebiassub'

Corresponds to SAVEBIASSUB.

'biaspx'

Corresponds to BIASPX.

'biaspy'

Corresponds to BIASPY.

'biasox'

Corresponds to BIASOX.

'biasoy'

Corresponds to BIASOY.

'biasconstant'

Corresponds to BIASCONSTANT.

'nocrmask'

Corresponds to NOCRMASK.

'skyalign'

Corresponds to SKYALIGN.

'oneskyoffset'

Corresponds to ONESKYOFFSET.

'maxskyoffset'

Corresponds to MAXSKYOFFSET.

'noobcheck'

Corresponds to NOOBCHECK.

'exmonitor'

Corresponds to EXMONITOR.

'recenterprofiles'

see RECENTERVAL.

'gapfile'

To read the spectrum gaps file; used when recentering profiles.

'methodextract'

see p3d_misc_extract_method.

'smowidth_ff'

To smooth the flat-field image, see the master user-parameter file.

'deg_polyfit_ff'

To replace the flat-field image with a polynomial fit, see the master user-parameter file.

'ff_originalnorm'

see the master user-parameter file.

'ff_normf2f'

To use a fiber-to-fiber transmission array, see the master user-parameter file.

'ff_normdisp'

see the master user-parameter file.

'ff_lowerpx'

see the master user-parameter file.

'ff_upperpx'

see the master user-parameter file.

'crval'

To set the wavelength of the bluemost pixel. Can also be suffixed with an underscore followed by the grating id keyword (see the parameter GRAT_ID in the instrument-keywords file).

'cdelt'

To set the pitch. Can also be suffixed with an underscore followed by the grating id keyword (see the parameter GRAT_ID in the instrument-keywords file).

'npix'

To set the number of pixels on the dispersion axis in the spectrum. Can also be suffixed with an underscore followed by the grating id keyword (see the parameter GRAT_ID in the instrument-keywords file).

Additionally, the following parameters are used:

'laimagemethod'

Corresponds to IMAGEMETHOD.

'laimageclean'

Corresponds to IMAGECLEAN.

'ladispmedian'

Corresponds to DISPMEDIAN.

'crwriteall'

Corresponds to WRITEALL.

'nocrc'

Corresponds to NOCRC.

'sigclip'

Corresponds to SIGCLIP.

'objlim'

Corresponds to OBJLIM.

'ratlim'

Corresponds to RATLIM.

'crfwhm'

Corresponds to FWHM.

'crgausskernelsize'

Corresponds to GAUSSKERNELSIZE.

'fwhm_tr'

Used to set the width of the Gaussian profile when crfwhm is unavailable and IMAGEMETHOD is unset.

'sigfrac'

Corresponds to SIGFRAC.

'growradius'

Corresponds to GROWRADIUS.

'maxiter'

Corresponds to MAXITER.

'cleansfx'

Corresponds to CLEANSFX.

'masksfx'

Corresponds to MASKSFX.

'laplsfx'

Corresponds to LAPLSFX.

'poflsfx'

Corresponds to POFLSFX.

'siprsfx'

Corresponds to SIPRSFX.

'laprsfx'

Corresponds to LAPRSFX.

'finesfx'

Corresponds to FINESFX.

'wallsfx'

Corresponds to WALLSFX.

Note! If any of the corresponding keywords is specified, that value takes precedence.
ofilenameXThis keyword returns the full name of the created extracted science object file.
exmonitorIf this keyword is set, a line-profiles viewer is shown after all fitting is done when using optimal extraction. The default value is: 1
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
bsw_ofilenameXn/a.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
logfileThis keyword specifies the name of an optional log file when p3d_cobjex is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
cinvSet this keyword to plot white lines on a black background instead of the default, which is to plot black lines on a white background.
recentervalSet this keyword or RECENTERLIMVAL to activate the use of profile recentering on science images (recenterprofiles). This keyword configures several of the recentering options. If RECENTERVAL is a:

string

the telluric line mode is used ('recenterusetelluric'). The default file is used when the keyword is set to 'telluric' while any other string is interpreted as a filename ('recentertellines').

value, -2.0 <= value <= 2.0

the value is used as a preset offset (recenteroffset).

other values

are used as a median-filter width with data; the used width is the integer of the absolute value.

recenterlimvalSet this keyword or RECENTERVAL to activate the use of profile recentering on science images ('recenterprofiles'). This keyword allows to filter out all spectra that have a lower maximum count than specified with this value [ADU], before calculating an offset using the median-filter method (when it is used). Note! This keyword is only considered if the user parameter 'recenterdpixpos' contains a single element.
nf2fSpectra are normalized fiber-to-fiber with a one-dimensional array if this keyword is set. The array is selected using the filename defined in the IMTRM header keyword of the flat-field image file (FLATFIELD). If the header keyword IMTRM is unavailable, and NF2F is set, data are normalized using a calculated transmission array of the flat-field image. In this case, data must not be normalized with the average spectrum (see the user parameter 'ff_normdisp'). For example, in the standard setup, if TRANSMISSION was set to the transmission-array file of a twilight flat while running p3d_cflatf, data are corrected for fiber-to-fiber transmission using that twilight-flat data here (see p3d_tool_flatfield), while the spectrum normalization is done using the flat-field image selected using FLATFIELD.
pcutlowA scalar floating-point value that specifies the extent of the total number of pixels on the dispersion axis removed from each spectrum at the lower (blue) end before calculating a fiber-to-fiber transmission array. This value is specified in per cent. Also see the user parameter 'ff_lowerpx'; This parameter os only used if NF2F is set. The default value is: 5.0
pcuthighA scalar floating-point value that specifies the extent of the total number of pixels on the dispersion axis removed from each spectrum at the upper (red) end before calculating a fiber-to-fiber transmission array. This value is specified in per cent. Also see the user parameter 'ff_upperpx'; this parameter is only used if NF2F is set. The default value is: 5.0
scattlightsubtractSet this keyword to first calculate and thereafter subtract a scattered-light image. Such an image is calculated by smoothing and interpolating the raw-data image, using regions on the CCD that lie far enough from nearby spectra; considering only the cross-dispersion axis. The scattered-light image is subtracted from the masterbias-subtracted raw image before (calculating the error image and) spectra are extracted. Note! This procedure only works with data where there is some "empty" space left near the edges, and preferrably also between spectrum traces on the CCD, on the cross-dispersion axis, which can be used to measure the extent of scattered light.
savefinalflatSet this keyword to write the final normalized flat-field image to the disk; this is otherwise not done. Also see the user parameter 'ff_save_final_image'.
satmaskSet this keyword to write saturated-pixels mask images to the disk. Such images indicate if any saturated pixels in the input images were used when calculating a spectrum image.
crnthreadsA scalar integer that specifies how many threads used when cleaning single input images of mic-ray hits. If this keyword is not set, READS is used instead. If nthreads was not set her, the default is to use the maximum number of ilable threads minus one, but the maximum number 16: (!cpu.hw_ncpu – 1) < 16. The default value is: max-1
nthreadsA scalar integer that specifies how many threads to use in the parallelized line profile calculations. The calculation speed scales nearly linearly with the number of threads. The default is to use the maximum number available, but no more than 16: max = !cpu.hw_ncpu < 16. The default value is: max
allinone

Unset this keyword to write the output data to separate files, instead of writing the extracted spectra to the first (DATA) extension, its error to the ERROR extension, the bad-pixels mask image to the MASK extension, the saturated-pixels mask image to the SATMASK extension, the data-quality mask image to the QUALITY extension, the normalized extracted flat-field image to the NORMFLAT extension, and the calculated scattered-light image to the SCATTLIG extension.

The normalized flat-field image is only written if the keyword SAVEFINALFLAT or the user parameter 'ff_save_final_image' are set. The scattered-light image is only written if the user parameter 'sls_writefile' is set.

The default value is: 1
noobcheckSet this keyword to avoid checking the OB header keyword; this only applies to the instruments that use it. To see if it applies to your instrument, check if the OBID parameter is defined in the instrument keywords file.
crcleanSet this keyword to clean all uncombined input images of cosmic-ray hits before the extraction procedure is started; either a master-bias image or prescan and overscan regions must be used to provide the bias level to use this functionality.
nocrmask

Set this keyword to avoid using the cosmic-ray hit mask image, which is created when CRCLEAN is set, when extracting the spectra optimally. Normally, it is recommended to keep this keyword unset; it is available to allow the user the ultimate choice.

The following keywords are used to configure the cosmic-ray cleaning procedure; they are only used if CRCLEAN is set; see p3d_cr and p3d_tool_crays_lapycosmic for more details:

sigclipA scalar decimal value that specifies the clipping sigma.
objlimA scalar decimal value that specifies the object limit value; this value is only used if IMAGEMETHOD is set.
ratlimA scalar decimal value that specifies the ratio limit value; this value is only used if IMAGEMETHOD is unset.
crfwhmA decimal scalar value, or a two-element array, that specifies the full width at half maximum of the gaussian kernel used to convolve the raw data. This keyword corresponds to FWHM in p3d_cr. This value is only used if IMAGEMETHOD is unset.
gausskernelsizeA scalar integer that specifies the size of the Gaussian kernel used when convolving the input image with a Gaussian kernel using the PyCosmic algorithm. This value is by default set to 2 * q + 1, where q is the maximum value of the x and the y pixel where a one-dimensional Gaussian reaches 15% of its maximum value (1.0); this condition makes use of the full-width-at-half-maximum value of FWHM. The value must be odd, and the maximum value is in any case 19. Be aware that convolution times increase strongly with the size of the Gaussian kernel.
sigfracA scalar decimal value that specifies a scale factor for pixels surrounding a cosmic-ray masked pixel; this parameter is only used if GROWRADIUS is non-zero. The default value is: 0.5
growradiusSet this keyword to grow all cosmic-ray hit pixels by a frame that is this many pixels wide; this is done to check if the surrounding pixels are affected by cosmic-ray hits as well. The default value is 1, the values 0 and 2 are also accepted. Surrounding pixels are checked if they are cosmic-ray affected using the first criterion S' > SIGFRAC * SIGCLIP. The default value is: 1
maxiterA scalar integer value that specifies the maximum number of iterationsd used when identifying cosmic-ray hits in each image.
imagemethodSet this keyword to use the original algorithm of P. van Dokkum (L.A. Cosmic), which was developed for cosmic-ray detection in images. The default is otherwise to use the IFS-modified algorithm with IMAGEMETHOD unset.
imagecleanSet this keyword to use the original [5 * 5 px²] two-dimensional median filter when replacing the flux in masked pixels with that of the surrounding pixels. An [11 px] one-dimensional median filter is used otherwise. The default value is: 1
dispmedianSet this keyword to a positive integer > 1 to subtract sampling flux along the dispersion axis. The value defines the width of the median window.
writeallSet this keyword to write all intermediate images in addition to the cleaned image and the cosmic-ray mask.
showcrguiSet this keyword to show a properties GUI of the cosmic-ray cleaning parameters before the algorithm is executed.
nocrcSet this keyword to force the use of the slower IDL-based cosmic-ray cleaning routine instead of the faster C-based routine.
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution this variable contains the extracted data when the routine exits.
bswoutXUpon a successful execution this variable contains the extracted beam switch data when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to your input science-object images, it is necessary to pass the name of the (provided) instrument-specific parameter file, a master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), a trace-mask image file, a dispersion-mask image file (if used), and an extracted flat-field image file (if used), as well as any additional arguments to p3d:

files = ['objectfile_1.fits.gz', 'objectfile_2.fits.gz']
parfile = 'larr2k.prm'
opath = 'Output'
masterbias  = opath + '/masterbias.fits.gz'
tracemask   = opath + '/tracemask.fits.gz'
flatfield   = opath + '/flatfield.fits.gz'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
detector = 0 ;; Required with multi-detector configurations.
p3d_cobjex, files, parfile, masterbias=masterbias, $
    tracemask=tracemask, flatfield=flatfield, $
    userparfile=userparfile, opath=opath, detector=detector, $
    logfile=opath + 'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your input science-object images, it is necessary to pass the name of the (provided) instrument-specific parameter file, a master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), a trace-mask image file, a dispersion-mask image file (if you use one), a flat-field image file (if you use one), as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_cobjex_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
tracemask  = ${opath}/tracemask.fits.gz
flatfield  = ${opath}/flatfield.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_cobjex_vm.sh \
    objectfile_1.fits.gz,objectfile_2.fits.gz,objectfile_3.fits.gz \
    $parfile masterbias=$masterbias tracemask=$tracemask \
    flatfield=$flatfield userparfile=$userparfile \
    opath=$opath detector=$detector logfile=$opath/dred.log \
    /verbose

# Using the Python tool, without using the p3d queue system:
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
tracemask  = ${opath}/tracemask.fits.gz
flatfield  = ${opath}/flatfield.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_dispatch.py --noqueue cobjex \
    objectfile_1.fits.gz,objectfile_2.fits.gz,objectfile_3.fits.gz \
    $parfile masterbias=$masterbias tracemask=$tracemask \
    flatfield=$flatfield userparfile=$userparfile \
    opath=$opath detector=$detector logfile=$opath/dred.log \
    /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cobjex_vm.sh tracemask=$tracemask" and -NOT- "p3d_cobjex_vm.sh tr=$tracemask".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the three output variable parameters OFILENAME, OUT, and BSWOUT.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_cobjex in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
parfile = 'larr2k.prm'
opath = 'Output'
masterbias = opath + '/masterbias.fits.gz'
tracemask  = opath + '/tracemask.fits.gz'
flatfield  = opath + '/flatfield.fits.gz'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
files = 'objectfile_1.fits.gz,objectfile_2.fits.gz,' +
        'objectfile_3.fits.gz'
commandline = files + ' ' + parfile + ' masterbias=' + masterbias +
  ' tracemask=' + tracemask + ' flatfield=' + flatfield +
  ' userparfile=' + userparfile + ' opath=' + opath +
  ' detector=' + detector +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('cobjex', commandline, noqueue=True)

p3d_cr.pro, line 1297, last changed at 2017-03-10 by christersandin (revision 4447)

p3d_cr, image, parfile, oimage, masterbias=, /biasadd, /biaspx, /biaspy, /biasox, /biasoy, biasconstant=, prescanx=, prescany=, overscanx=, overscany=, sigclip=, objlim=, ratlim=, fwhm=, gausskernelsize=, sigfrac=, growradius=, maxiter=, /imagemethod, /imageclean, dispmedian=, /writeall, sfx=, cleansfx=, masksfx=, laplsfx=, poflsfx=, siprsfx=, laprsfx=, finesfx=, wallsfx=, /compress, nthreads=, /showgui, /noC, filenames=, opath=, ron=, gain=, detsec=, userparfile=, detector=, /allinone, daxis=, logfile=, loglevel=, font=, cmdline=, /gui, /subroutine, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Cleans one or several raw spectrum images of fiber-fed Integral-Field Spectroscopy instruments of cosmic-ray hits.

This routine is a wrapper for the p3d IFS-capable version of the PyCosmic (Husemann et al. 2012, hereafter HKS12) and the L.A. Cosmic (van Dokkum 2001, hereafter vD01) algorithms. These routines clean single (uncombined) raw-data images of cosmic-ray hits. Requirements include a master-bias image, or (at least) one of the prescan and the overscan parameters, which is subtracted from the data before the cleaning algorithm is applied. All algorithm parameters are collected here, and data are also prepared and saved here. The algorithms themselves are placed in p3d_tool_crays_lapycosmic.

The most complete resource regarding the two supported algorithms – including a detailed comparison – is HKS12; this reference also compares the, so-called, DCR algorithm of Pych (2004). The default algorithm is PyCosmic; set the keyword IMAGEMETHOD or the user parameter 'laimagemethod' to use L.A. Cosmic instead. As is illustrated by HKS12, undersampled data are better cleaned with the L.A. Cosmic algorithm, in other cases it appears that PyCosmic does a better job.

To reproduce the results of HKS12 when using PyCosmic, it is necessary to set the grow-radius keyword GROWRADIUS or the user parameter 'growradius' to 0; the default value is 1. (This option is for data-reduction connaisseurs, as results appear to be very similar when using a non-zero grow radius.) With a zero grow radius, the outcome should be very similar, but not identical; in particular, single pixels with the lowest possible signal-to-noise are masked differently (for example where the calculated value is just above SIGCLIP). An algorithm difference between PyCosmic as implemented here and the Python-based routine, which might be the cause of differences between masks, is that the former (latter) performs one two-dimensional (two consecutive one-dimensional) convolutions with a Gaussian kernel.

Regardless of the selected algorithm, it is important to check that the cosmic-ray hit cleaning procedure only removes cosmic-ray hits, and does not affect telluric lines or object emission lines. It is often enough to compare the cleaned image with the cosmic-ray mask and the raw image. The keyword WRITEALL can be used to save all intermediate products, which allow a more detailed study of the quality of the cleaning procedure.

The output-data files consist of the cosmic-ray cleaned image and the cosmic-ray mask image. These are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE or the user-parameter 'allinone' is unset. There is no real advantage to either approach; although, if separate files are used and the cosmic-ray mask image file is renamed it will not be found by the other p3d tools.

This tool can be used in two ways:

  • Run this tool directly on raw-data images.

    Note: Regarding the PMAS 4kx4k CCD or other (future) multi-block instruments: the routine must be used with uncombined data; this also applies to the master-bias image.

    Note: This approach is (currently) unavailable with instruments that store data of different detectors in different extensions of the fits-file image. This applies to, for example, GMOS-S and GMOS-N.

  • Set the CRCLEAN keyword when executing the tools p3d_ctrace, p3d_cdmask, p3d_cflatf, and p3d_cobjex. The selected algorithm is then applied to the (combined [if required]) image (format) that p3d works with. All algorithm-critical keywords of this tool are also available from the respective calling tool.

    Note: If several images are fed to the respective tool, they will at first be combined using, for example, a median or an average; unless the GROUP keyword is used to prevent this (see p3d_cobjex). A resulting combined image will not be cleaned of cosmic-ray hits, even if CRCLEAN is set.

Warning: It may be important to check the quality of the cleaned outcome. All output images that are created through the use of the WRITEALL keyword, require a lot of disk space; probably hundreds of megabytes and perhaps even gigabytes.

Speed tests

Here are results of speed tests on some VIMOS data, using 1-8 threads on a 4-core machine with hyperthreading. In this case, the PyCosmic algorithm was used; speeds ought to be slightly longer with the L.A. Cosmic algorithm. All values in columns 2-6 are in seconds. The speedup //Sp// = //T1// / //Tp//, and the efficiency //Ep// = //Sp// / //p//, where //p// is the number of threads. Values are also compared to the IDL routine, //Ip// = //T//(IDL) / //Tp//.

Quadrant: 1/4 2/4 3/4 4/4 mean Sp Ep Ip IDL 1 th. 58.069 57.724 57.847 57.849 57.872 1.26 1.26 1.00 C 1 th. 73.032 72.886 72.243 72.624 72.696 1.00 1.00 0.80 C 2 th. 39.693 39.824 39.334 39.404 39.564 1.84 0.92 1.46 C 3 th. 27.804 27.896 27.722 27.783 27.801 2.61 0.87 2.08 C 4 th. 23.882 24.138 23.776 24.142 23.985 3.03 0.76 2.41 C 5 th. 24.410 24.165 24.146 24.033 24.189 3.01 0.60 2.39 C 6 th. 21.315 21.309 21.395 21.380 21.350 3.40 0.57 2.71 C 7 th. 18.987 18.647 18.796 18.891 18.830 3.86 0.55 3.07 C 8 th. 24.556 24.279 24.272 24.152 24.315 2.99 0.37 2.38

The table shows a speedup far from linear. In this case, it seems to be sufficient to use 4 threads if the number of threads is the limiting factor, and 7 threads if the execution time of the highest priority.

References

van Dokkum, P.G. 2001, PASP, 113, 1420, "Cosmic-Ray Rejection by Laplacian Edge Detection" (vD01)

Pych, W. 2004, PASP, 116, 148, "A Fast Algorithm for Cosmic-Ray Removal from Single Images"

Husemann, B., Kamann, S., Sandin, C., Sánchez, S., Mast, D. and Garcia-Benito, R. 2012, A&A, 545, A137, "PyCosmic: a robust method to detect cosmics in CALIFA and other fiber-fed integral-field spectroscopy datasets" (HKS12)

Links

The PyCosmic code of HKS12 can be found here: http://pycosmic.sf.net

The original L.A. Cosmic code(s) of vD01 can be found here: http://www.astro.yale.edu/dokkum/lacosmic/

The DCR code of Pych (2004) can be downloaded here: http://users.camk.edu.pl/pych/DCR/index.html

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This tool takes one, or several, raw images, or (block-)combined images spectrum images as input, along with a matching master-bias image.

In addition to the standard fits-file header keywords, the resulting image contains additional p3d-specific keywords:

IMTYPE

The p3d image type is set to 'p3d: cosmic-ray cleaned image'.

COMMENT

Multiple comments, which indicate values of used parameters, are added to the output data header.

Input parameters:
imageThis variable contains the images to be cleaned of cosmic-ray hits. The value can be specified in different ways:

  • A two-dimensional array (scalar image) or a three-dimensional array (array of images), where the array is counted in the first dimension. The array must be of floating-point type.

  • A scalar string or an array of strings that contain(s) the FITS-filename(s) of the images; the image files can be compressed using gzip or compress – this must be reflected in the filename suffix, i.e., each file must, if it is compressed, end with ".gz" or ".Z".

parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
masterbiasA scalar string with the name of a master-bias image file. This file is required if optimal extraction is used and none of the following four keywords is used: BIASPX, BIASPY, BIASOX, or BIASOY. The value can be specified in two different ways:

  • as a two-dimensional array (scalar image) of floating-point type.

  • as a scalar string that contains the name of the master-bias image file.

Note! A master-bias image can of course be neither subtracted nor added if it is not provided. Note! This keyword must not be set if the prescan and the overscan regions should be used to estimate the bias level.
biasaddSet this keyword to add the bias image to the cleaned bias-subtracted image before saving it. The default value is: 1
biaspxSet this keyword to use the x-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y-axis in the x-axis prescan region. The same array is used with all y-axis columns of data that are bias subtracted.
biaspySet this keyword to use the y-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x-axis in the y-axis prescan region. The same array is used with all x-axis rows of data that are bias subtracted.
biasoxSet this keyword to use the x-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y-axis in the x-axis overscan region. The same array is used with all y-axis columns the data that are bias subtracted.
biasoySet this keyword to use the y-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x-axis in the y-axis overscan region. The same array is used with all x-axis rows of data that are bias subtracted.
biasconstantThis keyword can be used in two ways:

  • Set this keyword to use a constant value as a bias value; otherwise an array is used. All prescan and overscan regions specified using the keywords BIASPX, BIASPY, BIASOX, and BIASOY can be used when this keyword is set; otherwise, only one of them.

  • If this keyword is set to a double type array or to a comma-separated string with values greater than one (when called from the shell), these values are used to create a bias. The number of elements must be the same as the number of blocks or extensions.

prescanxXThis keyword is used to define x-axis prescan regions when IMAGE is a floating-point-type-value array. In this case, and if this keyword is used through BIASPX, the keyword contains the x-axis prescan array as tags 'd1'–'dn', where //n// is short for the number of blocks. The array of the tag 'use' is set to 1 whenever data are present, otherwise 0.
prescanyXThis keyword is used to define y-axis prescan regions when IMAGE is a floating-point-type-value array. In this case, and if this keyword is used through BIASPY, the keyword contains the y-axis prescan array as tags 'd1'–'dn', where //n// is short for the number of blocks. The array of the tag 'use' is set to 1 whenever data are present, otherwise 0.
overscanxXThis keyword is used to define x-axis overscan regions when IMAGE is a floating-point-type-value array. In this case, and if this keyword is used through BIASOX, the keyword contains the x-axis overscan array as tags 'd1'–'dn', where //n// is short for the number of blocks. The array of the tag 'use' is set to 1 whenever data are present, otherwise 0.
overscanyXThis keyword is used to define y-axis overscan regions when IMAGE is a floating-point-type-value array. In this case, and if this keyword is used through BIASOY, the keyword contains the y-axis overscan array as tags 'd1'–'dn', where //n// is short for the number of blocks. The array of the tag 'use' is set to 1 whenever data are present, otherwise 0.
sigclipA scalar floating-point type value that specifies the clipping sigma, see p3d_tool_crays_lapycosmic. The default value is: 5.0
objlimA scalar floating-point type value that specifies the object limit value, see p3d_tool_crays_lapycosmic; this value is only used if IMAGEMETHOD is set. The default value is: 2.0
ratlimA scalar floating-point type value that specifies the ratio limit value, see p3d_tool_crays_lapycosmic.pro; this value is only used if IMAGEMETHOD is unset. The default value is: 1.2
fwhmA floating-point type scalar value, or a two-element value, that specifies the full width at half maximum of the gaussian kernel used to convolve the raw data. If only one value is specified, it is used with both axes. These two values are divided with the two CCD bin numbers. This keyword is only used if IMAGEMETHOD is unset. The default value is: fwhm_tr, fwhm_tr
gausskernelsizeA scalar integer that specifies the size of the Gaussian kernel used when convolving the input image with a Gaussian using the PyCosmic algorithm. The default value is 2 * //q// + 1, where //q// is the maximum value of the //x// and the //y// pixel where a one-dimensional Gaussian reaches 15 % of its maximum value (1.0); this condition makes use of FWHM. The value must be odd, and the maximum value is in any case 19. Be aware that convolution times increase strongly with the size of the Gaussian kernel.
sigfracA scalar floating-point type value that specifies a scale factor for pixels surrounding a cosmic-ray masked pixel. This parameter is only used if GROWRADIUS is non-zero. The default value is: 0.5
growradiusSet this scalar-integer keyword to grow all cosmic-ray hit pixels by a frame this many pixels wide; this is done to check if surrounding pixels are affected by cosmic-ray hits as well. The default value is 1; 0 and 2 are also accepted. The first criterion S' > SIGCLIP is replaced with S' > SIGFRAC * SIGCLIP when surrounding pixels are checked if they are cosmic-ray affected. To reproduce the PyCosmic-specific results of HKS12, it is necessary to set GROWRADIUS = 0. The default value is: 1
maxiterA scalar integer that specifies the maximum number of iterations used when identifying cosmic-ray hits in each image, see p3d_tool_crays_lapycosmic. The default value is: 4
imagemethodSet this keyword to use the original algorithm of P. van Dokkum (L.A. Cosmic), which was developed for cosmic-ray detection in images. The default is otherwise to use the IFS-modified algorithm with IMAGEMETHOD unset.
imagecleanSet this keyword to use the original [5 * 5 px²] two-dimensional median filter when replacing the flux in masked pixels with that of the surrounding pixels. otherwise an [11 px] one-dimensional median filter is used. The default value is: 1
dispmedianSet this keyword to a positive integer > 1 to subtract sampling flux along the dispersion axis. The value defines the width of the median window.
writeallSet this keyword to write all intermediate images in addition to the cleaned image and the cosmic-ray mask.
sfxA scalar string that specifies the output filename suffix without any file-compression suffix. The default value is: '.fits'
cleansfxA scalar string that specifies the output filename suffix appended to saved cleaned images. The default value is: '_crcl'
masksfxA scalar string that specifies the output filename suffix appended to saved cosmic-ray masks. The default value is: '_crmk'
laplsfxA scalar string that specifies the output filename suffix appended to saved //L^+// images; this is only used when WRITEALL is set. The default value is: '_crlp'
poflsfxA scalar string that specifies the output filename suffix appended to saved //S// images; this is only used when WRITEALL is set. The default value is: '_crpf'
siprsfxA scalar string that specifies the output filename suffix appended to saved //S'// = //L^+// / (2 * //N//) images; this is only used when WRITEALL is set. The default value is: '_crsp'
laprsfxA scalar string that specifies the output filename suffix appended to saved //L^+// (IMAGE / conv(IMAGE)) images; this is only used when WRITEALL is set and IMAGEMETHOD is unset. The default value is: '_crlr'
finesfxA scalar string that specifies the output filename suffix appended to saved fine-structure images; this is only used when WRITEALL is set and IMAGEMETHOD is set. The default value is: '_crfs'
wallsfxA scalar string that specifies the output filename suffix appended to saved multi-extension images; this is only used when WRITEALL is set. The default value is: '_crwa'
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
nthreadsA scalar integer that specifies how many threads re used when cleaning single input images of osmic-ray hits. The calculation speed scales early linearly with the number of threads. The efault is to use the maximum number of available hreads minus one, but the maximum number s 16: (!cpu.hw_ncpu – 1) > 1 < 16. The default value is: max-1
showguiSet this keyword to show a properties GUI before the algorithm is launched. The GUI allows an adjustment of the parameters critical to the algorithm.
noCSet this keyword to force the use of the slower IDL-based routine instead of the faster compiled C-based routine.
gainA floating-point type value with as many elements as IMAGE. The unit of GAIN must be electrons per analog-to-digital unit [e- / ADU]. The value is retrieved from the respective file if it is not provided (using IMAGE when it is a string, or FILENAMES when IMAGE is of floating-point type).
ronA floating-point type with as many elements as IMAGE. The unit of RON must be analog-to-digital units [ADU]. The value is retrieved from the respective file if it is not provided (using IMAGE when it is a string, or FILENAMES when IMAGE is of floating-point type).
filenamesXThe output filenames are created using the strings in this array if IMAGE is not a string array. The same files are used to retrieve the FITS header arrays.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detsecXA four-element by //n//-element integer array that specifies the detector region to use on the CCD. DETSEC must be present when the number of blocks is greater than one.
userparfileA scalar string with the name of a file with user- defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'compress'

Corresponds to COMPRESS.

'allinone'

Corresponds to ALLINONE.

'nthreads_cr'

Corresponds to NTHREADS.

'nthreads'

Corresponds to NTHREADS; this parameter is only read if 'nthreads_cr' is not found.

'biaspx'

Corresponds to BIASPX.

'biaspy'

Corresponds to BIASPY.

'biasox'

Corresponds to BIASOX.

'biasoy'

Corresponds to BIASOY.

'biasconstant'

Corresponds to BIASCONSTANT.

'laimagemethod'

Corresponds to IMAGEMETHOD.

'laimageclean'

Corresponds to IMAGECLEAN.

'ladispmedian'

Corresponds to DISPMEDIAN.

'crwriteall'

Corresponds to WRITEALL.

'nocrc'

Corresponds to NOC.

'sigclip'

Corresponds to SIGCLIP.

'objlim'

Corresponds to OBJLIM.

'ratlim'

Corresponds to RATLIM.

'crfwhm'

Corresponds to FWHM.

'crgausskernelsize'

Corresponds to GAUSSKERNELSIZE.

'fwhm_tr'

Used to set the width of the Gaussian profile when crfwhm is unavailable and IMAGEMETHOD is unset.

'sigfrac'

Corresponds to SIGFRAC.

'growradius'

Corresponds to GROWRADIUS.

'maxiter'

Corresponds to MAXITER.

'opfx'

Corresponds to OPFX.

'cleansfx'

Corresponds to CLEANSFX.

'masksfx'

Corresponds to MASKSFX.

'laplsfx'

Corresponds to LAPLSFX.

'poflsfx'

Corresponds to POFLSFX.

'siprsfx'

Corresponds to SIPRSFX.

'laprsfx'

Corresponds to LAPRSFX.

'finesfx'

Corresponds to FINESFX.

'wallsfx'

Corresponds to WALLSFX.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

Note! If any of the corresponding keywords is specified, that value takes precedence.
allinoneUnset this keyword to write the output data to separate files, instead of writing the cosmic-ray cleaned image to the first (DATA) extension and the cosmic-ray mask image to the MASK extension. The default value is: 1
daxisXA scalar integer that specifies the dispersion dimension in the input image(s). The default value is: 1
logfileThis keyword specifies the name of an optional log file when p3d_cobjex is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine opening message will not be written to the log file.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation, and then exit.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) As a stand-alone tool from the IDL command line

In addition to the raw spectrum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments to p3d_cr:

files = ['file_1.fits.gz', 'file_2.fits.gz']
parfile=  'nvimos_hr.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
p3d_cr, files, parfile, userparfile=userparfile, $
    opath=opath, logfile=opath+'dred.log', /verbose

ii) As a stand-alone tool from a terminal, using the IDL VM

In addition to the raw spectrum images, it is necessary to pass the name of the (provided) instrument-specific parameter file as well as any additional arguments to p3d_cr. The recommended approach is to use the provided shell script p3d_cr_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, without using the p3d queue system:
files = file_1.fits.gz,file_2.fits.gz
parfile = nvimos_hr.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_cr_vm.sh -s $files $parfile userparfile=$userparfile \
    opath=$opath logfile=$opath/dred.log /verbose

# Using the Python tool, using the p3d queue system:
files = file_1.fits.gz,file_2.fits.gz
parfile = nvimos_hr.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_dispatch.py cr $files $parfile userparfile=$userparfile \
    opath=$opath logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cr_vm.sh userparfile=$userparfile" and -NOT- "p3d_cr_vm.sh user=$userparfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the output variable parameter OIMAGE.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_cr in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'file_1.fits.gz,file_2.fits.gz'
parfile = 'nvimos_hr.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
commandline = files + ' ' + parfile' +
  ' userparfile=' + userparfile + ' opath=' + opath + ' nthreads=4' +
  ' detector=' + detector + ' logfile=dred.log loglevel=2 /verbose'
p3d_dispatch('cr', commandline, noqueue=True)

p3d_ctrace.pro, line 820, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_ctrace, filename, parfile, out, masterbias=, /biaspx, /biaspy, /biasox, /biasoy, biasconstant=, /savebiassub, userparfile=, ofilename=, opath=, opfx=, detector=, /exmonitor, crnthreads=, nthreads=, /compress, logfile=, loglevel=, detsepgrp=, detsepnum=, /gui, /subroutine, /cinv, /crclean, sigclip=, objlim=, ratlim=, crfwhm=, gausskernelsize=, sigfrac=, growradius=, maxiter=, /imagemethod, /imageclean, dispmedian=, /writeall, /showcrgui, /nocrc, /allinone, font=, cmdline=, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Creates a trace-mask image from a set of continuum images.

Before any spectra can be extracted from a raw-data image file, it is necessary to know their locations in the image. The tracing procedure of p3d locates all the spectra, for a pre-defined instrument, and saves a trace-mask image. In this image, the position on the cross- dispersion axis, starting at the (lower) edge of the image, is stored for each spectrum. Additionally, if requested, p3d also calculates cross-dispersion line profiles for each spectrum and wavelength bin (pixel). These profiles are used to extract more accurate spectrum positions (i.e. a more accurate trace mask), and to perform optimal extraction.

It is highly recommended to use an optimal spectrum extraction method that accounts for overlapping spectra (so-called fiber-to-fiber cross talk) for most instruments. The line profiles used in such an optimal extraction are calculated by this tool, after the calculation of a first-guess trace-mask image. Because the process of calculating line profiles is computationally very expensive, for most instruments, they are only calculated if this option is activated (see the master user-parameter file). However, if p3d is able to (automatically) compile the C version of the profile-calculation routine (which it in most cases can), the associated calculations will be much faster; some factor ten seems typical on a machine with eight threads.

The output data files, consisting of the trace-mask image and the line-profiles data cube are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE, or the user-parameter 'allinone', is unset. There is no real advantage to either approach; although, if separate files are used and the line-profiles data cube file is renamed, it will not be found by other tools.

The raw images should be continuum images, it is otherwise difficult - or even impossible – to trace the spectra in regions where there is no signal, or where there are very densely packed emission lines. Either a lamp continuum or a twilight flat (if available, and appropriate) could be used as an input image.

Please, see the p3d A&A paper as well as the master user-parameter file and the p3d documentation WIKI, for help on how to setup the parameters so that spectra in your data are located and traced properly.

This routine calls the routine p3d_cr, if requested to do so, which cleans single input images of cosmic-ray hits; see the CRCLEAN keyword. The outcome of the cleaning procedure depends both on properties of the input image as well as the algorithm parameters. Different parameters might produce a different outcome. There is no guarantee whatsoever that the cleaning removes only cosmic-ray hits. Check the output images to build your own opinion regarding the quality of the outcome. Also, see p3d_cr for more details.

General information on how to learn more about this routine

The nearly, and mostly, full automatic tracing procedure is described in detail in the p3d A&A paper: Sandin et al. 2010, A&A, 515, 35.

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes one, or several, continuum-data raw images as input. The images are combined (by default), but p3d can also calculate a trace-mask image for each file if combineimages_tr==yes (in the user-parameter file). Prescan and overscan regions must not be removed in the input data, they are removed by p3d during the reduction. Input files of instruments using several readout ports must also not be combined in advance, p3d combines the images automatically.

The output – the trace-mask image – consists of an image with as many pixels on the dispersion axis as there are pixels on the dispersion axis in the input image(s) – prescan and overscan regions are (mostly) kept. On the cross-dispersion axis, there are n rows, one for each spectrum. The number of rows in the output file is always the same, pre-defined, number. For each spectrum and pixel on the dispersion axis, the trace-mask image contains the distance to the edge of the CCD. In addition to the standard fits-file header keywords, the extracted spectrum images contain additional p3d-specific keywords:

IMRAW

The name(s) of the raw input continuum image file(s).

IMMBI

The name of the master-bias image file, if it is used.

IMPRF

The name of the line-profiles image file, if it is used. Note! This keyword is only written if the user-parameter 'allinone' is unset.

IMTYPE

The p3d image type is set to 'p3d: trace-mask image'.

EXTNAME

In case that ALLINONE is set, two extension names are used in the output-data file: the trace-mask image [DATA], and the line-profiles image [PROF].

The line-profiles data cube is written either to the PROF extension of the trace-mask image or to a separate file. Its header contains the following set of additional p3d-specific keywords:

IMTRC

The name of the trace-mask image file used as a first-guess for the centroids when calculating the line profiles. Note! This keyword is only written if ALLINONE is unset.

P3DFFUNC

The name of the profile function name.

IMTYPE

The p3d image type is set to 'p3d: line-profiles image'.

If several input files were specified, which were thereafter combined by p3d, before calculating a trace-mask image, the combined image contains the following p3d-specific header keywords:

IMCMBxxx

Is set to the name(s) of the input image file(s).

P3DDFLIP

Is set to 1 if the input images were flipped on the dispersion axis.

NCOMBINE

The number of raw images used to calculate the combined image. If you use the min/max-filtered average method, the minimum and maximum values at each pixel will be discarded before calculating the average, and NCOMBINE will therefore be smaller than the number of input images.

Saved, and optionally already combined, bias-subtracted raw-data images contain only one additional header keyword:

IMTYPE

The p3d image type is set to 'p3d: bias-subtracted image'.

Input parameters:
filenameA scalar string that specifies the name of the raw data file to be used when tracing spectra.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
masterbiasA scalar string with the name of a master-bias image file. This file is required if optimal extraction is used and none of the following four keywords is used: BIASPX, BIASPY, BIASOX, BIASOY. Note! This keyword must not be set if the prescan and the overscan regions should be used to estimate the bias level.
biaspxSet this keyword to use the x-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis prescan region. The same array is used with all y-axis columns of bias subtracted data.
biaspySet this keyword to use the y-axis prescan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis prescan region. The same array is used with all x-axis rows of bias subtracted data.
biasoxSet this keyword to use the x-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the y axis in the x-axis overscan region. The same array is used with all y-axis columns of bias subtracted data.
biasoySet this keyword to use the y-axis overscan region when subtracting a bias. Specifically, a median value is calculated for each pixel on the x axis in the y-axis overscan region. The same array is used with all x-axis rows of bias subtracted data.
biasconstantThis keyword can be used in two ways:

  • Set this keyword to use a constant value as a bias value; otherwise an array is used. All prescan and overscan regions specified using the keywords BIASPX, BIASPY, BIASOX, and BIASOY can be used when this keyword is set; otherwise only one of them.

  • If this keyword is set to a double type array or to a comma-separated string with values > 1 (when called from the shell), these values are used to create a bias; the number of elements must be the same as the number of blocks.

savebiassubSet this keyword to write the bias-subtracted raw-data image before spectrum extraction. The name of the output file is the same as the (combined) input file, with the added intermediate suffix '_subbias'. Also see the user parameter 'savebiassub'.
userparfileA scalar string with the name of a file with user- defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'combineimages_tr'

To specify if input images are to be combined or not. Corresponds to IMCOMBINE.

'compress'

Corresponds to COMPRESS.

'exmonitor'

Corresponds to EXMONITOR.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'psnumwave'

To select the total number of of wavelengths where a plot of the line profiles is made.

'detector_sepgrp'

Corresponds to DETSEPGRP.

'detector_sepnum'

Corresponds to DETSEPNUM.

'opfx'

Corresponds to OPFX.

'sfx'

To specify the data output file suffix.

'icsfx'

To specify the data output file intermediate suffix of combined files.

'trsfx'

To specify the data output file intermediate suffix of the created trace-mask image.

'methimcombine_tr'

To specify the method that is used to combine the input images.

'allinone'

Corresponds to ALLINONE.

'nthreads_cr'

Corresponds to CRNTHREADS.

'nthreads'

Corresponds to NTHREADS.

'crclean'

Corresponds to CRCLEAN.

'biaspx'

Corresponds to BIASPX.

'biaspy'

Corresponds to BIASPY.

'biasox'

Corresponds to BIASOX.

'biasoy'

Corresponds to BIASOY.

'biasconstant'

Corresponds to BIASCONSTANT.

'savebiassub'

Corresponds to SAVEBIASSUB.

'gapfile'

To read the sprectrum gaps file.

Additionally, the following user parameters are used by p3d_cr:

'laimagemethod'

Corresponds to IMAGEMETHOD.

'laimageclean'

Corresponds to IMAGECLEAN.

'ladispmedian'

Corresponds to DISPMEDIAN.

'crwriteall'

Corresponds to WRITEALL.

'nocrc'

Corresponds to NOCRC.

'sigclip'

Corresponds to SIGCLIP.

'objlim'

Corresponds to OBJLIM.

'ratlim'

Corresponds to RATLIM.

'crfwhm'

Corresponds to FWHM.

'crgausskernelsize'

Corresponds to GAUSSKERNELSIZE.

'fwhm_tr'

Used to set the width of the Gaussian profile when crfwhm is unavailable and IMAGEMETHOD is unset.

'sigfrac'

Corresponds to SIGFRAC.

'growradius'

Corresponds to GROWRADIUS.

'maxiter'

Corresponds to MAXITER.

'cleansfx'

Corresponds to CLEANSFX.

'masksfx'

Corresponds to MASKSFX.

'laplsfx'

Corresponds to LAPLSFX.

'poflsfx'

Corresponds to POFLSFX.

'siprsfx'

Corresponds to SIPRSFX.

'laprsfx'

Corresponds to LAPRSFX.

'finesfx'

Corresponds to FINESFX.

'wallsfx'

Corresponds to WALLSFX.

Note! If any of the corresponding keywords is specified, that value takes precedence.
ofilenameXThis keyword returns the full name of the created trace-mask file.
opathA scalar string that specifies the path, where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
exmonitorIf this keyword is set, a trace viewer is shown after all the tracing and the line-profile fitting is done when optimal extraction is used. The default value is: 1
imcombineSet this keyword to combine all input images before locating the spectra. The default value is: 1
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is then appended to all output filenames. The default value is: 0
crnthreadsA scalar integer that specifies how many threads used when cleaning single input images of mic-ray hits. If this keyword is not set, READS is used instead. If nthreads was not set her, the default is to use the maximum number of ilable threads minus one, but the maximum number 16: (!cpu.hw_ncpu – 1) < 16. The default value is: max-1
nthreadsA scalar integer that specifies how many threads to use in the parallelized line profile calculations. The calculation speed scales nearly linearly with the number of threads. The default is to use the maximum number available, but no more than 16: max = !cpu.hw_ncpu – 16. The default value is: max
logfileThis keyword specifies the name of an optional log file when p3d_ctrace is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, then this keyword specifies the log level; [1],2,3 (the higher the value, the more output). The default value is: 1
detsepgrp(Profile fitting) This keyword can be set in instrument setups where there are more than one detector on the spatial axis, and the background levels differ (making a line-profile fit tricky). The keyword should in this case be set to a positive non-zero integer that indicates beyond which spectrum the new detector begins. This value can also be set with the (instrument- or) user-parameter 'detector_sepgrp'. If this keyword, or the user-parameter is set, it is also mandatory to set DETSEPNUM.
detsepnum(Profile fitting) Together with DETSEPGRP, this value specifies the number of spectra in the group that falls within the first detector; the value must be a positive non-zero integer. If DETSEPGRP is set, it is also mandatory to set this value. This value can also be set with the (instrument- or) user-parameter 'detector_sepnum'.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine opening message will not be written to the log file.
cinvSet this keyword to plot white lines on a black background instead of the default, which is to plot black lines on a white background.
allinoneUnset this keyword to write the output data to separate files, instead of writing the trace-mask image to the DATA extension and the line-profiles image to the PROF extension. The default value is: 1
crclean

Set this keyword to clean all uncombined input images of cosmic-ray hits before the tracing procedure is started; either a master-bias image or prescan and overscan regions must be used to provide the bias level to use this functionality.

The following keywords are used to configure the cosmic-ray cleaning procedure; they are only used if CRCLEAN is set; see p3d_cr and p3d_tool_crays_lapycosmic for more details:

sigclipA scalar decimal value that specifies the clipping sigma.
objlimA scalar decimal value that specifies the object limit value; this value is only used if IMAGEMETHOD is set.
ratlimA scalar decimal value that specifies the ratio limit value; this value is only used if IMAGEMETHOD is unset.
crfwhmA decimal scalar value, or a two-element array, that specifies the full width at half maximum of the gaussian kernel used to convolve the raw data with. This keyword corresponds to FWHM in p3d_cr. This value is only used if IMAGEMETHOD is unset.
gausskernelsizeA scalar integer that specifies the size of the Gaussian kernel used when convolving the input image with a Gaussian kernel using the PyCosmic algorithm. This value is by default set to 2 * q + 1, where q is the maximum value of the x and the y pixel where a one-dimensional Gaussian reaches 15% of its maximum value (1.0); this condition makes use of the full-width-at-half-maximum value of FWHM. The value must be odd, and the maximum value is in any case 19. Be aware that convolution times increase strongly with the size of the Gaussian kernel.
sigfracA scalar decimal value that specifies a scale factor for pixels surrounding a cosmic-ray masked pixel. This parameter is only used if GROWRADIUS is non-zero. The default value is: 0.5
growradiusSet this keyword to grow all cosmic-ray hit pixels by a frame this many pixels wide; this is done to check if the surrounding pixels are affected by cosmic-ray hits as well. The default value is 1, the values 0 and 2 are also accepted. Surrounding pixels are checked if they are cosmic-ray affected using the first criterion S' > SIGFRAC * SIGCLIP. The default value is: 1
maxiterA scalar integer value that specifies the maximum number of iterations used when identifying cosmic-ray hits in each image.
imagemethodSet this keyword to use the original algorithm of P. van Dokkum (L.A. Cosmic), which was developed for cosmic-ray detection in images. The default is otherwise to use the IFS-modified algorithm with IMAGEMETHOD unset.
imagecleanSet this keyword to use the original two-dimensional median filter when replacing the flux in masked pixels with that of the surrounding pixels. Otherwise an [11 pixel] one-dimensional median filter is used. The default value is: 1
dispmedianSet this keyword to a positive integer > 1 to subtract sampling flux along the dispersion axis. The value defines the width of the median window.
writeallSet this keyword to write all intermediate images in addition to the cleaned image and the cosmic-ray mask.
showcrguiSet this keyword to show a properties GUI of the cosmic-ray cleaning parameters before the algorithm is executed.
nocrcSet this keyword to force the use of the slower IDL-based cosmic-ray cleaning routine instead of the faster C-based routine.
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined. This keyword must not be used if this routine is called as a script instead of using the GUI.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to avoid printing the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation and then exit.
Output parameters:
outXUpon a successful execution, this variable contains the trace mask (data) when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to your input continuum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, the master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), as well as any additional arguments to p3d.

files = ['tracefile_1.fits.gz', 'tracefile_2.fits.gz']
parfile = 'larr2k.prm'
opath = 'Output'
masterbias = opath + '/masterbias.fits.gz'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
detector = 0 ;; Required with multi-detector configurations.
p3d_ctrace, files, parfile, masterbias=masterbias, $
    userparfile=userparfile, opath=opath, detector=detector, $
    logfile=opath+'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your input continuum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, the master-bias image file or (at least) one of the bias prescan and overscan parameters (if you use optimal extraction), as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_ctrace_vm.sh or the Python tool p3d_dispatch.py.

# Using the shell script, making use of the p3d queue system:
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_ctrace_vm.sh tracefile_1.fits.gz,tracefilefile_2.fits.gz \
    $parfile masterbias=$masterbias userparfile=$userparfile \
    opath=$opath detector=$detector logfile=$opath/dred.log /verbose

# Using the Python tool, without using the p3d queue system:
parfile = larr2k.prm
opath = Output
masterbias = ${opath}/masterbias.fits.gz
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
p3d_dispatch.py --noqueue ctrace \
    tracefile_1.fits.gz,tracefilefile_2.fits.gz \
    $parfile masterbias=$masterbias userparfile=$userparfile \
    opath=$opath detector=$detector logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_ctrace_vm.sh masterbias=$masterbias" and -NOT- "p3d_ctrace_vm.sh ma=$masterbias".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the two output variable parameters OFILENAME, and OUT.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_ctrace in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'tracefile_1.fits.gz,tracefilefile_2.fits.gz'
parfile = 'larr2k.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
masterbias = opath + '/masterbias.fits.gz'
commandline = files + ' ' + parfile + ' masterbias=' + masterbias +
  ' userparfile=' + userparfile + ' opath=' + opath +
  ' detector=' + detector +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('ctrace', commandline, noqueue=True)

p3d_cvimos_combine.pro, line 754, last changed at 2016-03-15 by christersandin (revision 4037)

p3d_cvimos_combine, filenames, parfile, out, dfilenames=, /keeparrangement, /norenormalize, /nofcdgoodnorm, /onlynormquadrant, telluricline=, maxintensity=, corrffile=, /compress, /savee3d, userparfile=, ofilename=, opath=, opfx=, logfile=, loglevel=, /allinone, cmdline=, /gui, /subroutine, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, /help

Combines four extracted VIMOS spectrum images of the four separate detectors to create one RSS file for the entire field.

The purpose of this tool is to combine extracted science-object images. The four input images should all have been reduced separately, they should belong to the same observation block (OB), and they should preferrably all have been flux calibrated. Before the four input images are combined, the files – including the corresponding error files, output bad-pixels mask files, and saturated-mask files – are checked in the following ways:

Quadrant

Each of the four images must come from a separate quadrant.

Exposure time

The four images must all use the same exposure time; the difference between the value of all four quadrants must be < 1 s.

Number of spectra

The four images must each contain the correct number of of spectra. This number is defined in the fiber position-tables. (There must be exactly 400 spectra in each image.)

CRVAL

The four images must all use the same value on CRVAL (this value is set during the wavelength calibration [p3d_cobjex]).

CDELT

The four images must all use the same value on CDELT (this value is set during the wavelength calibration [p3d_cobjex]).

Flux calibration

All four images must be either flux calibrated or not flux calibrated. The maximum value of flux-calibrated images is assumed to be always smaller than 1, the maximum value of non flux-calibrated images is assumed to be always greater than 1.

The flux level varies across the four detectors, which is why data are "re-normalized" by default to get the same flux in a selected telluric line; the re-normalization is switched off when the NORENORMALIZE keyword is set. See the next subsection for details of this re-normalization procedure.

Spectra of the combined image are by default re-arranged to go from south to north, and from west to east – the resulting images should use the "vimos_positions_rer.dat" fiber position-table file. Set the KEEPARRANGEMENT keyword if the wish is instead to just concatenate the data, without re-arranging them – the resulting images should in this case use the "vimos_positions_coc.dat" fiber position-table file.

The output-data files, consisting of the combined spectra, the corresponding combined error image, the combined bad-pixels mask image, the combined saturated-pixels mask image, the combined data-quality mask image, and the re-normalization correction factor array (if it is used), are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE, or the user parameter 'allinone', is unset. There is no real advantage to either approach; although, if separate files are used and one or more of the error image file, the output bad-pixels mask image file, the saturated-pixels mask image file, or the re-normalization correction factor file are renamed, they will not be found by other tools.

Re-normalizing data

Spectra are normalized anew to account for imprecise flux calibration between the four detectors, as well as other issues that are difficult to handle. Without such an additional normalization, differences detector-to-detector are seen clearly in spatial maps of the combined image between the four separate quadrants.

The re-normalization procedure integrates the flux in one selected telluric line across all working fibers. Thereafter, all spectra are multiplied with the correction factor: median_intensity / intensity of each individual spatial element.

The following four subsections discuss all options and issues considered in the re-normalization procedure.

Spectra of dead and unused fibers and low-transmission fibers

Only fully working fibers (spatial elements) are considered in the re-normalization procedure. Information about dead and unused fibers, as well as low-transmission fibers, is retrieved using the 'deadfibersfile' parameters in the instrument-parameter file.

Selecting the telluric line

By default, p3d selects the telluric line from the default line-list file (p3d_path/data/tables/linelists/telluric_lines_lores.dat), where the strongest line is used (due to a simplified measurement, it is possible that the second strongest line is selected instead). Any other line-list file can be selected using the TELLURICLINE keyword or the user parameter 'telluricline'. However, to ensure that data are normalized with a telluric line that does not contain any emission from the object, it is instead recommended to specify the wavelength of a pre-selected emission line, using the same keyword TELLURICLINE (using the unit Angstrom [Å]). Nlended lines are not handled.

Setting the normalization intensity

The normalization of the intensity in the telluric line is by default calculated as the maximum value of the median intensity of the individual detectors; this is the median_intensity value above. Use a different value with the keyword MAXINTENSITY or the user parameter 'maxintensity' if you are able to determine this value more accurately by some other means.

Normalizing spectrum to spectrum or per quadrant

Also by default, all spectra are normalized individually. If you, for some reason, instead want to normalize all spectra of one quadrant with the median intensity in that quadrant, set the keyword ONLYNORMQUADRANT or the user parameter 'onlynormquadrant'.

Non flux-calibrated data

Data that are not flux calibrated are by default treated in the same way as flux-calibrated data. There is the option, however, to normalize spectra per quadrant using a total value of the mean flat-field image spectrum. The mean value must be available in the FFLMN keyword of the object spectra; this value is present in p3d-reduced data. Set the NOFCDGOODNORM keyword or the user parameter 'nofcdgoodnorm' to use this functionality.

General information on how to learn more about this routine

The image combination procedure is described in more detail on the p3d documentation WIKI (see below).

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

The output file always contains the sum of the pre-defined number of extracted spectra in each input image (with as many pixels on the cross-dispersion axis), which are wavelength calibrated and normalized with a flat-field image since earlier – see p3d_cobjex. Data are stored in the so-called row-stacked spectrum (RSS) format. To create a spatial map of the intensities at any wavelength, using this RSS file, it is necessary to make use of a fiber position-table file; depending on your setup, you should use either "vimos_positions_rer.dat" ([KEEPARRANGEMENT = 0]) or "vimos_positions_coc.dat" (KEEPARRANGEMENT = 1). Extracted data can also can also be saved using the Euro3D format (use the keyword SAVEE3D or the user parameter 'savee3d'), and data of instruments using square-shaped spatial elements are easily converted to a cube format using the tool p3d_rss2cube. In addition to the standard fits-file header keywords, the extracted spectrum images contain additional p3d-specific keywords:

IMRAW

The name(s) of the input science object-image file(s); this keyword is taken from the first file in the list.

IMMBI

The name of the master-bias image file, if it is used; this keyword is taken from the first file in the list.

IMTRC

The name of the trace-mask image file; this keyword is taken from the first file in the list.

IMDMK

The name of the dispersion-mask image file, if it is used; this keyword is taken from the first file in the list.

IMFFL

The name of the flat-field image file, if it is used; this keyword is taken from the first file in the list.

IMCRF

The name of the re-normalization correction-factor file.

IMERR

The name of the combined science object-image error file. Note! This keyword is only written if ALLINONE is unset.

IMONC

The name of the combined output bad-pixels mask image file; this keyword is only added if IMONC is set in all four input files. Note! This keyword is only written if ALLINONE is unset.

IMSMK

The name of the combined saturated-pixels mask file; this keyword is only added if IMSMK is set in all four input files. The saturated-pixels mask image in addition contains the keyword SATLIMIT, which specifies the pixel value used to define the saturation limit. Note! This keyword is only written if ALLINONE is unset.

IMTYPE

The p3d image type is set to 'p3d: combined object image'; the error image type is 'p3d: combined image – error'.

POSTAB

Is set to either "p3d: concatenated data" to indicate the use of KEEPARRANGEMENT = 1, or to "p3d: remapped data" to indicate the use of [KEEPARRANGEMENT = 0].

IMOBn

Is set to the n separate names of the input image files.

EXTNAME

Six extensions are used in the output-data file when ALLINONE is set:

  • The extracted spectrum image [DATA].

  • The extracted spectrum error image [ERROR].

  • The bad-pixels mask image [MASK].

  • The saturated-pixels mask image [SATMASK].

  • The data-quality mask image [QUALITY].

  • The re-normalization correction-factor binary-table array [CORRFACT].

The re-normalization correction-factor binary-table extension, or file, contains two additional p3d-specific keywords:

The name of the extracted flat-field image file. Note! This keyword is only written if ALLINONE is unset.

IMTYPE

The p3d image type is 'p3d: renorm. correction f. (VIMOS)'.

Input parameters:
filenamesA four-element string array that specifies the names of the extracted data files to combine. The array must contain one file per quadrant. The files can be in any order, the routine figures out by itself which quadrant each file belongs to.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
dfilenamesA four-element string array that specifies the names of the extracted error data files to combine. The array must contain one file per quadrant. The files must be specified in the same order as in FILENAMES. If DFILENAMES is not provided, the extracted error-data filenames are taken from the keyword IMERR in the data header of FILENAMES.
keeparrangementIf this keyword is set, spectrum data are simply combined (concatenated). The default behavior is otherwise to re-order them S->N, and W->E.
norenormalizeNo data re-normalization is performed if this keyword is set.
nofcdgoodnormSet this keyword to re-normalize non flux-calibrated data using a renormalized gain value. The default is otherwise to use a telluric line in the same fashion as with flux-calibrated data.
onlynormquadrantWith flux calibrated data, the re-normalization is y default performed on individual spectra (of the et of fully working fibers). Set this parameter to nstead perform the re-normalization with the mean ntensity value of a telluric line in each uadrant.
telluriclineThe re-normalization procedure of flux-calibrated data uses a telluric line, which is by default selected automatically in the high-resolution telluric-line list file ("telluric_lines_hires.dat"). Alternatively, set this parameter either to the name of any existing telluric line-list file, or to the wavelength of the particular line to use (the unit is Angstrom [Å]); example: TELLURICLINE = 5577. Note! The value must be within the wavelength range of the wavelength-calibrated and flux-calibrated spectrum image. Note! The exact wavelength is unimportant, but the value should be accurate enough that the line can be properly located by the fitting routine, i.e. the line center should be within a few pixels distance [CDELT] of the chosen value. Note! Ensure a telluric line is used that shows no signs of object emission. Otherwise, data quality is compromised. The p3d spectrum viewer p3d_sv could be used to check which line to use. Note! If this keyword is used with a filename, i.e. a string, the following search order of the path applies:

i.

The full path of an existing file must be specified if the string contains one or several path separators "/" or "\".

ii.

A file without any path separator is searched for in the user-parameter file directory, if such a file is used.

iii.

A file without any path separator is searched for in the p3d default line-list directory: data/tables/linelists/.

maxintensityA scalar floating-point value that specifies the intensity used to normalize all spectra. The default behavior is, when this value is unset, to instead first calculate a median intensity of the selected telluric line for each detector individually. Thereafter, MAXINTENSITY is set to the maximum value of the four median values.
corrffileA scalar string with the name of a file that contains the re-normalization correction factor array. The correction-factor array of this file is used if this parameter is set, otherwise a new array is calculated here. The format of this binary-table fits file is that it contains an array with four columns and 400 rows. Note! The following file search order of the path applies:

i.

The full path of an existing file must be specified if the string contains one or several path separators "/" or "\".

ii.

A file without any path separator is searched for in the same directory as (FILENAMES).

iii.

If the file was not found in the same directory as FILENAME, the parent directory is used instead.

compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
savee3dThe output data are saved in the Euro3D (e3d) format in addition to the usual RSS format when this keyword is set.
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file:

'keywordsfile'

allows the use of an alternative header keywords file.

'compress'

see COMPRESS.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'opfx'

see OPFX.

'vcsfx'

the file combination suffix ['_vmcmb'].

'vccorrf'

the correction file suffix ['_corrf'].

'mosfx'

the output bad-pixels mask file suffix ['_ocmsk'].

'mmsfx'

the saturated-pixels mask file suffix ['_smask'].

'keeparrangement'

see KEEPARRANGEMENT.

'norenormalize'

see NORENORMALIZE.

'nofcdgoodnorm'

see NOFCDGOODNORM.

'onlynormquadrant'

see ONLYNORMQUADRANT.

'telluricline'

see TELLURICLINE.

'maxintensity'

see MAXINTENSITY.

'deadfibersfile'

allows the use of an alternative dead-fibers file.

ofilenameXThis keyword returns the full name of the created combined spectrum-image file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
logfileThis keyword specifies the name of an optional log file when p3d_cvimos_combine is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
allinoneUnset this keyword to write the output data to separate files, instead of writing the extracted spectra to the first (DATA) extension, its error to the ERROR extension, the bad-pixels mask image to the MASK extension, the saturated-pixels mask image to the SATMASK extension, the data-quality mask image to the QUALITY extension, and the re-normalization correction-factor array binary table to the CORRFMHR and CORRFACT extensions. The default value is: 1
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine messages are not written to the log file.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution this variable contains the data image (odata) when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to the extracted spectrum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments to p3d:

parfile = 'nvimos_hr.prm'
 files = ['VIMOS_oextr1.fits.gz', 'VIMOS_oextr2.fits.gz', $
          'VIMOS_oextr3.fits.gz', 'VIMOS_oextr4.fits.gz']
dfiles = ['VIMOS_oextr1_err.fits.gz', 'VIMOS_oextr2_err.fits.gz', $
          'VIMOS_oextr3_err.fits.gz', 'VIMOS_oextr4_err.fits.gz']
userparfile = '/user_p3d.dat' ;; If you use this file
p3d_cvimos_combine, files, parfile, dfiles=dfiles, $
    userparfile=userparfile, logfile=opath + 'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to the extracted spectrum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_cvimos_combine_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
parfile = nvimos_hr.prm
 files = VIMOS_oextr1.fits.gz,VIMOS_oextr2.fits.gz,\
         VIMOS_oextr3.fits.gz,VIMOS_oextr4.fits.gz'
dfiles = VIMOS_oextr1_err.fits.gz,VIMOS_oextr2_err.fits.gz,\
         VIMOS_oextr3_err.fits.gz,VIMOS_oextr4_err.fits.gz
userparfile = user_p3d.dat # If you use this file
p3d_cvimos_combine_vm.sh $files $parfile dfiles=dfiles, \
    userparfile=$userparfile logfile=dred.log /verbose

# Using the Python tool, without using the p3d queue system:
parfile = nvimos_hr.prm
 files = VIMOS_oextr1.fits.gz,VIMOS_oextr2.fits.gz,\
         VIMOS_oextr3.fits.gz,VIMOS_oextr4.fits.gz'
dfiles = VIMOS_oextr1_err.fits.gz,VIMOS_oextr2_err.fits.gz,\
         VIMOS_oextr3_err.fits.gz,VIMOS_oextr4_err.fits.gz
userparfile = user_p3d.dat # If you use this file
p3d_dispatch.py --noqueue cvimos_combine \
    $files $parfile dfiles=dfiles, \
    userparfile=$userparfile logfile=dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_cvimos_combine_vm.sh dfilenames=$dfilenames" and -NOT- "p3d_cvimos_combine_vm.sh df=$dfilenames".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the output variable parameter OFILENAME.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_cvimos_combine in a single IDL session, without using
# the p3d queue system:
from p3d_dispatch import *
parfile = 'nvimos_hr.prm'
opath = 'Output'
 files = 'VIMOS_oextr1.fits.gz,VIMOS_oextr2.fits.gz,' +
         'VIMOS_oextr3.fits.gz,VIMOS_oextr4.fits.gz'
dfiles = 'VIMOS_oextr1_err.fits.gz,VIMOS_oextr2_err.fits.gz,' +
         'VIMOS_oextr3_err.fits.gz,VIMOS_oextr4_err.fits.gz'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
commandline = files + ' ' + parfile + ' dfiles=' + dfiles +
  ' userparfile=' + userparfile +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('cvimos_combine', commandline, noqueue=True)

p3d_darc.pro, line 1442, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_darc, filename, parfile, out, method=, refwl=, spaxelscale=, posang=, airmass=, parang=, pressure=, temperature=, relhumidity=, latitude=, obselevation=, templapserate=, empiricalfit=, fitx0=, fity0=, ra_offset=, dec_offset=, /nogui, deadfibersfile=, /nomask, maskval=, /masklowtr, /compress, /savee3d, postable=, userparfile=, ofilename=, opath=, opfx=, detector=, nthreads=, logfile=, loglevel=, /cinv, /allinone, /noC, cmdline=, /gui, /subroutine, stawid=, topwid=, logunit=, verbose=, /quiet, font=, error=, /debug, cdebug=, /help

Corrects densely arranged data for differential atmospheric refraction by shifting them spatially in each wavelength bin.

Unless observations were made with the object at zenith, data will show effects of differential atmospheric refraction (DAR), which means that the flux of any object is shifted spatially on the surface of the IFU, the amplitude and direction of the shift depends on the wavelength.

The purpose of this tool is to first to figure out what the amplitude and the angle of the DAR offset is at each wavelength, for a reduced (and perhaps also flux-calibrated) spectrum-image file. Thereafter, each image is shifted according to the DAR-vector, so that flux at all wavelengths, of one spatial element, originates in the same point on the sky.

The direction of the DAR vector is the parallactic angle. The amplitude of the DAR vector is calculated using two approaches:

  • Following the approach of Ciddor (1996) to calculate the wavelength-dependent refractive index (and then the refraction); this approach is very similar to that of Filippenko (1982), the difference is that Filippenko uses an older equation for the refractive index.

  • Following the approach of Walsh & Roy (1990), which uses a refraction-calculation routine of the Starlink project (SLA REFRO, see p3d_misc_sla_refro). To use another value on the troposphere temperature lapse rate than the default (0.0065°C/m), use the keyword TEMPLAPSERATE or the user parameter 'templapserate'.

The correction is performed using the DAR offset vector and a (fractional) image shift that uses bilinear interpolation.

In addition to the two theoretical methods, an empirical method is available as well. In this approach, which is described in some detail by Arribas et al. (1999), a two-dimensional (elliptical) Gaussian profile is fitted to the data at each wavelength. The center position of each fit is used to shift the data, after the respecive shift array has been fitted with a polynomial. It is important that the data contain some star-like, and probably bright enough, feature to use this method. The position of the star on the //x// and the //y// axes can be specified using the FITX0 and FITY0 keywords.

Select the method to use with the METHOD keyword, the theoretical approach using the refractive-index equation of Ciddor is the default. The empirical fit is made in any case, and is shown for reference in the automatically generated plots when the keyword EMPIRICALFIT is set.

The output-data files, consisting of the DAR-corrected spectrum image and the associated error image, the input bad-pixels mask image, the input saturated-pixels mask image, and the input data-quality mask image, are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE or the user parameter 'allinone' is unset. There is no real advantage to either approach; although, if separate files are used and the error file, the input bad-pixels mask image file, or the input saturated-pixels mask image file are renamed, they will not be found by other tools.

Used parts of the DAR routine of J. Walsh (ESO)

With the written permission of J. Walsh at ESO, most of the features of his publicly available DAR-correction code have been added here. Specifically, the following features and issues are considered properly in the entirety thanks to his work:

The angles

Some instruments require an offset angle added to the parallactic angle and the position angle. For example, with VIMOS data this offset is -90°. Also, the sign of the shifts on the //x// and the //y// axes varies.

The hour angle

When the azimuth angle is available, it is used to calculate the hour angle instead of using the regular hour-angle-formula.

Effective air mass

The effective air mass is calculated using the beginning and final parallactic angles, when available, see p3d_misc_obsprop.

The original routines of J. Walsh also include various methods to interpolate data; only bilinear interpolation is available here. The source code of the original routines (arshft.f) is available online at the IFS-WIKI.

DAR-related references

Edlén, B. 1966, Metrologia, 2, 71-80, "The refractive index of air".

Filippenko, A.V. 1982, PASP, 94, 715, "The importance of atmospheric differential refraction in spectrophotometry".

Walsh, J.R., Roy, J.-R. 1990, in proc. 2nd ESO/ST-ECF Data Analysis Workshop, eds. D. Baade, P. J. Grosbol, ESO, Garching, 34, 95, "Area spectroscopy and correction for differential atmospheric refraction".

Birch, K.P., Downs, M. 1993, Metrologia, 30, 155-162, "An updated Edlén equation for the refractive index of air".

Birch, K.P., Downs, M. 1993, Metrologia, 31, 315-316, "Correction to the updated Edlén equation for the refractive index of air".

Ciddor, P.E., Appl. Optics 1996, 35, 1566, "Refractive index of air: new equations for the visible and near infrared"

Arribas, S., Mediavilla, E., García-Lorenzo, B., del Burgo, C., and Fuensalida, J.J. 1999 A&AS, 136, 189, "Differential atmospheric refraction in integral-field spectroscopy: effects and correction".

Sandin, C., Schönberner, D., Roth, M.M., Steffen, M., Böhm, P., Monreal-Ibero, A. 2008, A&A, 486, 545, "Spatially resolved spectroscopy of planetary nebulae and their halos. I. Five galactic disk objects".

DAR-related Links

The IFS Wiki

http://ifs.wikidot.com

The Starlink project

http://starlink.jach.hawaii.edu/starlink

NIST

The National Institute of Standards and Technology on the determination of the refractive index of air: http://emtoolbox.nist.gov/Wavelength/Documentation.asp

The required physical parameters

The theoretical approach used here requires information on a number of physical properties. The following eight properties are always retrieved (these values are all setup in p3d_misc_obsprop):

  • The observatory latitude [LATITUDE].

  • The observatory elevation [OBSELEVATION].

  • The declination.

  • The right ascension.

  • The (local) sidereal time.

  • The hour angle.

  • The zenith distance (air mass).

  • The parallactic angle [PARANG].

To calculate the wavelength-dependent amplitude of the refraction, the following information is also required:

  • The air pressure [PRESSURE]

    The air pressure is sometimes available in the data header. The corresponding parameter name is set by the AIRPRESSURE parameter in the instrument-specific keywords file. In the case that a starting and an ending value are available, the mean of those two values is used instead; the two corresponding parameter names are set by the AIRPRESSURESTART and AIRPRESSUREEND parameters.

    The required unit of the pressure is millibar [mbar]. Some instruments provide a value using other units. In such cases, it might be possible to scale those values with a factor. The corresponding parameter name is set by the AIRPRESSUREFACTOR parameter in the instrument-specific keywords file.

    For those cases where the pressure is unavailable, it can be specified as a keyword, setting the user parameter 'airpressure', or in the property GUI shown by default when p3d_darc is launched.

    The default value is 800 mbar if no other value can be set.

  • The air temperature [TEMPERATURE]

    The air ambient temperature is sometimes available in the data header. The corresponding parameter name is set by the AIRTEMPERATURE parameter in the instrument-specific keywords file.

    The required unit of the temperature is degrees Celcius [°C].

    For those cases where the temperature is unavailable, it can be specified as a keyword, setting the user parameter 'airtemperature', or in the property GUI shown by default when p3d_darc is launched.

    The default value is 8.0 °C if no other value can be set.

  • The relative humidity [RELHUMIDITY]

    The relative humidity of the air is sometimes available in the data header. The corresponding parameter name is set by the AIRRHUMIDITY parameter in the instrument-specific keywords file.

    The required unit of the relative humidity is per cent [%].

    For those cases where the relative humidity is unavailable, it can be specified as a keyword, setting the user parameter 'airrhumidity', or in the property GUI shown by default when p3d_darc is launched.

    The default value is 10.0 % if no other value can be set.

Additionally, it necessary to provide p3d_darc with information on the following properties:

  • The reference wavelength [REFWL]

    The reference, or pivot, wavelength should always be defined by the user. The default is to use the center wavelength, which varies depending on the used image data.

    The required unit of the reference wavelength is Ångström [Å].

    For those cases where the reference wavelength is unavailable, it can be specified as a keyword or in the property GUI shown by default when p3d_darc is launched.

  • The spaxel scale [SPAXELSCALE]

    The spaxel scale is sometimes available in the data header. The corresponding parameter name is set by the SPAXELSCALE parameter in the instrument-specific keywords file.

    The required unit of the spaxel scale is arcsec/spatial element and the default value is 1.0 [arcsec/spatial element] (which might not work at all with the used data).

    For those cases where the spaxel scale is unavailable, it can be specified as a keyword or in the property GUI shown by default when p3d_darc is launched.

  • The IFU rotation angle [POSANG]

    The IFU rotation angle is often available in the data header. The corresponding parameter name is set by the IFUANGLE parameter in the instrument-specific keywords file.

    This angle is measured counterclockwise from the IFU zero position, where North is up and East to the left, as is seen when the IFU is viewed on the sky.

    The required unit of the rotation angle is degrees [°]. The default value is 0.0 °, which should be correct for all IFUs that do not allow a rotation.

    For those cases where the rotation angle is unavailable, it can be specified as a keyword or in the property GUI shown by default when p3d_darc is launched.

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below.

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrumental setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes one, or several, extracted science-object images as input. Data must (currently) be of the row-stacked-spectrum (RSS) format. p3d then converts data to a cube format, makes the correction, and converts data back to the RSS format. It is also required to specify an instrument-specific parameter file to allow the routine to find the relevant properties it needs to correct the data for the DAR.

The output consists of one new file (per entry in FILENAME), and, maybe also, a corrected error file (although the error is currently not properly calculated, as all correlation between pixels is neglected). Except for that data of the new file have been interpolated, there is no other difference compared to the input data. Extracted data can also be saved using the Euro3D format (use the keyword SAVEE3D or the user parameter 'savee3d'), and data of instruments using square-shaped spatial elements are easily converted to cube format using the tool p3d_rss2cube. In addition to the standard fits-file header keywords, plus the p3d-specific keywords added by previous routines, DAR-corrected spectrum images contain the following additional p3d-specific keywords:

IMOBX

The name of the input (extracted) science-object image file.

IMERR

The name of the DAR-corrected image error file.

IMTYPE

The p3d image type is set to 'p3d: DAR-corrected image'; the error-image type is 'p3d: DAR-corrected image – error'.

Input parameters:
filenameA string scalar or array that specifies the name of the extracted (and optionally flux-calibrated) spectrum-image file that will be DAR corrected.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
methodA scalar integer that specifies the method to use when shifting the data spatially. The three available methods are:

[0]

Use the theoretical equation of the wavelength-dependent refraction available in p3d_darc_amplitude; this is the default method.

1

Use the theoretical and height-integrated equation available in p3d_misc_sla_refro.

2

Use the polynomial fit to the empirically fitted center positions of a feature in the data themselves. The keyword EMPIRICALFIT is set automatically when this method is used.

The default value is: 0
refwlA scalar floating-point type value that specifies e reference wavelength. By default, the center velength is used (crval + cdelt * npix / 2). The default value is: wl.r. / 2
spaxelscaleA scalar floating-point type value that specifies e scale of each (square-shaped) spatial element, e unit is arcsec / element. The default value is: 1.0
posangA scalar floating-point type value that specifies the position angle of the IFU during the observations. The unit is degrees [°]. When this keyword is absent, the value is searched for in the data header. The default angle is 0°. The default value is: 0.0
airmassA scalar floating-point type value that specifies the (effective) air mass during the observations. When this keyword is absent, the value is searched for in the data header.
parangA scalar floating-point type value that specifies the (average) parallactic angle during the observations. The unit is degrees [°]. When this keyword is absent, the value is searched for in the data header. When it is not found there either, it is calculated using the properties that follow below.
pressureA scalar floating-point type value that specifies he (average) pressure during the observations. The nit is millibar [mbar]. When neither this keyword or the user parameter 'airpressure' is specified, he pressure is searched for in the data header. The default value is: 800.0
temperatureA scalar floating-point type value that specifies e temperature during the observations. The unit degrees Celcius [°C]. When neither this keyword r the user parameter 'airtemperature' is ecified, the temperature is searched for in the ta header. The default value is: 8.0
relhumidityA scalar floating-point type value that cifies the relative humidity during the ervations. The unit is per cent [%]. When ther this keyword nor the user parameter rrhumidity' is specified, the relative humidity searched for in the data header. The default value is: 10.0
latitudeA scalar floating-point type value that specifies the latitude of the observatory where the observations were made. The unit is degrees [°]. When this keyword is absent, the latitude is searched for in the data header.
obselevationA scalar floating-point type value that specifies the elevation of the observatory where the observations were made. The unit is meters [m]. When this keyword is absent, the elevation is searched for in the data header.
templapserateA scalar floating-point type value that es the temperature lapse rate in the here. The unit is degrees Celcius per meter This value is used with the alternative that calculates the refraction offset sc_sla_refro). The default value is: 0.0065
empiricalfitSet this keyword to attempt a fit to the data using a two-dimensional Gaussian profile. By default, a star is searched for at the center of the field; this can be modified using the FITX0 and FITY0 keywords. The empirical fit is shown together with the two theoretical solutions in the automatically generated postscript file. Note! The empirical fitting may fail if the FITX0 and FITY0 keywords are set to improper values, or if there is no clear peak in the data.
fitx0A scalar floating-point type value that specifies the starting //x// coordinate of the star that is to be fitted empirically in the data. Note! The coordinate refers to a frame where the data have been rotated to have North up and East to the left – disregarding the position angle! A nonzero position angle must be properly considered (and this might be a bit messy). Note! Only one value can be specified, even if FILENAME contains several entries.
fity0A scalar floating-point type value that specifies the starting //y// coordinate of the star that is to be fitted empirically in the data. Note! The coordinate refers to a frame where the data have been rotated to have North up and East to the left – disregarding the position angle! A nonzero position angle must be properly considered (and this might be a bit messy). Note! Only one value can be specified, even if FILENAME contains several entries.
ra_offsetThis scalar floating-point type value can be used to offset the value of the right ascension read from the header of the spectrum-image file(s). The default value is: 0.0
dec_offsetThis scalar floating-point type value can be used

  • offset the value of the declination read from

he header of the spectrum-image file(s). The default value is: 0.0
noguiThe properties GUI is not shown if this keyword is set.
deadfibersfileAn optional scalar string with the name of a file that lists dead, unused, and low-transmission fibers. The file must conform to the standard of the regular dead-fibers files of p3d. If this keyword is not set, the corresponding file is used as provided in the instrument-specific parameter file PARFILE.
nomaskIf this keyword is set, the spatial elements on the edges of the DAR-corrected data cube, which would have been using non-existing data, ARE NOT masked with the value set using MASKVAL. The default is otherwise to perform this masking.
maskvalA scalar floating-point type value used to mask spatial elements that should not to be used after the differential atmospheric refraction correction. The default value is: 0.0
masklowtrSet this keyword to also mask low-transmission marked fibers.
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
savee3dThe output data are saved in the Euro3D (e3d) format in addition to the usual RSS format when this keyword is set.
postableAn optional scalar string with the name of a fiber positition-table file. This keyword is only required if the data were not reduced using p3d. In that case, the header keywords IMTYPE (and POSTAB) are undefined; p3d uses these keywords to determine which file to use. This keyword is only considered if SAVEE3D is set.
userparfileA scalar string with the name of a file with user- defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'compress'

corresponds to COMPRESS.

'allinone'

corresponds to ALLINONE.

'nthreads'

corresponds to NTHREADS.

'rebuildlib'

forces a rebuild of the C library of routines.

'dontuseCroutines'

corresponds to NOC.

'savee3d'

corresponds to SAVEE3D.

'opfx'

corresponds to OPFX.

'darsfx'

To specify the data output file suffix.

'a2'

To use A3 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A3 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Letter instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'ra_offset'

corresponds to RA_OFFSET.

'dec_offset'

corresponds to DEC_OFFSET.

'templapserate'

corresponds to TEMPLAPSERATE.

'airpressure'

corresponds to PRESSURE.

'airtemperature'

corresponds to TEMPERATURE.

'airrhumidity'

corresponds to RELHUMIDITY.

ofilenameXThis keyword returns the full name of the DAR-corrected spectrum-image file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
nthreadsA scalar integer that specifies how many threads to use in the parallelized parts. The default is to use the maximum number available, but no more than 16: max = !cpu.hw_ncpu < 16. The default value is: max
logfileThis keyword specifies the name of an optional log file.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
cinvSet this keyword to plot white lines on a black background instead of the default, which is to plot black lines on a white background.
allinoneUnset this keyword to write the output data to separate files, instead of writing the DAR-corrected spectra to the first (DATA) extension, the DAR-corrected error image to the ERROR extension, the input bad-pixels mask to the MASK extension, the saturated-pixels mask image to the SATMASK extension, and the input data-quality mask image to the QUALITY extension. The default value is: 1
noCSet this keyword to not use any C routines.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine opening message will not be written to the log file.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution this variable contains the DAR-corrected spectrum image when the routine exits.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) As a stand-alone tool from the IDL command line

In addition to the extracted, and preferrably flux calibrated, spectrum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments:

files = ['objxfile_1.fits.gz', 'objxfile_2.fits.gz']
parfile = 'larr2k.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
detector = 0 ;; Required with multi-detector configurations.
p3d_darc, files, parfile, userparfile=userparfile, opath=opath, $
    logfile=opath + 'dred.log', /verbose, /empiricalfit

ii) As a stand-alone tool from a terminal, using the IDL VM

In addition to the extracted, and preferrably flux calibrated, spectrum images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments. The recommended approach is to use the provided shell script p3d_darc_vm.sh or the Python tool p3d_dispatch.py.

# Using the shell script, making use of the p3d queue system:
parfile = larr2k.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_darc_vm.sh exobjectfile.fits.gz $parfile \
    /empiricalfit userparfile=$userparfile opath=$opath \
    logfile=$opath/dred.log /verbose

# Using the Python tool, without using the p3d queue system:
parfile = larr2k.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_dispatch.py --noqueue darc exobjectfile.fits.gz $parfile \
    /empiricalfit userparfile=$userparfile opath=$opath \
    logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_darc_vm.sh userparfile=$userparfile" and -NOT- "p3d_darc_vm.sh us=$userparfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the two output variable parameters OFILENAME, and OUT.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to ii), but is executed from the Python command line. Here is an example:

# Launch p3d_darc in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
parfile = 'larr2k.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
commandline = 'exobjectfile.fits.gz ' + parfile + ' /empiricalfit' +
  ' userparfile=' + userparfile + 'opath=' + opath +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('darc', commandline, noqueue=True)

p3d_fluxcal.pro, line 618, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_fluxcal, filename, parfile, out, dout, dfilename=, sensfunc=, extinctionfile=, airmass=, exptime=, ext_waveunit=, detector=, /compress, /savee3d, postable=, userparfile=, ofilename=, opath=, opfx=, logfile=, loglevel=, /allinone, cmdline=, /gui, /subroutine, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, /help

This tool is used to flux calibrate extracted spectrum images using an already prepared flux-sensitivity function.

Observed data are provided in counts, using the so-called analog-to-digital unit (ADU). Flux calibration is the process where ADUs are converted to a physical unit. Spectra are in p3d flux calibrated in two steps. At first a sensitivity function is created with observations of a standard star, which are compared to already existing calibration data of the same star. The tool p3d_fluxsens is used to create such a sensitivity function; the same interactive functionality is provided in the spectrum viewer p3d_sv. In a second step, the sensitivity function is used to flux calibrate observed data of science objects; this tool is used in that second step.

In comparison to how flux calibration is performed using IRAF, the tool p3d_fluxsens corresponds to the modules "standard" and "sensfunc" in the "onedspec" package, while this tool, p3d_fluxcal, corresponds to the "calibrate" module in the same package. See the links section below for information on how to flux calibrate p3d data using these IRAF routines if there is need for it.

Output-data files, consisting of flux calibrated spectrum images and corresponding spectrum error images are by default written to the same file, in consecutive extensions. They are instead written to separate files if the keyword ALLINONE or the user parameter 'allinone' is unset. There is no real advantage to either approach; although, if separate files are used and the error file, the input bad-pixels mask image file, or the input saturated-pixels mask image file are renamed, they will not be found by other tools.

Required input to flux calibrate data

The extracted spectrum images that are to be flux calibrated must be provided in the row-stacked-spectrum (RSS) format; this is the format of spectrum images extracted with p3d. The spectrum images must also be wavelength calibrated.

The first requirement is a sensitivity-function (fits) file, which in its zeroth extension contains a (one-dimensional) spectrum in the magnitudes physical unit [mag]. The most straightforward approach to create such a sensitivity function is to use p3d_fluxsens, or the sensitivity-function tab in the spectrum-viewer tool p3d_sv. However, it is equally possible to use a sensitivity function created with the IRAF module "onedspec/sensfunc". The sensitivity-function file is specified with the keyword SENSFUNC.

A second option is to use a file with extinction data, as well as an appropriately estimated value of the air mass. The extinction-data file must contain the extinction as function of wavelength, in the unit [mag/airmass]. Please, see the routine p3d_misc_fluxcal_read_file_atmospheric_extinction for details on the required and accepted format of these files. Both fits files and plain-text files are accepted in general. The extinction data file is specified with the keyword EXTINCTIONFILE. It is possible to specify the wavelength unit using the keyword EXT_WAVEUNIT if the automatically determined wavelength unit is incorrect.

The air mass is estimated using keywords in the data header, see the routine p3d_misc_obsprop. Whenever possible – if enough header keywords are available – an attempt is made to calculate an effective air mass that accounts for observing conditions at the beginning, in the middle, and at the end of the exposure. It is possible to instead specify a value on the air mass explicitly using the keyword AIRMASS.

A final required value is the exposure time of the observed standard star. This value is normally read from the header keywords EXPTIME and EXPTIMEM; the latter keyword specifies an additional value in milliseconds and is only available with some instruments. It is possible to instead specify a value on the exposure time explicitly using the keyword EXPTIME.

Details on how flux calibration is performed in p3d

The flux calibration is performed by multiplying observed spectra with the following factor for each pixel //j//:

f_j = 10 ^ ( 0.4 * airmass * ext_j ) / ( exptime * cdelt * 10 ^ ( 0.4 * sensitivityfunction_j ) ),

where //airmass// is the air mass, //ext// [mag/airmass] the extinction, //exptime// [s] the exposure time, //cdelt// [Å] the dispersion pitch of the observed spectrum, and //sensitivityfunction// [mag] the sensitivity function. The nominator is set to unity when extinction data are not used. The unit of flux-calibrated spectra is [erg/s/cm²/Å]; this is recorded in the fits-header keyword BUNIT.

Output filenames of flux calibrated spectrum-image files are created by appending an intermediate suffix to the input filenames. The suffix is read from the user parameter 'fcsfx' [_fluxcal].

Links

An alternative to do the flux calibration in p3d is available with the "standard", "sensfunc", and "calibrate" modules of the IRAF package "onedspec":

http://iraf.net/

Instructions to use the IRAF modules with data extracted using p3d are available at the "General hints and tips" page on the p3d documentation WIKI:

http://sourceforge.net/apps/mediawiki/p3d/index.php

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below.

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This tool takes one, or several, row-stacked spectrum (RSS) images as input. The output files always contain as many pixels as the input files. To create a spatial map of the intensities at any wavelength using this RSS file, it is necessary to make use of a fiber position-table file, as the tool p3d_sv does. Please, use the fiber position-table file defined in the respective instrument-specific parameter file. Extracted data can also be saved using the Euro3D format (use the keyword SAVEE3D or the user parameter 'savee3d'). In addition to the standard fits-file header keywords, flux calibrated spectrum images contain additional p3d-specific keywords:

BUNIT

The physical unit of the flux: erg/(s*cm^2*Angstrom).

IMERR

The name of the flux calibrated spectrum-error file. Note! This keyword is only written if ALLINONE is unset.

IMSFF

The name of the used sensitivity-function file.

IMTYPE

The p3d image type is set to 'p3d: flux calibrated RSS image'; the error-image type is 'p3d: flux calibrated RSS image – error'.

EXTNAME

Five extensions are used in the output-data file when ALLINONE is set:

  • The flux-calibrated spectrum image [DATA].

  • The flux-calibrated spectrum error image [ERROR].

  • The copied bad-pixels mask image [MASK].

  • The copied saturated-pixels mask image [SATMASK].

  • The copied data-quality mask image [QUALITY].

Input parameters:
filenameA scalar or an array of strings that specifies the names of the extracted RSS-image files, which are to be flux calibrated using the sensitivity function provided with the keyword SENSFUNC.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
dfilenameA scalar or an array of strings that specifies the names of the extracted spectrum-error data files to flux calibrate. The files must be ordered as in FILENAME. p3d attempts to retrieve the names of the error files automatically if possible, either by reading the ERROR extension or using the IMERR header keyword. Use this keyword if this is unsuccessful, or if the intention is to use custom error files.
sensfuncA scalar string that specifies the name of a sensitivity-function file, which was created using p3d_sensfunc or the spectrum-viewer tool p3d_sv.
extinctionfileA scalar string that specifies the name of an atmospheric-extinction file. This string is passed on to the routine p3d_misc_fluxcal_read_file_atmospheric_extinction, see that routine for detailed information regarding required and accepted formats.
airmassA scalar floating-point value that specifies the (effective) air mass during the observations of the standard star. Normally, a qualified quess is made to retrieve this value using header keywords and observing conditions, see p3d_misc_obsprop. Use this keyword if the intention is to use a custom value.
exptimeA scalar floating-point value that specifies the exposure time of the observed spectrum. Normally, this value is retrieved from the file header, see the header keyword EXPTIME and EXPTIMEM (which is used with milli-second values) in the instrument-specific keywords-list file. Use this keyword if the intention is to use a custom value.
ext_waveunitA scalar string that specifies the unit of the wavelength array in the atmospheric-extinction file. An attempt is normally made to set the unit automatically. Use this keyword if the automatically determined unit is incorrect. See the keyword WAVEUNIT in the routine p3d_misc_fluxcal_read_file_atmospheric_extinction for information regarding required and accepted values.
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
savee3dThe output data are saved in the Euro3D (e3d) format in addition to the usual RSS format when this keyword is set.
postableAn optional scalar string with the name of a fiber-positition table file. This keyword is only required if data were not reduced using p3d. In that case the header keywords IMTYPE (and POSTAB) are (probably) not set correctly; p3d uses these keywords to determine which file to use automatically. This keyword is only considered if SAVEE3D is set.
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'compress'

Corresponds to COMPRESS.

'savee3d'

Corresponds to SAVEE3D.

'allinone'

Corresponds to ALLINONE.

'opfx'

Corresponds to OPFX.

'fcsfx'

To specify the output file intermediate suffix of flux-calibrated data.

'ckpostable'

Corresponds to POSTABLE.

'crpostable'

Corresponds to POSTABLE.

'postable'

Corresponds to POSTABLE.

ofilenameXThis keyword returns the full name of the flux-calibrated spectrum-image file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
logfileThis keyword specifies the name of an optional log file when p3d_fluxcal is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
allinoneUnset this keyword to write the output data to separate files, instead of writing the flux calibrated spectra to the first (DATA) extension, the error image to the ERROR extension, and copying the bad-pixels mask image [MASK], the saturated-pixels mask image [SATMASK], and the data-quality mask image [QUALITY] extensions from the input file, if they exist. The default value is: 1
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine messages are not written to the log file.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution this variable contains the data cube image of the last entry in FILENAME when the routine exits.
Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

There is no need to setup the input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

In addition to your files with extracted spectra, it is necessary to pass the name of the (provided) instrument-specific parameter file, the name of the sensitivity-function file created using p3d_fluxsens or p3d_sv, an extinction-data file, as well as any additional arguments:

files = ['file1_oextr1.fits.gz', 'file2_oextr1.fits.gz']
parfile = 'vimos_hr.prm'
sensfunc = 'std-hr-orange-H600_crcl_oextr1_spectrum_fluxsens.fits.gz'
extinctionfile = '../../extinction_paranal.fits'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
p3d_fluxcal, files, parfile, sensfunc=sensfunc, $
    extinctionfile=extinctionfile, userparfile=userparfile, $
    opath=opath, logfile=opath + 'dred.log', /verbose

iii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your files with extracted spectra, it is necessary to pass the name of the (provided) instrument-specific parameter file, the name of the sensitivity-function file created using p3d_fluxsens or p3d_sv, an extinction-data file, as well as any additional arguments. The recommended approach is to use the provided shell script p3d_fluxcal_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
files = file1_oextr1.fits.gz,file2_oextr1.fits.gz
parfile = vimos_hr.prm
sensfunc = std-hr-orange-H600_crcl_oextr1_spectrum_fluxsens.fits.gz
extinctionfile = ../../extinction_paranal.fits
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_fluxcal_vm.sh $files $parfile sensfunc=$sensfunc \
    extinctionfile=$extinctionfile userparfile=$userparfile \
    opath=$opath logfile=$opath/dred.log /verbose

# Using the Python tool, without using the p3d queue system:
files = file1_oextr1.fits.gz,file2_oextr1.fits.gz
parfile = vimos_hr.prm
sensfunc = std-hr-orange-H600_crcl_oextr1_spectrum_fluxsens.fits.gz
extinctionfile = ../../extinction_paranal.fits
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
p3d_dispatch.py --noqueue fluxcal $files $parfile \
    sensfunc=$sensfunc extinctionfile=$extinctionfile \
    userparfile=$userparfile opath=$opath \
    logfile=$opath/dred.log /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_fluxcal_vm.sh userparfile=$userparfile" and -NOT- "p3d_fluxcal_vm.sh user=$userparfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the three output variable parameters OFILENAME, OUT, and DOUT.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from the Python command line. Here is an example:

# Launch p3d_fluxcal in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'file1_oextr1.fits.gz,file2_oextr1.fits.gz'
parfile = 'vimos_hr.prm'
sensfunc = 'std-hr-orange-H600_crcl_oextr1_spectrum_fluxsens.fits.gz'
extinctionfile = '../../extinction_paranal.fits'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
commandline = files + ' ' + parfile +
  ' sensfunc=' + sensfunc + ' extinctionfile=' + extinctionfile +
  ' userparfile=' + userparfile + ' opath=' + opath +
  ' logfile=' + opath + '/dred.log /verbose'
p3d_dispatch('fluxcal', commandline, noqueue=True)

p3d_ifsfit.pro, line 1915, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_ifsfit, filename, conffile, dfilename=, parfile=, cext=, spaxelscale=, spaxelarea=, dwl=, z=, vel_z=, redshift=, sshift=, posoffset=, extinctionfile=, /interactive, /psplot, /bindata, targetsn=, binlow=, binupp=, /binflipx, binmaps=, binmap_extension=, col_binidx=, col_x=, col_y=, ofilename=, /compress, /a2, /a3, /a5, /letter, /legal, /tabloid, /landscape, nxplot=, nyplot=, savefit=, /onlysavefit, /fitparsave, mpmaxiter=, mpxtol=, mpftol=, mpgtol=, /nowritebintables, /nowritemaps, detector=, posfile=, deadfibersfile=, userparfile=, nthreads=, /noC, logfile=, loglevel=, font=, cmdline=, gui=, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Fits any number of emission and absorption lines in integral-field spectroscopy data using a simple plain-text configuration file.

For an extracted integral-field spectroscopy data set, this tool fits any number of lines in any number of wavelength ranges. The lines can be either emission lines or absorption lines, and the profile type can be Gaussian, Lorentzian, Moffat, or a parameterized Voigt. The input spectra are fitted one-by-one or they can be binned. The wavelengths of fitted lines can be fixed relative to each other, but they can also all be fitted separately. Any type of line blend is allowed, but the line separation and the spectral resolution determine together if the fit is reliable or not.

The results of the fits are saved to binary-table fits files (unless the keyword NOWRITEBINTABLES is set). And when the elements are square shaped, such as in a cube, they are also saved as regular image files where the fitted parameters and their errors are saved to different extensions (unless the keyword NOWRITEMAPS is set). The tool p3d_ifsfit_mosaic produces publication-ready plots and images using the binary-table fits files and the image files can be viewed with any fits-capable tool, for example ds9.

The input spectrum files

Two different kinds of fits files with extracted spectra can be used with p3d_ifsfit: data cubes and row-stacked spectra (RSS).

Data cubes

Cubes are identified automatically when there is a DATA extension where NAXIS == 3. The spectrum dimension is assumed as the last index. The IFU angle is also set to 0.0. Non-existent spectra are identified as those that contain NaNs; the dead-fibers file is not used with these data. Error data can be provided in two ways: through the ERROR extension, or if that is unavailable, through the STAT extension; or in separate files using the same names on the extension. Data are not loaded in full with cubes, but for each wavelength range separately.

The wavelength array is setup using the header keywords NAXIS3, CRPIX3, CRVAL3, CDELT3 or CD3_3, CTYPE3, NAXIS1 and NAXIS2 in the DATA extension. Linear as well as logarithmic wavelengths are handled transparently; a logarithmic array results when the keyword CTYPE3 contains the string 'AWAV-LOG'.

Note: The header keyword BUNIT is ignored, which means that in the case of, for example, MUSE data, the results need to be multiplied with the factor in this keyword.

RSS images

RSS data require more setup than cubes; the setup parameters are taken from the p3d-specific instrument-parameter file (PARFILE; it is, in principle, possible to use RSS images not reduced using p3d, but then care must be taken to create corresponding setup files). Here follows detailed information about what is required to setup the fitting of an RSS image. The instrument-parameter file (PARFILE) should contain the following keywords (some are optional):

daxis

An integer that specifies the dispersion axis; which is either the x-axis (1) or the y-axis (2); this is an optional parameter, where a guess is made to determine the dispersion axis if this value is unavailable. The guess is made in the way that the dispersion axis is at first set to 1. The other value, 2, is used when CRVAL3 is read and is found to be <= 1.

keywordsfile

A string that provides the filename of an instrument-specific header-keywords file; that file must exist, but it may be empty. Three entries are read in the header-keywords file, viz. IFUANGLE, RIGHTASCENSION, and DECLINATION that provide the name of the header keyword with the IFU angle, the right ascension and the declination coordinates. The IFU angle is set to 0.0 degrees if the header entry is unavailable and the default keyword for RIGHTASCENSION is "RA" and for DECLINATION it is "DEC".

ndetector

An integer that specifies the number of detectors in the instrument-setup. This keyword is used with p3d-reduced data to automatically figure out which fiber position- and dead-fibers tables to use. The parameter is optional and the value could in any case be set to 1 for all custom setups.

spnum

An integer that specifies the number of spectra; this is a required parameter.

postable

A scalar string that specifies the name of an existing instrument- and IFU-specific fiber position-table file; this parameter is not used if the keyword POSFILE is set in the call to p3d_ifsfit, but one of these options must be provided. The format of this file is described in p3d_misc_read_postable.

crpostable

A scalar string that specifies the name of an existing VIMOS-specific fiber position-table file for detector-combined data. There should be little need to set this keyword in custom setups.

ckpostable

A scalar string that specifies the name of an existing VIMOS-specific fiber position-table file for detector-combined data. There should be little need to set this keyword in custom setups.

The wavelength array is setup using the header keywords NAXISx, CRVALx, CRPIXx, CDELTx, CTYPEx, and NAXISy; where x (y) is either 1 (2) or 2 (1), depending on the configured or guessed dispersion axis. Linear as well as logarithmic wavelengths are handled transparently; a logarithmic array results when the keyword CTYPE3 contains the string 'AWAV-LOG'.

Dereddening the spectra

Data are dereddened if both the extinction factor //c// is set and a plain-text two-column extinction-data file is provided; see the keywords CEXT and EXTINCTIONFILE. Wavelengths are assumed to be Angstrom [Å] unless the string "nm" is found somewhere in the extinction-data file (in that case, a proper scaling is made of the wavelength array).

Spectra are dereddened by scaling them with the factor: 10 ^ (c * ( ext_tabulated(wavelength) / ext_tabulated(Hbeta) – 1)). All spectra are dereddened right away after they are loaded with RSS files if the extinction factor is a constant; each section of wavelengths is dereddened separately for all spectra when using cubes.

The fitting procedure

The fits are made file per file, group per group, line per line, and spectrum or bin one after another, in that order. Each line fit is made according to the specifications in the line-configuration file, which options and exact format are detailed in p3d_ifsfit_confread.

Separate fits for each spatial element

Spectra are only fitted when they are not marked to belong to dead or unused fibers; notably, the presence of NaN data indicate unused data in data cubes. Each used spectrum is searched for non-finite data points, as well as leading and trailing zeros, which are removed from each array. The data are additionally dereddened if requested.

The result of each fit in terms of fitting parameters and characteristics are saved to a binary-table fits file (see below regarding the format of this file), unless the keyword NOWRITEBINTABLES is set. An image fits file is also saved when the spatial elements are square shaped (unless the keyword /NOWRITEMAPS is set), or if the input file is a data cube; the different parameters and their errors are saved to subsequent extensions.

Optionally, only a subset of the spectra are fitted if the SAVEFIT keyword is set to an array of integers that identifies those spectra; the outcome is then saved in plain-text files. Set the keyword ONLYSAVEFIT to skip the binary-table fits files altogether. A final option is to save the fitted parameters using the keyword FITPARSAVE.

Binned fits

Instead of fitting the spectra one by one, they can be binned using three different approaches: 1, provide the name of a bin map; 2, let p3d_ifsfit calculate bins using the tool VORONOI_2D_BINNING; or 3, provide an RSS image to fit that was already binned. These approaches are described next.

Provide the name of a bin map

A file containing a bin map can be specified for each entry in the FILENAME argument using the keyword BINMAPS. The bin map file can be either a plain-text file or a fits file, which has been optionally compressed using the tool gzip. The keywords BINMAP_EXTENSION, COL_BINIDX, COL_X, and COL_Y can be used to adjust how fits files are read. The required format of the bin map file is described in more detail in the routine p3d_misc_bins_file_read.

If another tool than the p3d spectrum viewer was used to create the bin map file, the x-axis might be flipped with respect to how p3d_ifsfit expects the arrays to be ordered. If this should be the case, please set the keyword BINFLIPX.

Let p3d_ifsfit calculate the bin map

The read data are binned here by setting the keyword BINDATA. The bins are calculated with the tool VORONOI_2D_BINNING, but only if the routine voronoi_2d_binning.pro is placed in !p3d_path/contrib/ (download the file manually from its web page and place it in that directory if this functionality is wanted, see the links section).

The bins calculations are defined with the three keywords TARGETSN, BINLOW, and BINUPP. The first keyword defines the target signal-to-noise ratio used by the binning routine and the remaining two parameters define which bins to sum up on the dispersion axis before using those data with the Voronoi bins tool.

Provide p3d_ifsfit with an already binned RSS image

Under certain circumstances, p3d_ifsfit is able to identify binned cube data (which is then an RSS image) and thereafter reconstruct the two-dimensional maps of the fitted properties. These are the conditions to have p3d_ifsfit do this:

  • The header of the DATA extension must have the cunit keyword set to the value 'BINNUMBER', i.e., either one of the two keywords CUNIT1 or CUNIT2 must be set to this value. This keyword is used to identify the binned cube and also to figure out which axis is the dispersion axis.

  • The header of the DATA extension is searched for the presence of one of the three keywords CUBEFILE, CUBE, or ORIGIN, which should contain the name of the data cube file that was used to create the binned cube that has been binned to create an RSS image. The DATA extension header of that file is then used to read the CD1_1 and CD2_2 keywords, which define the size of the spatial elements. If this keyword is unpresent or if the file it points at cannot be read, it is necessary to specify the spatial-element size using the keyword SPAXELSCALE.

    The cube file header is used to retrieve and copy the world coordinate system (WCS) and to calculate an estimate of the total number of spatial elements in the bin map (regardless of if they are all used or not).

  • The required bin map is located using three approaches. At first, an attempt is made to read the extension set with the keyword BINMAP_EXTENSION from the binned cube image. If such an extension does not exist, a second attempt is made to read one of the three keywords BINMAP, BINFILE, or VORONOI from the DATA extension header. The contents of the read keyword must contain the name of a file where the bin map is stored. If such a file is found and it exists, an attempt is again made to read the extension with the keyword BINMAP_EXTENSION from that file. The third, and final, attempt is to use the files in the BINMAPS keyword.

    The format of the bin map must be the same as in the case when a bin map is provided with the BINMAPS keyword. The keyword BINFLIPX is set by default with these data.

    Should the second approach be successful and there is no information about the original cube file, an attempt is made to read the header of a bin map image from the same file. The name of the extension must be one of the following four strings: BINMAP_IMAGE, BIN_IMAGE, BINMAP, or VORONOI_IMAGE. The only keywords read are NAXIS1 and NAXIS2, which are used to calculate an estimate of the total number of spatial elements in the image (regardless of if they are all used or not).

All of the data in one file are read at once when the first axis is the dispersion axis. Otherwise, only the section used with the respective bins is read; this approach could result in drastically faster loading times with the biggest data sets.

Plots of the fits

Each group of line fits is optionally plotted to a PostScript file when the keyword PSPLOT is set. The PostScript file is formatted to contain NXPLOT columns and NYPLOT rows of plot panels, where the number of pages is set to accommodate all spatial elements or bins. The keywords NXPLOT, NYPLOT, and LANDSCAPE could be adjusted according to the used wavelength range to show as much detail as required.

Note: The number of pages becomes large using, for example, data cubes of MUSE. The size of the PostScript file is about 1GB with a regular 1x1 arcmin² data cube. The file is relatively quickly compressed using a tool such as pbzip2 (100M) or pigz (160M; use the keyword COMPRESS to do this automatically), but the conversion to a pdf file using ps2pdf (about 400M) takes much longer.

The format of the binary-table fits files

One binary-table fits file is written for each fitted line in each fitted wavelength-range group. The default prefix of the output filename is 'p3d_ifsfit', which can be changed using the OFILENAME keyword. The filename is suffixed with the id of the file, the group, and the line, for example, 'p3d_ifsfit_f1_g1_i101.fits'.

The zero-extension header of the fits file contains the following p3d-specific keywords:

IMTYPE

The p3d image type is set to 'p3d_ifsfit: fitted data'

USEBINS

An integer that specifies if data were binned (1) or not (0).

LINE_ID

An integer that specifies an identification number of the saved line; this positive value is freely chosen by the user in the line-configuration file. The number is also contained in the filename, directly after the sub-string '_i'.

GROUP_ID

An integer that specifies an identification number of the group that contained the line; this positive value is freely chosen by the user in the line-configuration file. The number is also contained in the filename, directly after the sub-string '_g'.

POLORDER

An integer that specifies the polynomial order of the fitted background.

TITLE

A string that provides a line-specific title string; the string can be specified in the line-configuration file; this keyword is used to label panels of different lines in p3d_ifsfit_mosaic automatically (using IDL-specific font commands). However, the string can be set to any value to match the needs of any plotting tool.

FUNCTION

A string that specifies the function used to fit the line. Possible values are: 'gaussian', 'lorentzian', 'moffat', and 'voigt'.

FUNCNPAR

An integer that specifies the number of free parameters in the fit. The value is 3 with the Gaussian and Lorentzian function types, and 4 with the Moffat and Voigt types.

POSANG

A floating-point type value that specifies the angle of the IFU.

SPXSCALE

A floating-point type value that specifies the scale of the spatial elements. The value was specified with the keyword SPAXELSCALE in the call to p3d_ifsfit, and alternatively, for cubes it is set automatically to the value |CD1_1| * 3600 when that keyword is present and is equal to CD2_2 * 3600 and is also non-zero. For square-shaped (circular) spatial elements, the scale is equal to the side (diameter) of the spatial element. For hexagons, the scale is twice the apothem.

SPXAREA

A floating-point type value that specifies the area of a spatial element. The value can be specified with the keyword SPAXELAREA upon calling p3d_ifsfit, but if it is unset it is calculated as follows. For square-shaped (circular) spatial elements, the area is the scale squared ((//scale// / 2) ^ 2 * pi). For hexagons, the area is sqrt(3) / 2 * //scale// ^ 2.

WAVELEN0

A floating-point type value that specifies the rest wavelength of the line as provided in the line-configuration file.

Z

A floating-point type value that specifies the redshift used in the calculations; this value was specified with the keyword Z or VEL_Z.

REDSHIFT

A floating-point type value that specifies a wavelength offset used in the calculations; this value was specified with the keyword REDSHIFT.

RA

A floating-point type value that specifies the right ascension coordinate of the center of the IFU (degrees; the sexagesimal string is provided in the comment field).

DEC

A floating-point type value that specifies the declination coordinate of the center of the IFU (degrees; the sexagesimal string is provided in the comment field).

DIDOFFST

An integer that specifies if the coordinates were offset (1) or not (0) using the keyword POSOFFSET.

The fitted parameters are contained as a binary table in the first extension, which has the name FITDATA. The parameters of the fit of each spatial element are contained in a row with the following twelve (binned data) or fifteen (unbinned data) columns:

1 - Bin index (Binidx)

Integers that specify the bin that the spatial element belongs to; this column is only used when the data are binned, and in that case the following four columns are not used.

1 - Row number (Rownum)

Integers that specify the row; this column is only used when the data are unbinned, and in that case the column above is not used.

2 - Spatial element id (Spatial_element_id)

Integers that specify the id of the spatial elements; this column is only used when the data are unbinned, and in that case the first column above, Binidx, is not used.

3 - E-W position (EW_position)

Floating-point type values that specify the x-axis coordinates of the spatial elements; this column is only used when the data are unbinned, and in that case the first column above, Binidx, is not used.

4 - S-N position (SN_position)

Floating-point type values that specify the y-axis coordinates of the spatial elements; this column is only used when the data are unbinned, and in that case the first column above, Binidx, is not used.

5 - Continuum

Floating-point type values that specify the value of the background fit at the wavelength of the fitted line.

6 - Intensity

Floating-point type values that specify the integrated line fluxes.

7 - Intensity error (Intensity_error)

Floating-point type values that specify the integrated line flux errors.

8 - Width

Floating-point type values that specify the line widths.

9 - Width error (Width_error)

Floating-point type values that specify the line width errors.

10 - Wavelength

Floating-point type values that specify the wavelengths.

11 - Wavelength error (Wavelength_error)

Floating-point type values that specify the wavelength errors.

12 - MPFIT status (MPFIT_status)

Integers that specify the return codes of the calls to mpfit.

13 - MPFIT niter (MPFIT_NITER)

Integers that specify the used numbers iterations of mpfit.

14 - MPFIT chisq (MPFIT_chisq)

Floating-point type values that specify the chi-square values of the fits.

15 - use

Byte-sized values that indicate if the fits are useful. The following codes are co-added to allow a further revised use: 0: The spatial element marked as "dead". 1: The fit for spatial element ok. 2: The fitted intensity is negative (positive) for emission (absorption) line. 4: The fitted intensity error is larger than the intensity.

The fitting algorithm

The profile-fitting procedure is based on the mpfit tool of Craig Markwardt (Markwardt 2009). The necessary files (mpfit.pro and the cmpfit package) can be downloaded separately from his website (see below), but are also downloaded semi-automatically if the installation instructions of p3d are followed. To speed up the calculations, the algorithm is implemented using the C-based tool cmpfit, which is parallelized using OpenMP, and implemented here as a dynamically loadable module (DLM).

The line-configuration file

The format of all components that go into the line-configuration file are described in the routine p3d_ifsfit_confread. An example file is available in p3d_path/data/master_ifsfit_lineconf.dat, which can be examined for examples on how to setup a custom configuration file. The file could be copied to your data-analysis directory, where it is first renamed to, for example, "p3d_ifsfit_lineconf.dat". Thereafter, its contents must be edited to work with the data it is supposed to be used with.

References

The p3d_ifsfit package is first mentioned in the following reference: C. Sandin, P. Weilbacher, F. Tabataba-Vakili, S. Kamann, and O. Streicher 2012, in: "Software and Cyberinfrastructure for Astronomy II", Radziwill N. M., Chiozzi G., eds, Proc. SPIE, 8451, 84510F-1~16, "Automated and generalized integral-field spectroscopy data reduction using p3d"

Markwardt, C.B. 2009, "Non-Linear Least Squares Fitting in IDL with MPFIT," in Astronomical Data Analysis Software and Systems XVIII, Quebec, Canada, eds. D. Bohlender, P. Dowler & D. Durand (ASP: San Francisco), ASP Conf. Ser., 411, 251

Links

ds9: http://ds9.si.edu

MPFIT: http://purl.com/net/mpfit

The Voronoi binning tool VORONOI_2D_BINNING (in the routine voronoi_2d_binning.pro) of Michele Cappellari can be downloaded here: http://www-astro.physics.ox.ac.uk/~mxc/software/

General information on how to learn more about this routine

Examples on how to use this tool are provided in the EXAMPLES section below.

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Input parameters:
filenameA string array with the filenames of FITS files with extracted spectra where line intensities are to be calculated.
conffileA scalar string with the name of a plain-text file that contains line-fitting configuration parameters, which can be read by p3d_ifsfit_confread.
Keyword parameters:
dfilenameAn optional string array with the names of error data files that correspond to the entries in FILENAME; the files must match. If this keyword is not specified, the error data file is searched for in the IMERR header keyword of each entry in FILENAME.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

cextThe use of this keyword specifies an extinction coefficient, which can be provided in several ways:

  • If cext is a constant floating-point type value, all data are dereddened using this value.

  • If cext is a pointer, it is assumed that it is an array of pointers that each contains a value on the extinction coeffifient for each spaxel. Note! This is only the case with binned data.

  • If cext is a string array with filenames, where each filename contains a value on the extinction coefficient for each spatial element, these files are used.

(Each file must be readable by p3d_ifsfit_io.)
spaxelscaleA floating-point type scalar value that specifies e size of individual elements [in arcsec]; this yword is unnecessary to specify with cubes that ntain a world coordinate system (WCS); in that se, the spaxelscale is set to |CD1_1| if D1_1| == CD2_2 and CD2_2 != 0. The default value is: 1.0
spaxelareaA floating-point scalar value that specifies the rea of an individual spatial element [in arcsec²]. t is unnecessary to specify this parameter if ARFILE is provided, in this case the area is alculated using the shape parameter in the iber-position table file. It is necessary to pecify SPAXELSCALE if the spatial-element shape is quare, otherwise the spaxelarea cannot be etermined. For circular and hexagon-shaped spatial lements it is not necessary to specify either PAXELSCALE or SPAXELAREA. The default value is: 1.0
dwlA scalar floating-point type value that defines a maximum allowed deviation from the provided center wavelengths (given in Ångström [Å]). The default value is: 1.0
zA scalar floating-point type value specifying the redshift of all regular lines. The default value is: 0.0
vel_zAs an alternative to Z, this scalar floating-point type value specifies the redshift of all regular lines as a velocity [km/s]. The redshift is recovered as: z = sqrt((1 + vel_z / c) / (1 – vel_z / c)) – 1 The default value is: 0.0
redshiftA scalar floating-point type value specifying an optional offset of all regular lines towards redder wavelengths; a negative value implies an offset towards bluer wavelengths. The default value is: 0.0
sshiftA scalar floating-point type value specifying an optional secondary redshift of all regular lines. The default value is: 0.0
posoffsetA two-element floating-point type value array that specifies an offset applied to the W-E, S-N coordinates when data are saved. The center of the the IFU is defined as [0.0, 0.0]. The default value is: 0.0, 0.0
extinctionfileA scalar string that specifies the name of a file that contains extinction data; with two columns, wavelength in Ångström [Å] and extinction, rows beginning with a semicolon are treated as comments; this variable must be set if CEXT is set.
interactiveIf this keyword is set, each line fit is followed by hitting the return key to continue.
psplotIf this keyword is set, all line fits are saved to PostScript files.
bindataSet this keyword to calculate Voronoi bins for the data before they are fitted. The Voronoi bins calculation is configured with the following three keywords. Note! This keyword is only available when the keyword BINMAPS is unset.
targetSNA scalar decimal value that defines the target ignal-to-noise value used when calculating Voronoi ins (using voronoi_2d_binning). Also see the note f the keyword BINDATA. ote! Together with the two keywords BINLOW and BINUPP, this keyword is only available when the routine file voronoi_2d_binning.pro is available in the directory !p3d_path/contrib/. The default value is: 200.0
binlowA scalar integer that defines the lower limiting bin of the wavelength range used when calculating Voronoi bins. The first bin is 1. The default bin is the middle bin (when the value is set to -1). Also see the notes of the two keywords BINDATA and TARGETSN. The default value is: -1
binuppA scalar integer that defines the upper limiting bin of the wavelength range used when calculating Voronoi bins. The first bin is 1. The default bin is the middle bin (when the value is set to -1). The default value is BINLOW. Also see the notes of the keywords BINDATA and TARGETSN. The default value is: binlow
binflipxSet this keyword to flip the x coordinates. This keyword is set by default when the input data are a binned cube. The default value is: 0, but 1 for binned cubes
binmaps

A string array – with the same number of elements as the FILENAMES argument – containing the names of plain-text or (binary-table) fits files with three-column rows associating each spatial element with a (Voronoi) bin.

The plain-text and fits-file format is defined in the routine p3d_misc_bins_file_write.

Fits files must contain either the extension BIN_TABLE or VORONOI_TABLE that contains the three columns: BINIDX, X, and Y. The name of the extension and the columns can be set with the keywords BINMAP_EXTENSION, COL_BINIDX, COL_X, and COL_Y.

Each property must contain as many rows as there are (unbinned) spatial elements in the data. Additionally, the following three fits-file header keywords are acknowledged: BINLOW, BINUPP, and TARGETSN.

Plain-text files must contain three columns, where the first two columns specify the X and Y coordinates of each bin and the third column specifies the bin number; negative values indicate that the spatial element is not binned. The number of rows must not exceed the number of elements in the loaded RSS images or data cubes.

Note: The four keywords BINDATA, TARGETSN, BINLOW, and BINUPP are ignored when this keyword is set.

binmap_extensionA scalar string that specifies the name of the extension of fits files specified with BINMAPS that contains the bins map. The default extension is BIN_TABLE or VORONOI_TABLE. See the keyword BINMAPS and the routine p3d_misc_bins_file_read for more details. The default value is: 'BIN_TABLE' || 'VORONOI_TABLE'
col_binidxA scalar string that specifies the name of the column that contains the bin indices for all spatial elements in fits files specified with the keyword BINMAPS. The default column name is BINIDX or BINNUM. See the keyword BINMAPS and the routine p3d_misc_bins_file_read for more details. The default value is: 'BINIDX' || 'BINNUM'
col_xA scalar string that specifies the name of the column that contains the //x// positions of all spatial elements in fits files specified with the keyword BINMAPS. The default column name is X. See the keyword BINMAPS and the routine p3d_misc_bins_file_read for more details. The default value is: 'X'
col_yA scalar string that specifies the name of the column that contains the //y// positions of all spatial elements in fits files specified with the keyword BINMAPS. The default column name is X. See the keyword BINMAPS and the routine p3d_misc_bins_file_read for more details. The default value is: 'Y'
ofilenameThis scalar string specifies a prefix of the output filenames. The default value is: 'p3d_ifsfit'
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
a2Set this keyword to use the A2 paper format with PostScript and PDF output instead of A4.
a3Set this keyword to use the A3 paper format with PostScript and PDF output instead of A4.
a5Set this keyword to use the A5 paper format with PostScript and PDF output instead of A4.
letterSet this keyword to use the US Letter paper format with PostScript and PDF output instead of A4.
legalSet this keyword to use the US Legal paper format with PostScript and PDF output instead of A4.
tabloidSet this keyword to use the US Tabloid paper format with PostScript and PDF output instead of A4.
landscapeSet this keyword to draw the plots in the landscape orientation instead of portrait. The default value is: 0
nxplotA scalar integer that specifies the number of plot panels horizontally of each page of PostScript output.
nyplotA scalar integer that specifies the number of plot panels vertically of each page of PostScript output.
savefitAn optional integer array specifying which spatial-element output is saved.
onlysavefitSet this keyword to inly save the data of the spatial elements of SAVEFIT, ignoring the rest.
fitparsaveSet this keyword to store the resulting fitting parameters in an IDL save file. The default value is: 1
mpmaxiterA scalar integer specifying the maximum number of iterations performed by MPFIT [MAXITER = 20].
mpxtolA scalar floating-point type value that specifies XTOL of MPFIT [XTOL = 1d-10].
mpftolA scalar floating-point type value that specifies FTOL of MPFIT [FTOL = 1d-10].
mpgtolA scalar floating-point type value that specifies GTOL of MPFIT [GTOL = 1d-10].
nowritebintablesSet this keyword to avoid writing the binary table utput files containing the fitted properties.
nowritemapsSet this keyword to avoid writing the spatial map images of the fitted properties; these files are, in any case, only written for data sets using square-shaped spatial elements (including data cubes).
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
posfileA scalar string that specifies the filename of the spatial element position-table file; this file must follow a specific format, see p3d_misc_read_postable.
deadfibersfileAn optional scalar string with the name of a file that lists dead/unused/low-transmission fibers. The file must conform to the standard of the regular dead-fibers files of p3d. If this keyword is not used, the corresponding file is used as provided in the instrument-specific parameter file PARFILE.
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file:

'compress'

See COMPRESS.

'a2'

To use A2 instead of A4.

'a3'

To use A3 instead of A4.

'a5'

To use A5 instead of A4.

'letter'

To use US Letter instead of A4 with PostScript and PDF output.

'legal'

To use US Legal instead of A4 with PostScript and PDF output.

'tabloid'

To use US Tabloid instead of A4 with PostScript and PDF output.

'psusletter'

To use US Letter instead of A4 with PostScript and PDF output.

'nthreads'

The number of threads to use in the parallel calculations.

'dontuseCroutines'

Corresponds to NOC.

'deadfibersfile'

A file that describes which fibers are either dead, unused, or have a low transmission value.

nthreadsA scalar integer that specifies how many threads to use in the parallelized line-profile fitting made using the C-based routine p3d_cifsfit. The calculation speed scales nearly linearly with the number of threads. The default is to use the maximum number available, max = !cpu.hw_ncpu. The default value is: max
noCSet this keyword to not use any C routines.
logfileThis keyword specifies the name of an optional log file when p3d_ifsfit is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpSet this keyword to show this routine documentation, and then exit.
Output parameters:
parA double array with the fitted parameters.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) As a stand-alone tool from the IDL command line

In addition to your extracted and calibrated spectrum-image files, which may be RSS images or datacubes, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments to p3d:

filenames = ['run157_00988_imcmb_oextr.fits.gz', $
             'run157_00999_imcmb_oextr.fits.gz']
lineconf = 'p3d_ifsfit_setup.dat'
parfile = 'larr2k.prm'
p3d_ifsfit, filenames, lineconf, parfile=parfile, redshift=2.5, $
    dwl=3.0, /verbose

With data cubes, such as those of MUSE it is not necessary to use the PARFILE keyword.

ii) As a stand-alone tool from a terminal, using the IDL VM

In addition to your extracted and calibrated spectrum-image files, which may be RSS images or datacubes, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments to p3d. The recommended approach is to use the provided shell script p3d_ifsfit_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
filenames = run157_00988_imcmb_oextr.fits.gz,\
            run157_00999_imcmb_oextr.fits.gz
lineconf = p3d_ifsfit_setup.dat
parfile = larr2k.prm
p3d_ifsfit_vm.sh $filenames $lineconf parfile=$parfile redshift=2.5 \
    dwl=3.0 /verbose

# Using the Python tool, without using the p3d queue system:
filenames = run157_00988_imcmb_oextr.fits.gz,\
            run157_00999_imcmb_oextr.fits.gz
lineconf = p3d_ifsfit_setup.dat
parfile = larr2k.prm
p3d_dispatch.py --noqueue ifsfit $filenames $lineconf \
    parfile=$parfile redshift=2.5 dwl=3.0 /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_ifsfit_vm.sh parfile=$parfile" and -NOT- "p3d_ifsfit_vm.sh pa=$parfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to ii), but is executed from the Python command line. Here is an example:

# Launch p3d_ifsfit in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'run157_00988_imcmb_oextr.fits.gz,' +
        'run157_00999_imcmb_oextr.fits.gz'
lineconf = 'p3d_ifsfit_setup.dat'
parfile = 'larr2k.prm'
commandline = files + ' ' + lineconf + ' parfile=' + parfile +
  ' redshift=2.5 dwl=3.0 /verbose'
p3d_dispatch('ifsfit', commandline, noqueue=True)

p3d_ifsfit_mosaic.pro, line 2054, last changed at 2016-07-06 by christersandin (revision 4290)

p3d_ifsfit_mosaic, filenames, lines, offset=, scale=, override=, nvertices=, /psplot, /logscale, contour=, colmini=, colmaxi, colminv=, colmaxv=, colminw=, colmaxw=, colminc=, colmaxc=, intensity=, kinematic=, continuum=, width=, colortable=, /invert, /cinv, bottom=, /sexagesimal, rightascension=, declination=, /samescale, /colorbar, /multibar, /reversebar, bartitle=, /barright, wcolorbar=, minor=, /yvertical, cstar=, csymbol=, csymsize=, cradius=, cspsym=, clinestyle=, symbolcolor=, /legendshow, legendoff=, /legendtop, /legendleft, /legendbox, set_character_size=, psfont_size=, c_triangulatemode=, c_levelsmfactor=, c_minvalue=, c_charsize=, c_color=, charsize=, charthick=, thick=, xomargin=, yomargin=, aspect=, /bmp, /gif, /jpeg, /quality, /png, /tiff, /transparent, outfilename=, pscolor=, /cmyk, /encapsulated, /a2, /a3, /a5, /letter, /legal, /tabloid, /landscape, paperxsize=, xsize=, xoffset=, paperysize=, ysize=, yoffset=, nxplot=, nyplot=, windex=, wxsize=, wysize=, /avantgarde, /bkman, /courier, /helvetica, /palatino, /schoolbook, /times, /bold, /book, /demi, /italic, /light, /medium, /narrow, /oblique, font=, set_font=, /tt_font, /isolatin1, bits_per_pixel=, logfile=, loglevel=, cmdline=, topwid=, logunit=, verbose=, /quiet, error=, /debug, /help

Produces publication-ready two-dimensional plots in the form of (encapsulated) PostScript, PDF, or bitmap files from data fitted with p3d_ifsfit.

Any number of binary-table fits files written by p3d_ifsfit are merged and the output can be presented in one panel per line, in nearly any geometrical configuration. Properties that can be viewed include maps of integrated intensities, velocities (from the line center), line widths, and continuum fluxes at the position of the selected line. One primary property is drawn in what appears as a bitmap image, and another secondary property can be drawn on top of the first using contours. It is only possible to select different properties for the primary and secondary plots.

The default is to write PostScript files; encapsulated output is used when ENCAPSULATED is set. The PostScript files are converted to pdf files on the fly when the keyword PDF is set (only available when the command ps2pdf (regular PostScript) or epstopdf (encapsulated PostScript) are available in the path). The output is instead a window on the screen when the keyword PSPLOT is unset. Alternatively, the window is written to a bitmap file using a default output filename when one of the keywords BMP, GIF, JPEG, PNG, or TIFF is set, or if the keyword OUTFILENAME contains one of the seven suffixes: '.bmp', '.gif', '.jpg', '.jpeg', '.png', '.tif', or '.tiff'.

Basic setup

The binary-table fits files to use in the plotting are specified with the FILENAMES argument, which can contain wildcards (as parsed by the IDL routine FILE_SEARCH). The array of strings can contain files for more lines than the ones to be plotted; the selection is made using the second argument LINES, which specifies which line id numbers to use.

The default primary property is the line intensity (INTENSITY). The three other properties that can be used instead are: the velocity (KINEMATIC), the line width (WIDTH), and the continuum flux (CONTINUUM). The primary property is drawn with one colored polygon per spatial element.

The secondary property is only shown if CONTOURS is set to 1 or 2, the default value is 0. The property is chosen by setting one of the property keywords to 2, for example, CONTINUUM = 2. Only one property can be used as either the primary or the secondary property.

All properties are by default shown with a linear scale. Logarithmic values are shown instead in all panels by setting the keyword LOGSCALE. Exactly which fitted values are shown, is selected with the keyword OVERRIDE.

Geometrical setup of plots in the figure

Plots are organized in columns and rows, according to how many panels are selected for plotting. The number of plots per column (row) is defined with the keyword NXPLOT (NYPLOT). The aspect is undefined by default and the plotted regions are very likely too wide or too high; it is therefore recommended to set ASPECT = 1.0. The aspect calculations account for the plotted regions, the font size and the titles, and one or multiple color bars with their own titles. Additional space for titles can be specified using the keywords XOMARGIN and YOMARGIN.

If the wish is that a final plot should contain different properties of the same line (instead of the same property for different lines), it is necessary to run p3d_ifsfit_mosaic once per property and then combine the output with an external tool (such as "xfig", "inkscape", or "Gimp" or "Photoshop" with bitmap files).

The size of the plotted region in (encapsulated) PostScript files is defined with the keywords XSIZE and YSIZE (centimeters; it is only necessary to set one of these keywords if the aspect ratio is also defined). The paper size is also defined with PostScript files, and it can be defined with the eight keywords: A2, A3, A5, LETTER, LEGAL, TABLOID, and PAPERXSIZE and PAPERYSIZE; the default is to use A4 paper (210x297mm). The plot region is by default centered on the paper, but the position can be set differently with the two keywords XOFFSET and YOFFSET. The default is also to plot the region in portrait orientation on the paper; set the keyword LANDSCAPE to instead use that orientation.

The size of the plotted region in windows and bitmap files is defined with the keywords WXSIZE and WYSIZE, in the pixel unit. If none of the bitmap-file keywords are set, the resulting plot is drawn in a window, which id is selected as the next free index. The window id can alternatively be set with the WINDEX keyword.

Colors

PostScript output is printed in color when PSCOLOR is set (this is the default). The default color model of PostScript output is RGB (red, green, blue), which is replaced with CMYK (cyan, magenta, yellow, and black) when the keyword CMYK is set.

The color table is set with the keyword COLORTABLE for both output to the screen and to PostScript files. Foreground and background colors are swapped by setting the keyword CINV. A colorbar is drawn when the keyword COLORBAR is set (see below).

Defining plotted intervals

The default interval covers the minimum to the maximum values of the plotted property. The interval can be defined for each property separately when the default value is not what is wanted by using the property-specific keywords COLMINx and COLMAXx, which can be provided as one value for all plotted panels or one value per panel.

All panels are plotted using the same scale, unless the keyword SAMESCALE is unset.

Axes

The unit of the two axes is arcsec. Set the keyword SEXAGESIMAL to use that representation instead. The right ascension and declination coordinates are taken from the binary-table fits file created by p3d_ifsfit, but alternative values can be specified with the keywords RIGHTASCENSION and DECLINATION. An offset can be applied to the coordinates using the keyword OFFSET.

The number of minor tick marks is set using the keyword MINOR. The vertical axis is drawn with rotated tick labels when the keyword YVERTICAL is set, otherwise they are horizontal as the horizontal-axis tick labels. The width of the lines that show the axes, as well as the drawn lines, is set with the keyword THICK. The length of the tick marks can be specified with the keyword TICKLEN; negative values produce tick marks that go out of the figure instead of into it.

Color bars

The value range can be visualized using a horizontal color bar on top of the plots through setting the keyword COLORBAR; the color bar is positioned vertically to the right of the panel(s) by setting the keyword BARRIGHT. One color bar is shown for all panels, unless the keyword MULTIBAR is set; the MULTIBAR keyword is set automatically when SAMESCALE is unset.

The title of the bar is set according to which property is plotted. It is also possible to use a personally defined string with the keyword BARTITLE. The width of the color bar is by default 1.0 character heights for horizontal bars and 2.0 character heights for vertical colorbars, but the width can be set freely with the keyword WCOLORBAR.

The color bar scale can be offset using the keyword SCALE; this is useful with, for example, MUSE data, where fitted intensities should be multiplied with the scale factor 10^-20 (see the header-keyword BUNIT in the MUSE data cube).

Labels and fonts

All three kinds of fonts can be used that IDL handles. Each font is setup using the following keywords:

Hershey (vector) fonts

This simple font is used by default with plots in window and bitmap files, and it can also be used with PostScript files (but the outcome is anything but neat). Use Hershey fonts by setting the keyword FONT = -1.

Device (PostScript) fonts

This is the default font when plotting to PostScript (or PDF) files, and this font scheme cannot be used with windows or bitmap files. Use device fonts by setting the keyword FONT = 0. The default font is HELVETICA. Six other device fonts are directly available through the keywords: AVANTGARDE, BKMAN, COURIER, PALATINO, SCHOOLBOOK, and TIMES. Further device-font properties can be set with the eight keywords: BOLD, BOOK, DEMI, ITALIC, LIGHT, MEDIUM, NARROW, and OBLIQUE. Additional fonts can be used through the keyword SET_FONT. The device font size is set using the keyword PSFONT_SIZE, the default value is 10.0 [pt].

Truetype fonts

These fonts can be used with both PostScript (PDF) output and plots to windows and bitmap files. Selected using the keyword FONT = 1. Additionally, it is necessary to set the keyword TT_FONT, and the specific font is selected using the keyword SET_FONT.

The character size is set with the keyword CHARSIZE, which is a scale factor applied to the default character size, see PSFONT_SIZE for device font output; the default value is CHARSIZE = 1.0. The character thickness is specified with the keyword CHARTHICK.

Each panel shows a label when multiple panels are shown, set the keyword LEGENDSHOW to show the label also in a single-panel plot. The label consists of a character with a closing parenthesis followed by the contents of the title field of the fitted line, which may contain format sequences, such as '!9a!x' that produces an alpha character with a device font (see p3d_ifsfit). The character of the first and second panels are 'a' and 'b'; a character offset index can be applied with the LEGENDOFF keyword. For example, the character label of the first panel is instead 'g' when LEGENDOFF = 6. The default position of the label is in the bottom right part of the panel; the position is adjusted with the keywords LEGENDTOP and LEGENDLEFT. A box is drawn below the legend in the background color by setting the keyword LEGENDBOX.

Note: Title strings formatted using device-font control strings unlikely look correct when using Hershey fonts. Therefore, it is necessary to setup the title string to use the expected font.

Contours

Contours can be set to indicate either the primary or the secondary component by setting the keyword CONTOUR to 1 or 2, respectively. The contour display can be adjusted with the following five keywords: C_TRIANGULATEMODE, C_LEVELSMFACTOR, C_MINVALUE, C_CHARSIZE, and C_COLOR.

Additional objects

In addition to the primary and secondary properties, it is possible to show pre-defined locations with a symbol. The coordinates of each symbol are specified with the keyword CSTAR. The symbol shape and size are set using the keywords CSYMBOL and CSYMSIZE. Additional circles are drawn centered on the first element in CSTAR by using the keyword CRADIUS. The circles can be drawn with symbols (see CSPSYM) or lines (see CLINESTYLE) or both, by setting CSPSYM to a negative value. The color of the symbols and circles is defined with the keyword SYMBOLCOLOR.

Input parameters:
filenamesA string array of names of binary-table FITS files created using p3d_ifsfit. Each filename must contain the following sub-strings:

'_f??_'

specify the origin of the data in the frames and the files.

'_i???.*'

specify line id numbers.

The variable can contain wildcards as it is piped through the IDL routine FILE_SEARCH; see the manual of that routine for information on how to specify the wildcards. Note! This array of filenames must not be exclusive to the used line ids; such ids are specified with the lines argument. Note! Using the tool from the command line, wildcards must be properly escaped using both apostrophes and backslash (POSIX-based platforms); see the examples section for an example.
linesAn integer array that specifies the line ids to use. These ids must be a part of the filename (as provided in FILENAMES).
Keyword parameters:
offsetA two-column floating-point type array with as many rows as there are elements in FILENAMES; this parameter specifies how data in each frame or file are offset in coordinates when they are loaded. The keyword is useful, for example, when putting the central star of a planetary nebula at the coordinates 0.0, 0.0.
override

A scalar integer which value allows more control on which data to show. All data are shown by default when the use flag is > 0. Set this flag to the sum of the bits to show the respective values:

1

Fitted intensity marked as ok.

2

Fitted intensity was negative (positive) for an emission (absorption) line.

4

Fitted intensity error was larger than the intensity.

The default value is OVERRIDE = 7 (1 + 2 + 4).

The default value is: 7
scaleA floating-point type value that allows data to be scaled, if necessary. The default scaling factor is SCALE = 1.0. The default value is: 1.0
nverticesSpecifies the number of vertices(– 1) to use with circular-shaped spatial elements (used when the spatial-element parameter shape = 1). The default value is: 20
psplotThis keyword determines if the output is drawn on the screen (0) or to a PostScript file. This keyword is set by default, unless one of the five bitmap-file keywords BMP, GIF, JPEG, PNG, or TIFF is set. The default value is: 1
logscaleLinear (logarithmic) data are used when this keyword is unset (set); can be a two-element array, in which case the first (second) element is used with the primary (secondary) property. The default value is: 0
contourThis integer parameter can be set to 0, 1, or 2. Use the setting 1 (2) to draw contours of the primary (secondary) property in the current panel. The default value is: 0
colminiThis floating-point type scalar value or array specifies the lower value considered in intensity plots (line flux). The number of elements must equal the number of lines if specified as an array.
colmaxiThis floating-point type scalar value or array specifies the upper value considered in intensity plots (line flux). The number of elements must equal the number of lines if specified as an array.
colminvThis floating-point type scalar value or array specifies the lower value considered in velocity plots (line center). The number of elements must equal the number of lines if specified as an array.
colmaxvThis floating-point type scalar value or array specifies the upper value considered in velocity plots (line center). The number of elements must equal the number of lines if specified as an array.
colminwThis floating-point type scalar value or array specifies the lower value considered in width plots (line width). The number of elements must equal the number of lines if specified as an array.
colmaxwThis floating-point type scalar value or array specifies the upper value considered in width plots (line width). The number of elements must equal the number of lines if specified as an array.
colmincThis floating-point type scalar value or array specifies the lower value considered in continuum plots. The number of elements must equal the number of lines if specified as an array.
colmaxcThis floating-point type scalar value or array specifies the upper value considered in continuum plots. The number of elements must equal the number of lines if specified as an array.
intensityA scalar integer that specifies if the intensity (the line flux) is to be plotted as the primary (1) or the secondary (2) property; not used when unset. The default value is: 1
kinematicA scalar integer that specifies if the velocity (from the line center) is to be plotted as the primary (1) or the secondary (2) property; not used when unset. The default value is: 0
widthA scalar integer that specifies if the fitted line width is to be plotted as the primary (1) or the secondary (2) property; unused when unset. The default value is: 0
continuumA scalar integer that specifies if the fitted continuum (at the center of the fitted line) is to be plotted as the primary (1) or the secondary (2) property; not used when unset. The default value is: 0
colortableA scalar string or integer that specifies which color table should be loaded. If COLORTABLE is a string, the string must contain two parts separated by a comma, without any whitespace between. The first part before the comma specifies the color-table number in the file name specified after the comma; the file must be formatted as described in the documentation for MODIFYCT. As an example, COLORTABLE='25,brewer.tbl'. Components in the ColorBrewer color-table file "brewer.tbl" found in the resource directory can be specified explicitly by preceeding the string with "CB"; for example, COLORTABLE='CB25'. If COLORTABLE is an integer, the integer must be given in the range -4, -3, -2, -1, 0, ..., up to the maximum number of available tables in IDL, plus the available tables in the ColorBrewer file (resource/) "brewer.tbl". Select -4, -3, -2, or -1 to load the cubehelix, 2 * Califa, and the Sauron color tables, the other values use the respective color map as defined by LOADCT. These are the permitted values:

-4

Loads the cubehelix color table.

-3

Loads the Califa project intensity color table, as defined September 2012.

-2

Loads the Califa project velocity field color table, as defined September 2012.

-1

Loads the Sauron color table (default).

0-74 (IDL version >= 8.3)

Loads the corresponding color table with LOADCT.

0-40 (IDL version < 8.3)

Loads the corresponding color table with LOADCT.

'CB0'-'CB85'

Loads the corresponding ColorBrewer color table; the color tables are defined in the file "resource/brewer.tbl"

75-159 (IDL version >= 8.3)

Loads the ColorBrewer color table corresponding to this number after 75 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

41-125 (IDL version < 8.3)

Loads the ColorBrewer color table corresponding to this number after 41 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

'x,ctfile'

Loads the x:th entry ('x' must be an integer) in the color-table file 'ctfile.

The default value is: -1
invertSet this keyword to invert the loaded color table.
cinvSet this keyword to use white on a black background instead of black on a white background. The default value is: 0
bottomReturns a scalar integer with the used bottom color index.
sexagesimalSet this keyword to use labels with the sexagesimal format instead of arcsec.
rightascensionA scalar floating-point type value used instead of the rightascension coordinate stored in the binary-table FITS file.
declinationA scalar floating-point type value used instead of the declination coordinate stored in the binary-table FITS file.
samescaleSet this keyword to plot the same property in all panels using the same scale.
colorbarSet this keyword to draw a color bar. The default value is: 1
multibarSet this keyword to draw one color bar for each panel; this keyword is only used if COLORBAR is set.
reversebarSet this keyword to reverse the color bar.
bartitle

Set this scalar string to use a custom color-bar title. Default strings are:

WIDTH is set

'Line width [FWHM]'

angstr = string("305b) ;; " (Å)

CONTINUUM is set

'Continuum Intensity ' + $ '[erg s!u-1!ncm!u-2!narcsec!u-2!n' + $ angstr + '!u-1!n]'

KINEMATIC is set

'Velocity [km s!u-1!n]'

INTENSITY is set

'Intensity ' + $ '[erg s!u-1!ncm!u-2!narcsec!u-2!n]'

barrightSet this keyword to put the color bar to the right of the panel(s) instead of on top of it (them).
wcolorbarA scalar floating-point type value that defines the width of the color bar in character height (width, when BARRIGHT is set). The default value is 2.0 (1.0) when BARRIGHT is set (unset). The default value is: BARRIGHT ? 2.0 : 1.0
minorA scalar integer that defines the number of minor tick marks between major tick marks; see p3d_ifsfit_colorbar. The default value is: 4
ticklenA scalar floating-point type value that specifies the length of tick marks as fraction of the window size. A negative value results in tick marks pointing out of the plot box instead of into it. The default value is: 0.02
yverticalThe vertical axis is drawn with tick labels that are rotated to follow the axis when this keyword is set. The default value is: 1
cstarA two-column floating-point type array with a free number of rows that specifies the coordinates of additional single symbols to draw. A typical use is to indicate the position of a central star.
csymbol

A scalar integer that specifies which symbol to use with the additional symbols drawn using the CTAR keyword. Accepted symbols are the standard symbols of IDL, plus the ones in the astro-lib routine PLOTSYM; the latter symbols are used by adding 10 to their numbers. Hence, the available numbers and symbols are the following:

1

plus sign (+)

2

asterisk (*)

3

period (.)

4

diamond

5

triangle

6

square

7

x

10

circle (o)

11

downward arrow (upper limit)

12

upward arrow (lower limt)

13

5 pointed star

14

triangle

15

upside down triangle

16

left pointing arrow

17

right pointing arrow

18

square

The default value is: 13
csymsizeA scalar floating-point type value that defines the size of the symbol of CSTAR (in characters). The default value is: 1.0
cradiusA floating-point type array that defines the radii of any number of circles to draw centered on the coordinates of the first element in CSTAR.
cspsymA scalar integer that defines the plotting symbol used with the circles drawn when CRADIUS is set. Valid values are 1-7 and -7-(-1). The default value is: 3
clinestyle

A scalar integer that specifies the line style used with the circles drawn when CRADIUS is set; these are the standard IDL line styles:

0

solid line

1

dotted line

2

dashed line

3

dash dotted line

4

dash tripple-dotted line

5

long dashed line

The default value is: 3
symbolcolorA scalar integer that specifies the color index of the CSTAR symbol(s) and the associated circles and labels. The colors are defined in p3d_misc_colortable (0 <= SYMBOLCOLOR <= 6). The default value is: CINV ? 1 : 0
legendshowSet this keyword to show a legend also in single-panel plots.
legendoffA scalar integer that specifies an offset in the legend character array. For example, if the (first) panel is supposed to begin with 'g)' and not 'a)', set LEGENDOFF = 6.
legendtopSet this keyword to put the legend at the top of the panel instead of at the bottom of the panel. A legend is added to the panel when more than one panel are drawn. The legend text contains the line title string.
legendleftSet this keyword to put the legend at the left of the panel instead of at the right of the panel. A legend is added to the panel when more than one panel are drawn. The legend text contains the line title string.
legendboxSet this keyword to draw a box using the foreground color beneath the legend text.
set_character_sizeA two-element integer array that specifies the character size of the device. The same size is used with all plots. The default value is: 10, 12
psfont_sizeA scalar floating-point type value that cifies the size of the PostScript font; this word is used with the FONT_SIZE keyword to the ICE procedure. The default value is: 10.0
c_triangulatemode

calar integer that specifies which approach to in the triangulation of irregularly gridded a. The three available methods are:

:: Uses the default method of the CONTOUR procedure of IDL.

:: Uses the TRIGRID function of IDL.

:: Uses the GRIDDATA function of IDL, with the 'InverseDistance' method.

The default value is: 1
c_levelsmfactorcalar integer used to define the number of tours to draw; C_LEVELSMFACTOR >= 1. The default value is: 2
c_minvaluecalar floating-point type value that defines the imum value used when drawing contours. The default value is: LOGSCALE ? – 18 : 1d-18
c_charsizecalar floating-point type value that specifies size of the characters in contour labels (in ctions of the CHARACTER value). The default value is: 0.8 * CHARSIZE
c_colorcalar integer that defines the color index of wn contours. The colors are defined in _misc_colortable (0 <= C_COLOR <= 6). The default value is: SYMBOLCOLOR
charsizeA scalar floating-point type value that specifies the size of the characters in all text but the contour labels (in characters). Note! Since !p.multi is used to generate the plots it is difficult to predict what the actual character size will be. The default value is: 1.0
charthickA scalar floating-point type value that specifies the width of the characters in all text but the contour labels. The default value is: 1.0
thickA scalar floating-point type value that specifies the width of the drawn lines and the axes lines. The default value is: 1.0
xomarginA two-element floating-point type array that specifies the lower and upper width of the outside-plot margins (!X.OMARGIN). The default value is: YVERTICAL ? 4.5 : 10.0, 0.2
yomarginA two-element floating-point type array that specifies the lower and upper height of the outside-plot margins (!Y.OMARGIN). The default value is: 3.0, 0.2
aspectA scalar floating-point type value that fixes the aspect ratio xsize / ysize (PostScript output) or wxsize / wysize (window) of plotted regions. The y-axis size is recalculated using the x-axis size and ASPECT when set. Note! It is recommended, but not mandatory to set ASPECT = 1.0. Note! Since hardware fonts are used by default, it is difficult to calculate an exact aspect ratio. The result should, however, be a good approximation. The size can be set manually using the XSIZE and YSIZE keywords (for PostScript output, see p3d_ifsfit_postscript) or the WXSIZE and WYSIZE keywords (for Windows).
bmpSet this keyword to write a Microsoft Windows Version 3 device-independent bitmap file; this keyword is set automaticlly if the file suffix in OUTFILENAME is equivalent to '.bmp'.
gifSet this keyword to write a bitmap file in the Graphics Interchange Format; this keyword is set automatically if the file suffix in OUTFILENAME is equivalent to '.gif'. Set the keyword TRANSPARENT to make the background transparent.
jpegSet this keyword to write a compressed bitmap file in the Joint Photographic Experts Group (JPEG) format; this keyword is set automtically if the file suffix in OUTFILENAME is equivalent to '.jpg' or '.jpeg'. The compression rate is set with the keyword QUALITY.
qualityA scalar integer that defines the quality index, in the range of 0 (terrible) to 100 (excellent) for the JPEG file. The default value is 95, which corresponds to excellent quality. Lower values of QUALITY produce higher compression ratios and smaller files. The default value is: 95
pngSet this keyword to write a Portable Network Graphics (PNG) bitmap file using lossless compression; this keyword is set automatically if the file suffix in OUTFILENAME is equivalent to '.png'. The output file contains a 24-bit image. Set the keyword TRANSPARENT to make the background transparent, in this case the file contains an 8-bit image.
tiffSet this keyword to write a Tagged Image File Format (TIFF) bitmap file; this keyword is set automatically if the file suffix in OUTFILENAME is equivalent to '.tif' or '.tiff'.
transparentSet this keyword to make the background color transparent in GIF and PNG files.
outfilename

A scalar string that specifies the name and format of the output file. The possible formats are all defined with the suffix, which can take any of the following 9 values:

.eps

Encapsulated PostScript; see the keyword ENCAPSULATED.

.ps

Postscript, which in comparison to encapsulated postscript also defines the paper size.

.bmp

Microsoft Windows Version 3 device independent bitmap file; see the keyword BMP.

.gif

Graphics Interchange Format bitmap file; see the keyword GIF.

.jpeg or .jpg

Compressed JPEG bitmap image; see the keyword JPEG.

.png

Portable Network Graphics (PNG) bitmap; see the keyword PNG.

.tif or .tiff

Tagged Image File Format (TIFF) bitmap; see the keyword TIFF.

Note: The keywords ENCAPSULATED, BMP, GIF, JPEG, PNG, and TIFF are neglected when this keyword is set.

The default value is: default filename depends
pscolorSet this keyword to use color in PostScript output. The default value is: 1
cmykSet this keyword to generate PostScript output using the CMYK (cyan, magenta, yellow, and black) color model. The default PostScript color model is otherwise RGB (red, green, blue).
encapsulatedSet this keyword to generate encapsulated ostScript output; and unset it to produce regular ostScript output. The default value is: 1
pdfSet this keyword to replace the PostScript output with a PDF file; this is only possible when both 'ps2pdf' and 'epstopdf' are available on the system.
a2Set this keyword to use the A2 paper format with the PostScript output instead of A4.
a3Set this keyword to use the A3 paper format with the PostScript output instead of A4.
a5Set this keyword to use the A5 paper format with the PostScript output instead of A4.
letterSet this keyword to use the US Letter paper format with the PostScript output instead of A4.
legalSet this keyword to use the US Legal paper format with the PostScript output instead of A4.
tabloidSet this keyword to use the US Tabloid paper format with the PostScript output instead of A4.
landscapeSet this keyword to produce PostScript landscape output instead of portrait.
paperxsizeA scalar floating-point type value that specifies the paper width (centimeter); this keyword is not used if A2, A3, A5, LETTER, LEGAL, or TABLOID is set.
xsize

A scalar floating-point type value that specifies the width of the plot area (centimeter). The default width depends on other keywords as follows:

A2

57.4 (LANDSCAPE), 40.0 (portrait)

A3

38.0 (LANDSCAPE), 26.0 (portrait)

A5

19.0 (LANDSCAPE), 12.8 (portrait)

LETTER

25.94 (LANDSCAPE), 19.59 (portrait)

LEGAL

33.6 (LANDSCAPE), 19.6 (portrait)

TABLOID

41.2 (LANDSCAPE, i.e., LEDGER), 25.9 (portrait)

else (A4)

27.7 (LANDSCAPE), 19.0 (portrait)

The default value is: various
xoffsetA scalar floating-point type value that specifies the //x// position of the plotted graphics on the paper. The value is set by default to center the graphics on the paper. The situation is more complex when LANDSCAPE is set, XOFFSET and YOFFSET are then swapped etc.
paperysizeA scalar floating-point type value that specifies the paper height (centimeter); this keyword is not used if A2, A3, A5, LETTER, LEGAL, or TABLOID is set.
ysize

A scalar floating-point type value that specifies the height of the plot area (centimeter). The default width depends on other keywords as follows:

A2

40.0 (LANDSCAPE), 57.4 (portrait)

A3

26.0 (LANDSCAPE), 38.0 (portrait)

A5

12.8 (LANDSCAPE), 19.0 (portrait)

LETTER

19.59 (LANDSCAPE), 25.94 (portrait)

LEGAL

19.6 (LANDSCAPE), 33.6 (portrait)

TABLOID

25.9 (LANDSCAPE, i.e., LEDGER), 41.2 (portrait)

else (A4)

19.0 (LANDSCAPE), 27.7 (portrait)

The default value is: various
yoffsetA scalar floating-point type value that specifies the //y// position of the plotted graphics on the paper. The value is set by default to center the graphics on the paper. The situation is more complex when LANDSCAPE is set, XOFFSET and YOFFSET are then swapped etc.
nxplotA scalar integer that specifies the number of panels on the x-axis. The default value is: 1
nyplotA scalar integer that specifies the number of panels on the y-axis. The default value is: number of fitted and shown lines
windexA scalar integer that specifies the window index to use with plots drawn on the screen. The default value is: 0 or !d.window
wxsizeA scalar integer that specifies the width of screen plots and bitmap images, in pixels. The width of the currently active window is used if possible. The default value is: !d.window == – 1 ? 600L : !d.x_size
wysizeA scalar integer that specifies the height of screen plots and bitmap images, in pixels. The height of the currently active window is used if possible. The default value is: !d.window == – 1 ? 500L : !d.x_size
avantgardeSet this keyword to select the ITC Avant Garde PostScript font.
bkmanSet this keyword to select the ITC Bookman PostScript font.
courierSet this keyword to select the Courier PostScript font.
helveticaSet this keyword to select the Helvetica PostScript font.
palatinoSet this keyword to select the Palatino PostScript font.
schoolbookSet this keyword to select the New Century Schoolbook PostScript font.
timesSet this keyword to select the Times-Roman PostScript font.
boldSet this keyword to specify that the bold version of the current PostScript font should be used.
bookSet this keyword to specify that the book version of the current PostScript font should be used.
demiSet this keyword to specify that the demi version of the current PostScript font should be used.
italicSet this keyword to specify that the italic version of the current PostScript font should be used.
lightSet this keyword to specify that the light version of the current PostScript font should be used.
mediumSet this keyword to specify that the medium version of the current PostScript font should be used.
narrowSet this keyword to specify that the narrow version of the current PostScript font should be used.
obliqueSet this keyword to specify that the oblique version of the current PostScript font should be used.
font

This scalar integer defines which font tables to use. The three available options are:

- 1

Scalable vector fonts, default with window plots.

0

Device fonts, for use with PostScript output.

1

Truetype fonts; see the keywords SET_FONT and TT_FONT.

The default value is: PSPLOT ? 0 : 1
set_fontSet this scalar string to the name of the font to use, see the corresponding keyword of the DEVICE procedure for more details.
tt_fontSet this keyword to indicate that the font specified with SET_FONT is a truetype font. This keyword should be set when FONT is set to 1.
isolatin1See the documentation for DEVICE; this keyword is set by default. The default value is: 1
bits_per_pixelSee the documentation for DEVICE. The default value is 8. The default value is: 8
logfileThis keyword specifies the name of an optional log file when p3d_ifsfit_mosaic is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
cmdlineXThis keyword is used then the tool is launched by the p3d server.
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR. encountered. Use this keyword when debugging p3d.
helpXSet this keyword to show this routine documentation, and then exit.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) As a stand-alone tool from the IDL command line

Besides the obligatory output files of p3d_ifsfit, pass any additional arguments to configure the output:

p3d_ifsfit_mosaic, 'p3d-ifsfit_f1_g3_i???.fits.gz', [200, 700], $
    aspect=1.0, /legendbox, scale=1d-20, colmini=1d-17, $
    colortable=-1, /sexagesimal, nxplot=2, nyplot=1, barr=0, $
    psplot=0, /multibar

p3d_ifsfit_mosaic, 'p3d-ifsfit_f1_g3_i???.fits.gz', [200, 700], $
    aspect=1.0, /legendbox, scale=1d-20, colmini=1d-17, $
    colortable=-1, /sexagesimal, nxplot=2, nyplot=1, /multibar, $
    /logscale, /barrright, /landscape, xsize=28.0

ii) As a stand-alone tool from a terminal, using the IDL VM

Besides the obligatory output files of p3d_ifsfit, pass any additional arguments to configure the output. The recommended approach is to use the provided shell script p3d_ifsfit_mosaic_vm.sh or the Python tool p3d_dispatch.py:

files = "p3d-ifsfit_f1_g3_i\*.fits.gz" # The * needs to be escaped.
p3d_ifsfit_mosaic_vm.sh $files 200,700 aspect=1.0 /legendbox \
    scale=1d-20 colmini=1d-17 colortable=-1 /sexagesimal \
    nxplot=2 nyplot=1 barright=0 psplot=0 /multibar

files = "p3d-ifsfit_f1_g3_i\*.fits.gz" # The * needs to be escaped.
p3d_ifsfit_mosaic_vm.sh $files 200,700 aspect=1.0 /legendbox \
    scale=1d-20 colmini=1d-17 colortable=-1 /sexagesimal nxplot=2 \
    nyplot=1 /multibar /logscale /barright /landscape xsize=28.0

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_ifsfit_mosaic_vm.sh bartitle=$bartitle" and -NOT- "p3d_ifsfit_mosaic_vm.sh bart=$bartitle".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to ii), but is executed from the Python command line. Here are two examples:

# Launch p3d_ifsfit_mosaic in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'p3d-ifsfit_f1_g3_i???.fits.gz' # The ??? need to be escaped.
commandline = files + ' 200,700 aspect=1.0 /legendbox scale=1e-20 ' +
  'colmini=1e-17 colortable=-1 /sexagesimal nxplot=2 nyplot=1 ' +
  'barright=0 psplot=0 /multibar'
p3d_dispatch('ifsfit_mosaic', commandline, noqueue=True)

files = 'p3d-ifsfit_f1_g3_i???.fits.gz' # The ??? need to be escaped.
commandline = files + ' 200,700 aspect=1.0 /legendbox scale=1e-20 ' +
  'colmini=1e-17 colortable=-1 /sexagesimal nxplot=2 nyplot=1 ' +
  '/multibar /logscale /barright /landscape xsize=28.0'
p3d_dispatch('ifsfit_mosaic', commandline, noqueue=True)

p3d_rss2cube.pro, line 597, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_rss2cube, filename, parfile, out, dout, pixstart=, pixend=, wavestart=, waveend=, postable=, rightascension=, declination=, ra_offset=, dec_offset=, spaxelscale=, detector=, /compress, userparfile=, ofilename=, opath=, opfx=, nthreads=, logfile=, loglevel=, /allinone, cmdline=, /gui, /subroutine, stawid=, topwid=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

Converts an extracted image file from the RSS format to a data cube.

Extracted spectrum images are in p3d always saved in the so-called row-stacked-spectrum (RSS) format. Here, each row or column, depending on the dispersion axis, of the two-dimensional image is a spectrum. To create a spatial map at any wavelength, it is necessary to use a fiber position-table file that contains information on where each spectrum is positioned on the sky. Alternatively, p3d can also save the output data in the E3D format (Kissler-Patig et al. 2004), which includes a fiber position table in the first fits-file extension.

p3d can for some instruments convert the RSS image and its associated error image into a three-dimensional cube, placing the dispersion axis in the third dimension. Specifically, this can be done for all instruments that use square-shaped spatial elements – for example, VIMOS, FLAMES/Argus, PMAS/LArr, MPFS, SPIRAL, and IMACS. The cube is, if possible, rotated to have North upwards and East leftwards. Some instruments allow a non-zero IFU position angle. The cube is only rotated when the rounded position angle is a multiple of 90 degrees. Furthermore, by default, all pixels on the dispersion axis are included in the cube.

It is possible to delimit the pixel (wavelength) range with the keywords PIXSTART and PIXEND (WAVESTART and WAVEEND [Å]); a wavelength delimiter is ignored if a pixel delimiter is specified. The resulting cube can be loaded with the p3d spectrum viewer p3d_sv or with SAOImage DS9 (Joye & Mandel 2003).

The output-data files, consisting of the data cube, the error data cube, the bad-pixels mask data cube, the saturated-pixels mask data cube, and the data-quality data cube, are by default written to the same file in consecutive extensions. They are instead written to separate files if the keyword ALLINONE or the user-parameter 'allinone' is unset. There is no real advantage to either approach; although, if separate files are used and the error file, the output bad-pixels mask file, or the saturated-pixels mask file are renamed, they will not be found by other tools.

It is also possible to save up to eight spatial-map images in a data cube using the p3d spectrum viewer p3d_sv. At first, the desired wavelengths are selected and saved on the small-image middle row with a mouse click. Thereafter, the menu entry "Save spatial maps" should be selected in the file menu.

Fiber position-table files for use with PINGSoft

As an extension, fiber position-table files prepared for use with the PINGSoft tool (Rosales-Ortega 2011) are provided since revision 1602. Current versions of PINGSoft only accept the horizontal axis as dispersion axis. Therefore, with instruments that use the vertical axis instead, it is necessary to transpose the RSS images, including the wavelength-array information keywords, before they are loaded into PINGSoft.

References

Joye, W.A. and Mandel, E. 2003, in ADASS XII, Payne, H.E., Jedrzejewski, R.I., and Hook, R.N., eds., ASP Conf. Series, 295, 489-492, "New Features of SAOImage DS9".

Kissler-Patig, M., Copin, Y., Ferruit, P., Pécontal-Rousset, A., Roth, M.M. 2004, AN, 325, 159-162, "The Euro3D data format: A common FITS data format for integral field spectrographs".

Rosales-Ortega, F.F. 2011, New Astronomy, 16, 220-228, "PINGSOFT: An IDL visualisation and manipulation tool for integral field spectroscopic data".

Links

DS9

http://hea-www.harvard.edu/RD/ds9/

General information on how to learn more about this routine

Instrument parameter files (*.prm), which are required to run this routine, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, check "${p3d_path}/data/instruments/*/").

A default (master) user-parameter file is available in "!p3d_path/data/master_userparfile.dat". Copy this file to your data-reduction output directory, rename the file to, for example, "user_p3d.dat" and edit it to suit your needs.

Examples on how to use this routine are provided in the EXAMPLES section below, but also see the p3d documentation WIKI for instrument-specific tips and advice: http://sourceforge.net/apps/mediawiki/p3d

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

This routine takes one or several row-stacked spectrum (RSS) images as input; this is the default format of all extracted data of p3d.

The cube image contains additional p3d-specific keywords in addition to the standard fits-file header keywords:

IMTYPE

The p3d image type is set to 'p3d: Cube-format image'; the error image type is 'p3d: Cube-format image – error'.

IMERR

The name of the science-object image error file. Note! This keyword is only written if ALLINONE is unset.

LOPIXEL

The lower limiting pixel of the RSS image copied to the output data cube. The first pixel is 1; this keyword is only written if LOPIXEL > 1.

LOWAVE

The lower limiting wavelength of the RSS image copied to the output data cube; this keyword is only set if both CRVAL and CDELT are non-zero in the input RSS image, as well as if LOPIXEL > 1.

HIPIXEL

The upper limiting pixel of the RSS image copied to the output data cube. The first pixel is 1; this keyword is only written if HIPIXEL < npix – 1, where npix is the number of pixels on the dispersion axis.

HIWAVE

The upper limiting wavelength of the RSS image copied to the output data cube; this keyword is only set if both CRVAL and CDELT are non-zero in the input RSS image, as well as if HIPIXEL < npix – 1, where npix is the number of pixels on the dispersion axis.

EXTNAME

Five extensions are used in the output-data file when ALLINONE is set:

  • The extracted spectrum data cube [DATA].

  • The extracted spectrum error data cube [ERROR].

  • The bad-pixels mask data cube [MASK].

  • The saturated-pixels mask data cube [SATMASK].

  • The data-quality mask data cube [QUALITY].

Input parameters:
filenameA scalar or an array of strings that specifies the names of the extracted RSS-image files, which are to be converted to a cube format.
parfile

A scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format.

Already properly formatted files are available for all configured instruments in the directories p3d_path/data/instruments/*/. It is sufficient to specify the file basename, without the full path of the p3d subdirectory; for example: "nvimos_hr.prm".

Keyword parameters:
pixstartA scalar integer that specifies the lower pixel bound on the dispersion axis copied to the output data cube. The first pixel is 1. The default value is: 1
pixendA scalar integer that specifies the upper pixel bound on the dispersion axis copied to the output data cube. The first pixel is 1. The default value is: last
wavestartA scalar decimal value that specifies the lower acket pixel on the dispersion axis copied to the tput data cube; the specified wavelength ngstrom] is converted to a pixel (by truncation). te! This keyword is only considered if PIXSTART is not provided. te! The two header keywords CRVAL and CDELT must be available in the data header. The default value is: crval
waveendscalar decimal value that specifies the upper acket pixel on the dispersion axis copied to the tput data cube; the specified wavelength ngstrom] is converted to a pixel (ceil()). te! This keyword is only considered if PIXEND is not provided. te! The two header keywords CRVAL and CDELT must be available in the data header. The default value is: crval + npix * cdelt
postableA scalar string that explicitly specifies which fiber position-table file to use. Otherwise, if this keyword is not specified, the routine uses the instrument-specific parameter file (PARFILE) to select the file.
rightascensionA scalar floating-point type value that specifies the center RA coordinate of the IFU; this keyword can be used if the value in the data header is inaccurate. The unit is degrees [°].
declinationA scalar floating-point type value that specifies the center DEC coordinate of the IFU; this keyword can be used if the value in the data header is inaccurate. The unit is degrees [°].
ra_offsetA scalar floating-point type value that specifies an optional offset of the IFU RA coordinate; this keyword can be used to specify an offset to the value provided in the data header. The unit is arcsec. This keyword is only used if RIGHTASCENSION is unspecified. The default value is: 0
dec_offsetA scalar floating-point type value that specifies an optional offset of the IFU DEC coordinate; this keyword can be used to specify an offset to the value provided in the data header. The unit is arcsec. This keyword is only used if DECLINATION is unspecified. The default value is: 0
spaxelscaleA scalar floating-point type value that specifies the size of the output pixels. This value must be specified for instruments that do not provide this value in the header. The unit is arcsec.
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
compressSet this keyword to compress all output files using gzip; the file suffix '.gz' is appended to all output filenames. The default value is: 0
userparfileA scalar string with the name of a file with user- defined parameters. The following parameters are read here from this file:

'keywordsfile'

The name of the file with instrument-specific header keywords.

'compress'

Corresponds to COMPRESS.

'opfx'

Corresponds to OPFX.

'cubsfx'

To specify the data output cube suffix.

'ckpostable'
'nthreads'

Corresponds to NTHREADS.

Note! If any of the corresponding keywords is specified, that value takes precedence.
ofilenameXThis keyword returns the full name of the created data cube file.
opathA scalar string that specifies the path where the output data are saved. The default value is: '.'
opfxA scalar string that specifies a prefix for the (automatically generated) output filename. The default value is: ''
nthreadsA scalar integer that specifies how many threads to use in the parallelized array reversal routine. The default is to use the maximum number available, but no more than 16: max = !cpu.hw_ncpu < 16. The default value is: max
logfileThis keyword specifies the name of an optional log file when p3d_cobjex is launched as a separate program.
loglevelIf this routine is launched as a separate program, and LOGFILE is used, this keyword specifies the log level; [1], 2, 3 (the higher the value, the more output). The default value is: 1
allinoneUnset this keyword to write the output data to separate files, instead of writing the extracted spectrum data cube to the first (DATA) extension, the error data cube to the ERROR extension, the bad-pixels mask data-cube to the MASK extension, the saturated-pixels mask data cube to the SATMASK extension, and the data-quality data cube to the QUALITY extension. The default value is: 1
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
subroutineXIf set, parts of the routine opening message will not be written to the log file.
stawidXIf set, various messages are written to the p3d GUI status line (this must be the widget id of that label widget).
topwidXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation, and then exit.
Output parameters:
outXUpon a successful execution, this variable contains the data cube image when the tool exits.
Examples:

This routine can be called in three different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) As a stand-alone tool from the IDL command line

In addition to the extracted and calibrated RSS images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments:

files = ['file_oextr1.fits.gz', $
         'file_oextr2.fits.gz', $
         'file_oextr3.fits.gz']
parfile = 'nvimos_hr.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' ;; If you use this file
p3d_rss2cube, files, parfile, userparfile=userparfile, $
    opath=opath, logfile=opath + 'dred.log', /verbose

ii) As a stand-alone tool from a terminal, using the IDL VM

In addition to the extracted and calibrated RSS images, it is necessary to pass the name of the (provided) instrument-specific parameter file, as well as any additional arguments. The recommended approach is to use the provided shell script p3d_rss2cube_vm.sh or the Python tool p3d_dispatch.py:

# Using the shell script, making use of the p3d queue system:
files = file_oextr1.fits.gz,file_oextr2.fits.gz,file_oextr3.fits.gz
parfile = nvimos_hr.prm
opath = Output
userparfile = ${opath}/user_p3d.prm # If you use this file
detector = 0 # Required with multi-detector configurations.
logfile = ${opath}/dred.log
p3d_rss2cube_vm.sh $files $parfile \
    userparfile=$userparfile opath=$opath logfile=$logfile /verbose

# Using the Python tool, without using the p3d queue system:
files = file_oextr1.fits.gz,file_oextr2.fits.gz,file_oextr3.fits.gz
parfile = nvimos_hr.prm
opath = Output
userparfile = ${opath}/user_p3d.dat # If you use this file
detector = 0 # Required with multi-detector configurations.
logfile = ${opath}/dred.log
p3d_dispatch.py --noqueue rss2cube $files $parfile \
    userparfile=$userparfile opath=$opath logfile=$logfile /verbose

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_rss2cube_vm.sh userparfile=$userparfile" and -NOT- "p3d_rss2cube_vm.sh user=$userparfile".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: When several files are specified in the first argument, they must be separated with either a comma or a semicolon.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach. This includes the three output variable parameters OFILENAME, OUT, and DOUT.

iii) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to ii), but is executed from the Python command line. Here is an example:

# Launch p3d_rss2cube in a single IDL session, without using the
# p3d queue system:
from p3d_dispatch import *
files = 'file_oextr1.fits.gz,file_oextr2.fits.gz,file_oextr3.fits.gz'
parfile = 'nvimos_hr.prm'
opath = 'Output'
userparfile = opath + '/user_p3d.dat' # If you use this file
detector = 0 # Required with multi-detector configurations.
logfile = opath + '/dred.log'
commandline = files + ' ' + parfile +
  ' userparfile=' + userparfile + ' opath=' + opath +
  ' logfile=' + logfile + ' /verbose'
p3d_dispatch('rss2cube', commandline, noqueue=True)

p3d_sv.pro, line 3275, last changed at 2017-04-13 by christersandin (revision 4547)

p3d_sv, image, posfile, dimage=, /noerrors, range=, hdr=, zhdr=, dhdr=, /andsky, colorcut=, specrange=, datapath=, orientation=, colortable=, /invert, /cinv, bottom=, cindex=, cindv=, coffset=, ch_start=, ch_rots=, ch_gamma=, ch_hue=, title=, parfile=, ofilename=, opath=, deadfibersfile=, binmap=, regions=, detector=, fitsheadertitle=, daxis=, spaxelscale=, winxysize=, sfobsfile=, sfcalfile=, sfextfile=, sfcatalog=, sfwaveunit=, sffluxunit=, sfext_waveunit=, redshift_vel=, redshift_z=, /nanometer, /micrometer, /wavenumber, raman_exc_wavel=, /noposreverse, /a2, /a3, /a5, /letter, /legal, /tabloid, emission_lines=, userparfile=, khdr=, kdhdr=, nthreads=, display_name=, /noC, font=, keep_cube=, /spatmap_flipx, cmdline=, tracking_events=, /gui, group_leader=, logunit=, verbose=, /quiet, error=, /debug, cdebug=, /help

The p3d interactive data cube and RSS-data spectrum viewer.

The p3d spectrum viewer allows a quick inspection of extracted spectra of fiber-fed integral field spectrographs (IFS), integral-field units (IFUs), as well as spectrum data cubes of any origin (for as long as necessary header keywords are available). The data to view are specified either using the name of a fits file or a data array with an accompanying header array. Using object spectra saved in the row-stacked spectrum (RSS) format (see the Input/Output section), the tool also requires a fiber position-table file to render correct spatial maps.

The graphical user interface of p3d_sv

The spectrum viewer is an entirely interactive tool that presents a graphical user interface (GUI, which is based on widgets). The GUI consists of two different regions, a menu bar, and a larger field that on separate tabs shows either a spectrum viewer, an enlarged version of the spatial map, or an interactive tool to create or inspect a sensitivity function for flux calibration. Each tab and the associated regions are described separately in the following subsections. The GUI can be resized from a minimum size of about 1000x750 px² to use any size; the exact size achievable depends on the used font.

The menu bar

Here is a description of all available (and unavailable) entries in the menu bar, which contains five sub menus: File, Image View, Spatial Map, Spectrum View, and Help.

The string in the last column shows the key-accelerator combination for tasks where such an accelerator is available. As an example of a key accelerator, the intended meaning of 'Ctrl+B' is to press the 'B' key while holding the 'Ctrl' key; thereafter, both keys are released.

Note: p3d is based on IDL, and therefore the Motif widget library (http://www.opengroup.org/motif/), which requires key accelerators (on UN*X-type platforms) to be placed in menu entries. Most options are therefore available in the menus listed above. Although, key accelerators are defined for most options, your window manager (such as, for example, those used by the KDE or GNOME environments) may have reserved any particular configuration, and if this is the case, the accelerator will not work.

File

  • View the data fits header [EXTENSION]

    Opens a text-viewer tool that displays the 80-column wide fits-file header (using XDISPLAYFILE). The name of the extension is shown with the EXTENSION keyword; typically, EXTENSION is set to 0, 1, or DATA.

  • View the 0-extension fits header [0]

    Opens a text-viewer tool that displays the 80-column wide zeroth extension fits-file header (using XDISPLAYFILE); this entry is only visible when the data extension is not equivalent with the zeroth extension.

  • View the errors fits header [EXTENSION]

    Opens a text-viewer tool that displays the 80-column wide fits-file header of the error data (using XDISPLAYFILE). The name of the extension is shown with the EXTENSION keyword; typically, EXTENSION is set to ERROR, SIGMA, or STAT.

  • Save Spatial Maps (Shift+W)

    Saves the spatial-map data array of the selected wavelength bins as a fits file.

    Note: The spatial map is -not- saved as an image. To reconstruct the spatial map, it is necessary to use the (proper) fiber position-table file.

  • Save The Current Spectrum (Shift+S)

    Saves the currently selected single, summed, or averaged, spectrum as a one-dimensional array in a fits file.

  • Save Both The Current And The Subtracted Spectra (Shift+Alt+S)

    Saves the currently selected single, summed, or averaged, spectrum as the first row in a fits file. Additionally, the subtracted spectrum is added as another row.

  • Select Color Table

    Opens a sub-menu where any of the pre-defined color tables, the available color tables of IDL, as well as the Color Brewer color tables can be selected; either interactively using XLOADCT, or directly. The first four color tables in the sub-menu are Cubehelix, Califa intensity, Califa velocity field, and Sauron; Sauron is the default color table of p3d.

  • Invert Color Table (Ctrl+I)

    The selected color table is reversed when this option is set.

  • Exit (Ctrl+Q)

    Exits the p3d spectrum viewer.

Image View

  • Color Map (F1-F12)

    The twelve available values in the pull-down menu are: 100, 99.9, 99.8, 99.7, 99.5, 99, 98, 97, 95, 90, 80, and 70 per cent of the pixel values.

  • Shown Wavelength Bin

    The wavelength bin showed in the spatial map is selected by clicking with the left mouse button in the spectrum image, or by entering a bin number on the "Bin 1" control panel. The six available options in the pull-down menu are:

    step 50 bins lower (Ctrl+Shift+Alt+F) step 10 bins lower (Ctrl+Shift+F) step 1 bin lower (Ctrl+F) step 1 bin higher (Ctrl+G) step 10 bins higher (Ctrl+Shift+G) step 50 bins higher (Ctrl+Shift+Alt+G)

  • Shown 1st Reference Wavelength Bin

    This menu option is only sensitive when the first reference wavelength bin has been set either by clicking in the spectrum image while holding the Ctrl key, or by entering a bin number on the "Bin 1" control panel. The six available options in the pull-down menu are:

    step 50 bins lower (Ctrl+Shift+Alt+H) step 10 bins lower (Ctrl+Shift+H) step 1 bin lower (Ctrl+H) step 1 bin higher (Ctrl+J) step 10 bins higher (Ctrl+Shift+J) step 50 bins higher (Ctrl+Shift+Alt+J)

  • Deactivate The 1st Reference Wavelength Bin (Alt+H)

    Deactivates the already activated first reference wavelength bin. Alternative ways to deactivate the bin are to enter -1 in the middle text field or click the middle-field delete-ref-bin button on the "Bin 1" control panel.

  • Shown 2nd Reference Wavelength Bin

    This menu option is only sensitive when the second reference wavelength bin has been set either by clicking in the spectrum image while holding the Ctrl and Shift keys, or by entering a bin number on the "Bin 1" control panel. The six available options in the pull-down menu are:

    step 50 bins lower (Ctrl+Shift+Alt+K) step 10 bins lower (Ctrl+Shift+K) step 1 bin lower (Ctrl+K) step 1 bin higher (Ctrl+L) step 10 bins higher (Ctrl+Shift+L) step 50 bins higher (Ctrl+Shift+Alt+L)

  • Deactivate The 2nd Reference Wavelength Bin (Alt+K)

    Deactivates the already activated second reference wavelength bin. Alternative ways to deactivate the bin are to enter -1 in the right text field or click the right-field delete-ref-bin button on the "Bin 1" control panel.

  • Blink Wavelengths: no (dT=0.30s) (Shift+B)

    When either one or both of the reference wavelength bins are set, this button allows a quick change between the chosen bins in the spatial map. The blink time period is indicated within parenthesis (in seconds); the time period can be adjusted on the "Bin 2" control panel.

  • Spatial Element ID: Select The Next Lower (Ctrl+Shift+N)

  • Spatial Element ID: Select The Next Higher (Ctrl+N)

  • Spatial Element Image Row: Select The Next Lower (Ctrl+Shift+M)

  • Spatial Element Image Row: Select The Next Higher (Ctrl+M)

Spatial Map

  • Spatial Elements: Use a Single Spectrum (Ctrl+Y)

  • Spatial Elements: Use a Summed Spectrum (Ctrl+Shift+Y)

  • Spatial Elements: Use a Mean Spectrum (Ctrl+Shift+Alt+Y)

  • Orientation: North [down], East [right]

    The four available values in the pull-down menu are: North [right], East [ up] (IFU rot. 90° clockwise) (Ctrl+Alt+O) North [ up], East [ left] (IFU not rotated) (Ctrl+O) North [ left], East [ down] (IFU rot. 90° cc.wise) (Ctrl+Shift+O) North [ down], East [right] (IFU rot. 180°) (Ctrl+Shift+Alt+O) when the IFU is set to point at the sky (or not).

    The orientation can also be set with the context menu in the spatial-map draw area that is activated with the right mouse button in the single-spectrum selection mode. Or, additionally, hold the Caps-Lock key and use the scroll wheel in the same area (remember to unlock the Caps Lock key when done).

  • Zoom: 100%

    Spatial maps of IFUs using square-shaped spatial elements that are drawn without polygons can be zoomed. The fifteen available values in the pull-down menu include: 100% (Ctrl+1) 200% (Ctrl+2) 300% (Ctrl+3) 400% (Ctrl+4) 500% (Ctrl+5) 600% (Ctrl+6) 800% (Ctrl+7) 1000% (Ctrl+8) 1200% (Ctrl+9) 1500% (Ctrl+0) 2000% (Ctrl+Shift+1) 2500% (Ctrl+Shift+2) 3000% (Ctrl+Shift+3) 3500% (Ctrl+Shift+4) 4000% (Ctrl+Shift+5) 4500% (Ctrl+Shift+6) 5000% (Ctrl+Shift+7) 6000% (Ctrl+Shift+8) 7000% (Ctrl+Shift+9) 8000% (Ctrl+Shift+0) 9000% (Ctrl+Shift+Alt+1) 10,000% (Ctrl+Shift+Alt+2) 12,000% (Ctrl+Shift+Alt+3)

    The zoom can also be set with the context menu in the spatial-map draw area that is activated with the right mouse button in the single-spectrum selection mode. Or, additionally, hold the Shift key pressed and use the scroll wheel in the same area.

  • Physical X Coordinate: West => East (Ctrl+X)

    The physical //x// coordinate shown in the coordinate field by default increases West => East (to the left). Select this option to toggle the //x// direction to instead have it increase East => West (as in the tool ds9); this option is also available through the keyword SPATMAP_FLIPX.

  • Save The Spatial Map To a Bitmap, PostScript, or PDF File (Ctrl+P)

    Opens a pick-file dialog that allows the selection of a name of a file. The PDF option is only shown in the button value string if both the tools 'epstopdf' and 'ps2pdf' are available in the system path.

  • Paper Size: ISO A4

    The paper size used with (non-encapsulated) PostScript and PDF output. The seven available options are: ISO A2 (Alt+1) ISO A3 (Alt+2) ISO A4 (Alt+3) ISO A5 (Alt+4) US Letter (Alt+5) US Legal (Alt+6) US Tabloid (Alt+7)

  • Annotate Saved Map With Spectrum Rows And Ids: yes (Ctrl+Shift+P)

    Prints a string on each spatial element in file output.

  • PostScript[/ PDF] Image Size: 7 x 7 cm

    The image size is used when saving the spatial map to a PostScript or PDF file. The thirteen available values in the pull-down menu include: 4 x 4, 5 x 5, 6 x 6, 7 x 7, 8 x 8, 9 x 9, 10 x 10, 11 x 11, 12 x 12, 14 x 14, 16 x 16, 18 x 18, and 20 x 20 cm². The PDF option in the button value string is only shown when both the tools 'epstopdf' and 'ps2pdf' are available in the system path.

  • Bitmap Image Size: 1000 x 1000 px²

    The image pixel size is used when saving the spatial map to a bitmap file. The thirteen available values in the pull-down menu include: 300 x 300, 400 x 400, 500 x 500, 600 x 600, 700 x 700, 800 x 800, 900 x 900, 1000 x 1000, 1200 x 1200, 1400 x 1400, 1500 x 1500, 1600 x 1600, and 2000 x 2000 px².

  • Save Encapsulated PDF Output: yes (Ctrl+Shift+Alt+P)

    Printing a PDF file, the output is configured to use the encapsulated option by default. Select this option to plot the output centered on a full page; this option is only available when both the tools 'epstopdf' and 'ps2pdf' are available in the system path.

  • Calculate a Voronoi Bins Map (Ctrl+Alt+B)

    Calculates Voronoi bins for the selected wavelength bin, using the selected target signal-to-noise value; the button is only sensitized when the file voronoi_2d_binning.pro is available in the path !p3d_path/contrib/ (the file can be downloaded from the web site of Michele Cappellari – see the Links section below).

  • Load a Bins Map of Spatial Elements (Ctrl+B)

    Opens a pick-file dialog that allows the selection of a spatial elements bins map to load; the selected file must be saved as plain text or to a binary-table fits file, but it can be compressed using gzip. It is also possible to load a bins map upon starting the spectrum viewer using the keyword BINMAP.

  • Save The Voronoi Bins Map of Spatial Elements (Ctrl+Shift+B)

    Opens a pick-file dialog that allows the selection of a plain-text or (binary-table) fits file name that is used to save the calculated spatial elements bins map; this option is only available after a Voronoi bins map has been calculated.

  • Use Bins: no (target S/N = 100.0, w.l. bins 00000:00000) (Alt+B)

    After a bins map was loaded or calculated, toggling this button shows the bins in the spatial map instead of the single elements.

  • Region Index: Select Previous (Ctrl+Shift+R)

    Selects the previous region index, the lowest index is 1; this option is only available when the spatial-elements selection mode is either sum or mean.

  • Region Index: Select Next (Ctrl+R)

    Selects the next region index of the already defined indices plus the next available value, the highest index is 1024; this option is only available when the spatial-elements selection mode is either sum or mean.

  • Region Index: Select Last Available (Ctrl+Shift+Alt+R)

    Selects the last available region index where no spatial elements are marked; this option is only available when the spatial-elements selection mode is either sum or mean.

  • Options for the current region

    The three available options in the pull-down menu are:

    Mark all spatial elements (Ctrl+A) Selects all spatial elements in the spatial map for the current region index.

    Select elements using one or several ROIs (Ctrl+Shift+A) Select spatial elements in the spatial map using the IDL region-of-interest tool XROI for the current region index.

    Unmark all spatial elements (Alt+A) Unmarks all selected spatial elements for the current region index.

  • All Regions: unmark all spatial elements (Alt+R)

    Unmarks all selected spatial elements for all region indices.

  • In the Spatial Map, show: Current Region

    The four available options in the pull-down menu are:

    Current Region (Alt+C) Using red symbols, indicate the current region.

    All Setup Regions (Ctrl+C) Using red (green) symbols, indicate the current (all other) region(s).

    All Setup Regions [+ DS9 Information] (Ctrl+Shift+C) Using red (green) symbols, indicate the current (all other) region(s). Also shows DS9 region properties of all regions (outlines, text, and colors).

    Current Region [+ DS9 Information] (Alt+Shift+C) Using red symbols, indicate the current region. Also shows DS9 region properties of all regions (outlines, text, and colors).

  • Restore a Saved ROI-Object File (Shift+T)

    One or multiple regions of interest (ROIs) can be setup when selecting groups of spatial elements. The ROI widget tool allows the ROI (object) data to be saved to a separate IDL save file. Files saved in that tool can be restored using this option. After the file is restored, it is necessary to use the ROI-selection option anew to view and actually use the loaded region(s).

  • Load (Append) a Regions File – p3d or DS9 Format (Ctrl+T)

    Loads a spatial-elements regions map saved using the next menu entry or DS9 (regions). Filenames ending with the suffix '.gz' are assumed to be compressed with gzip.

  • Save The Setup Regions (Ctrl+Shift+T)

    Saves the currently setup regions – in the range 1-1024 – to a plain-text file. The format of the file is specific to p3d. The file is compressed using gzip if the filename provided contains the suffix '.gz'.

  • Subtract The Current Spectrum From All Spectra (Alt+D)

    The average of the currently marked spectra is subtracted from all spectra; this option is unavailable with RSS images or data cubes with more than 2,000,000,000 elements.

  • Divide All Spectra With The Current Spectrum (Alt+Shift+D) The average of the currently marked spectra is divided from all spectra in the RSS image or data cube; this option is unavailable when there are more than 2,000,000,000 elements.

  • Restore The Spectrum image / cube To Its Entry State (Ctrl+D)

    The spectrum RSS image or data cube is restored to its state before the subtraction or the division.

Spectrum View

  • Y-range: 99.8% (Shift+F1-Shift+F12)

    The twelve available values in the pull-down menu are: 100, 99.9, 99.8, 99.7, 99.5, 99, 98, 97, 95, 90, 80, and 70 per cent of the pixel values.

    The Y-range can also be set with the widgets below the spectrum plot or by holding the Alt key and using the scroll wheel when the mouse pointer is kept in the spectrum-plot draw area.

  • X-axis Property: Wavelength [Å]

    The four available options in the pull-down menu are: Wavelength [Å] (Ctrl+U) Wavelength [nm] (Ctrl+Shift+U) Wavelength [µm] (Ctrl+Alt+U) Wavenumber [/cm] (Ctrl+Shift+Alt+U)

  • Shown Wavelength / Wavenumber Range

    The wavelength or wavenumber range showed in the spectrum can be changed with the options here; the six available options in the pull-down menu are:

    Step 10 × <pitch> [<unit>] Lower (Ctrl+Shift+Alt+J) Step 3 × <pitch> [<unit>] Lower (Alt+Shift+J) Step <pitch> [<unit>] Lower (Shift+J) Step <pitch> [<unit>] Higher (Ctrl+J) Step 3 × <pitch> [<unit>] Higher (Ctrl+Shift+J) Step 10 × <pitch> [<unit>] Higher (Ctrl+Alt+J)

    These options are available by using the scroll wheel in the spectrum-plot draw area (with the Ctrl and Shift modifiers) or the leftwards and rightwards pointing arrows below the plot.

  • X-axis Wavelength Range – Zoom In (Ctrl+Z)

    Zoom in on the wavelength / wavenumber range; this option is also available by using the scroll wheel and the Shift key in the spectrum-plot draw area or the downwards pointing arrow below the plot.

  • X-axis Wavelength Range – Zoom Out (Ctrl+Shift+Z)

    Zoom out of the wavelength / wavenumber range; this option is also available by using the scroll wheel and the Shift key in the spectrum-plot draw area or the upwards pointing arrow below the plot.

  • X-axis Wavelength Range – Reset (Alt+Z)

    Use the full available wavelength range.

  • Show: Spectrum – Background

    The four available options in the pulldown menu are: Spectrum: showing the selected spectrum (Alt+V) Spectrum – Background: (Ctrl+V) showing the background-subtracted spectrum Spectrum – Background, Background: (Ctrl+Shift+V) showing the background-subtracted spectrum (black) and the background spectrum (green) Spectrum – Background, Background, Spectrum: (Ctrl+Shift+Alt+V) showing the background-subtracted spectrum (black), the background spectrum (green), and the selected spectrum before the background subtraction (blue).

  • Open Line Fits Log File (Ctrl+Shift+,)

    Opens a dialog windows that allows the selection of a filename that is used to log line fits – using the next option.

  • Log Line Fit To File (Ctrl+,)

    Saves the just fitted values to the opened log file.

  • Clear Fitted Line (Alt+,)

    Clears the fitted line and removes it from the spectrum plot.

  • Show Region Statistics (Ctrl+S)

    Shows spectrum statistics at the selected wavelength in the Color control panel when the spectrum mode is either Sum or Mean.

  • Show Region Statistics – Also Calculate Median (Ctrl+Shift+S)

    Also calculate the median; such calculations are lengthy for large RSS images and data cubes.

  • Show Emission Lines in The Spectrum Plot (Ctrl+.)

    The locations of the entries in the selected emission-line file are indicated with vertical bars and diagonal tags.

  • View Emission Line Entries in a Window (Ctrl+Shift+.)

    Opens a text widget that shows the entries in the plain-text emission-line file as one entry per line.

Help

  • Polygons: no

    Spatial maps of instruments with IFUs using square-shaped spatial elements are by default rendered using regular images; spatial maps of IFUs with circular or hexagon-shaped spatial elements are always rendered using polygons. Set this option to render spatial maps of square-shaped spatial elements using polygons.

  • Calculate weighted mean: no

    Spectrum means across wavelength bins are either calculated as arithmetic averages or weighted averages using the 1-standard-deviation errors or variances. Set this option to use the weighted mean.

  • ROI NN by histogram: yes

    Calculating which spatial elements are selected using the ROI method, instruments and IFUs using square-shaped spatial elements by default use the IDL HISTOGRAM routine. Unset this option to instead use an algorithm that calculates the distance to the center of each spatial element; this approach is drastically slower than the histogram approach.

  • Nthreads: <number>

    The twenty available values in the pull-down menu are: 1-16, 24, 32, 48, and 64. However, the maximum value shown is never higher than the maximum number of available threads on the computer (!cpu.hw_ncpu).

  • Verbose: 0

    Allows the verbosity level to be set interactively. The four available values in the pull-down menu include: 0, no information; 1, more information; 2, most information; and 3, all information. The menu does not show higher values, which can only be set with the keyword VERBOSE.

  • Debug: no

    Set this option to toggle the debug state; this option is only available when the tool is run from the command line.

  • Debug C routines: no (0)

    Allows the C debug information level to be set interactively. The seven available values in the pull-down menu include: "0 – no debugging information", "1 – More information", "2 – .", "3 – .", "4 – .", "5 – .", and "6 – All debugging information". The initial value can be set with the CDEBUG keyword.

    Note: Different routines implement different levels of debugging verbosity.

  • About p3d_sv

    Shows a window with information about p3d_sv.

  • Copying

    Shows a window with the licensing-information content of the file COPYING.

The main-panel tab

The spectrum-viewer tab shows five different regions with related controls: a spectrum image; a set of four control panels including a color bar, region statistics, bin selection widgets, blinking setup, emission-line view setup, a spatial-map control panel, etc.; a spatial map in the top right corner; a status line with coordinates, and a spectrum panel with a control panel beneath it. All these regions are described next.

1. The spectrum image – the top left area

The spectrum image area shows the loaded data as an image, where each row represents (parts of) a single spectrum; the dispersion direction is always the x-axis with the wavelength increasing towards the right-hand side (see the description for the keyword DAXIS when input RSS data instead use the y-axis).

A green vertical bar marks the wavelength bin indicating the selected spatial map (see 3.). Two additional reference wavelength bins are shown with a dark green or red vertical bar when activated. The wavelength and reference bins can be changed by five different methods:

i)

Click on the new wavelength-bin (1st ref. bin; 2nd ref. bin) position using the left mouse button (while holding the Ctrl key; while holding the Ctrl+Shift keys).

ii)

Enter a new value for the respective bin using the three text fields in the "Bin 1" control panel (2.).

iii)

Click on the decrement and increment buttons on each side of the text fields in the "Bin 1" control panel to move 1 or 10 bins with one click. It is also possible to use the key accelerators explained in the "Image View" menu panel.

iv)

Click in the image with the left mouse button and then press the left and right arrow keys to decrement and increment the wavelength bin by 1 bin. Simultaneously use the Ctrl (Ctrl+Shift) key to instead decrement or increment by 10 (50) bins.

v)

The two reference wavelength bins can be deactivated by entering the value -1 in the respective text field in the "Bin 1" control panel, clicking the deactive button to the right of the respective text field, or by using the key accelerator Alt+I or Alt+K, respectively.

A white vertical bar indicates the wavelength (or wavenumber) of the mouse in this image as well as in the spectrum plot (5., where the bar is shown in red color) where the wavelength (or the wavenumber) is also shown as a value; this second bar is shown when the mouse pointer is dragged across the spectrum image or the spectrum plot.

In cases where the full wavelength range or all spectra do not fit in the spectrum image, the horizontal and vertical slider bars below and to the right of the image allow panning the full wavelength range and all spectra. The same functionality is achieved using the scroll wheel when the mouse pointer is inside the spectrum image; hold no key (the Alt key) to scroll horizontally (vertically) with the scroll wheel. A finer stepping is achieved by pressing the Ctrl or Ctrl+Shift simultaneously with the scroll wheel. The viewed wavelength range and set of shown spectra can also be changed with the left and right arrow buttons in the first two rows of the spatial-map control panel (2.). A mouse control help image is shown in the full spectrum image are when the mouse pointer is held above the question-mark field where the two scroll bars intersect in the lower right corner of the spectrum-image area.

The two narrow vertical image areas just to the left and to the right of the spectrum image, with white horizontal lines drawn on a black background, indicate the currently selected image rows – i.e. the currently chosen spectra – that are co-added to create the spectrum shown in the spectrum plot (5.). The number of lines can only be > 1 if the 'Sum' or the 'Mean' spectrum-plot modes are used (see 3.). Any other (single) spectrum can be selected by clicking in these areas; a spectrum will only be shown if it is marked as a science (or sky) spectrum in the spatial-element position table (for RSS images).

2. The row with multiple control panels – middle row

The widgets on the control panel row are split into four panels: Color, Bin 1, Bin 2, and Extra. The control panel for the spatial map remains visible on all panels. All five panels are described one by one next.

Control panel: Color

The "Color" control panel shows and sets the pixel-value range used to scale the color map. The two leftmost text fields below the image allow the lower and upper values to be changed. It is also possible to let p3d_sv calculate the range values based on a histogram-calculation where a certain percentage of the pixel values are used to set the lower and the upper limit values; use the droplist to the left of the two text fields – or the function key accelerators F1-F12 – to make use of this functionality. The same color map scaling is used with the spectrum image and the spatial-map area (3.). The upper rightmost area shows a color bar, which displays the color versus pixel value. The currently selected lower, middle, and upper pixel values are indicated below the color bar; additional pixel values are shown at 25% and 75% of the maximum pixel value if there is enough space to avoid overlapping values.

The lower part of the color control panel shows region statistics and allows logging of line fits. Region statistics can be shown when multiple spatial elements are co-added using the Sum or Mean options and the mouse pointer moves across the spectrum plot; they are activated by selecting the Show option. Statistics values are shown separately in two sub-panels for the main region (left-side panel) and the optional background region (right-side panel). Each panel presents statistics for the selected wavelength based on all co-added spectra using the following 13 fields (traversing to the right and down from the top left corner): the region ID, the number of used (total) co-added spatial elements at the current wavelength, the sum of the flux, the mean and the 1-standard-deviation, the skewness, the median, the variance, the kurtosis, the maximum absolute deviation, the minimum, and the maximum value. Because the median calculations can be extremely slow with large data sets, the median datum is only shown when the Median box above the statistics panels is ticked.

To save calculation time, the region statistics are only calculated for co-added spectra when the Show box is ticked. When spectra are selected one-by-one – through dragging the mouse pointer across the spatial map (with the left mouse button pressed) – the region statistics calculations are re-done a few seconds after the mouse button is released.

The three remaining widget buttons are used to log line fits made interactively in the spectrum plot:

  • Click the open-file widget to select a plain-text file that is used to log the fits; the format of the file is explained in the file itself, but the file basically contains all properties of the fit with additional information on the used data file and the time the file was opened and closed. An alternative approach to open the line-list file is to use the key accelerator Ctrl+Shift+,.

    Once the log file has been selected and opened, the open-file icon is replaced with a close-file (stop) icon.

  • The Log Fit button becomes sensitive when the line-fit log file has been opened and one has made a line fit in the spectrum plot. The fit is recorded in the log file by clicking on this button.

  • The Clear Fit button becomes sensitive as soon as one has made a line fit; at which point the fitted line is drawn atop the spectrum until this button is clicked.

Control panel: Bin 1

The top row contains three groups of widgets that allow selection of the wavelength and reference bins in the spectrum image. The reference bins are used when blinking the spatial map (see below in the Bin 2 panel) The three groups of widgets are the following:

  • The first group controls the wavelength bin. The buttons with arrows decrement or increment the bin number by 10 or 1 when one clicks the respective button. A value on the bin can also be entered in the text field.

  • The second group controls the first reference wavelength bin. The buttons with arrows decrement or increment the bin number by 10 or 1 when one clicks the respective button. A value on the bin can also be entered in the text field. The reference bin is deactivated by entering -1 in the text field or clicking the X button.

  • The third group controls the second reference wavelength bin. The buttons with arrows decrement or increment the bin number by 10 or 1 when one clicks the respective button. A value on the bin can also be entered in the text field. The reference bin is deactivated by entering -1 in the text field or clicking the X button.

The row with smaller images can contain up to 20 images, but the visible number depends on the screen size. The small spatial maps on the image row are empty at first. The currently shown spatial map is stored in one of the images when clicking with the left mouse button on the (button) bar situated below the respective image area; the used wavelength range is indicated on the same button bar. Each image is replaced with the current spatial map when the respective image buffer is clicked anew. The number of visible spatial maps depends on the width of the screen (the maximum possible number is

20.

.

For IFUs with square-shaped spatial elements, the complete set of used spatial maps can be stored with the 'Save spatial maps' option in the file menu or the key accelerator Shift+W. The output fits file contains an image (a two-dimensional array) if only one spatial-map image was saved on the spatial-maps row. Otherwise, it contains a set of images (a three-dimensional array), one per saved spatial map. Each pixel in the saved image corresponds to one spatial element. The pixels are arranged as they would appear in an image of the sky. If the IFU rotation angle is set to 0, 90, 180, or 270 [°], the image is oriented such that North is up and East left. For other angles, the rotation is relative to a 0° rotation angle.

Control panel: Bin 2

The control panel contains widgets that allow blinking between the activated wavelength bins and setting the bin width, and load a bin map or bin the data using Voronoi bins.

The spatial map can be blinked between the wavelength bin and the reference bins once at least one of the latter bins was defined. Click the Blink button or use the key accelerator Shift+B to start or end blinking between the wavelength bin and the first or the second reference bin, or both reference bins. The slider allows setting the blink time interval in the range 0.02 to 1.0 second. The text field allows the blink time interval to be specified as any value in the range 0.02 to 5.0 seconds. The rightmost blinking-related widget, a droplist, allows 14 preset blink-time intervals to be conveniently set: 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 1, 2, 3, 4, or 5 seconds.

The three widgets on the line control the wavelength bin width //w//; the actual width of the bin is 2 * //w// + 1. A higher value results in a smoother image. The maximum value on //w// is 100.

The remaining widgets on this panel control the use of maps of spatial elements, so-called bin maps. These widgets are split into three rows. On the first row, the first button widget showing a hand is sensitive if the file voronoi_2d_binning.pro is available in the directory !p3d_path/contrib/ (see the links section below). Click the button to create a Voronoi binned map using the currently selected wavelength bin (and bin width). Please note, that this can take quite some while with large data sets.

Click the open icon to open an existing bin map file, which doesn't necessarily need to be a Voronoi bin map. Four of the optional keywords used when reading fits files can be configured with the four text fields on the third row; these keywords are BINMAP_EXTENSION, COL_BINIDX, COL_X, and COL_Y. The text fields are filled with (some of) the default values for each keyword. The Reset button to the right of the text fields is sensitized when any of the fields is edited, and clicking it restores the default value for each property. The required format of the plain-text or fits file is described in the documentation to the routine p3d_misc_bins_file_read.

The save icon is sensitized once a bin map has been calculated. Click the button to save the map, including the //x// and //y// coordinates along with the bin index for each spatial element to a plain-text or a fits file. Tick the Use Bins box once Voronoi bins are calculated or a bin map file has been loaded to actually use the map. When bins are used, one selects and deselects all spatial elements of separate bins in the spatial map, instead of indiviual spatial elements. One use for the created bin map file is with the line-fitting tool p3d_ifsfit (using its keyword BINMAPS).

The most influential parameter to the Voronoi binning tool is the target signal-to-noise ratio and this parameter is set with the three widgets on the row below the buttons. The slider allows choosing values between 5.0 and 500.0, the text field allows any positive value < 10000.0, and the droplist is pre-configured with the following 12 target signal-to-noise ratios: 10, 20, 30, 40, 50, 100, 150, 200, 300, 500, 750, and 1000.

Control panel: Extra

The slider at the top allows adjustment of the relative height of the spatial-map panel with respect to the total height of the spectrum-viewer tab. Drag the slider to choose any relative height in the range 40-70%; the selected percentage is shown in the label to the right of the slider. Check the White Light box on the right-hand side of the slider to have the spatial map show an image that is the sum of all wavelength bins [this isn't that useful a function as the colors are, currently, not re-scaled to show the new image].

The row preceded with the label System Redshift allows the definition of a system redshift for the loaded data. The left text field shows the redshift //z// and the right-hand side text field the corresponding velocity (in km/s). Both fields can be edited. The button labeled "Fitted => System Redshift" allows the fitted redshift to be copied to the System Redshift value once a line fit has been made.

The final widgets allow a setup of shown emission-line data. Check the box Show Emission Lines to indicate the locations of the entries in the chosen emission-line list file atop the spectrum in the spectrum plot. The file browse icon allows the selection of any plain-text file where the first column contains wavelengths (Angstrom), the second column a string describing the line, and the third column a short version of the wavelength. The name of the selected file is shown in the label field in the lowest row of icons. Click the rightmost button widget to open a GUI that shows the contents of the currently chosen emission-line list file.

The emission lines are positioned using the system redshift value, but the redshift of a line fit is used instead when such a line fit is made. The positions revert to use the system redshift when the line fit is cleared.

The currently available line list files include the two files: emission_lines-ground_based-noFe.dat (default) and emission_lines-ground_based.dat, which are both available in the directory !p3d_path/data/tables/linelists/. Additional files could be added for use with data for wavelengths in vacuuum.

Control Panel: Spatial Map

The control panel contains three widget fields stacked vertically: the viewed wavelength range, the range of viewed spectra, and a part that selects spectra in the spatial map.

The first field controls the viewed wavelength range, which is changed by clicking on the six buttons to the left and right of the label field (also see the description of the slider bars and the key accelerators in 1. above). The buttons become unsensitive when they cannot be selected any more, and switch state to sensitive when they can be selected again:

  • The first button selects the bluemost range of pixels.

  • The second button steps 100 pixels lower, or as many pixels as possible.

  • The third button steps to the next lower pixel.

  • The fourth button steps to the next higher pixel.

  • The fifth button steps 100 pixels higher, or as many pixels as possible.

  • The sixth button selects the redmost range of pixels.

The second field controls the viewed range of spectra, which is changed by clicking on the six buttons to the left and right of the label field (also see the description of the slider bars and the key accelerators in 1. above). The buttons become unsensitive when they cannot be selected any more, and switch state to sensitive when they can be selected again:

  • The first button selects the first spectra in the RSS image.

  • The second button steps downwards 100 spectra, or as many spectra as possible.

  • The third button selects the next lower spectrum.

  • The fourth button selects the next higher spectrum.

  • The fifth button steps upwards 100 spectra, or as many spectra as possible.

  • The sixth button selects the last spectra in the RSS image.

The third and final field in the control panel controls the selection of spectra or bins in the spatial map (3.), where the co-added spectrum is shown in the spectrum plot (5.). There are three plot modes: 'Single', 'Sum', and 'Mean', which can be selected with the droplist; this is the leftmost button on the first row. Alternatively, change the plot mode with the "Spatial Map" menu bar or the key accelerators Ctrl+Y, Ctrl+Shift+Y2, and Ctrl+Shift+Alt+Y, respectively. Here follows a more detailed description of the available modes.

The Single spatial-element mode

The currently selected spatial element is indicated with a red symbol in the spatial map. Using polygons and the single-element plotting mode, the symbol is a red disk surrounded with a black ring; this is the default mode. With IFUs using square-shaped spatial elements, the symbol shape of the selected element depends on the zoom level; it is a red square surrounded by a thin black frame at larger zoom values.

The image row and the element ID are indicated in the two text fields on the right-hand side in the lower row. Select another element with a click on the left mouse button in the spatial map. Alternatively, it is possible to enter the number of another image row, or a spectrum ID, in the respective text field. The spatial element of the previous (next) image row can also be selected with the key accelerator Ctrl+Shift+M (Ctrl+M). Likewise, the element ID of the previous (next) spatial element can be selected with the key accelerator Ctrl+Shift+N (Ctrl+N). As a last method, the spatial elements immediately to the left, right, above, and below, of the currently selected element can be selected using the corresponding arrow key when the mouse pointer is held above the spatial map draw area.

The Sum and Mean spatial-element modes

When the plotting mode is chosen as 'Sum', the required spatial elements or bins at first need to be selected by clicking on them one by one in the spatial map or using the region-of-interest functionality; alternatively, it is possible to load a previously created / saved spatial-elements region map. The resulting summed spectrum is shown in the spectrum plot (5.). The spectrum plot shows the mean spectrum of all selected spectra if the plotting mode is instead set to 'Mean'. The default is an arithmetic mean; it is possible to instead calculate a weighted mean by checking the box "Calculate weighted means" in the Help menu.

Spatial elements or bins, i.e. spectra, are selected with a click on the left mouse button, and they are deselected with a click on the right mouse button; marked elements are shown as squares for square-shaped elements, and as red-black disks for circular and hexagon-shaped elements. Each spatial element or bin can be assigned two region numbers, a main region and an optional region that is used as background. The regions may overlap. The default region is 1. At any time, it is only possible to view spatial elements marked to belong to one selected region. The two regions are selected using the two text fields in the lowermost row. Possible values are 1-1024. The background region of the selected region is deactivated by entering 0 or any invalid text in its text field. Using a background region, the co-added and scaled background spectrum (scaled to use as many spatial elements as the main region) is subtracted from the main spectrum.

The droplist in the lower left part of the control panel allows to change what is seen in the spatial map. There are four options:

  • Current: Only show spatial elements marked as belonging to the current region.

  • All: Show all regions. Spatial elements of the current (other) region are indicated with a red (green) color.

  • All+DS9: Show all regions. Spatial elements of the current (other) region are indicated with a red (green) color. Additionally, properties of regions loaded from DS9 region files are plotted on top of the spatial map.

  • Cur+DS9: Only show spatial elements marked as belonging to the current region. Additionally, properties of regions loaded from DS9 region files are plotted atop the spatial map.

Different combinations of pre-selected spatial elements or bins of these can be chosen for any region with the following fourteen choices (the entries with sky, low-transmission spectra, and dead/undead spectra are only available if the used instrument has such spectra [and p3d_sv has been launched using the PARFILE keyword]):

" all "

Mark all elements. (Ctrl+A)

" ROI "

Mark elements using a ROI (Ctrl+Shift+A)

" no "

Unmark all elements. (Alt+A)

"all- L"

Mark all but low-transmission elements.

"ROI- L"

Mark all ROI-selected elements, but low-transmission elements.

" L"

Mark all low-transmission elements.

"all-dL"

Mark all but dead and low-transmission elements.

"ROI-dL"

Mark all ROI-selected elements, but dead and low-transmission elements.

"all-d "

Mark all but dead elements.

"ROI-d "

Mark all ROI-selected elements, but dead elements.

" d "

Mark all dead elements.

" L"

Mark all dead and low-transmission elements.

"all-s "

Mark all but sky elements.

" sky "

Mark all sky elements.

The four fields labeled ROI open up a GUI (based the IDL tool XROI) that allows an interactive selection of one or several regions of interest (ROIs). Spatial elements and bins are hereby selected easily inside regions in the form of rectangles, ellipses, polygons, and freehand drawn shapes. By a click on the right mouse button on a created region, while inside the XROI tool, it is possible to grow the region by a threshold value or a standard deviation multiplier. The spatial elements are marked once the tool is closed; only those spatial elements are chosen that lie roughly inside the selected regions.

Here are some further remarks of the ROI GUI tool. Selected regions are used in consecutive calls of the ROI function and they can also be saved to a file inside the tool; such files can later be loaded using the spatial-map menu option "Restore a saved ROI object file". The import image function in the file menu and the vertical image flip option are not used in the spectrum viewer. In the ROI information window, the area, perimeter, and # pixels options refer to the subsampled spatial-map image used in the ROI widget tool – i.e. not to spatial elements; the histogram function is also unused. The ROI GUI XROI is described more completely in the IDL documentation.

The total number of selected spectra is always indicated in the label field on the top right-hand side of the control panel. The same selected spectra are also indicated on the vertical green wavelength bar with a darker green shade (this is more easily seen with a non-zero bandwidth), and on the sides of the spectrum image.

The Subtr and Divid buttons

The currently selected spectrum, or an average of the currently selected spectra, can be subtracted or divided from the entire spectrum image. This could be a useful function if it is needed to check what is left after the sky has been subtracted, or if it is necessary to create a full-flux spectrum of a standard star for flux calibration, or if the spectra are to be normalized prior to checking a flat-field image. This functionality is accessed by pressing the Subtr button (Alt+D) to subtract the selected spectrum from all spectra. Alternatively, select the Divid button (Shift+Alt+D) to divide all spectra with the selected spectrum. The original spectrum is thereafter restored by selecting either button anew (or by using the key accelerator Ctrl+D). [This function is not as up-to-date as the regions and the background regions functionality.]

3. The spatial map of one selected wavelength – top right area

The spatial-map region shows an image using the currently selected wavelength bin as it appears on the sky (except instruments not used on the sky). To put each spatial element in the right place, and use the right shape, it is necessary to use the correct fiber position-table file with RSS data; no such file is required with cube data. Spatial elements of instruments that use circular or hexagon-shaped elements are rendered with polygons; in this case, the spatial map always shows the entire IFU (and the scroll bars remain insensitive).

Spatial elements of instruments that use square-shaped elements are represented by pixels in a regular image. The spatial map can be zoomed using the pull-down menu in the Single element mode, the Zoom option in the spatial-map menu bar, or using the mouse scroll wheel together with the Shift key. The vertical and the horizontal scroll bars can be used to pan across the IFU when the zoom scale is so large that the entire does not fit in the spatial-map window; the same functionality is achieved using the scroll wheel (pan horizontally) or the scroll wheel and the Alt key (pan vertically). A mouse help image is shown in the spatial-map image area when the mouse pointer is held above the question-mark field where the two scroll bars intersect in the lower right corner.

The image orientation can be changed through a click on the right mouse button in the spatial map; this change is only possible in the single-element mode. The orientation is by default set so that North is upwards and East leftwards (ORIENTATION = 1) when the IFU position angle equals zero degrees. An orientation arrow indicator is also shown atop the spatial map in the single-element mode (when the IFU is pointed at the sky). The orientation can always be changed with the Orientation pull-down menu in the Spatial-map menu bar.

4. The status line

The status line consists of two regions: a label field on the left-hand side and a part showing different kinds of coordinates on the right-hand side. The left-hand side label field shows help on different components of the GUI when the mouse pointer is held over the respective widget. The right-hand side part shows different kinds of coordinates depending on which draw area the mouse pointer is in.

Coordinates: spectrum image

A comma-separated string shows four values: the pixel, the wavelength or wavenumber (in the used unit), the flux, and the spectrum number.

Coordinates: spatial map

The coordinates are split into five separate fields:

i.

The right ascension and declination coordinates (Ra, Dec) are shown when the loaded data contain enough information to determine a world coordinate system (WCS). Data cubes should always contain such information, while there might not be enough information to calculate one for RSS data. Unless the instrument parameter files contain information about the spatial-element scale, it might be necessary to specify it with the SPAXELSCALE keyword.

ii.

The spatial element id and RSS-image row numbers.

iii.

The region number of the spatial element below the mouse pointer.

iv.

The physical coordinates. The //x// coordinate can be flipped by checking the "Physical X Coordinate: West => East" box in the Spatial Map menu or setting the SPATMAP_FLIPX keyword.

v.

The window coordinates (pixels).

== 5. The spectrum plot – the image area at the bottom ==

The currently selected spectrum is shown in the image area in the bottom part of the spectrum-viewer tool as a histogram plot (each pixel is a separate bin). When the mouse pointer is moved across the plot, the current wavelength (or wavenumber) is indicated with a vertical bar, both in the spectrum plot and in the spectrum image (1.). Positions of emission lines for a particular redshift are shown when the Show Emission Lines box is checked in the Extra control panel. A mouse help image is shown in the spectrum-plot image area when the mouse pointer is held above the question-mark field below the image area to the right.

Showing intensity error bars

Error bars are by default hidden. The 'Show errors' button – the third button from the left in the control panel below the plot window - becomes sensitive if error data were loaded. Click the button to show error bars for all shown wavelength bins in the spectrum plot; the button is thereby renamed to 'Hide error bars'. Click the same button repeatedly to toggle the display of the error bars.

The shown intensity range

The shown range of intensity values (the y-range) is, by default, set to be a histogram-calculated range, where 99.8% of all pixel values of the shown spectrum and wavelength range are used. The range can be changed by using any of the other values in the second drop list from the left below the spectrum plot (the fourth widget element); alternatively, use the key accelerators 'Shift+F1' – 'Shift+F12' or the scroll wheel while holding the Alt key to choose any of the preset histogram pixel-value percentages. An intensity range can also be set manually by entering any (floating-point type) value in the two leftmost text fields next to the drop list. The histogram-calculation is switched off whenever a value is entered in these text fields; it is switched on again when any of the drop list pixel-value percentages is selected anew.

The shown wavelength range

The entire (loaded) wavelength range is shown by default. The wavelength range can be changed manually by editing the two text fields found below the spectrum plot to the right of the "x:" label. The wavelength range is reset to the full range by clicking the Reset button (or when using the Alt+Z key accelerator).

Shifting the wavelength range

The wavelength range can be shifted to lower or higher value ranges, i.e. panned, by pressing the increment and decrement buttons below the spectrum plot – on the left-hand and right-hand sides of the second last text field. The same functionality is achieved by using the respective key accelerator Ctrl+W and Ctrl+E, the left and right arrow key( accelerator)s, or the mouse scroll wheel when the mouse pointer is placed in the spectrum-plot area. A finer pan is achieved by pressing the Ctrl or Ctrl+Shift keys while rolling the scroll wheel.

The panning stride is by default set to 1 / 10 of the full wavelength range; this value can be changed by entering a new value in the text field between the decrement and the increment buttons. A similar functionality, where the stride is decreased (increased) by 5% is achieved by using the key accelerator Ctrl+left (Ctrl+right). The value is reset to the default value by clicking the 'Reset' button.

Decreasing or increasing the range; zooming in and out

It is possible to zoom in or out on the used wavelength range with the buttons below the spectrum plot; these are the last two buttons and the text field on the right-hand side. Press the downwards (upwards) pointing arrow to zoom in (out), the same functionality is achieved by using the key accelerators Ctrl+Z and Ctrl+X, the up and down arrow key( accelerator)s, or by using the scroll wheel with the Shift key when the mouse pointer is in the spectrum-plot area. The zoom is made with respect to the center except when using the scroll wheel, where it is made at the position of the mouse pointer.

The currently shown wavelength range values are replaced with new values that show 90% (111%) of the currently visible range when zooming in (out). This value can be changed by entering a new value in the text field between the zoom-in/out buttons; alternatively, the percentage can be changed in steps of 1% by using the key accelerators Ctrl+down and Ctrl+up, respectively. The value is reset to the default value by clicking the 'Reset' button.

Calculating simple line fits

A simple line fit using a Gaussian profile is calculated using the following four-step approach:

i.

Click at the lower background point of the fit.

ii.

Click at the higher background point of the fit.

iii.

Click at the center position of the line fit and hold the mouse button. Select the emission line in the popup menu if it is visible. If the required line is not visible in the seven available line-entry options, select the first option that opens another GUI where the emission line can be selected among all lines in the used emission-line list.

The values of the fit are shown in the coordinate line to the right of the status line; in a comma-separated list. And the fitted wavelength is used to calculate a redshift together with the information on the selected emission line. The fitted values can be logged, if a line-fit log file was opened before making the line fit.

When the line fit values are no longer required, it is recommended to click the Clear Fit button in the Color control panel (to remove the line fit from the spectrum plot).

The enlarged spatial-map tab

The enlarged spatial-map tab provides a larger representation of the spatial map than in the main tab. The spatial map and its controls as well as the control panel are used in the same way. A mouse help image is shown in the spatial-map image area when the mouse pointer is held above the question-mark field where the two scroll bars intersect in the lower right corner.

The tab shows a vertical color bar on the left side of the spatial map, and a control panel on its right side at the top. The status line is shown at the bottom of the tab, where coordinates are shown to the left and help messages in the right-side field.

The sensitivity-function tab

The widgets on this tab support the interactive creation of a sensitivity function for flux calibration; the corresponding non-interactive creation is handled using the separate tool p3d_fluxsens. The complete and detailed documentation on how a sensitivity function is calculated is also covered in that module.

The general procedure in the spectrum viewer is to first select a calibration standard-star spectrum file, and an extinction-data file, if needed. Thereafter, it is possible to mask entries (bandpasses) in the calibration data, and adjust properties of the fit. Finally, the sensitivity function can be saved to a fits file, and the plots to a PostScript file.

The sensitivity-function tab shows four different regions with related controls: a file-selection section, a file-configuration section, a settings adjustment column, and a plot region. These regions are described next.

1. The file-selection section – top left-hand side area

The file-selection section contains widgets stacked on top of each other to select three different kinds of files: an observed spectrum, a calibration standard-star spectrum, and an extinction-data spectrum.

Selecting an observed spectrum

A sensitivity function is calculated by comparing an observed standard-star spectrum with a calibration standard-star spectrum. The currently selected spectrum in the spectrum-viewer tab is used by default. Click the "Use an External File"-button to toggle the use of an external file instead. There are three methods to select such an external file: use the file-browser button on the right-hand side of the text field, enter a filename in the text field, or set the keyword SFOBSFILE to the name of the observed standard-star spectrum file before starting the spectrum viewer.

Any external spectrum file must be a one-dimensional spectrum fits file with the wavelength information stored in header keywords; an optional second spectrum, in the same file, may contain a sky spectrum. It is, in any case, expected that the summed spectrum contains as much as possible of the flux spread out across the integral-field unit, and that the spectrum was sky subtracted.

Selecting a calibration standard-star spectrum file

The first requirement is to select a calibration standard-star file. As with the observed spectrum file, there are three methods to select such a file: use the file-browser button on the right-hand side of the text field, enter a filename in the text field, or set the keyword SFCALFILE to the name of the calibration standard-star spectrum file before starting the spectrum viewer. All details about the accepted file format are provided in the module p3d_misc_fluxcal_read_file_standard_star.

Depending on the used file, it might be necessary to provide p3d_sv with some details on the calibration standard-star spectrum file format. Such details can be specified with the widgets in the file-configuration section – or using the keywords SFCATALOG, SFWAVEUNIT, and SFFLUXUNIT, before starting the spectrum viewer.

It might be necessary to mask some bandpasses in the calibration data; this is most easily achieved by selecting bandpasses in any of the four plots. One bandpass entry is removed by clicking near it, and a group of bandpasses are removed through a click on the left mouse button and then dragging out a box. The last selected bandpass or group of bandpasses is unmarked by clicking the middle mouse button. The modified list of bandpasses can be saved to a plain-text file by clicking on the button with a disk icon.

It is possible to use multi-extension calibration files, such as the one that comes with the MUSE pipeline. Using such a file, the selected extension number and contents of the related EXTNAME header keyword are indicated in the field to the right of the disk icon.

Selecting an extinction-data spectrum file

An extinction spectrum file can be specified with the widgets in the last row. As with the other two kinds of input files, there are three methods to select such a file: use the file-browser button on the right-hand side of the text field, enter a filename in the text field, or set the keyword SFEXTFILE to the name of the extinction data file before starting the spectrum viewer. All details about the accepted file format are provided in the module p3d_misc_fluxcal_read_file_atmoshperic_extinction.

The unit of the extinction-data wavelength array can – in case it is incorrectly determined – be set with the corresponding widget in the file-configuration section, or with the keyword SFEXT_WAVEUNIT before starting the spectrum viewer.

Click the "Use Extinction Data"-button to toggle the use of any (loaded) extinction-data file.

2. The file-configuration section – top right-hand side area

The input file configuration section allows some control of how units of input files are interpreted. From the top to the bottom, the four drop list options are:

  • Catalog

    This option corresponds to the CATALOG keyword of the module p3d_fluxsens, see that module for more details. By selecting a catalog option, the wavelength unit and the flux unit of the calibration standard-star file are defined with one keyword. The seven options are: "iraf", "oke", "oke", "hst", "text_eso", "fits_eso", and "none"; the two "oke" options refer to the same setup, and the "fits_eso" option is the fits-file version of the "text_eso" option. The default option is "none"; in this case, the wavelength-unit and flux-unit widgets are used.

  • Wavelength Unit

    The module that reads the calibration standard-star file makes an attempt to guess the wavelength unit, unless CATALOG has been specified (as something else than the drop-list option "none"). The two options are: "Angstrom [Å]" [default] and "nanometer [nm]". This drop-list widget is only sensitive when the catalog drop-list widget is set to "none". This option can also be set with the keyword SFWAVEUNIT before starting the spectrum viewer. See the module p3d_misc_fluxcal_read_file_standard_star for more details.

  • Flux Unit

    The module that reads the calibration standard-star file makes an attempt to guess the flux unit, unless CATALOG has been specified (as something else than the drop-list option "none"). The three options are: "Magnitudes" [default], "erg/s/cm*2/Å, and "10**-16*erg/s/cm*2/Å". This drop-list widget is only sensitive when the catalog drop-list widget is set to "none". This option can also be set with the keyword SFFLUXUNIT before starting the spectrum viewer. See the module p3d_misc_fluxcal_read_file_standard_star for more details.

  • Extinction Data Wavelength Unit

    The module that reads the extinction-data file makes an attempt to guess the wavelength unit. The two options available in the drop-list widget are: "Angstrom [Å]" [default] and "nanometer [nm]". This option can also be set with the keyword SFEXT_WAVEUNIT before starting the spectrum viewer. See the module p3d_misc_fluxcal_read_atmospheric_extinction for more details.

3. The control panel – bottom left-hand side area

The control panel allows further control of the calculation and the saving of the sensitivity function. The seven options are the following, listed from the top to the bottom:

Read the Parameter File anew

Click to force a the instrument-parameter file as well as the user-parameter file – if it is used – to be read anew.

Reset the Bandpass Selection

Click to restore all bandpasses of the input calibration standard-star spectrum file if any bandpasses were interactively removed.

Setting the fitting function

Select the fitting function that should be used to fit the sensitivity function. All options of the keyword INPFUNCTION of the module p3d_tool_fluxcal_sensitivity_function are available in the drop list. The widgets immediately below the drop list are active if a polynomial or 'Spline' is selected. The default function is a regular polynomial.

Polynomial order / Spline tension

The text-field label is set to "Polynomial Order" when the fitting function is set to a polynomial. The currently selected polynomial order is visible in the text field, and also in the drop list on the right-hand side of the text field. The polynomial order is easily changed by entering a new value in the text field; the value must be chosen >= 0 and <= 20. Both widgets are updated to show the same value. The default value is 6.

The text-field label is set to "Spline Tension" when the fitting function is set to 'Spline'. This value defines whether to fit a cubic spline (low values) or a rational polynomial (high values). The drop list on the right-hand side of the text field is insensitive in this case.

The text field as well as the drop list are insensitive if the fitting function is neither a polynomial nor 'Spline'.

Linear extrapolation

Sensitivity-function values that fall outside the selected bandpasses of the calibration standard-star spectrum have to be extrapolated from values that fall inside the outermost bandpasses. Click to toggle the use of a linear fit to the outermost two values [default] or simply use the outermost value.

Save the Flux Calibrated Spectrum

When a sensitivity function is saved, it is possible to save the flux-calibrated observed spectrum to a separate file as well. Click to toggle the behavior of not saving such a file [default] or saving it. The automatically created filename is composed by adding a file suffix to the name of the observed standard-star spectrum file; the file suffix is taken from the user parameter 'fcsfx'.

Save the Sensitivity Function

Click to save the currently viewed sensitivity function to a fits file, using an automatically created filename. The automatically created filename is composed by adding a file suffix to the name of the observed standard-star spectrum file; the file suffix is taken from the user parameter 'fssfx'.

Save the Sensitivity Function As...

Click to save the currently viewed sensitivity function to a fits file, using a file dialog with an automatically created default filename. The automatically created filename is composed by adding a file suffix to the name of the observed standard-star spectrum file; the file suffix is taken from the user parameter 'fssfx'.

Save Plots to a Postscript File

Click to save the plots to a PostScript file, using a file dialog with an automatically created default filename. The automatically created filename is composed by first adding a file suffix to the name of the observed standard-star spectrum file; the file suffix is taken from the user parameter 'fssfx'. Thereafter, the string '_sensfunc.ps' is appended to the end of the filename.

4. The plotting section – bottom right-hand side area

Once a calibration standard-star spectrum file has been selected, plots are drawn each time a file is loaded, the file format changes, or any of the fitting related widgets are used. The plot window is divided into four parts, starting from the top left-hand side they are: the sensitivity function, the residual between the fit and the calibration spectrum, the calibration standard-star spectrum, and the extinction function. All properties are plotted versus wavelength.

The sensitivity function – top left-hand side plot

The final sensitivity function is drawn with a black solid line, while the calibration-factor function of the calibration standard-star spectrum is drawn with diamond symbols. Entries in the calibration standard-star data that were deleted are plotted with a red X symbol.

The residual – top right-hand side plot

A residual is calculated as the difference between calibration standard-star bandpass entries and interpolated values of the fitted sensitivity function. The residual of each entry is drawn with a diamond symbol. Entries in the calibration standard-star data that were deleted are plotted with a red X symbol.

The calibration spectrum – bottom left-hand side plot

The calibration standard-star spectrum is drawn with both a black solid line, and diamond symbols. Entries in the calibration standard-star data that were deleted are plotted with a red X symbol. The observed and flux-calibrated spectrum is drawn with a light green solid line while errors, if present, are drawn with a dark-green solid line.

The extinction function – bottom right-hand side

This panel is only drawn if extinction data are used. Extinction-data entries are drawn with filled dark green disks. Extinction data interpolated to use the same wavelength array as the observed spectrum (calibration standard-star spectrum) are drawn with a dark-green solid line (black diamond symbols). The interpolation function is in this case always a rational spline.

General information on how to learn more about this routine

Instrument-specific fiber position tables, which are required to use this tool, are found in the directory "!p3d_path/data/instruments/*/" (outside of IDL, instead check "${p3d_path}/data/instruments/*/").

Examples on how to use this tool are provided in the EXAMPLES section below.

See the file README in the root of the p3d package for installation instructions, if p3d is not already installed on the computer.

Additional information, documentation, new versions, etc., can be found at the p3d project web site "http://p3d.sourceforge.net".

Requirements and format of the input and the output data

p3d only works with FITS-formatted images, i.e. fits files. Such files have a header that contains additional information about the observing conditions, the telescope, the instrument setup, and other useful information. Both input and output fits files can be compressed using gzip, to save storage space (also on Windows platforms); p3d handles such compressed files transparently.

The spectrum viewer only works with data cubes or images where each row or column is a separate spectrum; this is the so-called row-stacked spectrum (RSS) format. Files, which are saved in the e3d format cannot be used with this tool (unless they are at first converted to the RSS format). To create a spatial map for RSS images, the spectrum viewer requires a fiber position-table file formatted according to the specifications of p3d (see p3d_misc_read_postable and the already existing position-table files in "${p3d_path}/data/instruments/*/").

Note: The position table -must- be properly formatted, otherwise the spatial map is poorly defined.

Links

See p3d_misc_colortable regarding the use in p3d of the Color Brewer color tables; licensing information are found at the following link: http://colorbrewer2.org/

The Voronoi binning routine voronoi_2d_binning.pro of Michele Cappellari can be downloaded here: http://www-astro.physics.ox.ac.uk/~mxc/software/

Input parameters:
imageA scalar string with the name of a reduced data-cube or RSS-image file or a two-dimensional array of floating-point type. In case this variable is set to an array, one dimension is the dispersion axis (DAXIS) and the other the spatial axis with as many elements as there are spectra in the RSS image. When launched as a separate tool, this argument must be a string; in this case, the error image is loaded from the ERROR, SIGMA, or STAT extension, or the the IMERR header keyword is used.
Optional parameters:
posfileA scalar string that specifies the name of the position-table file; this file must follow a specific format, see p3d_misc_read_postable. Using an RSS data file, the parameter PARFILE must be set if this parameter is unset.
Keyword parameters:
dimageA scalar string with the name of a reduced error data-cube or RSS-image file or a two-dimensional array of floating-point type. To show the error bars in the spectrum plot, it is necessary to click the 'Show errors' button in the lower left part of the control panel. When launched as a separate tool, this argument must be a string. However, unless it is necessary to use a dedicated error-image file, it is better to leave this keyword empty and let p3d_sv retrieve the error data using IMAGE.
noerrorsSet this keyword to avoid loading error data; this could be useful when loading Bragdingnagian datacubes (of MUSE) and there is too little RAM.
rangeAn optional two-element integer or floating-point type array that specifies the pixel or wavelength range to load from the data and their errors when these are specified as files. Pixels are specified with positive integers, for example RANGE = [100, 1250], and wavelengths with floating-point type values, for example RANGE = [-5000.0, -6500.0].
hdrXA string array with the FITS-file header of the data provided with IMAGE; this header is primarily used to read the wavelength information through the CRVALx and CDELTx keywords (x = DAXIS). The header is also used when spectra or spectrum images (slices) are saved within the tool.
zhdrXAs HDR, but of the zero extension, when KHDR is not 0.
dhdrXAs HDR, but for the error data.
andskyThe sky fiber positions are also shown for instruments that use such fibers if this keyword is set.
colorcutA two-element array of floating-point type that specifies the lower and upper values used when scaling the color map for the spectrum image and the spatial map. The minimum to maximum range of the values within 99.8% of all pixel values is used by default when this keyword is unset.
datapathA scalar string that specifies in which directory spectra and spectrum images are to be saved. The default value is: .
specrangeA two-element array of floating-point type that specifies the lower and upper values used when scaling the color map for the spectrum plot. The minimum to maximum range of the values within 98% of all pixel values in the currently shown spectrum is used by default if this keyword is unset.
orientationA scalar integer that specifies the orientation in the spatial map; 0 <= ORIENTATION <= 3. The default value 1 results in North up, and East left. The default value is: 1
colortableA scalar string or integer that specifies which color table should be loaded. If COLORTABLE is a string, the string must contain two parts separated by a comma, without any whitespace between. The first part before the comma specifies the color-table number in the file name specified after the comma; the file must be formatted as described in the documentation for MODIFYCT. As an example, COLORTABLE='25,brewer.tbl'. Components in the ColorBrewer color-table file "brewer.tbl" found in the resource directory can be specified explicitly by preceeding the string with "CB"; for example, COLORTABLE='CB25'. If COLORTABLE is an integer, the integer must be given in the range -4, -3, -2, -1, 0, ..., up to the maximum number of available tables in IDL, plus the available tables in the ColorBrewer file (resource/) "brewer.tbl". Select -4, -3, -2, or -1 to load the cubehelix, 2 * Califa, and the Sauron color tables, the other values use the respective color map as defined by LOADCT. These are the permitted values:

-4

Loads the cubehelix color table.

-3

Loads the Califa project intensity color table, as defined September 2012.

-2

Loads the Califa project velocity field color table, as defined September 2012.

-1

Loads the Sauron color table (default).

0-74 (IDL version >= 8.3)

Loads the corresponding color table with LOADCT.

0-40 (IDL version < 8.3)

Loads the corresponding color table with LOADCT.

'CB0'-'CB85'

Loads the corresponding ColorBrewer color table; the color tables are defined in the file "resource/brewer.tbl"

75-160 (IDL version >= 8.3)

Loads the ColorBrewer color table corresponding to this number after 75 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

41-125 (IDL version < 8.3)

Loads the ColorBrewer color table corresponding to this number after 41 is subtracted; the color tables are defined in the file "resource/brewer.tbl"

'x,ctfile'

Loads the x:th entry ('x' must be an integer) in the color-table file 'ctfile.

The default value is: -1
invertSet this keyword to invert the loaded color table.
cinvSet this keyword to plot white lines on a black background instead of the default, which is to plot black lines on a white background.
bottomA scalar integer that specifies the bottom color map index to use with the data; 0 <= BOTTOM <= !d.table_size – 1.
cindexAn array of integers specifying the color indices used as reserved colors. CINDEX ideally has 7 elements, P3D_SV uses 7 reserved colors. If the number of elements is smaller, the upper range of reserved colors will use the largest index available (also see CINDV).
cindvAn array of 7 integers specifying which of the indices in CINDEX are used as the reserved colors: If CINDEX has 7 elements, CINDV = CINDEX[0:6]. If CINDEX has 3 elements, CINDV = [CINDEX[0], CINDEX[1], CINDEX[2], CINDEX[2], CINDEX[2], CINDEX[2], CINDEX[2]].
coffsetA scalar integer that specifies how many additional color tables are specified in addition to the default ones.
ch_startThis scalar decimal value defines the color when COLORTABLE = – 4. The default value is: 0.5
ch_rotsThis scalar decimal value defines the rotation in color when COLORTABLE = – 4. The default value is: -1.5
ch_hueThis scalar decimal value defines the hue intensity scaling when COLORTABLE = – 4. The default value is: 1.2
ch_gammaThis scalar decimal value defines the gamma correction factor when COLORTABLE = – 4. The default value is: 1.0
titleA string used in the title of the base widget. The default value is: 'p3d Spectrum Viewer'
parfileA scalar string that specifies the name of a file with instrument-specific setup parameters. These files follow a strict format. Already properly formatted files are available for all configured instruments in the p3d data/instruments/*/ directories. It is sufficient to specify the file basename, without the full path of the p3d subdirectory. Note! Using RSS image data in the first argument, this parameter must be specified if POSFILE is unset. The file is used to select the fiber position-table and dead-fibers files, if used.
ofilenameThis keyword, which can be set to a scalar string, allows a specification of an output-filename prefix for files written with the tool. The output-filename prefix is otherwise taken from the name of the input file. In any case, when run separately using the IDL VM or from the p3d GUI, the suffix '_spectrum' ('_spatialmaps') is appended before the file-type suffix for saved spectra (spatial maps). If the tool is started from the command line, the default output filename is 'p3d_spectrum.fits.gz' ('p3d_spatialmaps.fits.gz'). Output files are compressed using gzip if the output filename ends with the string '.gz'.
opathA scalar string that specifies the path where the output spectrum and spatial-map files are saved. The default value is: '.'
deadfibersfileAn optional scalar string with the name of a file that lists dead/unused/low-transmission fibers. The file must conform to the standard of the regular dead-fibers files of p3d. If this keyword is not used, the corresponding file is used as provided in the instrument-specific parameter file PARFILE.
binmapA scalar string containing the name of a plain-text file with three-column rows identifying each spatial element with a bin. The file must contain three columns, where the first two columns specify the X and Y coordinates of each bin and the third column specifies the bin number; negative values indicate that the spatial element is not binned. The number of rows must not exceed the number of elements in the loaded RSS image or data cube.
regionsSpecify one or several optional regions files to load when the spectrum viewer is launched. Available file formats are the p3d-specific format and DS9 region files.
detectorA scalar integer that specifies the currently selected detector; DETECTOR is a zero-based value. The default value is: 0
fitsheadertitleThis scalar string is used when displaying the fits header. The default value is: 'Fits header'
daxisDefines the dispersion direction dimension of IMAGE. The default, 1, is in the x-direction. This parameter must only be provided if PARFILE is unset. The default value is: 1
spaxelscaleFor RSS data without any information on the physical scale of a square-shaped spatial element, this keyword allows it to be specified. The value is used to calculate a World Coordinate System (WCS), which in turn is used to show world coordinates when dragging the mouse pointer across the spatial map windows). The unit is arcsec. See the keyword SPAXELSCALE in the instrument-keyword file for instruments that provide this information in the data header.
winxysizeA two-element integer array that specifies a requested size on the spectrum-viewer window. The maximum possible size is used if this keyword is unpresent.
sfobsfileA scalar string that specifies the name of an observed spectrum file. This spectrum file is selected and loaded as a starting point to create a sensitivity function. Use this keyword instead of the file-selection widgets on the sensitivity-function tab if you want to select an external file before starting the spectrum viewer. The default is otherwise to use the spectrum selected in the spectrum viewer.
sfcalfileA scalar string that specifies the name of a calibration standard-star spectrum file. This spectrum file is selected and loaded as a starting point to create a sensitivity function. Use this keyword instead of the file-selection widgets on the sensitivity-function tab if you want to select a file before starting the spectrum viewer.
sfextfileA scalar string that specifies the name of an extinction data file. This file is selected and loaded when creating a sensitivity function. Use this keyword instead of the file-selection widgets on the sensitivity-function tab if you want to select a file before starting the spectrum viewer.
sfcatalogA scalar string that specifies the pre-defined file format of plain-text calibration standard-star spectrum files, which are specified with the keyword SFCALFILE. Use this keyword to specify the format before starting the spectrum viewer.
sfwaveunitA scalar string that specifies the unit of the wavelength array in the plain-text calibration standard-star spectrum file specified with the keyword SFCALFILE. This keyword is only used if SFCATALOG is unset.
sffluxunitA scalar string that specifies the unit of the flux array in the plain-text calibration standard-star spectrum file specified with the keyword SFCALFILE. This keyword is only used if SFCATALOG is unset.
sfext_waveunitA scalar string that specifies the unit of the wavelength array in the plain-text extinction-data file specified with the keyword SFEXTFILE.
redshift_velA scalar floating-point value that specifies the redshift in the data using a velocity, the unit is km/s.
redshift_zA scalar floating-point value that specifies the redshift in the data; this value isn't considered if REDSHIFT_VEL is set.
nanometerSet this keyword to plot the spectrum x-axes using nanometer instead of the default Ångström (Å).
micrometerSet this keyword to plot the spectrum x-axes using micrometer (µm) instead of the default Ångström (Å).
wavenumberSet this keyword to plot the spectrum x-axes using the wavenumber (/cm) instead of the default wavelength Ångström (Å).
raman_exc_wavelThis scalar floating-point type keyword defines the Raman excitation wavelength (in Ångström [Å]), which is used to plot the upper x-axis (wavenumber shift) when WAVENUMBER is set. The default value is: 7853
noposreverseSet this keyword to avoid reversing fiber position arrays with instruments where the parameter 'posarr_reverse' is set.
a2Set this keyword to use the A2 paper format with PostScript and PDF output instead of A4.
a3Set this keyword to use the A3 paper format with PostScript and PDF output instead of A4.
a5Set this keyword to use the A5 paper format with PostScript and PDF output instead of A4.
letterSet this keyword to use the US Letter paper format with PostScript and PDF output instead of A4.
legalSet this keyword to use the US Legal paper format with PostScript and PDF output instead of A4.
tabloidSet this keyword to use the US Tabloid paper format with PostScript and PDF output instead of A4.
emission_linesSet this keyword to a scalar string with the name of a file that contains wavelengths of emission lines to use in the line identification during a line fit. The default file is searched for in the current directory and the p3d line-list directory "!p3d_path/data/tables/linelists/". The file must contain three columns: the wavelength in Å, an identification string, and a truncated wavelength. The default value is: 'emission_lines-ground_based-noFe.dat'
userparfileA scalar string with the name of a file with user-defined parameters. The following parameters are read here from this file: to-be-filled-in
khdrXA scalar string used to identify the source extension of the shown data. The default value is: 'keyword'
kdhdrXA scalar string used to identify the source extension of the shown error data. The default value is: 'keyword'
nthreadsA scalar integer that specifies how many threads to use in various parallelized operations used here. The default is to use 1 thread, as it seems to be the optimal value. The default value is: 1
display_nameSet this keyword equal to a string that specifies the name of the X Windows display on which the base should be displayed; this keyword has no effect on Microsoft Windows platforms.
noCSet this keyword to not use any C routines.
tracking_eventsIf this keyword is set, hints on the function he various widgets and actions are given on a us line. The default value is: 1
fontSet this scalar string to the name of the font to be used with all widget components of this tool.
keep_cubeSet this keyword to keep data cubes as cubes in the p3d_sv state structure, instead of first converting them to RSS images; the two approaches should give the same result, and this keyword is only used for debugging purposes, i.e., leave it as it is. The default value is: 1
spatmap_flipxUnset this keyword to have the physical (x, y) coordinates in the spatial map to work as in DS9 where a lower x value is always found on the East (left) side. The corresponding user parameter is 'spatmap_flip_x_coord'. The default is to have the East-West x coordinate increase towards East [spatmap_flipx = 1]. Note! The coordinates are always based at (0, 0).
cmdlineXThis keyword is used then the tool is launched by the p3d server.
guiXIf this keyword is set, the routine is assumed to be called from the p3d GUI – meaning that there is no initialization of the p3d system variables.
group_leaderXIf set, error messages are displayed using DIALOG_MESSAGE, using this widget id as DIALOG_PARENT, instead of MESSAGE.
logunitXMessages are saved to the file pointed to by this logical file unit, if it is defined.
verboseSet this parameter to a positive scalar integer to make p3d write some information on STDOUT about what is going on. The following four values are acknowledged:

0

Writes no information at all. This is the default.

1

Writes the more important information; regarding subroutine configurations, mostly.

2

Writes most information; includes a more verbose output than 1.

3

Writes all information, including information on the execution state of GUI subroutines. This may be a useful mode when debugging the code.

quietSet this keyword to not print the disclaimer on the screen when this tool is started from the command line or from the shell.
errorXThis scalar integer returns an error code if set. Any value different from zero means that an error has occurred.
debugXAn error handler is not setup if this keyword is set. Otherwise, the error handler is setup using the routine CATCH, which makes each subroutine exit quietly when a bug is encountered. Use this keyword when debugging p3d.
cdebugA keyword as DEBUG used with the called C routines; valid values are 0-6 where a higher number results in more output to STDERR.
helpXSet this keyword to show this routine documentation, and then exit.
Side effects:

None, unless a spectrum or a (set of) spectrum image(s) is saved.

Examples:

This routine can be called in four different ways (for use on WINDOWS platforms, replace the path character '/' with '\'):

i) Using the p3d GUI

The spectrum viewer is available for each type of reduced product once an object image has been reduced, a dispersion-mask image has been created, or a flat-field image has been created – or when an already created image of these types has been opened. Click on the rightmost diagram symbol, for each type, to open the spectrum viewer. There is no need to setup any input parameters, p3d handles this transparently.

ii) As a stand-alone tool from the IDL command line

The spectrum viewer p3d_sv can be launched using either a filename or already loaded data. Unless the data are present in the form of a cube, it is necessary to also supply the name of a provided instrument-specific fiber position-table file or an instrument-specific parameter file. Some features, such as locations of dead and low-transmission fibers are only available using the instrument-specific parameter file. It is recommended to use this latter approach, in particular if the data were reduced using p3d.

; Using a file:
image = readfits(, header)
parfile = 'larr2k.prm'
p3d_sv, 'object_oextr.fits.gz', parfile=parfile, colortable=-1

; Loading the data before launching the spectrum viewer:
image = readfits('object_oextr.fits.gz', header)
parfile = 'larr2k.prm'
p3d_sv, image, parfile=parfile, hdr=header, colortable=-1, /verbose

; Loading the data, using a fiber position-table file:
image = readfits('object_oextr.fits.gz', header)
posfile = !p3d_path + 'data/instruments/pmas/larr2k_positions.dat'
p3d_sv, image, posfile, hdr=header, colortable=-1, /verbose

Reduced datacubes of, for example, the ESO/VLT/MUSE instrument, are opened simply by providing a filename, and it might be useful to set the RANGE keyword to delimit the amount of loaded data.

; Load a MUSE data cube; delimit the wavelength range to 4800-7000Å:
image = 'muse_cube.fits'
p3d_sv, image, range=[4800.,7000.]

; Load a MUSE data cube; delimit the wavelength range to use the
; pixel range 0-2000, use a different color table:
image = 'muse_cube.fits'
p3d_sv, image, range=[0, 2000L], colortable=-4

; Load a MUSE data cube; save memory by not loading any error data,
;   show information about the loading process, etc.:
image = 'muse_cube.fits'
p3d_sv, image, /noerrors, /verbose, colortable=-4

iii) As a stand-alone tool from a terminal, using the IDL VM

The extracted spectrum image file is in this approach always specified with a filename. Additionally, it is necessary to pass either the name of a (provided) instrument-specific fiber position-table file, or an instrument-specific parameter file. Some features, such as the location of dead and low-transmission fibers, are only available using the instrument-specific parameter file. It is recommended to use this latter approach, in particular if the data were reduced using p3d. Use either the provided shell script p3d_sv_vm.sh or the Python tool p3d_dispatch.py.

# Using an instrument-parameter file, and don't use the
#   p3d queue system:
image = object_oextr.fits.gz
parfile = larr2k.prm
p3d_sv_vm.sh --noqueue $image parfile=$parfile colortable=-1

# Using the Python tool and an instrument-parameter file, and do use
#   the p3d queue system:
image = object_oextr.fits.gz
parfile = larr2k.prm
p3d_dispatch.py sv $image parfile=$parfile colortable=-1

# Using a fiber position-table file, and don't use the
#   p3d queue system:
image = object_oextr.fits.gz
posfile = ${p3d_path}/data/instruments/pmas/larr2k_positions.dat
p3d_sv_vm.sh -s $image $posfile colortable=-1 /verbose

Reduced datacubes of, for example, the ESO/VLT/MUSE instrument, are opened simply by providing a filename, and it might be useful to set the RANGE keyword to delimit the amount of loaded data.

# Load a MUSE data cube; delimit the wavelength range to 4800-7000Å,
#   and don't use the p3d queue system:
image = muse_cube.fits
p3d_sv_vm.sh --noqueue $image range=4800.,7000.

# Load a MUSE data cube using the Python tool; delimit the wavelength
# range to use the pixel range 0-2000, use another color table, and
# do use the p3d queue system:
image = muse_cube.fits
p3d_dispatch.py sv $image range=0,2000 colortable=-4

# Load a MUSE data cube; save memory by not loading any error data,
#   show information about the loading process, and don't use the
#   p3d queue system:
image = muse_cube.fits
p3d_sv_vm.sh -s $image /noerrors /verbose colortable=-4

Note: In this approach, it is necessary to specify all options using the full names of the keywords. Partially correct keyword names are not permitted, as they are inside IDL. Hence, as an example, you must type "p3d_sv_vm.sh $image $posfile orientation=1" and -NOT- "p3d_sv_vm.sh $image $posfile ori=1".

Note: The tool launches in the current directory, which is why it is unnecessary to specify full paths if files reside in the current directory.

Note: Parameters indicated with an 'X' in the list above are unavailable in this approach.

iv) As a stand-alone tool from Python, using the IDL VM

This approach is analogous to iii), but is executed from Python. Here are two examples:

# Launch p3d_sv in a single IDL session, without using the p3d
#   queue system:
import p3d_dispatch
image = 'object_oextr.fits.gz'
parfile = 'larr2k.prm'
commandline = image + ' parfile=' + parfile + ' colortable=-4'
p3d_dispatch('sv', commandline, noqueue=True)

# Set the NOERRORS keyword to avoid loading the MUSE error data:
image = 'DATACUBEFINAL_expcombine.fits.gz'
commandline = image + ' /noerrors /verbose'
p3d_dispatch('sv', commandline, noqueue=True)

page last updated on: Friday April 21, 2017 13:31:40; retrieved on: Friday May 26, 2017 07:23:58