  Installation and testing guidelines for the MUSE data reduction pipeline

================================================================================

  Prerequisites
 ^^^^^^^^^^^^^^^

In case you do not install from the pipeline kit, you need to make sure that
some prerequisites are met:

- Download CFITSIO, wcslib, CPL, and EsoRex from
     http://www.eso.org/sci/software/cpl/download.html
  and install them.
  Notes:
  . CFITSIO needs to be configured with --enable-reentrant (for multithreading).
  . CPL v6.6, configured for multithreading (the default), is required.
  . CPL needs to be set up to work with wcslib (configure switch --with-wcs)
    and FFTW (configure switch --with-fftw).

  CFITSIO = stand-alone C library for easy FITS file access
  wcslib  = Astronomical World Coordinate System transformations library
  FFTW    = C subroutine library for computing the discrete Fourier transform
  CPL     = Common Pipeline Library for VLT
  EsoRex  = ESO Recipe Execution Tool

- libxslt should be installed, too, for the xsltproc command to work. On most
  (Linux) systems it is installed by default.

- In the following we assume that you want to install the pipeline into a
  directory ${MUSE_DIR} and that you have installed CPL into ${CPL_DIR},
  respectively. You should replace these with real paths while carrying out the
  instructions below.

- Set up esorex by editing ~/.esorex/esorex.rc (run esorex --create-config if it
  doesn't exist yet) to contain at least
     esorex.caller.recipe-dir=${MUSE_DIR}
  Perhaps you also want
     esorex.caller.msg-level=debug
  to get more output while running the recipes.

- GCC 4.3 or later for parallelization, GCC 4.7 or later for full performance
  You need a recent version of the GNU Compiler Collection to be able to use
  OpenMP parallelization in some functions of the MUSE library. A compiler
  supporting OpenMP v3.1 is needed to use the self-calibration function in
  the muse_scipost recipe with full performance.


================================================================================

  Installing the pipeline from a release tarball
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Unpack the sources:
     tar xvjf musep-2.6.tar.bz2

- Configure the sources (and keep a log of the output):
     cd musep-2.6
     ./configure --prefix=${MUSE_DIR} --with-cpl=${CPL_DIR} --with-cext=${CPL_DIR} 2>&1 | tee build.log
  where ${CPL_DIR} is the location, CPL was installed to.

- Compile and install the sources:
     make         2>&1 | tee -a build.log
     make install 2>&1 | tee -a build.log


================================================================================

  Installing the pipeline from SVN
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Check out the MUSE pipeline source tree from SVN:
     svn co http://svnhq2.hq.eso.org/p2/trunk/Pipelines/muse muse
  (Or use another tool like |git svn| or |hg svn|.)

- Configure the muse pipeline recipes code (I usually use /tmp/musetest as
  ${MUSE_DIR}):
     cd musep
     ./bootstrap
  (this will fail if not the right versions of autoconf, automake, aclocal, and
  libtool are installed)
     ./configure --prefix=${MUSE_DIR} [ --with-cpl=${CPL_DIR} ] [ --with-cext=${CPL_DIR} ]
  possibly add
     --enable-debug                (compile with debugging symbols)
     --enable-strict               (more warnings during compilation)
     --enable-maintainer-mode      (enables compilation of documentation)
  Compile it:
     make
     make html                     (to generate the HTML documentation)
  Install it (into ${MUSE_DIR} which I usually set to /tmp/musetest)
     make install
     make install-html             (installs HTML documentation in
                                    ${MUSE_DIR}/share/doc/muse/html)
  Note: The install targets also carry out the normal compilation if
        it was not done before seperately.


================================================================================

  Automated Testing
 ^^^^^^^^^^^^^^^^^^^

To ensure that the pipeline was installed correctly, one can run the automated
test programs.

- Unpack the current tarball of test data (named "muse_tests_YYYY-MM-DD.tar.bz2")
  in the top level _destination_ directory so that the test data ends up in
  ${MUSE_DIR}/tests:
     cd ${MUSE_DIR}
     tar xvjf muse_tests_YYYY-MM-DD.tar.bz2

- Now change back to the _source_ directory of the MUSE pipeline and run the
  tests (and keep a log of the output):
     cd <path>/musep-2.6/      (if you install from a release tarball)
  or
     cd <path>/musep/          (if you install from SVN)
  and then
     make check 2>&1 | tee check.output

- If the output says that any tests failed, please mail check.output to the
  pipeline maintainers.

- Now you can again remove data generated by this check:
    rm tests/*.log tests/*.fits


================================================================================

  Restrictions
 ^^^^^^^^^^^^^^

The following recipes work, with some restrictions:
  muse_lingain    (experimental, to derive gain and non-linearity correction;
                   needs special, long exposure sequence)
  muse_bias
  muse_dark
  muse_flat
  muse_wavecal    (care has to be taken to select inputs from all lamps)
  muse_lsf
  muse_geometry   (care needs to be taken to select a full series and
                   corresponding calibrations, as well as to limit the memory
                   usage by setting OpenMP thread limits; not strictly part of
                   the science pipeline)
  muse_ampl       (not part of the science pipeline)
  muse_qi_mask    (special recipe for the mask creation on the instrument
                   workstation; not part of the science pipeline)

  muse_illum      (dummy recipe, only useful for the online pipeline at Paranal)
  muse_twilight
  muse_create_sky
  muse_standard   (no support for multiple stars)
  muse_astrometry (works for defined fields)

  muse_scibasic
  muse_scipost
  muse_exp_align   (works to compute offsets for most exposure sequences with
                    large overlaps, but not all)
  muse_exp_combine (care has to be taken to limit memory usage, e.g. by
                    specifying wavelength ranges)

The following recipes provide functionality that is integrated into the
muse_scipost recipe. They were split off to be able to test each step
separately, but are not thoroughly tested:
  muse_scipost_correct_dar
  muse_scipost_raman
  muse_scipost_subtract_sky
  muse_scipost_subtract_sky_simple
  muse_scipost_correct_rv
  muse_scipost_calibrate_flux
  muse_scipost_apply_astrometry
  muse_scipost_combine_pixtables
  muse_scipost_make_cube


================================================================================

  Troubleshooting
 ^^^^^^^^^^^^^^^^^

- Routines in the MUSE library obey the following environment variables to
  switch on experimental features:
    MUSE_AIT_HACK_SLICE_NUMBER
                       Set this to an integer between 1 and 48 to restrict the
                       tracing search to this number of slices.
                       the tracing module will try to interpolate the missing
                       slices afterwards, but this is bound to give bad results
                       and will cause many error messages to be printed!
    MUSE_BASICPROC_SKIP_GAIN_OVERRIDE
                       If you give a NONLINEARITY_GAIN file you can use it to
                       only correct for nonlinearities and skip applying the
                       gain by setting this variable to a positive integer.
    MUSE_BASICPROC_SKIP_NONLIN_CORR
                       If you give a NONLINEARITY_GAIN file you can use it to
                       only override the gain and skip applying the nonlinearity
                       correction by setting this to a positive integer.
    MUSE_TWILIGHT_SCALES
                       A comma-delimited string of 24 numbers which are
                       interpreted as relative scales for each IFU, before
                       constructing the DATACUBE_SKYFLAT.
    MUSE_TWILIGHT_SKIP Set this to a filename of a DATACUBE_SKYFLAT as saved by
                       the muse_twilight recipe, to start from that instead of
                       spending a lot of time to reconstruct it again; to be
                       used for debugging!
    MUSE_GEOMETRY_SKIP Set this to a positive integer to skip part of the
                       loading/processing. 1 means to skip raw image processing
                       and start from reduced data (which has to be present in the
                       current dir!).  2 means to skip both raw image processing
                       and the spot measurements (needs first reduced file and
                       the spots tables in the current dir!).
    MUSE_GEOMETRY_NO_INVERSION
                       Set this to a positive integer to deactivate after-the-fact
                       flipping of the slices along the vertical direction.
    MUSE_GEOMETRY_MASK_ROTATION
                       If the multi-pinhole mask was aligned along a diagonal,
                       set this variable to the angle of this rotation (so
                       either 11.7675 or 22.6185 degrees), so that such global
                       misalignment can be taken into account when comparing to
                       the expected mask hole distances.
    MUSE_GEOMETRY_PINHOLE_DY
                       Use the value of this variable as the pinhole distance in
                       vertical direction [mm]. This overrides the internally
                       computed value (normally accurate to about 0.0001 mm).
    MUSE_GEOMETRY_STD_GAP
                       If this variable is set to a positive integer, the
                       horizontal geometrical computation will use an older
                       way to compute the gaps.
    MUSE_GEOMETRY_HORI_OFFSETS
                       Set this to a comma-delimited list of floating point
                       values to apply additional offsets to the internal
                       horizontal IFU location refinement. The values are
                       interpreted as horizontal shifts in units of pixels.
    MUSE_DAR_CORRECT_METHOD
                       Switch methods to correct differential atmopheric
                       refraction (DAR). Four values are valid:
                       - "Filippenko": use the DAR correction method from
                          Filippenko (1982); this is the default
                       - "Owens": use the DAR correction method from Owens
                         (1967) which is also used by Marchetti (2001)
                       - "Edlen": use the refractive index calculation from
                         Edlen (1966) and Birch & Downs (1993)
                       - "Ciddor": use the refractive index calculation from
                         Ciddor (1966); this is supposed to give the most
                         accurate results
    MUSE_PIXTABLE_SAVE_AS_TABLE
                       When this variable is set to a positive integer, pixel
                       tables are saved as FITS binary tables instead of multi-
                       extension FITS images. On most hardware it should be
                       faster to use the default. (Loading any pixel table in a
                       later processing stage detects the file format and acts
                       accordingly.)
    MUSE_COLLAPSE_PIXTABLE
                       set this to a positive value to get the collapsed image
                       of the field of view created from data before resampling;
                       per filter this takes as long as the resampling to cube;
                       This only works, if the output format is not Euro3D.
    MUSE_COLLAPSE_USE_VARIANCE
                       set this to a positive value to weight datapoints using
                       inverse variance (to suppress low S/N regions) while
                       collapsing data to an image. This applies both in normal
                       operation and when MUSE_COLLAPSE_PIXTABLE is set.
    MUSE_CPL_ERRORSTATE_NDUMP
                       The MUSE recipes output an error summary at the end of
                       the processing. By default this lists the last 20 errors
                       that happened during processing. Set this variable to a
                       positive value to output any other number.
                       (Note that even when setting this to a high value, most
                       of the errors will display as 'The actual error was
                       lost', since CPL only keeps a certain number of errors in
                       its history.)
    MUSE_EXPERT_USER   Set this environment variable if you are familiar with
                       the code of the MUSE pipeline and are happy to check the
                       output/log for WARNINGs and ERRORs, can deal with partly
                       filled products, occasional crashes, and unexpected
                       behavior.


- Some routines in the MUSE library obey the following environment variables to
  enable extra debugging output:
    MUSE_DEBUG_QUADRANTS
                       switch on additional debugging messages about the
                       location of quadrants on the CCD and trimmed regions
    MUSE_DEBUG_TRACE   switch on additional debugging messages in tracing
    MUSE_DEBUG_WAVECAL set this to positive integers to get subsequently more
                       debug output, a (max) value of 3 or higher will produce
                       an ASCII output file "MUSE_DEBUG_WAVE_LINES-nn.ascii"
                       (where nn is the IFU number) with all detected lines and
                       their associated wavelengths
    MUSE_DEBUG_IGNORE_READOUT and MUSE_DEBUG_IGNORE_INSMODE
                       These variables can be used to override pipeline guards
                       against applying unsuitable calibrations. Specifically,
                       if the first variable is set to a positive integer,
                       a master bias of a different read-out mode can be applied
                       to another raw data set. The second variable can be used
                       to apply a flat-field to a dataset of a different
                       instrument mode.
    MUSE_DEBUG_LSF_FIT set this to a positive integer to get lots of debugging
                       output for every iteration of the LSF fit procedure
    MUSE_DEBUG_AUTOCALIB
                       set this variable, and the pipeline saves an extra column
                       "MASK" with the pixel table during autocalibration; this
                       also causes the autocalibration to output a lot more
                       debug-level info and statistics for all slices of all
                       IFUs for all lambda ranges; otherwise only progress
                       messages are given.
    MUSE_DEBUG_SKY     set this to a positive integer to more debugging output
                       from sky subtraction
    MUSE_PLOT_TRACE    will try to use gnuplot to visualize the detected slices
                       and relevant limits (if value is 1)
    MUSE_DEBUG_DCR     switch on additional debugging messages in the cosmic ray
                       detection following the DCR approach (values of 1, 2, 3,
                       and 4 result in subsequently more debug output), some may
                       also write extra files with debugging output
    MUSE_DEBUG_FLUX    set this to to a positive integer to get debug output and
                       files while carrying out flux calibration:
                         >= 1: files flux__sens_*.ascii containing different
                               sensitivity calculation stages
                         >= 2: files flux__cube.fits, flux__intimage.fits, and
                               possiblty flux__tellregions.fits containing the
                               reconstructed cube, a special image file with
                               measurements of integrated flux, and the table
                               of the builtin telluric regions (if used)
                         > 2:  processed pixel table used to reconstruct the cube
    MUSE_DEBUG_PIXTABLE_LIMITS
                       if set to a non-zero value, output coordinate limits of a
                       pixel table after computing them
    MUSE_DEBUG_GEO_VERIFY_DY
                       if set to a positive integer, verify vertical pinhole
                       mask distances using a set of output files
    MUSE_DEBUG_GEO_VERTICAL
                       if set to a positive integer, output extra debugging
                       about the vertical distances between slices in each stack
    MUSE_DEBUG_MEMORY_PROGRAM
                       set to a executable name for which to print memory
                       information in selected parts of the pipeline
    MUSE_DEBUG_WCS     if set to a positive integer, output debugging info when
                       creating new WCS info from a FITS header and carrying out
                       the astrometric calibration. In the latter case, extra
                       files are saved for the following values:
                         >= 2: files wcs__cube.fits, wcs__image_median.fits, and
                               wcs__detections.fits, containing the cube, a
                               median detection image, and a table of detected
                               sources for the astrometric calibration, as well
                               as ASCII files wcs__{detections,references}.ascii
                               containing tables of detected and reference
                               sources
                         > 2:  processed pixel table used to reconstruct the cube
    MUSE_DEBUG_GRID_CONVERSION
                       set to a non-zero value in builds with --enable-debug
                       will output information about which pixtable row ends up
                       where in the 3D grid when resampling to a datacube
    MUSE_DEBUG_NEAREST set to a non-zero value in builds with --enable-debug
                       to get information about nearest-neighbor resampling
    MUSE_DEBUG_WEIGHTED
                       The following non-zero values cause special debug output
                       in the resampling routines in builds with --enable-debug:
                            1: display sums                           3D case
                            2: display contributing pixels            3D case
                               (needs X, Y, Z coordinates, see below)
                            4: display individual pixels              3D case
                            8: detailed info about weights of all     3D case
                               contributing pixels
                          128: display contributing pixels         2D case
                               (needs X, Z coordinates, see below)
                          256: display individual pixels           2D case
                       The values can be combined, i.e. a value of 3 means to
                       output both sums and contributing pixels.
    MUSE_DEBUG_WEIGHTED_X
    MUSE_DEBUG_WEIGHTED_Y
    MUSE_DEBUG_WEIGHTED_Z
                       when all of these variables are set to a positive value,
                       then they are used to display pixtable entries
                       contributing to the output pixel at this grid point in
                       the output cube (values starting at 1).
    MUSE_DEBUG_WEIGHTED_GRID
                       Set this to a valid filename to produce an ASCII file
                       of the constructed grid. The columns in that table are
                       the grid cell numbers (starting from 1) and the three
                       coordinates of the cell centers.
    MUSE_DEBUG_WEIGHT_CUBE
                       set this to a valid filename to produce a weight cube
                       (analogous to a weight map in imaging) in addition to a
                       final datacube when carrying out cube reconstruction
                       using a weighted resampling method
    MUSE_DEBUG_CRREJECT
                       The following non-zero values cause special debug output
                       in the cosmic-ray rejection during final resampling, in
                       builds with --enable-debug:
                            1: display pixel table entry for rejected cosmic ray
                            2: display statistics of surrounding pixels
                            4: debug filling of statistics object
                       All need X, Y, Z coordinates, see below.
    MUSE_DEBUG_CRREJECT_X
    MUSE_DEBUG_CRREJECT_Y
    MUSE_DEBUG_CRREJECT_Z
                       Debug coordinates, similar to MUSE_DEBUG_WEIGHTED_*.

  If you have problems with some functions, switch them on to see more, possibly
  helpful output. If you cannot solve a problem, ask for help. If you are part
  of the MUSE consortium, use the CRAL gitlab ticket system at
     https://git-cral.univ-lyon1.fr/MUSE/DRS/issues
  (An account is required.)
  otherwise write to usd-help@eso.org.
