Documention on how to run the PQ realtime pipeline.

The first step towards running the real-time pipeline is to load the photometry database with the data for the next observing run (unless it was loaded earlier). The second step is to stage the photometry data onto disk and the final is to run the pipeline.
Note: before doing anything else, first login to tg-login and type newgrp quest then source ~ajd/.cshrc.

For details of pipeline individual tasks, look here
For a diagram of the system architecture and here.

-----------------------------------------------

Quick HOWTOs

-----------------------------------------------

How do I load photometry into the database?

  • /home/ajd/runphot.pl (RA) (Dec) (delta RA) (delta Dec) /mnt/dw2-08/rawdata A 10.0 dque 10
    • Using 10 nodes, it will take about a week to load the photometry for a PQ observing run.

    -----------------------------------------------

    How do I stage the photometry data?

  • /home/ajd/bin/stage_mssql.pl (UT Date= 200#####) (RA) (Dec) (delta RA) /mnt/dw2-07/rawdata 1 28 7
    • Allow at least 12 hours for staging photometry files.

    -----------------------------------------------

    How do I run the actual real-time pipeline?

    First, kill any pipeline processes that may be running.
    Then, in one xterm type:

  • /home/ajd/pipeline_realtime/bin/realtime_server.pl /mnt/dw2-07/rawdata

    • Finally, in another xterm type:

  • /home/ajd/pipeline_realtime/runpipe.pl /mnt/dw2-07 10 7

    • Order is important and both scripts will continue to run until you kill them if runtime is not given.

    -----------------------------------------------

    How do I reanalyze data received in real-time?

    First, kill any pipeline processes that may be running.
    Next, move the raw photometry data files data to /mnt/dw2-07/rawdata/checked.
    Then, in one xterm type:

  • /home/ajd/pipeline_realtime/bin/realtime_server.pl /mnt/dw2-07/rawdata (UT Date=200#####) 8.0

    • Finally, in another xterm type:

  • /home/ajd/pipeline_realtime/runpipe.pl /mnt/dw2-07 10 7 8.0 (UT Date=200#####)

    • Order is important and both scripts will continue to run until you kill them if runtime is not given.

    -----------------------------------------------

    How do I fix the WCS for many clean images?

  • /home/ajd/pipeline_realtime/bin/runwcs.pl filename 18 8.0

    • The file must contains a list of the images and their root location
    e.g. cln_do20050809.009.C.JB.03.069.fits.gz /mnt/dw2-03

    -----------------------------------------------