8.- Data Reduction Machine
8.1 - General Information
As an add-on to the WIRC System, a reduction machine is configured for
the data pipeline.
The idea is that this machine does not affect and it is completelly transparent
to the acquisition machine.
There is a task running on this machine for getting all the new images
acquired through the net, automatically, using sftp. This means that the
reduction machine pulls the data out instead of having the acquisition machine
pushing it.
In this way, if the link is not working, or the reduction machine is down,
the acquisition machine is not affected at all. We may say that the acquisition
machine does not even know of the existance of the reduction machine.
The system works basically in this way:
There is a database on the ArcVIEW system, a module called DBS. This database
is a general purpose module which holds variables of general interest for
all the rest of the system. This DBS module has, besides the internal port
for the other modules, an external TCP/IP port for servinbg external requests.
One of the things the database can keep track of, is the images that has
been written to disk. If set for doing so, the FITS writter module can send
to the Database any new image that was written to disk. In this way, the
DBS mantains a list with all the images that have been written. There is
a set of commands that the DBS module accept for handling this list of images.
In the Data Reduction Machine, then, can run any software for communicating
with the DBS module on the acquisition machine, so it can know which images
have been written and where they are; this is the only information this
machine needs in order to pull the desired data out.
Any software that can use a socket for communicating can then do this.
As examples, two "applications" has been created, intended onlly for starting
and demo purposes; more complex applications can be built in the future
by the interested parts:
a) Iraf scripts: a set of Iraf scripts for polling the data has been created.
Documentation on this Iraf scripts can be found on the Scripting section, and on the scripts
files itselfs ($HOME/irafpkgs/arcview/imageList/*)
b) an ArcVIEW-based application was created too. An specific arcview module
was created for remotelly getting the data. This application is a module
that can be loaded on any ArcVIEW application. This module basically knows
how to talk with the DBS module through the socket. The idea then, the reduction
machine can starts a local ArcVIEW with just this module (IMPUL module)
loaded. This module will talk with the remote DBS module and can talk also
with a local DBS module, if running. So in a more complex situation, it
can bring the images from the acquisition machine, and in turn send the
names of the images gotten to the local DBS, which in turn will keep track
of all the images now on the local disk. A third machine can now run
another IMPUL module and get the images from the local machine, forming a
daisy-chain. Of course normal application will not requiere such complexity,
but it can be naturally done.
A little GUI was created for handling this module on the reduction machine.
Documentation on that GUI can be found here
Another thing that now can be done os to mix the rpovided Iraf scripts
with the provided ArcVIEW module provided. Since the Iraf scripts can talk
to a DBS, if the local ArcVIEW is running a DBS module (besides the IMPUL
one), the script can talk to the local DBS and process the images as they
are arriving, so we will have a task just getting images (ArcVIEW)
and another postprocessing it in parallel (Iraf scripts).
As was said before, all this are just demos that can be improved, modified
or completelly replaced as needed.
For a more general information, the list of available DBS commands for
handling the image list through the net are described here
8.2- Starting the Data Reduction Software for pulling the data
It is important to note that both machines, the acquisition and the reduction
machine has exactly the same software.
This means that both machines can be run as either reduction or acquisition
machine, meaning that the purpose of the reduction machine is dual: in normal
times, it will be used as data reduction machine, but in case the image acquisition
machine dies, for any reason, it can be swapped and act as data acquisition
machine.
There is nothing to do in order to configure the machine as reduction or
acquisition; it will detect inmediatelly if the acquisition PCI card is installed
or not, and will autoconfigure according to that.
So for starting the software for getting the images:
a) If you will use that iraf scripts, then just start a cl windows and type
"getimages" on it ("epar getimages" for configuration)
b) If you are going to use the GUI, then just start ArcVIEW in the same
way it is done on the acquisition software, and it will start ArcVIEW
configured as an "image puller" rather than as an "image acquisition"
system
8.3 - Technical Information
The WIRC data reduction machine is a dual Xeon Processor running at 3.06
GHz. It has 3 GB of RAM and about 130 GB of free HD.
It is a dual head monitor, configured as a unique monitor (Twin View).
It has a SONY DRU500A DVD writter drive, which speeds are:
DVD-R, DVD+R: 4x
DVD-RW, DVD+RW: 2x
Operating system: Linux RedHat 8.0
Software packages installed for recution purposes includes Iraf 2.12 and
IDL 5.6