Mercurial > public > sccdocs / file revision

Overview
========

The "Data processing" section of the SCC is used to upload measurements 
and ancillary files, monitor the processing progress, and view the output 
results of the SCC.

The data processing section consist of three subsections:

* The procedures of submitting new data for processing are described 
  in the :ref:`uploading_measurements` section.

* The procedures of submitting ancillary files (i.e. sounding, overlap, 
  and lidar ratio files) that are used in the processing are described 
  in the :ref:`uploading_ancillary` section.
  
* The procedure to find and explore existing measurements and ancillary 
  files are detailed in the :ref:`viewing_measurements` section.

Quick start
-----------

.. note::

   The following discussion assumed that you have already set-up the system 
   settings in the "Admin" section (see :ref:`administration` for details).

In order to start processing lidar data, you will need to upload them 
on the SCC server. The files should be in netcdf format, following the 
specific format described in :ref:`netcdf_file`. There is no restriction imposed
in the filename, but we suggest to use the format <measurement_ID>.nc. You can 
upload using the "Quick upload" link in the menu.

When you submit the processed measurements in the quick upload form, you will 
need to specify the system configuration that was used to perform the 
measurement. The *channels* that are defined in the file should be the
ones defined in the selected system. The uploading of ancillary files or the
selection of categories for the measurement are optional.

When the file is uploaded, it is checked if it conforms to the needed
netcdf format. The check guarantees that the mandatory variables are present
in the file and that they have the correct format. Note that the check is not 
exhaustive, in the sense it does not guarantee that the file is able to be 
correctly processed; the aim of this check is to detect as early as possible
common problems in the file. If some errors are found, the file will be 
deleted from the server.

If ancillary files are needed for the processing two things should be done:

#. The ancillary file names should be defined in the submitted netcdf file. In 
   contrast to the measurement files, the file name of the ancillary files
   should follow a specified format, described in :ref:`netcdf_file`. In brief,
   a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file a name
   ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, where
   YYYY is a year, MM is the month, DD is the day, cc is the earlinet station 
   call sign and NN is a number.

#. The ancillary files should be uploaded on the server. The uploading can be
   done in the same form as before, or independently in the "Upload ancillary"
   form. If an ancillary file required by a measurement is not present in the
   database, this file will appear as "Missing" and the processing of the 
   measurement will not start and will be marked as "pending". When the missing 
   file is uploaded, its status will change to "OK" and the measurement will be 
   processed.