ioannis@37: Overview
ioannis@37: ========
ioannis@37: 
ioannis@37: The "Data processing" section of the SCC is used to upload measurements 
ioannis@37: and ancillary files, monitor the processing progress, and view the output 
ioannis@37: results of the SCC.
ioannis@37: 
ioannis@37: The data processing section consist of three subsections:
ioannis@37: 
ioannis@37: * The procedures of submitting new data for processing are described 
ioannis@37:   in the :ref:`uploading_measurements` section.
ioannis@37: 
ioannis@37: * The procedures of submitting ancillary files (i.e. sounding, overlap, 
ioannis@37:   and lidar ratio files) that are used in the processing are described 
ioannis@37:   in the :ref:`uploading_ancillary` section.
ioannis@37:   
ioannis@37: * The procedure to find and explore existing measurements and ancillary 
ioannis@37:   files are detailed in the :ref:`viewing_measurements` section.
ioannis@37: 
ioannis@37: Quick start
ioannis@37: -----------
ioannis@37: 
ioannis@37: .. note::
ioannis@37: 
ioannis@37:    The following discussion assumed that you have already set-up the system 
ioannis@37:    settings in the "Admin" section (see :ref:`administration` for details).
ioannis@37: 
ioannis@37: In order to start processing lidar data, you will need to upload them 
ioannis@37: on the SCC server. The files should be in netcdf format, following the 
ioannis@37: specific format described in :ref:`netcdf_file`. There is no restriction imposed
ioannis@37: in the filename, but we suggest to use the format <measurement_ID>.nc. You can 
ioannis@37: upload using the "Quick upload" link in the menu.
ioannis@37: 
ioannis@37: When you submit the processed measurements in the quick upload form, you will 
ioannis@37: need to specify the system configuration that was used to perform the 
ioannis@37: measurement. The *channels* that are defined in the file should be the
ioannis@37: ones defined in the selected system. The uploading of ancillary files or the
ioannis@37: selection of categories for the measurement are optional.
ioannis@37: 
ioannis@37: When the file is uploaded, it is checked if it conforms to the needed
ioannis@37: netcdf format. The check guarantees that the mandatory variables are present
ioannis@37: in the file and that they have the correct format. Note that the check is not 
ioannis@37: exhaustive, in the sense it does not guarantee that the file is able to be 
ioannis@37: correctly processed; the aim of this check is to detect as early as possible
ioannis@37: common problems in the file. If some errors are found, the file will be 
ioannis@37: deleted from the server.
ioannis@37: 
ioannis@37: If ancillary files are needed for the processing two things should be done:
ioannis@37: 
ioannis@37: #. The ancillary file names should be defined in the submitted netcdf file. In 
ioannis@37:    contrast to the measurement files, the file name of the ancillary files
ioannis@37:    should follow a specified format, described in :ref:`netcdf_file`. In brief,
ioannis@37:    a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file a name
ioannis@37:    ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, where
ioannis@37:    YYYY is a year, MM is the month, DD is the day, cc is the earlinet station 
ioannis@37:    call sign and NN is a number.
ioannis@37: 
ioannis@37: #. The ancillary files should be uploaded on the server. The uploading can be
ioannis@37:    done in the same form as before, or independently in the "Upload ancillary"
ioannis@37:    form. If an ancillary file required by a measurement is not present in the
ioannis@37:    database, this file will appear as "Missing" and the processing of the 
ioannis@37:    measurement will not start and will be marked as "pending". When the missing 
ioannis@37:    file is uploaded, its status will change to "OK" and the measurement will be 
ioannis@37:    processed.
ioannis@37:   
ioannis@37: 
ioannis@37: