docs/data_processing/introduction.rst

Mon, 10 Mar 2014 16:57:08 +0200

author
Iannis <ioannis@inoe.ro>
date
Mon, 10 Mar 2014 16:57:08 +0200
changeset 38
a7523d43ddc0
parent 37
4e3f530ac84e
child 41
c9ba4c828051
permissions
-rw-r--r--

New version and structure for the docs.

ioannis@37 1 Overview
ioannis@37 2 ========
ioannis@37 3
ioannis@37 4 The "Data processing" section of the SCC is used to upload measurements
ioannis@37 5 and ancillary files, monitor the processing progress, and view the output
ioannis@37 6 results of the SCC.
ioannis@37 7
ioannis@37 8 The data processing section consist of three subsections:
ioannis@37 9
ioannis@37 10 * The procedures of submitting new data for processing are described
ioannis@37 11 in the :ref:`uploading_measurements` section.
ioannis@37 12
ioannis@37 13 * The procedures of submitting ancillary files (i.e. sounding, overlap,
ioannis@37 14 and lidar ratio files) that are used in the processing are described
ioannis@37 15 in the :ref:`uploading_ancillary` section.
ioannis@37 16
ioannis@37 17 * The procedure to find and explore existing measurements and ancillary
ioannis@37 18 files are detailed in the :ref:`viewing_measurements` section.
ioannis@37 19
ioannis@37 20 Quick start
ioannis@37 21 -----------
ioannis@37 22
ioannis@37 23 .. note::
ioannis@37 24
ioannis@37 25 The following discussion assumed that you have already set-up the system
ioannis@37 26 settings in the "Admin" section (see :ref:`administration` for details).
ioannis@37 27
ioannis@37 28 In order to start processing lidar data, you will need to upload them
ioannis@37 29 on the SCC server. The files should be in netcdf format, following the
ioannis@37 30 specific format described in :ref:`netcdf_file`. There is no restriction imposed
ioannis@37 31 in the filename, but we suggest to use the format <measurement_ID>.nc. You can
ioannis@37 32 upload using the "Quick upload" link in the menu.
ioannis@37 33
ioannis@37 34 When you submit the processed measurements in the quick upload form, you will
ioannis@37 35 need to specify the system configuration that was used to perform the
ioannis@37 36 measurement. The *channels* that are defined in the file should be the
ioannis@37 37 ones defined in the selected system. The uploading of ancillary files or the
ioannis@37 38 selection of categories for the measurement are optional.
ioannis@37 39
ioannis@37 40 When the file is uploaded, it is checked if it conforms to the needed
ioannis@37 41 netcdf format. The check guarantees that the mandatory variables are present
ioannis@37 42 in the file and that they have the correct format. Note that the check is not
ioannis@37 43 exhaustive, in the sense it does not guarantee that the file is able to be
ioannis@37 44 correctly processed; the aim of this check is to detect as early as possible
ioannis@37 45 common problems in the file. If some errors are found, the file will be
ioannis@37 46 deleted from the server.
ioannis@37 47
ioannis@37 48 If ancillary files are needed for the processing two things should be done:
ioannis@37 49
ioannis@37 50 #. The ancillary file names should be defined in the submitted netcdf file. In
ioannis@37 51 contrast to the measurement files, the file name of the ancillary files
ioannis@37 52 should follow a specified format, described in :ref:`netcdf_file`. In brief,
ioannis@37 53 a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file a name
ioannis@37 54 ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, where
ioannis@37 55 YYYY is a year, MM is the month, DD is the day, cc is the earlinet station
ioannis@37 56 call sign and NN is a number.
ioannis@37 57
ioannis@37 58 #. The ancillary files should be uploaded on the server. The uploading can be
ioannis@37 59 done in the same form as before, or independently in the "Upload ancillary"
ioannis@37 60 form. If an ancillary file required by a measurement is not present in the
ioannis@37 61 database, this file will appear as "Missing" and the processing of the
ioannis@37 62 measurement will not start and will be marked as "pending". When the missing
ioannis@37 63 file is uploaded, its status will change to "OK" and the measurement will be
ioannis@37 64 processed.
ioannis@37 65
ioannis@37 66
ioannis@37 67

mercurial