docs/data_processing/introduction.rst@4e3f530ac84e (annotated)
docs/data_processing/introduction.rst
Mon, 10 Mar 2014 16:56:47 +0200
- author
- Iannis <ioannis@inoe.ro>
- date
- Mon, 10 Mar 2014 16:56:47 +0200
- changeset 37
- 4e3f530ac84e
- child 41
- c9ba4c828051
- permissions
- -rw-r--r--
Added files for the new doc structure.
| ioannis@37 | 1 | Overview |
| ioannis@37 | 2 | ======== |
| ioannis@37 | 3 | |
| ioannis@37 | 4 | The "Data processing" section of the SCC is used to upload measurements |
| ioannis@37 | 5 | and ancillary files, monitor the processing progress, and view the output |
| ioannis@37 | 6 | results of the SCC. |
| ioannis@37 | 7 | |
| ioannis@37 | 8 | The data processing section consist of three subsections: |
| ioannis@37 | 9 | |
| ioannis@37 | 10 | * The procedures of submitting new data for processing are described |
| ioannis@37 | 11 | in the :ref:`uploading_measurements` section. |
| ioannis@37 | 12 | |
| ioannis@37 | 13 | * The procedures of submitting ancillary files (i.e. sounding, overlap, |
| ioannis@37 | 14 | and lidar ratio files) that are used in the processing are described |
| ioannis@37 | 15 | in the :ref:`uploading_ancillary` section. |
| ioannis@37 | 16 | |
| ioannis@37 | 17 | * The procedure to find and explore existing measurements and ancillary |
| ioannis@37 | 18 | files are detailed in the :ref:`viewing_measurements` section. |
| ioannis@37 | 19 | |
| ioannis@37 | 20 | Quick start |
| ioannis@37 | 21 | ----------- |
| ioannis@37 | 22 | |
| ioannis@37 | 23 | .. note:: |
| ioannis@37 | 24 | |
| ioannis@37 | 25 | The following discussion assumed that you have already set-up the system |
| ioannis@37 | 26 | settings in the "Admin" section (see :ref:`administration` for details). |
| ioannis@37 | 27 | |
| ioannis@37 | 28 | In order to start processing lidar data, you will need to upload them |
| ioannis@37 | 29 | on the SCC server. The files should be in netcdf format, following the |
| ioannis@37 | 30 | specific format described in :ref:`netcdf_file`. There is no restriction imposed |
| ioannis@37 | 31 | in the filename, but we suggest to use the format <measurement_ID>.nc. You can |
| ioannis@37 | 32 | upload using the "Quick upload" link in the menu. |
| ioannis@37 | 33 | |
| ioannis@37 | 34 | When you submit the processed measurements in the quick upload form, you will |
| ioannis@37 | 35 | need to specify the system configuration that was used to perform the |
| ioannis@37 | 36 | measurement. The *channels* that are defined in the file should be the |
| ioannis@37 | 37 | ones defined in the selected system. The uploading of ancillary files or the |
| ioannis@37 | 38 | selection of categories for the measurement are optional. |
| ioannis@37 | 39 | |
| ioannis@37 | 40 | When the file is uploaded, it is checked if it conforms to the needed |
| ioannis@37 | 41 | netcdf format. The check guarantees that the mandatory variables are present |
| ioannis@37 | 42 | in the file and that they have the correct format. Note that the check is not |
| ioannis@37 | 43 | exhaustive, in the sense it does not guarantee that the file is able to be |
| ioannis@37 | 44 | correctly processed; the aim of this check is to detect as early as possible |
| ioannis@37 | 45 | common problems in the file. If some errors are found, the file will be |
| ioannis@37 | 46 | deleted from the server. |
| ioannis@37 | 47 | |
| ioannis@37 | 48 | If ancillary files are needed for the processing two things should be done: |
| ioannis@37 | 49 | |
| ioannis@37 | 50 | #. The ancillary file names should be defined in the submitted netcdf file. In |
| ioannis@37 | 51 | contrast to the measurement files, the file name of the ancillary files |
| ioannis@37 | 52 | should follow a specified format, described in :ref:`netcdf_file`. In brief, |
| ioannis@37 | 53 | a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file a name |
| ioannis@37 | 54 | ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, where |
| ioannis@37 | 55 | YYYY is a year, MM is the month, DD is the day, cc is the earlinet station |
| ioannis@37 | 56 | call sign and NN is a number. |
| ioannis@37 | 57 | |
| ioannis@37 | 58 | #. The ancillary files should be uploaded on the server. The uploading can be |
| ioannis@37 | 59 | done in the same form as before, or independently in the "Upload ancillary" |
| ioannis@37 | 60 | form. If an ancillary file required by a measurement is not present in the |
| ioannis@37 | 61 | database, this file will appear as "Missing" and the processing of the |
| ioannis@37 | 62 | measurement will not start and will be marked as "pending". When the missing |
| ioannis@37 | 63 | file is uploaded, its status will change to "OK" and the measurement will be |
| ioannis@37 | 64 | processed. |
| ioannis@37 | 65 | |
| ioannis@37 | 66 | |
| ioannis@37 | 67 |
