comparison: docs/data_processing/introduction.rst
docs/data_processing/introduction.rst
- changeset 37
- 4e3f530ac84e
- child 41
- c9ba4c828051
equal
deleted
inserted
replaced
| 36:98a251a75e80 | 37:4e3f530ac84e |
|---|---|
| 1 Overview | |
| 2 ======== | |
| 3 | |
| 4 The "Data processing" section of the SCC is used to upload measurements | |
| 5 and ancillary files, monitor the processing progress, and view the output | |
| 6 results of the SCC. | |
| 7 | |
| 8 The data processing section consist of three subsections: | |
| 9 | |
| 10 * The procedures of submitting new data for processing are described | |
| 11 in the :ref:`uploading_measurements` section. | |
| 12 | |
| 13 * The procedures of submitting ancillary files (i.e. sounding, overlap, | |
| 14 and lidar ratio files) that are used in the processing are described | |
| 15 in the :ref:`uploading_ancillary` section. | |
| 16 | |
| 17 * The procedure to find and explore existing measurements and ancillary | |
| 18 files are detailed in the :ref:`viewing_measurements` section. | |
| 19 | |
| 20 Quick start | |
| 21 ----------- | |
| 22 | |
| 23 .. note:: | |
| 24 | |
| 25 The following discussion assumed that you have already set-up the system | |
| 26 settings in the "Admin" section (see :ref:`administration` for details). | |
| 27 | |
| 28 In order to start processing lidar data, you will need to upload them | |
| 29 on the SCC server. The files should be in netcdf format, following the | |
| 30 specific format described in :ref:`netcdf_file`. There is no restriction imposed | |
| 31 in the filename, but we suggest to use the format <measurement_ID>.nc. You can | |
| 32 upload using the "Quick upload" link in the menu. | |
| 33 | |
| 34 When you submit the processed measurements in the quick upload form, you will | |
| 35 need to specify the system configuration that was used to perform the | |
| 36 measurement. The *channels* that are defined in the file should be the | |
| 37 ones defined in the selected system. The uploading of ancillary files or the | |
| 38 selection of categories for the measurement are optional. | |
| 39 | |
| 40 When the file is uploaded, it is checked if it conforms to the needed | |
| 41 netcdf format. The check guarantees that the mandatory variables are present | |
| 42 in the file and that they have the correct format. Note that the check is not | |
| 43 exhaustive, in the sense it does not guarantee that the file is able to be | |
| 44 correctly processed; the aim of this check is to detect as early as possible | |
| 45 common problems in the file. If some errors are found, the file will be | |
| 46 deleted from the server. | |
| 47 | |
| 48 If ancillary files are needed for the processing two things should be done: | |
| 49 | |
| 50 #. The ancillary file names should be defined in the submitted netcdf file. In | |
| 51 contrast to the measurement files, the file name of the ancillary files | |
| 52 should follow a specified format, described in :ref:`netcdf_file`. In brief, | |
| 53 a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file a name | |
| 54 ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, where | |
| 55 YYYY is a year, MM is the month, DD is the day, cc is the earlinet station | |
| 56 call sign and NN is a number. | |
| 57 | |
| 58 #. The ancillary files should be uploaded on the server. The uploading can be | |
| 59 done in the same form as before, or independently in the "Upload ancillary" | |
| 60 form. If an ancillary file required by a measurement is not present in the | |
| 61 database, this file will appear as "Missing" and the processing of the | |
| 62 measurement will not start and will be marked as "pending". When the missing | |
| 63 file is uploaded, its status will change to "OK" and the measurement will be | |
| 64 processed. | |
| 65 | |
| 66 | |
| 67 |
