comparison: docs/data_processing/introduction.rst
docs/data_processing/introduction.rst
- changeset 41
- c9ba4c828051
- parent 37
- 4e3f530ac84e
- child 42
- 01cfcb387e7f
equal
deleted
inserted
replaced
| 40:fed9320bfbd2 | 41:c9ba4c828051 |
|---|---|
| 23 .. note:: | 23 .. note:: |
| 24 | 24 |
| 25 The following discussion assumed that you have already set-up the system | 25 The following discussion assumed that you have already set-up the system |
| 26 settings in the "Admin" section (see :ref:`administration` for details). | 26 settings in the "Admin" section (see :ref:`administration` for details). |
| 27 | 27 |
| 28 Upload a measurement | |
| 29 ~~~~~~~~~~~~~~~~~~~~ | |
| 30 | |
| 28 In order to start processing lidar data, you will need to upload them | 31 In order to start processing lidar data, you will need to upload them |
| 29 on the SCC server. The files should be in netcdf format, following the | 32 on the SCC server. The files should be in netcdf format, following the |
| 30 specific format described in :ref:`netcdf_file`. There is no restriction imposed | 33 specific format described in :ref:`netcdf_file`. There is no restriction imposed |
| 31 in the filename, but we suggest to use the format <measurement_ID>.nc. You can | 34 in the filename, but we suggest to use the format <measurement_ID>.nc. You can |
| 32 upload using the "Quick upload" link in the menu. | 35 upload using the "Quick upload" link in the menu. |
| 43 exhaustive, in the sense it does not guarantee that the file is able to be | 46 exhaustive, in the sense it does not guarantee that the file is able to be |
| 44 correctly processed; the aim of this check is to detect as early as possible | 47 correctly processed; the aim of this check is to detect as early as possible |
| 45 common problems in the file. If some errors are found, the file will be | 48 common problems in the file. If some errors are found, the file will be |
| 46 deleted from the server. | 49 deleted from the server. |
| 47 | 50 |
| 51 | |
| 52 Ancillary files | |
| 53 ~~~~~~~~~~~~~~~ | |
| 54 | |
| 48 If ancillary files are needed for the processing two things should be done: | 55 If ancillary files are needed for the processing two things should be done: |
| 49 | 56 |
| 50 #. The ancillary file names should be defined in the submitted netcdf file. In | 57 #. **The ancillary file names should be defined in the submitted measurement |
| 51 contrast to the measurement files, the file name of the ancillary files | 58 file**. In contrast to the measurement files, the file name of the ancillary |
| 52 should follow a specified format, described in :ref:`netcdf_file`. In brief, | 59 files should follow a specified format, described in :ref:`netcdf_file`. In |
| 53 a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file a name | 60 brief, a sounding file should have a name rs_YYYYMMDDccNN.nc, a overlap file |
| 54 ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, where | 61 a name ov_YYYYMMDDccNN.nc, and a lidar ratio file a name lr_YYYYMMDDccNN.nc, |
| 55 YYYY is a year, MM is the month, DD is the day, cc is the earlinet station | 62 where YYYY is a year, MM is the month, DD is the day, cc is the earlinet |
| 56 call sign and NN is a number. | 63 station call sign and NN is a number. |
| 57 | 64 |
| 58 #. The ancillary files should be uploaded on the server. The uploading can be | 65 #. **The ancillary files should be uploaded on the server**. The uploading can be |
| 59 done in the same form as before, or independently in the "Upload ancillary" | 66 done in the same form as before, or independently in the "Upload ancillary" |
| 60 form. If an ancillary file required by a measurement is not present in the | 67 form. If an ancillary file required by a measurement is not present in the |
| 61 database, this file will appear as "Missing" and the processing of the | 68 database, this file will appear as "Missing" and the processing of the |
| 62 measurement will not start and will be marked as "pending". When the missing | 69 measurement will not start and will be marked as "pending". When the missing |
| 63 file is uploaded, its status will change to "OK" and the measurement will be | 70 file is uploaded, its status will change to "OK" and the measurement will be |
| 64 processed. | 71 processed. |
| 65 | |
| 66 | 72 |
| 67 | 73 |
| 74 Monitor processing progress | |
| 75 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| 76 | |
| 77 When uploading finishes successfully, you will be transfered to the | |
| 78 **measurement status page**. There you can monitor the progress of the | |
| 79 processing through the status symbols. The three symbols correspond to the | |
| 80 phase of processing: | |
| 81 | |
| 82 Uploading | |
| 83 It indicates if everything needed for processing is provided; practically | |
| 84 this indicates if the measurement file and all needed ancillary files have | |
| 85 been uploaded. | |
| 86 | |
| 87 Preprocessing | |
| 88 This indicates the progress of the preprocessor module. | |
| 89 | |
| 90 Optical processing | |
| 91 This indicates if progress of the optical processing module (ELDA). | |
| 92 | |
| 93 | |
| 94 These three indicators currently support four states: | |
| 95 | |
| 96 Gray (Not started) | |
| 97 A gray indicator means that the process did not start yet. | |
| 98 | |
| 99 Orange (In progress) | |
| 100 An orange indicator means that the process is currently performed. | |
| 101 | |
| 102 Green (Success) | |
| 103 A green indicator means that the process finished successfully. | |
| 104 | |
| 105 Red (Fail) | |
| 106 A red indicator means that the process failed. | |
| 107 | |
| 108 | |
| 109 While the processing is in progress the page will automatically refresh every | |
| 110 10 seconds. | |
| 111 | |
| 112 | |
| 113 View output | |
| 114 ~~~~~~~~~~~ | |
| 115 | |
| 116 When the processing of your measurement finishes successfully, you can browse | |
| 117 and download the results. | |
| 118 | |
| 119 To **browse** the results, you must select the "Output" tab at the bottom part | |
| 120 of the page. There you will find links to view graphs of the resulting optical | |
| 121 products. | |
| 122 | |
| 123 To **download** the results, you can use the links on the "File actions" | |
| 124 submenu. Depending on the state of the processing you can downlaod | |
| 125 *preprocessed singals*, *optical products*, and *optical product plots*. | |
| 126 | |
| 127 | |
| 128 | |
| 129 | |
| 130 | |
| 131 | |
| 132 |
