docs/data_processing/uploading_ancillary.rst@bb1b88add508 (annotated)
docs/data_processing/uploading_ancillary.rst
Tue, 09 Mar 2021 21:49:27 +0200
- author
- ioannis@ioannis-VirtualBox
- date
- Tue, 09 Mar 2021 21:49:27 +0200
- changeset 121
- bb1b88add508
- parent 49
- 815fe2598c15
- permissions
- -rw-r--r--
New SCC input PDF file.
New link to SCC netcdf checker script.
| ioannis@37 | 1 | .. _uploading_ancillary: |
| ioannis@37 | 2 | |
| ioannis@47 | 3 | Ancillary files |
| ioannis@47 | 4 | =============== |
| ioannis@47 | 5 | |
| ioannis@47 | 6 | Together with your measurement data, you can submit ancillary files that will |
| ioannis@47 | 7 | be used in the processing procedure. The files that are currently supported |
| ioannis@47 | 8 | are **sounding files**, **overlap files**, and **lidar ratio files**. |
| ioannis@47 | 9 | |
| ioannis@47 | 10 | These files can be submitted either together with a measurement, using the |
| ioannis@47 | 11 | "Quick upload" form, or on their own, using the "Upload ancillary" form. Both |
| ioannis@47 | 12 | methods are equivalent and will not affect which files is used for processing |
| ioannis@47 | 13 | your measurements. The file that will be used are defined in the *measurement |
| ioannis@47 | 14 | file* using the appropriate attributes as described in the :ref:`netcdf_file` |
| ioannis@47 | 15 | section. |
| ioannis@47 | 16 | |
| ioannis@47 | 17 | If one of your measurements requires ancillary files for its processing, the |
| ioannis@47 | 18 | actual processing of the data will start only after all the required files |
| ioannis@47 | 19 | have been uploaded. The files that are missing will appear in the ancillary file |
| ioannis@47 | 20 | list with "Missing" status. When all these files have been uploaded the |
| ioannis@47 | 21 | processing of the measurement will start automatically. |
| ioannis@47 | 22 | |
| ioannis@47 | 23 | |
| ioannis@47 | 24 | Upload ancillary |
| ioannis@47 | 25 | ---------------- |
| ioannis@47 | 26 | In the "Upload ancillary" form you need to specify to submit at least one |
| ioannis@47 | 27 | ancillary file. |
| ioannis@47 | 28 | |
| ioannis@47 | 29 | Form fields |
| ioannis@47 | 30 | ~~~~~~~~~~~ |
| ioannis@47 | 31 | |
| ioannis@47 | 32 | Station (Required) |
| ioannis@47 | 33 | You need to specify the EARLINET station that is related to this file. In |
| ioannis@47 | 34 | most cases you will just have to select your station. |
| ioannis@47 | 35 | |
| ioannis@47 | 36 | Sounding file (Optional) |
| ioannis@47 | 37 | In this optional field you can submit a sounding file that will be used |
| ioannis@47 | 38 | by your measurements in the processing. The file needs to follow the netCDF |
| ioannis@47 | 39 | file format specified in the pdf file found in the :ref:`netcdf_file` section. |
| ioannis@47 | 40 | The file you submit will not be necessarily connected with the |
| ioannis@47 | 41 | submitted measurement; which sounding file will be used in the processing of |
| ioannis@47 | 42 | each measurement is specified in the measurement's file. The file you upload |
| ioannis@47 | 43 | will, nevertheless, remain on the SCC and will be used whenever needed. |
| ioannis@47 | 44 | |
| ioannis@47 | 45 | Overlap file (Optional) |
| ioannis@47 | 46 | In this optional field you can submit a overlap file that will be used |
| ioannis@47 | 47 | by your measurements in the processing. The file needs to follow the netCDF |
| ioannis@47 | 48 | file format specified in the pdf file found in the :ref:`netcdf_file` section. |
| ioannis@47 | 49 | The file you submit will not be necessarily connected with the |
| ioannis@47 | 50 | submitted measurement; which overlap file will be used in the processing of |
| ioannis@47 | 51 | each measurement is specified in the measurement's file. The file you upload |
| ioannis@47 | 52 | will, nevertheless, remain on the SCC and will be used whenever needed. |
| ioannis@47 | 53 | |
| ioannis@47 | 54 | Lidar ratio file (Optional) |
| ioannis@47 | 55 | In this optional field you can submit a lidar ratio file that will be used |
| ioannis@47 | 56 | by your measurements in the processing. The file needs to follow the netCDF |
| ioannis@47 | 57 | file format specified in the pdf file found in the :ref:`netcdf_file` section. |
| ioannis@47 | 58 | The file you submit will not be necessarily connected with the |
| ioannis@47 | 59 | submitted measurement; which lidar ratio file will be used in the processing |
| ioannis@47 | 60 | of each measurement is specified in the measurement's file. The file you |
| ioannis@47 | 61 | upload will, nevertheless, remain on the SCC and will be used whenever needed. |
| ioannis@47 | 62 | |
| ioannis@47 | 63 | Form Validation |
| ioannis@47 | 64 | ~~~~~~~~~~~~~~~ |
| ioannis@47 | 65 | |
| ioannis@47 | 66 | The aim of form validation is to detect errors in your submitted data as early |
| ioannis@47 | 67 | as possible. In this way you can save time by correcting errors before the |
| ioannis@47 | 68 | processing of your data fails. |
| ioannis@47 | 69 | |
| ioannis@47 | 70 | When the files are submitted the following checks are performed: |
| ioannis@47 | 71 | |
| ioannis@47 | 72 | * At least one file needs to be submitted. |
| ioannis@47 | 73 | |
| ioannis@47 | 74 | * The submitted measurement file should have the correct netCDF format. All |
| ioannis@47 | 75 | mandatory variables and attributes should be present in the file. All |
| ioannis@47 | 76 | variables and attributes should have the correct type (float, integer, etc). |
| ioannis@47 | 77 | |
| ioannis@47 | 78 | * A file with the same filename should not exist in the SCC or, if it exists, |
| ioannis@47 | 79 | it should have status "Missing" or "Error". If a file with the same filename |
| ioannis@47 | 80 | exists, and its status is "OK" the submitted file is rejected. |
| ioannis@47 | 81 | |
| ioannis@47 | 82 | |
| ioannis@47 | 83 | Browse ancillary files |
| ioannis@47 | 84 | ---------------------- |
| ioannis@47 | 85 | |
| ioannis@47 | 86 | You can browse existing and missing ancillary files in the "Ancillary files" |
| ioannis@47 | 87 | page of the Data processing sub menu. |
| ioannis@47 | 88 | |
| ioannis@47 | 89 | You can filter the results of the list using the **Filter** box above the form. |
| ioannis@47 | 90 | The filter searches for matches in all columns of the table, i.e you can use it |
| ioannis@47 | 91 | to filter the results by filename, stations, data, and status. |
| ioannis@47 | 92 | |
| ioannis@47 | 93 | You can view more information about the ancillary file following the link on the |
| ioannis@47 | 94 | name of the file. |
| ioannis@47 | 95 | |
| ioannis@47 | 96 | |
| ioannis@49 | 97 | .. _ancillary_details: |
| ioannis@49 | 98 | |
| ioannis@48 | 99 | View ancillary file details |
| ioannis@48 | 100 | --------------------------- |
| ioannis@47 | 101 | |
| ioannis@47 | 102 | In the ancillary file status page you can view more detailed information about |
| ioannis@47 | 103 | the selected file. The following info are provided. |
| ioannis@47 | 104 | |
| ioannis@47 | 105 | Status |
| ioannis@47 | 106 | The status of the file can have three values. **Ok** means that the file |
| ioannis@47 | 107 | has been uploaded successfully and is ready to be used in the processing. |
| ioannis@47 | 108 | **Missing** status means that one or more of the submitted measurements |
| ioannis@47 | 109 | have requested this file, but the file has not been submitted yet. An |
| ioannis@48 | 110 | **Error** status indicates that the file has been submitted but there was |
| ioannis@47 | 111 | an error while reading its content; you will need to correct its content |
| ioannis@47 | 112 | and resubmit the file. |
| ioannis@47 | 113 | |
| ioannis@47 | 114 | Interpolation |
| ioannis@47 | 115 | This indicates the interpolation type that will be used on the content of |
| ioannis@47 | 116 | the ancillary file. You can change the interpolation type in the admin |
| ioannis@47 | 117 | section following the "Edit in admin" link in the File actions sub menu. |
| ioannis@49 | 118 | The algorithms currently implemented are "Linear interpolation" and |
| ioannis@49 | 119 | "Natural Cubic Spline". If no value is selected the natural cubic spline |
| ioannis@49 | 120 | algorithm will be used. |
| ioannis@47 | 121 | |
| ioannis@47 | 122 | Station |
| ioannis@47 | 123 | The EARLINET station that is the related to this ancillary file |
| ioannis@47 | 124 | |
| ioannis@47 | 125 | Submitted on |
| ioannis@47 | 126 | The date and time that the file was submitted. If the file is missing, but is |
| ioannis@47 | 127 | required by an uploaded measurement, this date indicated the date and time |
| ioannis@47 | 128 | of the measurement upload. |
| ioannis@47 | 129 | |
| ioannis@47 | 130 | At the bottom part of the page you can see a list of **Related measurements**. |
| ioannis@47 | 131 | These are measurements that use the specific ancillary file for their |
| ioannis@47 | 132 | processing. |
| ioannis@47 | 133 |
