docs/data_processing/uploading_ancillary.rst

Tue, 05 Apr 2022 12:16:26 +0200

author
Giuseppe D'Amico <giuseppe.damico@imaa.cnr.it>
date
Tue, 05 Apr 2022 12:16:26 +0200
changeset 143
5358586fe387
parent 49
815fe2598c15
permissions
-rw-r--r--

Added missing variables in HiRELPP product

ioannis@37 1 .. _uploading_ancillary:
ioannis@37 2
ioannis@47 3 Ancillary files
ioannis@47 4 ===============
ioannis@47 5
ioannis@47 6 Together with your measurement data, you can submit ancillary files that will
ioannis@47 7 be used in the processing procedure. The files that are currently supported
ioannis@47 8 are **sounding files**, **overlap files**, and **lidar ratio files**.
ioannis@47 9
ioannis@47 10 These files can be submitted either together with a measurement, using the
ioannis@47 11 "Quick upload" form, or on their own, using the "Upload ancillary" form. Both
ioannis@47 12 methods are equivalent and will not affect which files is used for processing
ioannis@47 13 your measurements. The file that will be used are defined in the *measurement
ioannis@47 14 file* using the appropriate attributes as described in the :ref:`netcdf_file`
ioannis@47 15 section.
ioannis@47 16
ioannis@47 17 If one of your measurements requires ancillary files for its processing, the
ioannis@47 18 actual processing of the data will start only after all the required files
ioannis@47 19 have been uploaded. The files that are missing will appear in the ancillary file
ioannis@47 20 list with "Missing" status. When all these files have been uploaded the
ioannis@47 21 processing of the measurement will start automatically.
ioannis@47 22
ioannis@47 23
ioannis@47 24 Upload ancillary
ioannis@47 25 ----------------
ioannis@47 26 In the "Upload ancillary" form you need to specify to submit at least one
ioannis@47 27 ancillary file.
ioannis@47 28
ioannis@47 29 Form fields
ioannis@47 30 ~~~~~~~~~~~
ioannis@47 31
ioannis@47 32 Station (Required)
ioannis@47 33 You need to specify the EARLINET station that is related to this file. In
ioannis@47 34 most cases you will just have to select your station.
ioannis@47 35
ioannis@47 36 Sounding file (Optional)
ioannis@47 37 In this optional field you can submit a sounding file that will be used
ioannis@47 38 by your measurements in the processing. The file needs to follow the netCDF
ioannis@47 39 file format specified in the pdf file found in the :ref:`netcdf_file` section.
ioannis@47 40 The file you submit will not be necessarily connected with the
ioannis@47 41 submitted measurement; which sounding file will be used in the processing of
ioannis@47 42 each measurement is specified in the measurement's file. The file you upload
ioannis@47 43 will, nevertheless, remain on the SCC and will be used whenever needed.
ioannis@47 44
ioannis@47 45 Overlap file (Optional)
ioannis@47 46 In this optional field you can submit a overlap file that will be used
ioannis@47 47 by your measurements in the processing. The file needs to follow the netCDF
ioannis@47 48 file format specified in the pdf file found in the :ref:`netcdf_file` section.
ioannis@47 49 The file you submit will not be necessarily connected with the
ioannis@47 50 submitted measurement; which overlap file will be used in the processing of
ioannis@47 51 each measurement is specified in the measurement's file. The file you upload
ioannis@47 52 will, nevertheless, remain on the SCC and will be used whenever needed.
ioannis@47 53
ioannis@47 54 Lidar ratio file (Optional)
ioannis@47 55 In this optional field you can submit a lidar ratio file that will be used
ioannis@47 56 by your measurements in the processing. The file needs to follow the netCDF
ioannis@47 57 file format specified in the pdf file found in the :ref:`netcdf_file` section.
ioannis@47 58 The file you submit will not be necessarily connected with the
ioannis@47 59 submitted measurement; which lidar ratio file will be used in the processing
ioannis@47 60 of each measurement is specified in the measurement's file. The file you
ioannis@47 61 upload will, nevertheless, remain on the SCC and will be used whenever needed.
ioannis@47 62
ioannis@47 63 Form Validation
ioannis@47 64 ~~~~~~~~~~~~~~~
ioannis@47 65
ioannis@47 66 The aim of form validation is to detect errors in your submitted data as early
ioannis@47 67 as possible. In this way you can save time by correcting errors before the
ioannis@47 68 processing of your data fails.
ioannis@47 69
ioannis@47 70 When the files are submitted the following checks are performed:
ioannis@47 71
ioannis@47 72 * At least one file needs to be submitted.
ioannis@47 73
ioannis@47 74 * The submitted measurement file should have the correct netCDF format. All
ioannis@47 75 mandatory variables and attributes should be present in the file. All
ioannis@47 76 variables and attributes should have the correct type (float, integer, etc).
ioannis@47 77
ioannis@47 78 * A file with the same filename should not exist in the SCC or, if it exists,
ioannis@47 79 it should have status "Missing" or "Error". If a file with the same filename
ioannis@47 80 exists, and its status is "OK" the submitted file is rejected.
ioannis@47 81
ioannis@47 82
ioannis@47 83 Browse ancillary files
ioannis@47 84 ----------------------
ioannis@47 85
ioannis@47 86 You can browse existing and missing ancillary files in the "Ancillary files"
ioannis@47 87 page of the Data processing sub menu.
ioannis@47 88
ioannis@47 89 You can filter the results of the list using the **Filter** box above the form.
ioannis@47 90 The filter searches for matches in all columns of the table, i.e you can use it
ioannis@47 91 to filter the results by filename, stations, data, and status.
ioannis@47 92
ioannis@47 93 You can view more information about the ancillary file following the link on the
ioannis@47 94 name of the file.
ioannis@47 95
ioannis@47 96
ioannis@49 97 .. _ancillary_details:
ioannis@49 98
ioannis@48 99 View ancillary file details
ioannis@48 100 ---------------------------
ioannis@47 101
ioannis@47 102 In the ancillary file status page you can view more detailed information about
ioannis@47 103 the selected file. The following info are provided.
ioannis@47 104
ioannis@47 105 Status
ioannis@47 106 The status of the file can have three values. **Ok** means that the file
ioannis@47 107 has been uploaded successfully and is ready to be used in the processing.
ioannis@47 108 **Missing** status means that one or more of the submitted measurements
ioannis@47 109 have requested this file, but the file has not been submitted yet. An
ioannis@48 110 **Error** status indicates that the file has been submitted but there was
ioannis@47 111 an error while reading its content; you will need to correct its content
ioannis@47 112 and resubmit the file.
ioannis@47 113
ioannis@47 114 Interpolation
ioannis@47 115 This indicates the interpolation type that will be used on the content of
ioannis@47 116 the ancillary file. You can change the interpolation type in the admin
ioannis@47 117 section following the "Edit in admin" link in the File actions sub menu.
ioannis@49 118 The algorithms currently implemented are "Linear interpolation" and
ioannis@49 119 "Natural Cubic Spline". If no value is selected the natural cubic spline
ioannis@49 120 algorithm will be used.
ioannis@47 121
ioannis@47 122 Station
ioannis@47 123 The EARLINET station that is the related to this ancillary file
ioannis@47 124
ioannis@47 125 Submitted on
ioannis@47 126 The date and time that the file was submitted. If the file is missing, but is
ioannis@47 127 required by an uploaded measurement, this date indicated the date and time
ioannis@47 128 of the measurement upload.
ioannis@47 129
ioannis@47 130 At the bottom part of the page you can see a list of **Related measurements**.
ioannis@47 131 These are measurements that use the specific ancillary file for their
ioannis@47 132 processing.
ioannis@47 133

mercurial