• file
• latest
• revisions
• annotate
• diff
• comparison
• raw

docs/data_processing/uploading_measurements.rst@a059b0d40a80 (annotated)

docs/data_processing/uploading_measurements.rst

Tue, 06 Jul 2021 18:57:09 +0200

author

Giuseppe D'Amico <giuseppe.damico@imaa.cnr.it>

date

Tue, 06 Jul 2021 18:57:09 +0200

changeset 129

a059b0d40a80

parent 50

e48b82a9b741

permissions

-rw-r--r--

Updated ELPP exit codes

ioannis@37	1	.. _uploading_measurements:
ioannis@37	2
ioannis@37	3	Uploading measurements
ioannis@37	4	======================
ioannis@42	5
ioannis@42	6	New measurements can be uploaded using the "Quick Upload" link in the data
ioannis@42	7	processing submenu. There, you will need to fill and submit the provided form.
ioannis@42	8
ioannis@42	9
ioannis@42	10	Form fields
ioannis@42	11	-----------
ioannis@42	12
ioannis@42	13	System (Required)
ioannis@42	14	This field specifies the system configuration that was used to perform the
ioannis@42	15	measurement. The available systems can be managed in the "Station Admin"
ioannis@42	16	section of the site. You can later change the system used in the processing
ioannis@42	17	of the measurement again in the "Station Admin" section.
ioannis@42	18
ioannis@42	19	Data file (Required)
ioannis@42	20	In this field you have to select the measurement file that will be processed.
ioannis@42	21	The file has to be in the specific format described in :ref:`netcdf_file`.
ioannis@42	22
ioannis@42	23	Sounding file (Optional)
ioannis@42	24	In this optional field you can submit a sounding file that will be used
ioannis@44	25	by your measurements in the processing. The file needs to follow the netCDF
ioannis@42	26	file format specified in the pdf file found in the :ref:`netcdf_file` section.
ioannis@42	27	The file you submit will not be necessarily connected with the
ioannis@42	28	submitted measurement; which sounding file will be used in the processing of
ioannis@42	29	each measurement is specified in the measurement's file. The file you upload
ioannis@42	30	will, nevertheless, remain on the SCC and will be used whenever needed.
ioannis@42	31
ioannis@42	32	Overlap file (Optional)
ioannis@42	33	In this optional field you can submit a overlap file that will be used
ioannis@44	34	by your measurements in the processing. The file needs to follow the netCDF
ioannis@42	35	file format specified in the pdf file found in the :ref:`netcdf_file` section.
ioannis@42	36	The file you submit will not be necessarily connected with the
ioannis@42	37	submitted measurement; which overlap file will be used in the processing of
ioannis@42	38	each measurement is specified in the measurement's file. The file you upload
ioannis@42	39	will, nevertheless, remain on the SCC and will be used whenever needed.
ioannis@42	40
ioannis@42	41	Lidar ratio file (Optional)
ioannis@42	42	In this optional field you can submit a lidar ratio file that will be used
ioannis@44	43	by your measurements in the processing. The file needs to follow the netCDF
ioannis@42	44	file format specified in the pdf file found in the :ref:`netcdf_file` section.
ioannis@42	45	The file you submit will not be necessarily connected with the
ioannis@42	46	submitted measurement; which lidar ratio file will be used in the processing
ioannis@42	47	of each measurement is specified in the measurement's file. The file you
ioannis@42	48	upload will, nevertheless, remain on the SCC and will be used whenever needed.
ioannis@42	49
ioannis@50	50	Comments (Optional)
ioannis@50	51	In this field you can add a small comment (up to 100 characters) to the
ioannis@50	52	measurement.
ioannis@50	53
ioannis@42	54	Categories (Optional)
ioannis@42	55	You can assign a number of categories to the uploaded measurement. The
ioannis@42	56	categories you assign can be lated changed in the in the "Station Admin"
ioannis@42	57	section of the site. You can select multiple categories by holding down the
ioannis@42	58	"Control" key (PC) or "Command" key (MAC) while selecting.
ioannis@42	59
ioannis@42	60	.. note::
ioannis@42	61
ioannis@42	62	Uploading the measurement file can take considerable time, depending on
ioannis@42	63	the size of the file and the speed of your Internet connection. In the
ioannis@42	64	current version of the site, no feedback is provided during the upload.
ioannis@43	65	You can get information on your upload progress depending on your browser.
ioannis@42	66
ioannis@42	67	Firefox
ioannis@43	68	You can install this `add-on <https://addons.mozilla.org/en-US/firefox/addon/uploadprogress/>`_.
ioannis@43	69
ioannis@42	70	Chrome
ioannis@45	71	A progress indicator is built-in.
ioannis@42	72
ioannis@42	73	IE 9-
ioannis@42	74	No progress bar is available (as far as we know).
ioannis@42	75
ioannis@44	76
ioannis@44	77	Form Validation
ioannis@44	78	---------------
ioannis@44	79
ioannis@46	80	The aim of form validation is to detect errors in your submitted data as early
ioannis@46	81	as possible. In this way you can save time by correcting errors before the
ioannis@46	82	processing of your data fails.
ioannis@46	83
ioannis@46	84	The data you provide in the form are check in two stages, before you submit
ioannis@46	85	and after you submit the form.
ioannis@44	86
ioannis@44	87	Before you submit the form
ioannis@44	88	~~~~~~~~~~~~~~~~~~~~~~~~~~
ioannis@44	89
ioannis@44	90	Before you submit the form the following checks are performed:
ioannis@44	91
ioannis@44	92	* All the necessary fields (i.e. "system" and "data file") need to be submitted.
ioannis@44	93
ioannis@44	94	* The ancillary file names have to be in the appropriate format described
ioannis@44	95	in :ref:`netcdf_file`. In brief, a sounding file should have a name
ioannis@44	96	rs_YYYYMMDDccNN.nc, a overlap file a name ov_YYYYMMDDccNN.nc, and a lidar ratio
ioannis@44	97	file a name lr_YYYYMMDDccNN.nc, where YYYY is a year, MM is the month, DD is
ioannis@44	98	the day, cc is the EARLINET station call sign and NN is a number.
ioannis@44	99
ioannis@44	100
ioannis@44	101	After you submit the form
ioannis@44	102	~~~~~~~~~~~~~~~~~~~~~~~~~~
ioannis@44	103
ioannis@44	104	After you submit the form:
ioannis@44	105
ioannis@44	106	* The uploaded **data file** needs to have a unique name.
ioannis@44	107
ioannis@44	108	* The **measurement id** defined in the file should *not* exist in the SCC.
ioannis@44	109
ioannis@44	110	* All channels declared in the measurement file should correspond to declared
ioannis@44	111	channels in the SCC.
ioannis@44	112
ioannis@44	113	* The submitted measurement file should have the correct netCDF format. All
ioannis@44	114	mandatory variables and attributes should be present in the file. All
ioannis@44	115	variables and attributes should have the correct type (float, integer, etc).
ioannis@44	116	This check is not exhaustive, and it is possible that some errors are **not**
ioannis@44	117	detected. For example "conditionally" mandatory variables are not checked
ioannis@44	118	(as, for example "LR_Input", that is mandatory if elastic backscatter retrievals
ioannis@44	119	have to be done).
ioannis@44	120
ioannis@44	121	If ancillary files are submitted
ioannis@44	122	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ioannis@44	123
ioannis@44	124	Additionally, if ancillary files are also uploaded, the following checks are
ioannis@44	125	performed:
ioannis@44	126
ioannis@44	127	* The submitted measurement file should have the correct netCDF format. All
ioannis@44	128	mandatory variables and attributes should be present in the file. All
ioannis@44	129	variables and attributes should have the correct type (float, integer, etc).
ioannis@44	130
ioannis@44	131	* A file with the same filename should not exist in the SCC or, if it exists,
ioannis@44	132	it should have status "Missing" or "Error". If a file with the same filename
ioannis@47	133	exists, and its status is "OK" the submitted file is rejected.