.. _uploading_ancillary:

Ancillary files
===============

Together with your measurement data, you can submit ancillary files that will
be used in the processing procedure. The files that are currently supported 
are **sounding files**, **overlap files**, and **lidar ratio files**. 

These files can be submitted either together with a measurement, using the
"Quick upload" form, or on their own, using the "Upload ancillary" form. Both
methods are equivalent and will not affect which files is used for processing 
your measurements. The file that will be used are defined in the *measurement
file* using the appropriate attributes as described in the :ref:`netcdf_file` 
section.

If one of your measurements requires ancillary files for its processing, the
actual processing of the data will start only after all the required files 
have been uploaded. The files that are missing will appear in the ancillary file
list with "Missing" status. When all these files have been uploaded the 
processing of the measurement will start automatically.


Upload ancillary
----------------
In the "Upload ancillary" form you need to specify to submit at least one 
ancillary file.

Form fields
~~~~~~~~~~~

Station (Required)
    You need to specify the EARLINET station that is related to this file. In 
    most cases you will just have to select your station.
    
Sounding file (Optional)
   In this optional field you can submit a sounding file that will be used
   by your measurements in the processing. The file needs to follow the netCDF
   file format specified in the pdf file found in the :ref:`netcdf_file` section. 
   The file you submit will not be  necessarily connected with the 
   submitted measurement; which sounding file will be used in the processing of 
   each measurement is specified in the measurement's file. The file you upload 
   will, nevertheless, remain on the SCC and will be used whenever needed.
   
Overlap file (Optional)
   In this optional field you can submit a overlap file that will be used
   by your measurements in the processing. The file needs to follow the netCDF
   file format specified in the pdf file found in the :ref:`netcdf_file` section. 
   The file you submit will not be  necessarily connected with the 
   submitted measurement; which overlap file will be used in the processing of 
   each measurement is specified in the measurement's file. The file you upload 
   will, nevertheless, remain on the SCC and will be used whenever needed.
   
Lidar ratio file (Optional)
   In this optional field you can submit a lidar ratio file that will be used
   by your measurements in the processing. The file needs to follow the netCDF
   file format specified in the pdf file found in the :ref:`netcdf_file` section. 
   The file you submit will not be  necessarily connected with the 
   submitted measurement; which lidar ratio file will be used in the processing 
   of each measurement is specified in the measurement's file. The file you 
   upload will, nevertheless, remain on the SCC and will be used whenever needed.    
    
Form Validation
~~~~~~~~~~~~~~~

The aim of form validation is to detect errors in your submitted data as early
as possible. In this way you can save time by correcting errors before the 
processing of your data fails. 

When the files are submitted the following checks are performed:

* At least one file needs to be submitted.

* The submitted measurement file should have the correct netCDF format. All 
  mandatory variables and attributes should be present in the file. All 
  variables and attributes should have the correct type (float, integer, etc).

* A file with the same filename should not exist in the SCC or, if it exists, 
  it should have status "Missing" or "Error". If a file with the same filename
  exists, and its status is "OK" the submitted file is rejected.


Browse ancillary files
----------------------

You can browse existing and missing ancillary files in the "Ancillary files"
page of the Data processing sub menu.

You can filter the results of the list using the **Filter** box above the form.
The filter searches for matches in all columns of the table, i.e you can use it 
to filter the results by filename, stations, data, and status. 

You can view more information about the ancillary file following the link on the
name of the file.


.. _ancillary_details:
 
View ancillary file details
---------------------------

In the ancillary file status page you can view more detailed information about 
the selected file. The following info are provided.

Status
   The status of the file can have three values. **Ok** means that the file
   has been uploaded successfully and is ready to be used in the processing.
   **Missing** status means that one or more of the submitted measurements
   have requested this file, but the file has not been submitted yet. An
   **Error** status indicates that the file has been submitted but there was 
   an error while reading its content; you will need to correct its content
   and resubmit the file.
  
Interpolation
   This indicates the interpolation type that will be used on the content of 
   the ancillary file. You can change the interpolation type in the admin 
   section following the "Edit in admin" link in the File actions sub menu.
   The algorithms currently implemented are "Linear interpolation" and 
   "Natural Cubic Spline". If no value is selected the natural cubic spline 
   algorithm will be used.
    
Station
   The EARLINET station that is the related to this ancillary file
    
Submitted on
   The date and time that the file was submitted. If the file is missing, but is
   required by an uploaded measurement, this date indicated the date and time 
   of the measurement upload.

At the bottom part of the page you can see  a list of **Related measurements**.
These are measurements that use the specific ancillary file for their 
processing.  
 
