Wed, 18 Nov 2020 15:11:18 +0000
Added custom_info field support
ioannis@56 | 1 | Overview |
ioannis@56 | 2 | ======== |
binietoglou@38 | 3 | |
ioannis@56 | 4 | This package provides utilities to handle raw (atmospheric) lidar input data. |
ioannis@56 | 5 | The main format supported are Licel binary files (including the Raymetrics modified format). |
ioannis@56 | 6 | |
ioannis@56 | 7 | The package provides a single command line tool, called licel2scc that can convert Licel binary files to the |
ioannis@56 | 8 | EARLINET's Single Calculus Chain NetCDF format. |
binietoglou@38 | 9 | |
ulalume3@68 | 10 | Installation |
ulalume3@68 | 11 | ============ |
ulalume3@68 | 12 | |
ulalume3@68 | 13 | The easiest way to install this module is from the python package index using ``pip``:: |
ulalume3@68 | 14 | |
ulalume3@68 | 15 | pip install atmospheric-lidar |
binietoglou@38 | 16 | |
ioannis@56 | 17 | Using it as a Licel to SCC converter |
ioannis@56 | 18 | ==================================== |
binietoglou@38 | 19 | |
binietoglou@43 | 20 | Parameter file |
ioannis@56 | 21 | -------------- |
ioannis@56 | 22 | Before converting Licel binary to SCC format, you need to create a file linking Licel channels to SCC channels. |
ioannis@56 | 23 | |
ioannis@56 | 24 | As an example, you can start by changing the file “cf_netcdf_parameters.py” that describe such |
ioannis@56 | 25 | parameters for the Clermont Ferrand lidar. |
ioannis@56 | 26 | |
ioannis@56 | 27 | Command line interface |
ioannis@56 | 28 | ---------------------- |
ulalume3@68 | 29 | The usage of the ``licel2scc`` program is described below:: |
ioannis@56 | 30 | |
ulalume3@68 | 31 | A program to convert Licel binary files to the SCC NetCDF format. |
ioannis@56 | 32 | |
ioannis@56 | 33 | positional arguments: |
ioannis@56 | 34 | parameter_file The path to a parameter file linking licel and SCC |
ioannis@56 | 35 | channels. |
victor@75 | 36 | files Location of licel files. Use relative path and |
victor@75 | 37 | filename wildcards. (default './*.*') |
binietoglou@38 | 38 | |
ioannis@56 | 39 | optional arguments: |
ioannis@56 | 40 | -h, --help show this help message and exit |
ioannis@56 | 41 | -i, --id_as_name Use transient digitizer ids as channel names, instead |
ioannis@56 | 42 | of descriptive names |
ioannis@56 | 43 | -m MEASUREMENT_ID, --measurement_id MEASUREMENT_ID |
ioannis@56 | 44 | The new measurement id |
ioannis@56 | 45 | -n MEASUREMENT_NUMBER, --measurement_number MEASUREMENT_NUMBER |
ioannis@56 | 46 | The measurement number for the date from 00 to 99. |
ioannis@56 | 47 | Used if no id is provided |
ioannis@56 | 48 | -t TEMPERATURE, --temperature TEMPERATURE |
ioannis@56 | 49 | The temperature (in C) at lidar level, required if |
ioannis@56 | 50 | using US Standard atmosphere |
ioannis@56 | 51 | -p PRESSURE, --pressure PRESSURE |
ioannis@56 | 52 | The pressure (in hPa) at lidar level, required if |
ioannis@56 | 53 | using US Standard atmosphere |
victor@85 | 54 | -D DARK_FILES, --dark_files DARK_FILES |
victor@85 | 55 | Location of files containing dark measurements. |
victor@85 | 56 | Use relative path and filename wildcars, see 'files' |
victor@85 | 57 | parameter for example. |
ioannis@56 | 58 | -d, --debug Print dubuging information. |
ioannis@56 | 59 | -s, --silent Show only warning and error messages. |
ulalume3@68 | 60 | --version Show current version. |
binietoglou@38 | 61 | |
ulalume3@68 | 62 | Similarly, the ``licel2scc-depol`` program can be used to convert |
ulalume3@68 | 63 | Licel files from Delta45 depolarization calibration measurements:: |
ulalume3@68 | 64 | |
ulalume3@68 | 65 | A program to convert Licel binary files from depolarization calibration |
ulalume3@68 | 66 | measurements to the SCC NetCDF format. |
ulalume3@68 | 67 | |
ulalume3@68 | 68 | positional arguments: |
ulalume3@68 | 69 | parameter_file The path to a parameter file linking licel and SCC |
ulalume3@68 | 70 | channels. |
ulalume3@68 | 71 | plus45_string Search string for plus 45 degree files (default '*.*') |
ulalume3@68 | 72 | minus45_string Search string for minus 45 degree files (default |
ulalume3@68 | 73 | '*.*') |
ulalume3@68 | 74 | |
ulalume3@68 | 75 | optional arguments: |
ulalume3@68 | 76 | -h, --help show this help message and exit |
ulalume3@68 | 77 | -i, --id_as_name Use transient digitizer ids as channel names, instead |
ulalume3@68 | 78 | of descriptive names |
ulalume3@68 | 79 | -m MEASUREMENT_ID, --measurement_id MEASUREMENT_ID |
ulalume3@68 | 80 | The new measurement id |
ulalume3@68 | 81 | -n MEASUREMENT_NUMBER, --measurement_number MEASUREMENT_NUMBER |
ulalume3@68 | 82 | The measurement number for the date from 00 to 99. |
ulalume3@68 | 83 | Used if no id is provided |
ulalume3@68 | 84 | -t TEMPERATURE, --temperature TEMPERATURE |
ulalume3@68 | 85 | The temperature (in C) at lidar level, required if |
ulalume3@68 | 86 | using US Standard atmosphere |
ulalume3@68 | 87 | -p PRESSURE, --pressure PRESSURE |
ulalume3@68 | 88 | The pressure (in hPa) at lidar level, required if |
ulalume3@68 | 89 | using US Standard atmosphere |
ulalume3@68 | 90 | -d, --debug Print dubuging information. |
ulalume3@68 | 91 | -s, --silent Show only warning and error messages. |
ulalume3@68 | 92 | --version Show current version. |
ioannis@56 | 93 | |
ioannis@56 | 94 | Usage in python code |
ioannis@56 | 95 | -------------------- |
binietoglou@43 | 96 | System class |
binietoglou@43 | 97 | ~~~~~~~~~~~~ |
ioannis@56 | 98 | To read data from a system, you need create a class that describes you system. |
ioannis@56 | 99 | This is very simple if your lidar data are in the Licel format, as you only need to specify |
ioannis@57 | 100 | the external file with the extra SCC parameters. You can use as an example the file ``cf_netcdf_parameters.py``: |
binietoglou@38 | 101 | |
ioannis@56 | 102 | .. code-block:: python |
binietoglou@38 | 103 | |
ioannis@56 | 104 | from licel import LicelLidarMeasurement |
ioannis@56 | 105 | import cf_netcdf_parameters |
ioannis@56 | 106 | |
ioannis@56 | 107 | class CfLidarMeasurement(LicelLidarMeasurement): |
ioannis@56 | 108 | extra_netcdf_parameters = cf_netcdf_parameters |
ioannis@56 | 109 | |
ioannis@57 | 110 | This code assumes that the ``cf_netcdf_parameters.py`` is in your python path. |
binietoglou@38 | 111 | |
binietoglou@38 | 112 | Using the class |
ioannis@56 | 113 | ~~~~~~~~~~~~~~~ |
binietoglou@38 | 114 | |
ioannis@56 | 115 | Once you have made the above setup you can start using it. The best way to understand how |
ioannis@56 | 116 | it works is through an interactive shell (I suggest [ipython](http://ipython.org/)). |
ioannis@56 | 117 | In the following example I use the cf_raymetrics setup: |
binietoglou@38 | 118 | |
ioannis@56 | 119 | .. code-block:: python |
binietoglou@38 | 120 | |
ioannis@56 | 121 | import glob # This is needed to read a list of filenames |
ioannis@56 | 122 | import cf_lidar |
binietoglou@38 | 123 | |
ioannis@56 | 124 | # Go to the folder where you files are stored |
ioannis@56 | 125 | cd /path/to/lidar/files |
ulalume3@45 | 126 | |
ioannis@56 | 127 | # Read the filenames |
ioannis@56 | 128 | files = glob.glob("*") # The * reads all the files in the folder. |
binietoglou@38 | 129 | |
ioannis@56 | 130 | # Read the files |
ioannis@56 | 131 | my_measurement = cf_lidar.CfLidarMeasurement(files) |
binietoglou@38 | 132 | |
ioannis@56 | 133 | # Now the data have been read, and you have a measurement object to work with: |
ioannis@56 | 134 | # See what channels are present |
ioannis@56 | 135 | print(my_measurement.channels) |
binietoglou@38 | 136 | |
ioannis@56 | 137 | # Quicklooks of all the channels |
ioannis@56 | 138 | my_measurements.plot() |
binietoglou@38 | 139 | |
binietoglou@38 | 140 | Converting to SCC format |
ioannis@56 | 141 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
binietoglou@38 | 142 | |
binietoglou@38 | 143 | There are some extra info you need to put in before converting to SCC format, "Measurement_ID", "Temperature", "Pressure": |
binietoglou@38 | 144 | |
ioannis@56 | 145 | .. code-block:: python |
ulalume3@45 | 146 | |
ioannis@56 | 147 | my_measurement.info["Measurement_ID"] = "20101229op00" |
ioannis@56 | 148 | my_measurement.info["Temperature"] = "14" |
ioannis@56 | 149 | my_measurement.info["Pressure"] = "1010" |
binietoglou@38 | 150 | |
binietoglou@38 | 151 | You can use standard values of temperature and pressure by just calling: |
binietoglou@38 | 152 | |
ioannis@56 | 153 | .. code-block:: python |
ioannis@56 | 154 | |
ioannis@56 | 155 | my_measurement.get_PT() |
ioannis@56 | 156 | |
ioannis@56 | 157 | You can specify the standard values by overriding your system's ``get_PT`` method: |
ioannis@56 | 158 | |
ioannis@56 | 159 | .. code-block:: python |
binietoglou@38 | 160 | |
ioannis@56 | 161 | from licel import LicelLidarMeasurement |
ioannis@56 | 162 | import cf_netcdf_parameters |
ioannis@56 | 163 | |
ioannis@56 | 164 | class CfLidarMeasurement(LicelLidarMeasurement): |
ioannis@56 | 165 | extra_netcdf_parameters = cf_netcdf_parameters |
ioannis@56 | 166 | |
ioannis@56 | 167 | def get_PT(): |
ioannis@56 | 168 | self.info['Temperature'] = 25.0 |
ioannis@56 | 169 | self.info['Pressure'] = 1020.0 |
ioannis@56 | 170 | |
ioannis@56 | 171 | If you have an external source of temperature and pressure information (a meteorological station) you can automate |
ioannis@56 | 172 | this by reading the appropriate code in the ``get_PT`` method . |
binietoglou@38 | 173 | |
binietoglou@38 | 174 | |
binietoglou@38 | 175 | After you have used this extra input, you save the file using this command: |
binietoglou@38 | 176 | |
ioannis@56 | 177 | .. code-block:: python |
binietoglou@38 | 178 | |
ulalume3@92 | 179 | my_measurement.save_as_SCC_netcdf("filename") |
ioannis@56 | 180 | |
ioannis@56 | 181 | where you change the output filename to the filename you want to use. |