Sat, 09 Jan 2021 18:46:13 +0200
Updated readme file.
victor@7 | 1 | Overview |
i@15 | 2 | ======== |
ioannis@0 | 3 | |
i@15 | 4 | This package provides a tool for interacting with the |
victor@7 | 5 | Single Calculus Chain through the command line. Specifically, with the script you can: |
ioannis@0 | 6 | |
ioannis@0 | 7 | * Upload a file to the SCC for processing |
ioannis@0 | 8 | * Download the processed files and graphs |
ioannis@0 | 9 | * Delete an existing measurement from the SCC (with appropriate privileges) |
ioannis@0 | 10 | |
ioannis@0 | 11 | The main functions are implemented in a class (SCC) that you can also import |
ioannis@0 | 12 | and use in your custom scripts. |
ioannis@0 | 13 | |
ioannis@0 | 14 | The script does not provide any feedback if a file upload fails. Before using |
ioannis@0 | 15 | the script, you will need to upload some files manually and be confident that |
ioannis@0 | 16 | your SCC file format and processing settings are correct. |
ioannis@0 | 17 | |
i@12 | 18 | Please note that this is not part of the "official" SCC tools. |
ioannis@0 | 19 | |
ioannis@0 | 20 | Any suggestions for improvements and new features are more than welcome. |
ioannis@0 | 21 | |
ioannis@0 | 22 | Installation |
ioannis@0 | 23 | ------------ |
ioannis@0 | 24 | |
victor@9 | 25 | The easiest way to install this module is from the python package index using pip:: |
victor@8 | 26 | |
ioannis@49 | 27 | pip install hg+https://repositories.imaa.cnr.it/public/scc_access#egg=scc-access |
ioannis@0 | 28 | |
i@36 | 29 | or, if you don't have mercurial on your system:: |
i@36 | 30 | |
ioannis@49 | 31 | pip install https://repositories.imaa.cnr.it/public/scc_access/archive/tip.zip |
i@36 | 32 | |
i@15 | 33 | You can also use the script by cloning this mercurial repository. |
ioannis@1 | 34 | |
ioannis@0 | 35 | Settings |
ioannis@0 | 36 | -------- |
ioannis@49 | 37 | The required user-defined settings need to be specified in a .yaml file. |
ioannis@49 | 38 | |
ioannis@49 | 39 | The following parameters should be specified:: |
ioannis@0 | 40 | |
ioannis@49 | 41 | basic_credentials: ['username', 'password'] # The HTTP user name and password that is needed to access the SCC site. |
ioannis@49 | 42 | website_credentials: ['username', 'password'] # The user name and password that is needed to log in to the SCC site. |
ioannis@49 | 43 | output_dir: /path/to/files/scc_output/ # The directory to download the files |
ioannis@49 | 44 | base_url: https://scc.imaa.cnr.it/ # SCC base URL. Normally you shouldn't need to change that. |
ioannis@0 | 45 | |
ioannis@49 | 46 | |
ioannis@49 | 47 | The repository includes a `settings_sample.yaml` file that you can use as a starting point. Rename the file, e.g. to |
ioannis@49 | 48 | `settings.yaml` and input the required parameters. If you don't want to specify the file path every time |
ioannis@49 | 49 | your run the `scc_access` script, you can name the file `.scc_access.yaml` and place it in your home directory. |
ioannis@0 | 50 | |
ioannis@49 | 51 | Please note that it's not a good idea to store your stations management credentials in the settings |
ioannis@49 | 52 | file. The standard user has "Station Management" privileges and, if the credentials |
ioannis@0 | 53 | are stolen, someone could change/delete the stations settings from the SCC database. |
ioannis@49 | 54 | For this, it is better to use a user account with minimum access settings, i.e. that |
ioannis@0 | 55 | can only upload files and measurements. |
ioannis@0 | 56 | |
ioannis@0 | 57 | |
ioannis@0 | 58 | Usage |
ioannis@0 | 59 | ----- |
ioannis@0 | 60 | |
ioannis@0 | 61 | You can upload a file specifying the username and the system id:: |
victor@8 | 62 | |
ioannis@49 | 63 | scc_access upload-file 20110101po01.nc 125 |
ioannis@0 | 64 | |
ioannis@0 | 65 | If you want to wait for the processing to finish and download the resulting files |
ioannis@49 | 66 | you can use add the `-p` or `--process` flag:: |
victor@8 | 67 | |
ioannis@49 | 68 | scc_access upload-file 20110101po01.nc 125 -p |
ioannis@49 | 69 | |
ioannis@49 | 70 | The two command above assume that you have placed your setting file in the default location. You can specify a |
ioannis@49 | 71 | custom location using the -c flag:: |
ioannis@49 | 72 | |
ioannis@49 | 73 | scc_access -c ./path/to/settings.yaml upload-file 20110101po01.nc 125 |
ioannis@49 | 74 | |
ioannis@49 | 75 | By default, the SCC will reject an uploaded file, if the specified measurement id already exists on the server. You |
ioannis@49 | 76 | can instruct the script to delete any existing measurement before uploading using the `--force_upload` flag:: |
ioannis@49 | 77 | |
ioannis@49 | 78 | scc_access upload-file 20110101po01.nc 125 -p --force_upload |
ioannis@0 | 79 | |
moritz@29 | 80 | If you want to delete an existing measurement id from the database use the `delete` |
moritz@29 | 81 | command and the measurement id:: |
victor@8 | 82 | |
moritz@29 | 83 | scc_access delete 20110101po01 |
moritz@29 | 84 | |
moritz@29 | 85 | You can list available measurements with the `list` command:: |
moritz@29 | 86 | |
moritz@29 | 87 | scc_access list |
ioannis@0 | 88 | |
ioannis@49 | 89 | .. note:: |
ioannis@49 | 90 | The `list` command needs to be updated. Cross-check the results before using them. |
ioannis@49 | 91 | |
ioannis@0 | 92 | For more information on the syntax type:: |
victor@8 | 93 | |
victor@9 | 94 | scc_access -h |
ioannis@49 | 95 | |
ioannis@49 | 96 | This will produce the following help text:: |
ioannis@49 | 97 | |
ioannis@49 | 98 | usage: scc_access [-h] [-d] [-s] [-c CONFIG] |
ioannis@49 | 99 | {delete,rerun-all,rerun-elpp,upload-file,list,download} ... |
ioannis@49 | 100 | |
ioannis@49 | 101 | positional arguments: |
ioannis@49 | 102 | {delete,rerun-all,rerun-elpp,upload-file,list,download} |
ioannis@49 | 103 | delete Deletes a measurement. |
ioannis@49 | 104 | rerun-all Rerun all processing steps for the provided |
ioannis@49 | 105 | measurement IDs. |
ioannis@49 | 106 | rerun-elpp Rerun low-resolution processing steps for the provided |
ioannis@49 | 107 | measurement ID. |
ioannis@49 | 108 | upload-file Submit a file and, optionally, download the output |
ioannis@49 | 109 | products. |
ioannis@49 | 110 | list List measurements registered on the SCC. |
ioannis@49 | 111 | download Download selected measurements. |
ioannis@49 | 112 | |
ioannis@49 | 113 | optional arguments: |
ioannis@49 | 114 | -h, --help show this help message and exit |
ioannis@49 | 115 | -d, --debug Print debugging information. |
ioannis@49 | 116 | -s, --silent Show only warning and error messages. |
ioannis@49 | 117 | -c CONFIG, --config CONFIG |
ioannis@49 | 118 | Path to the config file. |
ioannis@49 | 119 | |
ioannis@49 | 120 | You can find out more information about each command e.g. like this:: |
ioannis@49 | 121 | |
ioannis@49 | 122 | scc_access upload-file -h |
ioannis@49 | 123 | |
ioannis@49 | 124 | In this case, the help text will give more details about the `upload-file` command:: |
ioannis@49 | 125 | |
ioannis@49 | 126 | usage: scc_access upload-file [-h] [-p] [--force_upload] |
ioannis@49 | 127 | [--radiosounding RADIOSOUNDING] |
ioannis@49 | 128 | [--overlap OVERLAP] [--lidarratio LIDARRATIO] |
ioannis@49 | 129 | filename system |
ioannis@49 | 130 | |
ioannis@49 | 131 | positional arguments: |
ioannis@49 | 132 | filename Measurement file name or path. |
ioannis@49 | 133 | system Processing system id. |
ioannis@49 | 134 | |
ioannis@49 | 135 | optional arguments: |
ioannis@49 | 136 | -h, --help show this help message and exit |
ioannis@49 | 137 | -p, --process Wait for the processing results. |
ioannis@49 | 138 | --force_upload If measurement ID exists on SCC, delete before |
ioannis@49 | 139 | uploading. |
ioannis@49 | 140 | --radiosounding RADIOSOUNDING |
ioannis@49 | 141 | Radiosounding file name or path |
ioannis@49 | 142 | --overlap OVERLAP Overlap file name or path |
ioannis@49 | 143 | --lidarratio LIDARRATIO |
ioannis@49 | 144 | Lidar ratio file name or path |