API reference¶
High-level functions¶
CloudnetPy’s high-level functions provide a simple mechanism to process cloud remote sensing measurements into Cloudnet products. A full processing goes in steps. Each step produces a file which used as an input for the next step.
Raw data conversion¶
Cloudnet instruments provide raw data in various formats that need to be first converted into netCDF with standardized metadata.
-
instruments.
mira2nc
(mmclx_file, output_file, site_meta, rebin_data=False)[source]¶ Converts METEK MIRA-35 cloud radar Level 1 file into netCDF file.
This function converts raw cloud radar file into a much smaller file that contains only the relevant data and can be used in further processing steps.
Parameters: - mmclx_file (str) – Raw radar file in netCDF format.
- output_file (str) – Output file name.
- site_meta (dict) – Dictionary containing information about the site. Required key value pair is name.
- rebin_data (bool, optional) – If True, rebins data to 30s resolution. Otherwise keeps the native resolution. Default is False.
Examples
>>> from cloudnetpy.instruments import mira2nc >>> site_meta = {'name': 'Vehmasmaki'} >>> mira2nc('raw_radar.mmclx', 'radar.nc', site_meta)
-
instruments.
rpg2nc
(path_to_l1_files, output_file, site_meta)[source]¶ Converts RPG cloud radar binary files into netCDF file.
This function reads one day of RPG Level 1 cloud radar binary files, concatenates the data and writes it into netCDF file.
Parameters: - path_to_l1_files (str) – Folder containing one day of RPG LV1 files.
- output_file (str) – Output file name.
- site_meta (dict) – Dictionary containing information about the site. Required key value pairs are altitude (metres above mean sea level) and name.
Raises: RuntimeError
– Failed to read the binary data.Examples
>>> from cloudnetpy.instruments import rpg2nc >>> site_meta = {'name': 'Hyytiala', 'altitude': 174} >>> rpg2nc('/path/to/files/', 'test.nc', site_meta)
-
instruments.
ceilo2nc
(input_file, output_file, site_meta)[source]¶ Converts Vaisala and Jenoptik raw files into netCDF file.
This function reads raw Vaisala (CT25k, CL31, CL51) and Jenoptik (CHM15k) ceilometer files and writes the data into netCDF file. Three variants of the attenuated backscatter are saved in the file:
- Raw backscatter, beta_raw
- Signal-to-noise screened backscatter, beta
- SNR-screened backscatter with smoothed weak background, beta_smooth
Parameters: - input_file (str) – Ceilometer file name. For Vaisala it is a text file, for Jenoptik it is a netCDF file.
- output_file (str) – Output file name, e.g. ‘ceilo.nc’.
- site_meta (dict) – Dictionary containing information about the site. Required key value pairs are name and altitude (metres above mean sea level).
Raises: RuntimeError
– Failed to read or process raw ceilometer data.Examples
>>> from cloudnetpy.instruments import ceilo2nc >>> site_meta = {'name': 'Mace-Head', 'altitude':5} >>> ceilo2nc('vaisala_raw.txt', 'vaisala.nc', site_meta) >>> ceilo2nc('jenoptik_raw.nc', 'jenoptik.nc', site_meta)
The categorize file¶
The categorize file concatenates all input data into common time / height grid.
-
categorize.
generate_categorize
(input_files, output_file)[source]¶ Generates Cloudnet categorize file.
The measurements are rebinned into a common height / time grid, and classified as different types of scatterers such as ice, liquid, insects, etc. Next, the radar signal is corrected for atmospheric attenuation, and error estimates are computed. Results are saved in ouput_file which is a compressed netCDF4 file.
Parameters: - input_files (dict) – dict containing file names for calibrated radar, lidar, model and mwr files.
- output_file (str) – Full path of the output file.
Raises: RuntimeError
– Failed to create the categorize file.Notes
Separate mwr-file is not needed when using RPG cloud radar which measures liquid water path. Then, the radar file can be used as a mwr-file as well, i.e. {‘mwr’: ‘radar.nc’}.
Examples
>>> from cloudnetpy.categorize import generate_categorize >>> input_files = {'radar': 'radar.nc', 'lidar': 'lidar.nc', 'model': 'model.nc', 'mwr': 'mwr.nc'} >>> generate_categorize(input_files, 'output.nc')
Product generation¶
Starting from the categorize file, several geophysical products can be generated.
-
products.
generate_classification
(categorize_file, output_file)[source]¶ Generates Cloudnet classification product.
This function reads the initial classification masks from a categorize file and creates a more comprehensive classification for different atmospheric targets. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_classification >>> generate_classification('categorize.nc', 'classification.nc')
-
products.
generate_iwc
(categorize_file, output_file)[source]¶ Generates Cloudnet ice water content product.
This function calculates ice water content using the so-called Z-T method. In this method, ice water content is calculated from attenuated-corrected radar reflectivity and model temperature. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_iwc >>> generate_iwc('categorize.nc', 'iwc.nc')
References
Hogan, R.J., M.P. Mittermaier, and A.J. Illingworth, 2006: The Retrieval of Ice Water Content from Radar Reflectivity Factor and Temperature and Its Use in Evaluating a Mesoscale Model. J. Appl. Meteor. Climatol., 45, 301–317, https://doi.org/10.1175/JAM2340.1
-
products.
generate_lwc
(categorize_file, output_file)[source]¶ Generates Cloudnet liquid water content product.
This function calculates cloud liquid water content using the so-called adiabatic-scaled method. In this method, liquid water content measured by microwave radiometer is used to constrain the theoretical liquid water content of observed liquid clouds. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_lwc >>> generate_lwc('categorize.nc', 'lwc.nc')
References
Illingworth, A.J., R.J. Hogan, E. O’Connor, D. Bouniol, M.E. Brooks, J. Delanoé, D.P. Donovan, J.D. Eastment, N. Gaussiat, J.W. Goddard, M. Haeffelin, H.K. Baltink, O.A. Krasnov, J. Pelon, J. Piriou, A. Protat, H.W. Russchenberg, A. Seifert, A.M. Tompkins, G. van Zadelhoff, F. Vinit, U. Willén, D.R. Wilson, and C.L. Wrench, 2007: Cloudnet. Bull. Amer. Meteor. Soc., 88, 883–898, https://doi.org/10.1175/BAMS-88-6-883
-
products.
generate_drizzle
(categorize_file, output_file)[source]¶ Generates Cloudnet drizzle product.
This function calculates different drizzle properties from cloud radar and lidar measurements. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_drizzle >>> generate_drizzle('categorize.nc', 'drizzle.nc')
References
O’Connor, E.J., R.J. Hogan, and A.J. Illingworth, 2005: Retrieving Stratocumulus Drizzle Parameters Using Doppler Radar and Lidar. J. Appl. Meteor., 44, 14–27, https://doi.org/10.1175/JAM-2181.1
Visualizing results¶
CloudnetPy offers an easy-to-use plotting interface:
-
plotting.
generate_figure
(nc_file, field_names, show=True, save_path=None, max_y=12, dpi=200, image_name=None)[source]¶ Generates a Cloudnet figure.
Parameters: - nc_file (str) – Input file.
- field_names (list) – Variable names to be plotted.
- show (bool, optional) – If True, shows the figure. Default is True.
- save_path (str, optional) – Setting this path will save the figure (in the given path). Default is None, when the figure is not saved.
- max_y (int, optional) – Upper limit in the plots (km). Default is 12.
- dpi (int, optional) – Figure quality (if saved). Higher value means more pixels, i.e., better image quality. Default is 200.
- image_name (str, optional) – Name (and full path) of the output image. Overrides the save_path option. Default is None.
Examples
>>> from cloudnetpy.plotting import generate_figure >>> generate_figure('categorize_file.nc', ['Z', 'v', 'width', 'ldr', 'beta', 'lwp']) >>> generate_figure('iwc_file.nc', ['iwc', 'iwc_error', 'iwc_retrieval_status']) >>> generate_figure('lwc_file.nc', ['lwc', 'lwc_error', 'lwc_retrieval_status'], max_y=4) >>> generate_figure('classification_file.nc', ['target_classification', 'detection_status']) >>> generate_figure('drizzle_file.nc', ['Do', 'mu', 'S'], max_y=3)
There is also possibility to compare CloundetPy files with the Matlab-processed old files which are available from devcloudnet.fmi.fi:
-
plotting.
compare_files
(nc_files, field_name, show=True, relative_err=False, save_path=None, max_y=12, dpi=200, image_name=None)[source]¶ Plots one particular field from old and new cloudnet files.
Parameters: - nc_files (tuple) – Tuple of strings of the two files to compare [0] = old Cloudnet file [1] = new CloudnetPy file
- field_name (str) – Name of variable to be plotted.
- show (bool, optional) – If True, shows the plot.
- relative_err (bool, optional) – If True, plots also relative error. Makes sense only for continuous variables. Default is False.
- save_path (str, optional) – If defined, saves the image to this path. Default is None.
- max_y (int, optional) – Upper limit of images (km). Default is 12.
- dpi (int, optional) – Quality of plots. Default is 200.
- image_name (str, optional) – Name (and full path) of the output image. Overrides the save_path option. Default is None.
Quality control¶
CloudnetPy Github source contains functions to check the quality of the processed files. These can be used to validate the metadata of the files and perform various checks for the data.
-
tests.
check_metadata
(file, log_file=None)[source]¶ Runs metadata checks for the given file.
Parameters: - file (str) – Name of the file to be tested.
- log_file (str, optional) – Name of the log file where warning messages are stored.
-
tests.
check_data_quality
(file, log_file=None)[source]¶ Runs data quality checks for the given file.
Parameters: - file (str) – Name of the file to be tested.
- log_file (str, optional) – Name of the log file where warning messages are stored.
Note
Quality control routines are not included in the CloudnetPy PyPI package.
Categorize modules¶
Categorize is CloudnetPy’s subpackage. It contains several modules that are used when creating the Cloudnet categorize file.
datasource¶
Datasource module, containing the DataSource
and ProfileDataSource
classes.
-
class
categorize.datasource.
DataSource
(filename, radar=False)[source]¶ Base class for all Cloudnet measurements and model data.
Parameters: - filename (str) – Calibrated instrument / model NetCDF file.
- radar (bool, optional) – Indicates if data is from cloud radar. Default is False.
-
filename
¶ Filename of the input file.
Type: str
-
dataset
¶ A netCDF4 Dataset instance.
Type: Dataset
-
source
¶ Global attribute source read from the input file.
Type: str
-
time
¶ Time array of the instrument.
Type: MaskedArray
-
altitude
¶ Altitude of instrument above mean sea level (m).
Type: float
-
data
¶ Dictionary containing
CloudnetArray
instances.Type: dict
-
getvar
(*args)[source]¶ Returns data array from the source file variables.
Returns just the data (and no attributes) from the original variables dictionary, fetched from the input NetCDF file.
Parameters: *args – possible names of the variable. The first match is returned. Returns: The actual data. Return type: MaskedArray Raises: RuntimeError
– The variable is not found.
-
append_data
(array, key, name=None, units=None)[source]¶ Adds new CloudnetVariable into data attribute.
Parameters: - array (ndarray) – Array to be added.
- key (str) – Key used with array when added to data attribute (which is a dictionary).
- name (str, optional) – CloudnetArray.name attribute. Default value is key.
- units (str, optional) – CloudnetArray.units attribute.
-
class
categorize.datasource.
ProfileDataSource
(filename, radar=False)[source]¶ The
ProfileDataSource
class, child ofDataSource
.Parameters: - filename (str) – Raw lidar or radar file.
- radar (bool, optional) – Indicates if data is from cloud radar. Default is False.
-
height
¶ Measurement height grid above mean sea level (m).
Type: ndarray
radar¶
Radar module, containing the Radar
class.
-
class
categorize.radar.
Radar
(radar_file)[source]¶ Radar class, child of ProfileDataSource.
Parameters: radar_file (str) – File name of the calibrated radar netCDF file. -
radar_frequency
¶ Radar frequency (GHz).
Type: float
-
wl_band
¶ Int corresponding to frequency 0 = 35.5 GHz, 1 = 94 GHz.
Type: int
-
folding_velocity
¶ Radar’s folding velocity (m/s).
Type: float
-
location
¶ Location of the radar, copied from the global attribute location of the radar_file.
Type: str
-
sequence_indices
¶ Indices denoting the different altitude regimes of the radar.
Type: list
-
type
¶ Type of the radar, e.g. ‘RPG-FMCW’, ‘METEK MIRA-36’
Type: str
-
rebin_to_grid
(time_new)[source]¶ Rebins radar data in time using mean.
Parameters: time_new (ndarray) – Target time array as fraction hour. Updates time attribute.
-
remove_incomplete_pixels
()[source]¶ Removes pixels where some of the (required) variables are existing.
-
correct_atten
(attenuations)[source]¶ Corrects radar echo for liquid and gas attenuation.
Parameters: attenuations (dict) – 2-D attenuations due to atmospheric gases and liquid: radar_gas_atten, radar_liquid_atten.
-
calc_errors
(attenuations, classification)[source]¶ Calculates uncertainties of radar echo.
Calculates and adds Z_error, Z_sensitivity and Z_bias
CloudnetArray
instances to data attribute.Parameters: - attenuations (dict) – 2-D attenuations due to atmospheric gases.
- classification (ClassificationResult) – The
ClassificationResult
instance.
-
lidar¶
Lidar module, containing the Lidar
class.
model¶
Model module, containing the Model
class.
-
class
categorize.model.
Model
(model_file, alt_site)[source]¶ Model class, child of DataSource.
Parameters: - model_file (str) – File name of the NWP model file.
- alt_site (float) – Altitude of the site above mean sea level (m).
-
type
¶ Model type, e.g. ‘gdas1’ or ‘ecwmf’.
Type: str
-
model_heights
¶ 2-D array of model heights (one for each time step).
Type: ndarray
-
mean_height
¶ Mean of model_heights.
Type: ndarray
-
data_sparse
¶ Model variables in common height grid but without interpolation in time.
Type: dict
-
data_dense
¶ Model variables interpolated to Cloudnet’s dense time / height grid.
Type: dict
-
interpolate_to_common_height
(wl_band)[source]¶ Interpolates model variables to common height grid.
Parameters: wl_band (int) – Integer denoting the wavelength band of the radar (0=35.5 GHz, 1=94 GHz).
classify¶
Module containing low-level functions to classify gridded radar / lidar measurements.
-
categorize.classify.
classify_measurements
(radar, lidar, model, mwr)[source]¶ Classifies radar/lidar observations.
This function classifies atmospheric scatterers from the input data. The input data needs to be averaged or interpolated to the common time / height grid before calling this function.
Parameters: Returns: The
ClassificationResult
instance.Return type:
-
categorize.classify.
fetch_quality
(radar, lidar, classification, attenuations)[source]¶ Returns Cloudnet quality bits.
Parameters: - radar (Radar) – The
Radar
instance. - lidar (Lidar) – The
Lidar
instance. - classification (ClassificationResult) – The
ClassificationResult
instance. - attenuations (dict) – Dictionary containing keys liquid_corrected, liquid_uncorrected.
Returns: Dictionary containing quality_bits, an integer array with the bits:
- bit 0: Pixel contains radar data
- bit 1: Pixel contains lidar data
- bit 2: Pixel contaminated by radar clutter
- bit 3: Molecular scattering present (currently not implemented!)
- bit 4: Pixel was affected by liquid attenuation
- bit 5: Liquid attenuation was corrected
Return type: dict
- radar (Radar) – The
-
class
categorize.classify.
ClassData
(radar, lidar, model, mwr)[source]¶ Container for observations that are used in the classification.
Parameters: -
z
¶ 2D radar echo.
Type: ndarray
-
ldr
¶ 2D linear depolarization ratio.
Type: ndarray
-
v
¶ 2D radar velocity.
Type: ndarray
-
width
¶ 2D radar width.
Type: ndarray
-
v_sigma
¶ 2D standard deviation of the velocity.
Type: ndarray
-
tw
¶ 2D wet bulb temperature.
Type: ndarray
-
beta
¶ 2D lidar backscatter.
Type: ndarray
-
lwp
¶ 1D liquid water path.
Type: ndarray
-
time
¶ 1D fraction hour.
Type: ndarray
-
height
¶ 1D height vector (m).
Type: ndarray
-
model_type
¶ Model identifier.
Type: str
-
radar_type
¶ Radar identifier.
Type: str
-
is_rain
¶ 2D boolean array denoting rain.
Type: ndarray
-
is_clutter
¶ 2D boolean array denoting clutter.
Type: ndarray
-
-
class
categorize.classify.
ClassificationResult
[source]¶ Result of classification
-
category_bits
¶ Array of integers concatenating all the individual boolean bit arrays.
Type: ndarray
-
is_rain
¶ 1D array denoting presence of rain.
Type: ndarray
-
is_clutter
¶ 2D array denoting presence of clutter.
Type: ndarray
-
insect_prob
¶ 2D array denoting 0-1 probability of insects.
Type: ndarray
-
liquid_bases
¶ 2D array denoting bases of liquid clouds.
Type: ndarray
-
is_undetected_melting
¶ 1D array denoting profiles that should contain melting layer but was not detected from the data.
Type: ndarray
-
melting¶
Functions to find melting layer from data.
-
categorize.melting.
find_melting_layer
(obs, smooth=True)[source]¶ Finds melting layer from model temperature, ldr, and velocity.
Melting layer is detected using linear depolarization ratio, ldr, Doppler velocity, v, and wet-bulb temperature, Tw.
The algorithm is based on ldr having a clear Gaussian peak around the melting layer. This signature is caused by the growth of ice crystals into snowflakes that are much larger. In addition, when snow and ice melt, emerging heavy water droplets start to drop rapidly towards ground. Thus, there is also a similar positive peak in the first difference of v.
The peak in ldr is the primary parameter we analyze. If ldr has a proper peak, and v < -1 m/s in the base, melting layer has been found. If ldr is missing we only analyze the behaviour of v, which is always present, to detect the melting layer.
Model temperature is used to limit the melting layer search to a certain temperature range around 0 C. For ECMWF the range is -4..+3, and for the rest -8..+6.
Notes
There might be some detection problems with strong updrafts of air. In these cases the absolute values for speed do not make sense (rain drops can even move upwards instead of down).
Parameters: - obs (ClassData) – The
ClassData
instance. - smooth (bool, optional) – If True, apply a small Gaussian smoother to the melting layer. Default is True.
Returns: 2-D boolean array denoting the melting layer.
Return type: ndarray
- obs (ClassData) – The
freezing¶
Module to find freezing region from data.
-
categorize.freezing.
find_freezing_region
(obs, melting_layer)[source]¶ Finds freezing region using the model temperature and melting layer.
Every profile that contains melting layer, subzero region starts from the mean melting layer height. If there are (long) time windows where no melting layer is present, model temperature is used in the middle of the time window. Finally, the subzero altitudes are linearly interpolated for all profiles.
Parameters: - obs (ClassData) – The
ClassData
instance. - melting_layer (ndarray) – 2-D boolean array denoting melting layer.
Returns: 2-D boolean array denoting the sub-zero region.
Return type: ndarray
Notes
It is not clear how model temperature and melting layer should be ideally combined to determine the sub-zero region.
- obs (ClassData) – The
falling¶
Module to find falling hydrometeors from data.
-
categorize.falling.
find_falling_hydrometeors
(obs, is_liquid, is_insects)[source]¶ Finds falling hydrometeors.
Falling hydrometeors are radar signals that are a) not insects b) not clutter. Furthermore, falling hydrometeors are strong lidar pixels excluding liquid layers (thus these pixels are ice or rain). They are also weak radar signals in very cold temperatures.
Parameters: - obs (ClassData) – The
ClassData
instance. - is_liquid (ndarray) – 2-D boolean array of liquid droplets.
- is_insects (ndarray) – 2-D boolean array of insects.
Returns: 2-D boolean array containing falling hydrometeors.
Return type: ndarray
- obs (ClassData) – The
insects¶
Module to find insects from data.
-
categorize.insects.
find_insects
(obs, melting_layer, liquid_layers, prob_lim=0.8)[source]¶ Returns insect probability and boolean array of insect presence.
Insects are classified by estimating heuristic probability of insects from various individual radar parameters and combining these probabilities. Insects typically yield small echo and spectral width but high linear depolarization ratio (ldr), and they are present in warm temperatures.
The combination of echo, ldr and temperature is generally the best proxy for insects. If ldr is not available, we use other radar parameters.
Insects are finally screened from liquid layers and melting layer - and above melting layer.
Parameters: - obs (ClassData) – The
ClassData
instance. - melting_layer (ndarray) – 2D array denoting melting layer.
- liquid_layers (ndarray) – 2D array denoting liquid layers.
- prob_lim (float, optional) – Probability higher than this will lead to positive detection. Default is 0.8.
Returns: 2-element tuple containing
- ndarray: 2-D probability of pixel containing insects.
- ndarray: 2-D boolean flag of insects presence.
Return type: tuple
- obs (ClassData) – The
atmos¶
This module contains functions to calculate various atmospheric parameters.
-
categorize.atmos.
calc_lwc_change_rate
(temperature, pressure)[source]¶ Returns rate of change of condensable water (LWC).
Calculates the theoretical adiabatic rate of increase of LWC with height, given the cloud base temperature and pressure.
Parameters: - temperature (ndarray) – Temperature of cloud base (K).
- pressure (ndarray) – Pressure of cloud base (Pa).
Returns: dlwc/dz (g m-3 m-1)
Return type: ndarray
References
Brenguier, 1991, https://bit.ly/2QCSJtb
-
categorize.atmos.
calc_saturation_vapor_pressure
(temperature)[source]¶ Goff-Gratch formula for saturation vapor pressure over water adopted by WMO.
Parameters: temperature (ndarray) – Temperature (K). Returns: Saturation vapor pressure (Pa). Return type: ndarray
-
categorize.atmos.
calc_mixing_ratio
(svp, pressure)[source]¶ Calculates mixing ratio from saturation vapor pressure and pressure.
Parameters: - svp (ndarray) – Saturation vapor pressure (Pa).
- pressure (ndarray) – Atmospheric pressure (Pa).
Returns: Mixing ratio (kg kg-1).
Return type: ndarray
-
categorize.atmos.
calc_air_density
(pressure, temperature, svp_mixing_ratio)[source]¶ Calculates air density (kg m-3).
Parameters: - pressure (ndarray) – Pressure (Pa).
- temperature (ndarray) – Temperature (K).
- svp_mixing_ratio (ndarray) – Saturation vapor pressure mixing ratio (kg/kg).
Returns: Air density (kg m-3).
Return type: ndarray
-
categorize.atmos.
calc_psychrometric_constant
(pressure)[source]¶ Returns psychrometric constant.
Psychrometric constant relates the partial pressure of water in air to the air temperature.
Parameters: pressure (ndarray) – Atmospheric pressure (Pa). Returns: Psychrometric constant value (Pa K-1) Return type: ndarray
-
categorize.atmos.
calc_wet_bulb_temperature
(model_data)[source]¶ Returns wet bulb temperature.
Returns wet bulb temperature for given temperature, pressure and relative humidity. Algorithm is based on a Taylor expansion of a simple expression for the saturated vapour pressure.
Parameters: model_data (dict) – Model variables temperature, pressure, rh. Returns: Wet bulb temperature (K). Return type: ndarray References
J. Sullivan and L. D. Sanders: Method for obtaining wet-bulb temperatures by modifying the psychrometric formula.
-
categorize.atmos.
calc_dew_point_temperature
(vapor_pressure)[source]¶ Returns dew point temperature.
Parameters: vapor_pressure (ndarray) – Water vapor pressure (Pa). Returns: Dew point temperature (K). Return type: ndarray Notes
Method from Vaisala’s white paper: “Humidity conversion formulas”.
-
categorize.atmos.
get_attenuations
(model, mwr, classification)[source]¶ Calculates attenuations due to atmospheric gases and liquid water.
Parameters: - model (Model) – The
Model
instance. - mwr (Mwr) – The
Mwr
instance. - classification (ClassificationResult) – The
ClassificationResult
instance.
Returns: Dictionary containing radar_gas_atten, radar_liquid_atten, liquid_atten_err, liquid_corrected and liquid_uncorrected fields.
Return type: dict
- model (Model) – The
-
class
categorize.atmos.
Attenuation
(model, classification)[source]¶ Base class for gas and liquid attenuations.
Parameters: - model (Model) – The
Model
instance. - classification (ClassificationResult) – The
ClassificationResult
instance.
-
classification
¶ The
ClassificationResult
instance.Type: ClassificationResult
- model (Model) – The
-
class
categorize.atmos.
GasAttenuation
(model, classification)[source]¶ Radar gas attenuation class. Child of Attenuation.
Parameters: - model (Model) – The
Model
instance. - classification (ClassificationResult) – The
ClassificationResult
instance.
-
atten
¶ Gas attenuation (dB).
Type: ndarray
- model (Model) – The
-
class
categorize.atmos.
LiquidAttenuation
(model, classification, mwr)[source]¶ Radar liquid attenuation class. Child of Attenuation.
Parameters: - model (Model) – The
Model
instance. - classification (ClassificationResult) – The
ClassificationResult
instance. - mwr (Mwr) – The
Mwr
instance.
-
atten
¶ Radar liquid attenuation (dB).
Type: ndarray
-
atten_err
¶ Error of radar liquid attenuation (dB).
Type: ndarray
-
uncorrected
¶ Boolean array denoting uncorrected pixels.
Type: ndarray
-
corrected
¶ Boolean array denoting corrected pixels.
Type: ndarray
- model (Model) – The
-
categorize.atmos.
fill_clouds_with_lwc_dz
(atmosphere, is_liquid)[source]¶ Fills liquid clouds with lwc change rate at the cloud bases.
Parameters: - atmosphere (tuple) – 2-element tuple containing temperature (K) and pressure (Pa).
- is_liquid (ndarray) – Boolean array indicating presence of liquid clouds.
Returns: liquid water content change rate (g/m3/m), so that for each cloud the base value is filled for the whole cloud.
Return type: ndarray
-
categorize.atmos.
get_lwc_change_rate_at_bases
(atmosphere, is_liquid)[source]¶ Finds LWC change rate in liquid cloud bases.
Parameters: - atmosphere (tuple) – 2-element tuple containing temperature (K) and pressure (Pa).
- is_liquid (ndarray) – Boolean array indicating presence of liquid clouds.
Returns: liquid water content change rate at cloud bases (kg/m3/m).
Return type: ndarray
-
categorize.atmos.
find_cloud_bases
(array)[source]¶ Finds bases of clouds.
Parameters: array (ndarray) – 2D boolean array denoting clouds or some other similar field. Returns: Boolean array indicating bases of the individual clouds. Return type: ndarray
-
categorize.atmos.
find_cloud_tops
(array)[source]¶ Finds tops of clouds.
Parameters: array (ndarray) – 2D boolean array denoting clouds or some other similar field. Returns: Boolean array indicating tops of the individual clouds. Return type: ndarray
-
categorize.atmos.
calc_adiabatic_lwc
(lwc_change_rate, dheight)[source]¶ Calculates adiabatic liquid water content (g/m3).
Parameters: - lwc_change_rate (ndarray) – Liquid water content change rate (g/m3/m) calculated at the base of each cloud and filled to that cloud.
- dheight (float) – Median difference of the height vector (m).
Returns: Liquid water content (g/m3).
-
categorize.atmos.
distribute_lwp_to_liquid_clouds
(lwc, lwp)[source]¶ Finds LWC that would produce measured LWP.
Calculates LWP-weighted, normalized LWC. This is the measured LWP distributed to liquid cloud pixels according to their theoretical proportion, i.e., sum(scaled LWC) = measured LWP.
Parameters: - lwc (ndarray) – 2D liquid water content (g/m3).
- lwp (ndarray) – 1D liquid water path (g/m2).
Returns: 2D LWP-weighted, normalized LWC (g/m2).
Return type: ndarray
droplet¶
This module has functions for liquid layer detection.
-
categorize.droplet.
correct_liquid_top
(obs, liquid, is_freezing, limit=200)[source]¶ Corrects lidar detected liquid cloud top using radar data.
Parameters: - obs (ClassData) – The
ClassData
instance. - liquid (dict) – Dictionary about liquid clouds including tops and presence.
- is_freezing (ndarray) – 2-D boolean array of sub-zero temperature, derived from the model temperature and melting layer based on radar data.
- limit (float) – The maximum correction distance (m) above liquid cloud top.
Returns: Corrected liquid cloud array.
Return type: ndarray
- obs (ClassData) – The
-
categorize.droplet.
find_liquid
(obs, peak_amp=1e-06, max_width=300, min_points=3, min_top_der=1e-07, min_lwp=0)[source]¶ Estimate liquid layers from SNR-screened attenuated backscatter.
Parameters: - obs (ClassData) – The
ClassData
instance. - peak_amp (float, optional) – Minimum value of peak. Default is 2e-5.
- max_width (float, optional) – Maximum width of peak. Default is 300 (m).
- min_points (int, optional) – Minimum number of valid points in peak. Default is 3.
- min_top_der (float, optional) – Minimum derivative above peak, defined as (beta_peak-beta_top) / (alt_top-alt_peak), which is always positive. Default is 2e-7.
- min_lwp (float, optional) – Minimum value from linearly interpolated lwp measured by the mwr. Default is 0.
Returns: Dict containing presence, bases and tops.
Return type: dict
- obs (ClassData) – The
-
categorize.droplet.
ind_base
(dprof, ind_peak, dist, lim)[source]¶ Finds base index of a peak in profile.
Return the lowermost index of profile where 1st order differences below the peak exceed a threshold value.
Parameters: - dprof (ndarray) – 1-D array of 1st discrete difference. Masked values should be 0, e.g. dprof = np.diff(masked_prof).filled(0)
- ind_peak (int) – Index of (possibly local) peak in the original profile. Note that the peak must be found with some other method before calling this function.
- dist (int) – Number of elements investigated below p. If ( p - dist)<0, search starts from index 0.
- lim (float) – Parameter for base index. Values greater than 1.0 are valid. Values close to 1 most likely return the point right below the maximum 1st order difference (within dist points below p). Values larger than 1 more likely accept some other point, lower in the profile.
Returns: Base index of the peak.
Return type: int
Raises: IndexError
– Can’t find proper base index (probably too many masked values in the profile).Examples
Consider a profile
>>> x = np.array([0, 0.5, 1, -99, 4, 8, 5])
that contains one bad, masked value
>>> mx = ma.masked_array(x, mask=[0, 0, 0, 1, 0, 0, 0]) [0, 0.5, 1.0, --, 4.0, 8.0, 5.0]
The 1st order difference is now
>>> dx = np.diff(mx).filled(0) [0.5, 0.5, 0, 0, 4, -3]
From the original profile we see that the peak index is 5. Let’s assume our base can’t be more than 4 elements below peak and the threshold value is 2. Thus we call
>>> ind_base(dx, 5, 4, 2) 4
When x[4] is the lowermost point that satisfies the condition. Changing the threshold value would alter the result
>>> ind_base(dx, 5, 4, 10) 1
See also
droplet.ind_top()
-
categorize.droplet.
ind_top
(dprof, ind_peak, nprof, dist, lim)[source]¶ Finds top index of a peak in profile.
Return the uppermost index of profile where 1st order differences above the peak exceed a threshold value.
Parameters: - dprof (ndarray) – 1-D array of 1st discrete difference. Masked values should be 0, e.g. dprof = np.diff(masked_prof).filled(0)
- nprof (int) – Length of the profile. Top index can’t be higher than this.
- ind_peak (int) – Index of (possibly local) peak in the profile. Note that the peak must be found with some other method before calling this function.
- dist (int) – Number of elements investigated above p. If (p + dist) > nprof, search ends to nprof.
- lim (float) – Parameter for top index. Values greater than 1.0 are valid. Values close to 1 most likely return the point right above the maximum 1st order difference (within dist points above p). Values larger than 1 more likely accept some other point, higher in the profile.
Returns: Top index of the peak.
Return type: int
Raises: IndexError
– Can not find proper top index (probably too many masked values in the profile).See also
droplet.ind_base()
Products modules¶
Products is CloudnetPy’s subpackage. It contains several modules that correspond to different Cloudnet products.
classification¶
Module for creating classification file.
-
products.classification.
generate_classification
(categorize_file, output_file)[source]¶ Generates Cloudnet classification product.
This function reads the initial classification masks from a categorize file and creates a more comprehensive classification for different atmospheric targets. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_classification >>> generate_classification('categorize.nc', 'classification.nc')
iwc¶
Module for creating Cloudnet ice water content file using Z-T method.
-
products.iwc.
generate_iwc
(categorize_file, output_file)[source]¶ Generates Cloudnet ice water content product.
This function calculates ice water content using the so-called Z-T method. In this method, ice water content is calculated from attenuated-corrected radar reflectivity and model temperature. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_iwc >>> generate_iwc('categorize.nc', 'iwc.nc')
References
Hogan, R.J., M.P. Mittermaier, and A.J. Illingworth, 2006: The Retrieval of Ice Water Content from Radar Reflectivity Factor and Temperature and Its Use in Evaluating a Mesoscale Model. J. Appl. Meteor. Climatol., 45, 301–317, https://doi.org/10.1175/JAM2340.1
lwc¶
Module for creating Cloudnet liquid water content file using scaled-adiabatic method.
-
products.lwc.
generate_lwc
(categorize_file, output_file)[source]¶ Generates Cloudnet liquid water content product.
This function calculates cloud liquid water content using the so-called adiabatic-scaled method. In this method, liquid water content measured by microwave radiometer is used to constrain the theoretical liquid water content of observed liquid clouds. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_lwc >>> generate_lwc('categorize.nc', 'lwc.nc')
References
Illingworth, A.J., R.J. Hogan, E. O’Connor, D. Bouniol, M.E. Brooks, J. Delanoé, D.P. Donovan, J.D. Eastment, N. Gaussiat, J.W. Goddard, M. Haeffelin, H.K. Baltink, O.A. Krasnov, J. Pelon, J. Piriou, A. Protat, H.W. Russchenberg, A. Seifert, A.M. Tompkins, G. van Zadelhoff, F. Vinit, U. Willén, D.R. Wilson, and C.L. Wrench, 2007: Cloudnet. Bull. Amer. Meteor. Soc., 88, 883–898, https://doi.org/10.1175/BAMS-88-6-883
-
class
products.lwc.
LwcSource
(categorize_file)[source]¶ Data container for liquid water content calculations. Child of DataSource.
This class reads input data from a categorize file and provides data structures and methods for holding the results.
Parameters: categorize_file (str) – Categorize file name. -
lwp
¶ 1D liquid water path.
Type: ndarray
-
lwp_error
¶ 1D error of liquid water path.
Type: ndarray
-
is_rain
¶ 1D array denoting presence of rain.
Type: ndarray
-
dheight
¶ Median difference in height vector.
Type: float
-
atmosphere
¶ Dictionary containing interpolated fields temperature and pressure.
Type: dict
-
categorize_bits
¶ The
CategorizeBits
instance.Type: CategorizeBits
-
-
class
products.lwc.
Lwc
(lwc_source)[source]¶ Class handling the actual LWC calculations.
Parameters: lwc_source (LwcSource) – The LwcSource
instance.-
dheight
¶ Median difference in height vector.
Type: float
-
echo
¶ Dictionary containing radar and lidar that are 2D arrays denoting which instrument detected the pixel.
Type: dict
-
is_liquid
¶ 2D array denoting liquid.
Type: ndarray
-
lwc_adiabatic
¶ 2D array storing adiabatic lwc.
Type: ndarray
-
lwc
¶ 2D array of liquid water content (scaled with lwp).
Type: ndarray
-
status
¶ Indicates where is liquid, and where it was adjusted.
Type: ndarray
-
lwc_error
¶ 2D lwc error estimate.
Type: ndarray
-
-
products.lwc.
find_topmost_clouds
(is_cloud)[source]¶ From 2d binary cloud field, return the uppermost cloud layer only.
Parameters: is_cloud (ndarray) – Boolean array denoting presence of clouds. Returns: Copy of input array containing only the uppermost cloud layer in each profile. Return type: ndarray
drizzle¶
Module for creating Cloudnet drizzle product.
-
products.drizzle.
generate_drizzle
(categorize_file, output_file)[source]¶ Generates Cloudnet drizzle product.
This function calculates different drizzle properties from cloud radar and lidar measurements. The results are written in a netCDF file.
Parameters: - categorize_file (str) – Categorize file name.
- output_file (str) – Output file name.
Examples
>>> from cloudnetpy.products import generate_drizzle >>> generate_drizzle('categorize.nc', 'drizzle.nc')
References
O’Connor, E.J., R.J. Hogan, and A.J. Illingworth, 2005: Retrieving Stratocumulus Drizzle Parameters Using Doppler Radar and Lidar. J. Appl. Meteor., 44, 14–27, https://doi.org/10.1175/JAM-2181.1
-
class
products.drizzle.
DrizzleSource
(categorize_file)[source]¶ Class holding the input data for drizzle calculations.
-
class
products.drizzle.
DrizzleClassification
(categorize_file)[source]¶ Class storing the information about different drizzle types, child of
ProductClassification
.Parameters: categorize_file (str) – Categorize file name. -
is_v_sigma
¶ 2D array denoting finite v_sigma.
Type: ndarray
-
warm_liquid
¶ 2D array denoting warm liquid.
Type: ndarray
-
drizzle
¶ 2D array denoting drizzle presence.
Type: ndarray
-
would_be_drizzle
¶ 2D array denoting possible drizzle pixels.
Type: ndarray
-
cold_rain
¶ 1D array denoting profiles with melting layer.
Type: ndarray
-
-
products.drizzle.
correct_spectral_width
(cat_file)[source]¶ Corrects spectral width.
Removes the effect of turbulence and horizontal wind that cause spectral broadening of the Doppler velocity.
Parameters: cat_file (str) – Categorize file name. Returns: Spectral width containing the correction for turbulence broadening. Return type: ndarray
-
products.drizzle.
calc_horizontal_wind
(cat_file)[source]¶ Calculates magnitude of horizontal wind.
Parameters: cat_file – Categorize file name. Returns: Horizontal wind (m s-1). Return type: ndarray
-
products.drizzle.
drizzle_solve
(data, drizzle_class, width_ht)[source]¶ Estimates drizzle parameters.
Parameters: - data (DrizzleSource) – The
DrizzleSource
instance. - drizzle_class (DrizzleClassification) – The
DrizzleClassification
instance. - width_ht (ndarray) – 2D corrected spectral width.
Returns: Dictionary of retrieved drizzle parameters, Do, mu, S, beta_corr.
Return type: dict
- data (DrizzleSource) – The
-
products.drizzle.
calc_dia
(beta_z_ratio, mu=0, ray=1, k=1)[source]¶ Drizzle diameter calculation.
Parameters: - beta_z_ratio (ndarray) – Beta to z ratio, multiplied by (2 / pi).
- mu (ndarray, optional) – Shape parameter for gamma calculations. Default is 0.
- ray (ndarray, optional) – Mie to Rayleigh ratio for z. Default is 1.
- k (ndarray, optional) – Alpha to beta ratio . Default is 1.
Returns: Drizzle diameter.
Return type: ndarray
References
product_tools¶
General helper classes and functions for all products.
-
class
products.product_tools.
CategorizeBits
(categorize_file)[source]¶ Class holding information about category and quality bits.
Parameters: categorize_file (str) – Categorize file name. -
category_bits
¶ Dictionary containing boolean fields for droplet, falling, cold, melting, aerosol, insect.
Type: dict
-
quality_bits
¶ Dictionary containing boolean fields for radar, lidar, clutter, molecular, attenuated, corrected.
Type: dict
-
-
class
products.product_tools.
ProductClassification
(categorize_file)[source]¶ Base class for creating different classifications in the child classes of various Cloudnet products. Child of CategorizeBits class.
Parameters: categorize_file (str) – Categorize file name. -
is_rain
¶ 1D array denoting rainy profiles.
Type: ndarray
-
is_undetected_melting
¶ 1D array denoting profiles which should contain melting layer but was not detected from the data.
Type: ndarray
-
-
products.product_tools.
get_source
(data_handler)[source]¶ Returns uuid (or filename if uuid not found) of the source file.
Parameters: data_handler (netCDF Dataset) – The netCDF Dataset instance. Returns: The file_uuid attribute, if exits. If missing, return the filename attribute. Return type: str
-
products.product_tools.
read_nc_fields
(nc_file, names)[source]¶ Reads selected variables from a netCDF file.
Parameters: - nc_file (str) – netCDF file name.
- names (str/list) – Variables to be read, e.g. ‘temperature’ or [‘ldr’, ‘lwp’].
Returns: Array in case of one variable passed as a string. List of arrays otherwise.
Return type: ndarray/list
-
products.product_tools.
interpolate_model
(cat_file, names)[source]¶ Interpolates 2D model field into dense Cloudnet grid.
Parameters: - cat_file (str) – Categorize file name.
- names (str/list) – Model variable to be interpolated, e.g. ‘temperature’ or [‘temperature’, ‘pressure’].
Returns: Array in case of one variable passed as a string. List of arrays otherwise.
Return type: ndarray/list
Misc¶
Documentation for various modules with low-level functionality.
utils¶
This module contains general helper functions.
-
utils.
seconds2hours
(time_in_seconds)[source]¶ Converts seconds since some epoch to fraction hour.
Parameters: time_in_seconds (ndarray) – 1-D array of seconds since some epoch that starts on midnight. Returns: Time as fraction hour. Return type: ndarray Notes
Excludes leap seconds.
-
utils.
time_grid
(time_step=30)[source]¶ Returns decimal hour array between 0 and 24.
Computes fraction hour time vector 0-24 with user-given resolution (in seconds).
Parameters: time_step (int, optional) – Time resolution in seconds, greater than 1. Default is 30. Returns: Time vector between 0 and 24. Return type: ndarray Raises: ValueError
– Bad resolution as input.
-
utils.
binvec
(x)[source]¶ Converts 1-D center points to bins with even spacing.
Parameters: x (array_like) – 1-D array of N real values. Returns: N + 1 edge values. Return type: ndarray Examples
>>> binvec([1, 2, 3]) [0.5, 1.5, 2.5, 3.5]
-
utils.
rebin_2d
(x_in, data, x_new, statistic='mean', n_min=1)[source]¶ Rebins 2-D data in one dimension.
Parameters: - x_in (ndarray) – 1-D array with shape (n,).
- data (MaskedArray) – 2-D input data with shape (n, m).
- x_new (ndarray) – 1-D target vector (center points) with shape (N,).
- statistic (str, optional) – Statistic to be calculated. Possible statistics are ‘mean’, ‘std’. Default is ‘mean’.
- n_min (int) – Minimum number of points to have good statistics in a bin. Default is 1.
Returns: Rebinned data with shape (N, m).
Return type: MaskedArray
Notes: 0-values are masked in the returned array.
-
utils.
rebin_1d
(x_in, data, x_new, statistic='mean')[source]¶ Rebins 1D array.
Parameters: - x_in (ndarray) – 1-D array with shape (n,).
- data (MaskedArray) – 1-D input data with shape (m,).
- x_new (ndarray) – 1-D target vector (center points) with shape (N,).
- statistic (str, optional) – Statistic to be calculated. Possible statistics are ‘mean’, ‘std’. Default is ‘mean’.
Returns: Rebinned data with shape (N,).
Return type: MaskedArray
-
utils.
filter_isolated_pixels
(array)[source]¶ From a 2D boolean array, remove completely isolated single cells.
Parameters: array (ndarray) – 2-D boolean array (numpy array or list) containing isolated values. Returns: Cleaned array. Return type: ndarray Examples
>>> filter_isolated_pixels([[0, 0, 0], [0, 1, 0], [0, 0, 0]]) array([[0, 0, 0], [0, 0, 0], [0, 0, 0]])
-
utils.
filter_x_pixels
(array)[source]¶ From a 2D boolean array, remove cells isolated in x-direction.
Parameters: array (ndarray) – 2-D boolean array containing isolated pixels in x-direction. Returns: Cleaned array. Return type: ndarray Examples
>>> filter_x_pixels([[1, 0, 0], [0, 1, 0], [0, 1, 1]]) array([[0, 0, 0], [0, 1, 0], [0, 1, 0]])
-
utils.
isbit
(integer, nth_bit)[source]¶ Tests if nth bit (0,1,2..) is on for the input number.
Parameters: - integer (int) – A number.
- nth_bit (int) – Investigated bit.
Returns: True if set, otherwise False.
Return type: bool
Raises: ValueError
– negative bit as input.Examples
>>> isbit(4, 1) False >>> isbit(4, 2) True
See also
utils.setbit()
-
utils.
setbit
(integer, nth_bit)[source]¶ Sets nth bit (0, 1, 2..) on input number.
Parameters: - integer (int) – A number.
- nth_bit (int) – Bit to be set.
Returns: Integer where nth bit is set.
Return type: int
Raises: ValueError
– negative bit as input.Examples
>>> setbit(1, 1) 3 >>> setbit(0, 2) 4
See also
utils.isbit()
-
utils.
interpolate_2d
(x, y, z, x_new, y_new)[source]¶ Linear interpolation of gridded 2d data.
Parameters: - x (ndarray) – 1-D array.
- y (ndarray) – 1-D array.
- z (ndarray) – 2-D array at points (x, y).
- x_new (ndarray) – 1-D array.
- y_new (ndarray) – 1-D array.
Returns: Interpolated data.
Return type: ndarray
Notes
Does not work with nans. Ignores mask of masked data. Does not extrapolate.
-
utils.
interpolate_2d_masked
(array, ax_values, ax_values_new)[source]¶ Interpolates 2D array preserving the mask.
Parameters: - array (ndarray) – 2D masked array.
- ax_values (tuple) – 2-element tuple containing x and y values of the input array.
- ax_values_new (tuple) – 2-element tuple containing new x and y values.
Returns: Interpolated 2D masked array.
Return type: ndarray
Notes
Uses linear interpolation.
-
utils.
l2norm
(*args)[source]¶ Returns l2 norm.
Parameters: *args – Variable number of data (array_like) with the same shape. Returns: The l2 norm. Return type: MaskedArray
-
utils.
l2norm_weighted
(values, overall_scale, term_weights)[source]¶ Calculates scaled and weighted Euclidean distance.
Calculated distance is of form: scale * sqrt((a1*a)**2 + (b1*b)**2 + …) where a, b, … are terms to be summed and a1, a2, … are optional weights for the terms.
Parameters: - values (tuple) – Tuple containing the values.
- overall_scale (float) – Scale factor for the calculated Euclidean distance.
- term_weights (tuple) – Weights for the terms. Must be single float or a list of numbers (one per term).
Returns: Scaled and weighted Euclidean distance.
Return type: float
TODO: Probably better use masked arrays instead of tuples.
-
utils.
cumsumr
(x, axis=0)[source]¶ Finds cumulative sum that resets on 0.
Parameters: - x (ndarray) – Input array.
- axis (int, optional) – Axis where the sum is calculated. Default is 0.
Returns: Cumulative sum, restarted at 0.
Return type: ndarray
Examples
>>> x = np.array([0, 0, 1, 1, 0, 0, 0, 1, 1, 1]) >>> cumsumr(x) [0, 0, 1, 2, 0, 0, 0, 1, 2, 3]
-
utils.
ffill
(arr, value=0)[source]¶ Forward fills a numpy array.
Parameters: - arr (ndarray) – 1-D or 2-D array.
- value (int) – Value to be filled. Default is 0.
Returns: Forward-filled array.
Return type: ndarray
Examples
>>> x = np.array([0, 5, 0, 0, 2, 0]) >>> ffill(x) [0, 5, 5, 5, 2, 2]
Notes
Works only in axis=1 direction.
-
utils.
init
(nvars, shape, dtype=<class 'float'>, masked=True)[source]¶ Initializes several numpy arrays.
Parameters: - nvars (int) – Number of arrays to be generated.
- shape (tuple) – Shape of the arrays, e.g. (2, 3).
- dtype (data-type, optional) – The desired data-type for the arrays, e.g., int. Default is float.
- masked (bool) – If True, generated arrays are masked arrays, else ordinary numpy arrays. Default is True.
Yields: Iterator containing the empty arrays.
Examples
>>> a, b = init(2, (2, 3)) >>> a masked_array( data=[[0., 0., 0.], [0., 0., 0.]], mask=False, fill_value=1e+20)
-
utils.
n_elements
(x, dist, var=None)[source]¶ Returns the number of elements that cover certain distance.
Parameters: - x (ndarray) – Input array with arbitrary units or time in fraction hour. x should be evenly spaced or at least close to.
- dist (float) – Distance to be covered. If x is fraction time, dist is in minutes. Otherwise x and dist should have the same units.
- var (str, optional) – ‘time’ or None. If None, inputs have the same units. If ‘time’, input is fraction hour and distance in minutes. Default is None.
Returns: Number of elements in the input array that cover dist.
Return type: int
Examples
>>> x = np.array([2, 4, 6, 8, 10]) >>> n_elements(x, 6) 3
The result is rounded to the closest integer, so:
>>> n_elements(x, 6.9) 3 >>> n_elements(x, 7) 4
With fraction hour time vector:
>>> x = np.linspace(0, 1, 61) >>> n_elements(x, 10, 'time') 10
-
utils.
isscalar
(array)[source]¶ Tests if input is scalar.
By “scalar” we mean that array has a single value.
Examples
>>> isscalar(1) True >>> isscalar([1]) True >>> isscalar(np.array(1)) True >>> isscalar(np.array([1])) True
-
utils.
get_wl_band
(radar_frequency)[source]¶ Returns integer corresponding to radar frequency.
Parameters: radar_frequency (float) – Radar frequency (GHz). Returns: 0=35GHz radar, 1=94Ghz radar. Return type: int
-
utils.
del_dict_keys
(dict_in, keys)[source]¶ Removes multiple keys from dictionary.
Parameters: - dict_in (dict) – A dictionary.
- keys (list / tuple) – list of keys to be deleted.
Returns: Dictionary without the deleted keys.
Return type: dict
-
utils.
get_site_information
(site, *args)[source]¶ Reads site information from Cloudnet http API.
Parameters: - site (str) – Site identifier, e.g. ‘mace-head’ or ‘lindenberg’.
- args – Variable number of field names to be queried. Possible field names are ‘latitude’, ‘longitude’, ‘altitude’ and ‘site_name’.
Returns: List of return values as floats.
Return type: list
Examples
>>> get_site_information('mace-head', 'latitude', 'longitude')
-
utils.
array_to_probability
(arr_in, loc, scale, invert=False)[source]¶ Converts continuous variable into 0-1 probability.
Parameters: - arr_in (MaskedArray) – Masked numpy array.
- loc (float) – Center of the distribution. Values smaller than this will have small probability. Values greater than this will have large probability.
- scale (float) – Width of the distribution, i.e., how fast the probability drops or increases from the peak.
- invert (bool, optional) – If True, large values have small probability and vice versa. Default is False.
Returns: Probability.
Return type: ndarray
-
utils.
range_to_height
(range_los, tilt_angle)[source]¶ Converts distances from a tilted instrument to height above the ground.
Parameters: - range_los (ndarray) – Distances towards the line of sign from the instrument.
- tilt_angle (float) – Angle in degrees from the zenith.
Returns: Altitudes of the LOS points.
Return type: ndarray
cloudnetarray¶
CloudnetArray class.
-
class
cloudnetarray.
CloudnetArray
(netcdf4_variable, name, units_from_user=None)[source]¶ Stores netCDF4 variables as CloudnetArrays.
Parameters: - netcdf4_variable (netCDF4 Variable) – The netCDF4
Variable
instance. - name (str) – Name of the variable.
- units_from_user (str, optional) – Units of the variable.
-
name
¶ Name of the variable.
Type: str
-
data
¶ The actual data.
Type: array_like
-
data_type
¶ ‘i4’ for integers, ‘f4’ for floats.
Type: str
-
units
¶ The units_from_user argument if it is given. Otherwise copied from the original netcdf4 variable.
Type: str
-
mask_indices
(ind)[source]¶ Masks data from given indices.
Parameters: ind (tuple) – Indices to be masked.
-
rebin_data
(time, time_new, height=None, height_new=None)[source]¶ Rebins data in time and optionally interpolates in height.
Parameters: - time (ndarray) – 1D time array.
- time_new (ndarray) – 1D new time array.
- height (ndarray, optional) – 1D height array.
- height_new (ndarray, optional) – 1D new height array. Should be given if also height is given.
- netcdf4_variable (netCDF4 Variable) – The netCDF4
-
class
cloudnetarray.
RadarArray
(netcdf4_variable, name, units_from_user=None)[source]¶ The
RadarArray
class, child ofCloudnetArray
.This class contains additional, cloud radar -specific methods.
Parameters: - netcdf4_variable (netCDF4 Variable) – The netCDF4
Variable
instance. - name (str) – Name of the variable.
- units_from_user (str, optional) – Units of the variable.
-
filter_isolated_pixels
()[source]¶ Filter vertical artifacts in radar data.
Notes
These kind of artifacts are seen in RPG data.
-
calc_linear_std
(time, time_new)[source]¶ Calculates std of radar velocity.
Parameters: - time (ndarray) – 1D time array.
- time_new (ndarray) – 1D new time array.
Notes
The result is masked if the bin contains masked values.
-
rebin_velocity
(time, time_new, folding_velocity, sequence_indices)[source]¶ Rebins radar velocity in polar coordinates.
Parameters: - time (ndarray) – 1D time array.
- time_new (ndarray) – 1D new time array.
- folding_velocity (float / list) – Folding velocity (m/s). Can be float when it’s the same for all altitudes, or list when it matches difference altitude regions (defined in sequence_indices).
- sequence_indices (list) – List containing indices of different folding regions, e.g. [[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10]].
- netcdf4_variable (netCDF4 Variable) – The netCDF4
output¶
Functions for file writing.
-
output.
update_attributes
(cloudnet_variables, attributes)[source]¶ Overrides existing CloudnetArray-attributes.
Overrides existing attributes using hard-coded values. New attributes are added.
Parameters: - cloudnet_variables (dict) – CloudnetArray instances.
- attributes (dict) – Product-specific attributes.
-
output.
save_product_file
(short_id, obj, file_name, copy_from_cat=())[source]¶ Saves a standard Cloudnet product file.
Parameters: - short_id (str) – Short file identifier, e.g. ‘lwc’, ‘iwc’, ‘drizzle’, ‘classification’.
- obj (object) – Instance containing product specific attributes: time, dataset, data.
- file_name (str) – Name of the output file to be generated.
- copy_from_cat (tuple, optional) – Variables to be copied from the categorize file.
-
output.
merge_history
(root_group, file_type, *sources)[source]¶ Merges history fields from one or several files and creates a new record.
Parameters: - root_group (netCDF Dataset) – The netCDF Dataset instance.
- file_type (str) – Long description of the file.
- *sources (obj) – Objects that were used to generate this product. Their history attribute will be copied to the new product.
-
output.
init_file
(file_name, dimensions, obs)[source]¶ Initializes a Cloudnet file for writing.
Parameters: - file_name (str) – File name to be generated.
- dimensions (dict) – Dictionary containing dimension for this file.
- obs (dict) – Dictionary containing
CloudnetArray
instances.
-
output.
copy_variables
(source, target, var_list)[source]¶ Copies variables (and their attributes) from one file to another.
Parameters: - source (object) – Source object.
- target (object) – Target object.
- var_list (list) – List of variables to be copied.


CrimPapers
Articles published on 7 September 2019
On 7 September 2019, 11 new articles were published in 9 journals.
Cross National Collaboration in Implementing Evidence-Based Practices by Jessica J. Warner, Devyn Rorem, Lisa Cooke, Jennifer Pealer, Myrinda Schweitzer Smith
The Back-Door Governance of Crime: Confiscating Criminal Assets in the UK
Intersectionality of Race, Class, and Gender in Predicting Police Satisfaction by Ahmet Kule, Vic W. Bumphus, Gale Iles
Policing People With Mental Illness: Experimental Evaluation of Online Training to De-Escalate Mental Health Crises
On the Unreliability of Multiple Systems Estimation For Estimating the Number of Potential Victims of Modern Slavery in the UK by John Whitehead, James Jackson, Alex Balch, Brian Francis
Teaching Health Versus Treating Illness: the Efficacy of Intensive Three Principles Correctional Counseling For Improving the Mental Health/resilience of People in An English Prison by Thomas M. Kelley, Jacqueline Hollows, Dennis M. Savard
When Gender Goes to the Dogs: Perceptions of Masculinity and Identity Change Among Male Participants in A Prison-Based Animal Program by Jennifer K. Wesely
C. J. Najdowski and M. Stevenson: Criminal Juries in the 21st Century: Contemporary Issues, Psychological Science, and the Law
Muslim Converts Who Turn to Global Jihad: Radicalization Characteristics and Countermeasures by Eitan Azani, Liram Koblentz-Stenzler
Pseudo-Radicalism and the De-Radicalization of Educated Youth in Indonesia by Bagong Suyanto, Mun’im Sirry, Rahma Sugihartati
Victims Without A Voice: Measuring Worst Forms of Child Labor in the Indian State of Bihar by Sheldon X. Zhang, Meredith Dank, Kyle Vincent, Pradeep Narayanan, Sowmyaa Bharadwaj, Sudharsanam Manni Balasubramaniam


Microsoft delays launch of its Surface Earbuds until 2020
by Mark GurmanMicrosoft has delayed the launch of its Surface Earbuds, missing the 2019 holiday shopping season.
The software giant was due to be the latest company to enter the increasingly crowded true wireless earbuds market, dominated by Apple 's popular AirPods, with a late 2019 release.
But the Surface Earbuds will now come out in the first half of 2020. The announcement was made by Panos Panay, the company's chief product officer, on Twitter. "Product-making is about the relentless pursuit to get all the details right, which takes time…sometimes more than we planned on," he said.
The wireless earbuds were announced earlier this year. They have a circular shape, integrate with a voice assistant and can be used to control Microsoft software like Office.
The global market for true wireless earbuds has grown exponentially in 2019, according to research from Counterpoint, reaching 33 million units in the third quarter. Apple's market share fell amid rising competition from Xiaomi, Samsung, Amazon and others, but AirPods were still by far the most popular with an estimated 45 per cent share.
Apple has recently launched the new AirPods Pro, becoming on of the few manufacturers, alongside Sony, to produce true wireless earbuds with noice cancellation.
At $US249 ($367), the Surface Earbuds are priced the same as Apple's new AirPods Pro in the US, but the delay means Microsoft will be missing out on a key category this holiday season.
Google has also been working to upgrade its wireless earbuds. That product will also be missing this holiday season. The company is aiming for a release in early 2019 at a price of $US179.
Bloomberg, with staff reporters


Psychic Game
Guess what letter I am thinking of:
Scoreboard
Wins:
Losses:
Guesses left:
Guesses so far:


|
mlpack
1.0.12
|
The exponential cooling schedule cools the temperature T at every step according to the equation. More...
Public Member Functions | |
| ExponentialSchedule (const double lambda=0.001) | |
| double | Lambda () const |
| Get the cooling speed, lambda. More... | |
| double & | Lambda () |
| Modify the cooling speed, lambda. More... | |
| double | NextTemperature (const double currentTemperature, const double) |
| Returns the next temperature given current status. More... | |
Private Attributes | |
| double | lambda |
| The cooling speed. More... | |
The exponential cooling schedule cools the temperature T at every step according to the equation.
where is the cooling speed. The smaller is, the slower the cooling speed, and better the final result will be. Some literature uses instead. In practice, is very close to 1 and will be awkward to input (e.g. alpha = 0.999999 vs lambda = 1e-6).
Definition at line 34 of file exponential_schedule.hpp.
|
inline |
Definition at line 42 of file exponential_schedule.hpp.
|
inline |
Get the cooling speed, lambda.
Definition at line 59 of file exponential_schedule.hpp.
References lambda.
|
inline |
Modify the cooling speed, lambda.
Definition at line 61 of file exponential_schedule.hpp.
References lambda.
|
inline |
Returns the next temperature given current status.
The current system's energy is not used in this calculation.
| currentTemperature | Current temperature of system. |
| currentEnergy | Current energy of system (not used). |
Definition at line 51 of file exponential_schedule.hpp.
References lambda.
|
private |
The cooling speed.
Definition at line 65 of file exponential_schedule.hpp.
Referenced by Lambda(), and NextTemperature().


{%
with
heading = "Add service"
%}
{% include "toolkit/page-heading.html" %}
{% endwith %}
{% if form.errors|length > 0 %}
There was a problem with your answer to the following questions
{% for field_name, field_errors in form.errors|dictsort if field_errors %}
{% for error in field_errors %}
{{ form[field_name].label }}
{% endfor %}
{% endfor %}


×
Modal title
JobGroup & JobName
Job Description
TriggerGroup & TriggerName
Trigger Description
Trigger Type
Cron Trigger
Simple Trigger
Calendar Trigger
Daily Interval Trigger
Cron Expression
Cron-Maker
Time Zone
Repeat Count
Repeat Interval / Unit
Millisecond
Second
Minute
Hour
Day
Week
Month
Year
Misfire Instruction
Priority
Start Date
×
Edit Job Data
Key
Value
Type
String
Boolean
Number
×
Remove Trigger
Do you really want to delete the trigger?


--- layout: post cover: assets/images/cuckoos-blog-cover.jpg navigation: True title: Installing Cuckoo Sandbox on macOS date: 2020-03-22 04:20:00 tags: [Cuckoos] class: post-template subclass: 'post tag-cuckoos' author: tomato ---
In this section we will cover only installing Cuckoos and not the full configuration process, since this is really the most difficult hurdle I have been hearing from Security Analyst. Please use the Cuckoos Project for recommendations in configuring Cuckoos Sandbox.
On a new macOS system there are a couple things we need to install, we will call section pre-install. For manual install we will need multiple packages, some will have issues. After fixing them we will move onto the install and startup of Cuckoos.
$ sudo easy_install pip
$ cd cuckoo_sanbox/
$ virtualenv venv
virtualenv venv
created virtual environment…
…
activators PythonActivator…
$ source ./venv/bin/activate
$ cd venv
(venv)$ pip install -U setuptools
So, when you try to install Cuckoos from pip, as seen below, you will get an error on capstone and unicorn packages:
(venv)$ pip install -U cuckoo
Processing
…
Collecting django==1.8.4
Using cached Django-1.8.4-py2.py3-none-any.whl (6.2 MB)
…
Building wheels for collected packages: unicorn, capstone
Building wheel for unicorn (setup.py) ... error
ERROR: Command errored out with exit status 1:
…
Running setup.py clean for unicorn
Building wheel for capstone (setup.py) ... error
…
Running setup.py clean for capstone
Failed to build unicorn capstone
…
command output.
We will need to download Cuckoos, edit capstone package, and modify Cuckoo criteria list of the package, but lets walk through the steps
(venv)$ pip download cuckoo
Collecting cuckoo
Using cached Cuckoo-2.0.7.tar.gz (8.6 MB)
Saved ./Cuckoo-2.0.7.tar.gz
Collecting alembic==1.0.10
Using cached alembic-1.0.10.tar.gz (1.0 MB)
Saved ./alembic-1.0.10.tar.gz
…
Collecting scandir; python_version < "3.5"
Using cached scandir-1.10.0.tar.gz (33 kB)
Saved ./scandir-1.10.0.tar.gz
Successfully downloaded cuckoo…
(venv)$ ls | xargs -n 1 | grep 'Cuck\|cap\|uni'
Cuckoo-2.0.7.tar.gz
capstone-3.0.5rc2.tar.gz
unicorn-1.0.1.tar.gz
Try install and it will fail on some packages, particularly on unicorn and captone:
(venv)$ pip install Cuckoo-2.0.7.tar.gz
…
…
Building wheels for collected packages: Cuckoo, unicorn, capstone
Building wheel for Cuckoo (setup.py) ... done
Created wheel for Cuckoo: filename=Cuckoo-2.0.7-py2-none-any.whl size=8895737 sha256=b3f7dc2782e96e126070bd662901c3d4cfeff51a717fe7ab6ebc626cc62927f5
Stored in directory: /Users/bebop/Library/Caches/pip/wheels/ee/5c/8e/d734a9fc27f0c5a9ebc4b0f96a33f84d4f7b9d2e173ba18b8a
Building wheel for unicorn (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /Users/bebop/cuckoo_sanbox/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/zc/bxw1rn3x75d8p4yqgy7ksy640000gn/T/pip-install-TFjH5H/unicorn/setup.py'"'"'; __file__='"'"'/private/var/folders/zc/bxw1rn3x75d8p4yqgy7ksy640000gn/T/pip-install-TFjH5H/unicorn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/zc/bxw1rn3x75d8p4yqgy7ksy640000gn/T/pip-wheel-192nun
cwd: /private/var/folders/zc/bxw1rn3x75d8p4yqgy7ksy640000gn/T/pip-install-TFjH5H/unicorn/
…
In order to fix issue, a few tweaks are needed on capstone. Let’s make some edits on the packages but since clutter is messy lets isolate our work by creating a temp directory called Ein
(venv)$ pwd
/Users/bebop/cuckoo_sanbox/
(venv)$ mkdir Ein
(venv)$ ls | xargs -n 1 | grep 'Cuck\|cap'
Cuckoo-2.0.7.tar.gz
capstone-3.0.5rc2.tar.gz
(venv)$ tar -xf Cuckoo-2.0.7.tar.gz -C Ein/
(venv)$ tar -xf capstone-3.0.5rc2.tar.gz -C Ein/
(venv)$ ls Ein/ | xargs -n 1
Cuckoo-2.0.7
capstone-3.0.5rc2
(venv)$ ls Ein/capstone-3.0.5rc2 | xargs -n 1
BUILDING.txt
LICENSE.TXT
Makefile
PKG-INFO
README.txt
capstone
setup.py
src
(venv)$ cd ~/cuckoo_sanbox/Ein/capstone-3.0.5rc2/src
(venv)$ grep -r i386 .
./src/make.sh: export LIBARCHS="i386 x86_64"
delete “i386” from line 63 in the make.sh file, should look like below when edited:
(venv)$grep -r 'export LIBARCHS' ~/cuckoo_sanbox/Ein/capstone-3.0.5rc2/src
export LIBARCHS="x86_64"
Verify, make sure you get no results then redo remove i386:
(venv)$ grep -r i386 .
(venv)$ pip install Ein/capstone-3.0.5rc2/
Processing ./Ein/capstone-3.0.5rc2
Building wheels for collected packages: capstone
Building wheel for capstone (setup.py) ... done
Created wheel for capstone: filename=capstone-3.0.5rc2-py2-none-macosx_10_15_x86_64.whl size=1345456 sha256=eb07a63b73a8c7a83978b440612d9bf57aa73a8ccef5f4060ebfb43a323c0ae4
Stored in directory: /Users/bebop/Library/Caches/pip/wheels/db/63/b5/31d6086133188e5446730f2ba4ffde6564c1f0e9b3fc1ecc3e
Successfully built capstone
Installing collected packages: capstone
Successfully installed capstone-3.0.5rc2
#Note that if you have any issues with libmagic (python-libmagic), install version 0.4.12
(venv)$ pip uninstall python-libmagic
Found existing installation: python-libmagic 0.4.0
Uninstalling python-libmagic-0.4.0:
Would remove:
/Users/bebop/cuckoo_sanbox/venv/lib/python2.7/site-packages/magic/*
/Users/bebop/cuckoo_sanbox/venv/lib/python2.7/site-packages/python_libmagic-0.4.0.dist-info/*
Proceed (y/n)? y
Successfully uninstalled python-libmagic-0.4.0
(venv)$ pip -qq install python-magic==0.4.12
Processing /Users/bebop/Library/Caches/pip/wheels/6f/b3/f6/fcc71a6a0c531119ba3be3070a0ca7fb08fb7fa5e666e4d9b7/python_magic-0.4.12-py2-none-any.whl
Installing collected packages: python-magic
Successfully installed python-magic-0.4.12
Next edit the Cuckoo install script, just modify the package version for unicorn and nothing else, dont forget to modify setup file!!!
(venv)$ grep -A34 'install_requires' Ein/Cuckoo-2.0.7/setup.py
install_requires=[
"alembic==1.0.10",
"androguard==3.0.1",
"beautifulsoup4==4.5.3",
"chardet==2.3.0",
"click==6.6",
"django==1.8.4",
"django_extensions==1.6.7",
"dpkt==1.8.7",
"egghatch>=0.2.3, <0.3",
"elasticsearch==5.3.0",
"flask==0.12.2",
"flask-sqlalchemy==2.4.0",
"httpreplay>=0.2.4, <0.3",
"ipaddress>=1.0.22",
"gevent>=1.2, <1.3",
"jinja2==2.9.6",
"jsbeautifier==1.6.2",
"oletools==0.51",
"peepdf>=0.4.2, <0.5",
"pefile2==1.2.11",
"pillow==3.2",
"pyelftools==0.24",
"pyguacamole==0.6",
"pymisp==2.4.106",
"pymongo==3.0.3",
"python-dateutil==2.4.2",
"python-magic==0.4.12",
"roach>=0.1.2, <0.2",
"sflock>=0.3.10, <0.4",
"sqlalchemy==1.3.3",
"unicorn==1.0.2rc1",
"wakeonlan==0.2.2",
"yara-python==3.6.3",
],
(venv)$ cd ~/cuckoo_sanbox/
(venv)$ pip install Ein/Cuckoo-2.0.7
Processing ./Ein/Cuckoo-2.0.7
Requirement already satisfied: alembic==1.0.10 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.0.10)
Requirement already satisfied: androguard==3.0.1 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (3.0.1)
Requirement already satisfied: beautifulsoup4==4.5.3 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (4.5.3)
Requirement already satisfied: chardet==2.3.0 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (2.3.0)
Requirement already satisfied: click==6.6 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (6.6)
Requirement already satisfied: django==1.8.4 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.8.4)
Requirement already satisfied: django_extensions==1.6.7 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.6.7)
Requirement already satisfied: dpkt==1.8.7 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.8.7)
Requirement already satisfied: egghatch<0.3,>=0.2.3 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.2.3)
Requirement already satisfied: elasticsearch==5.3.0 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (5.3.0)
Requirement already satisfied: flask==0.12.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.12.2)
Requirement already satisfied: flask-sqlalchemy==2.4.0 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (2.4.0)
Requirement already satisfied: httpreplay<0.3,>=0.2.4 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.2.6)
Requirement already satisfied: ipaddress>=1.0.22 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.0.23)
Requirement already satisfied: gevent<1.3,>=1.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.2.2)
Requirement already satisfied: jinja2==2.9.6 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (2.9.6)
Requirement already satisfied: jsbeautifier==1.6.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.6.2)
Requirement already satisfied: oletools==0.51 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.51)
Requirement already satisfied: peepdf<0.5,>=0.4.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.4.2)
Requirement already satisfied: pefile2==1.2.11 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.2.11)
Requirement already satisfied: pillow==3.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (3.2.0)
Requirement already satisfied: pyelftools==0.24 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.24)
Requirement already satisfied: pyguacamole==0.6 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.6)
Requirement already satisfied: pymisp==2.4.106 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (2.4.106)
Requirement already satisfied: pymongo==3.0.3 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (3.0.3)
Requirement already satisfied: python-dateutil==2.4.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (2.4.2)
Requirement already satisfied: python-magic==0.4.12 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.4.12)
Requirement already satisfied: roach<0.2,>=0.1.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.1.2)
Requirement already satisfied: sflock<0.4,>=0.3.10 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.3.10)
Requirement already satisfied: sqlalchemy==1.3.3 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.3.3)
Requirement already satisfied: unicorn==1.0.2rc1 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (1.0.2rc1)
Requirement already satisfied: wakeonlan==0.2.2 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (0.2.2)
Requirement already satisfied: yara-python==3.6.3 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (3.6.3)
Requirement already satisfied: requests==2.13.0 in ./venv/lib/python2.7/site-packages (from Cuckoo==2.0.7) (2.13.0)
Requirement already satisfied: python-editor>=0.3 in ./venv/lib/python2.7/site-packages (from alembic==1.0.10->Cuckoo==2.0.7) (1.0.4)
Requirement already satisfied: Mako in ./venv/lib/python2.7/site-packages (from alembic==1.0.10->Cuckoo==2.0.7) (1.1.2)
Requirement already satisfied: six>=1.2 in ./venv/lib/python2.7/site-packages (from django_extensions==1.6.7->Cuckoo==2.0.7) (1.14.0)
Requirement already satisfied: capstone==3.0.5rc2; sys_platform == "darwin" in ./venv/lib/python2.7/site-packages (from egghatch<0.3,>=0.2.3->Cuckoo==2.0.7) (3.0.5rc2)
Requirement already satisfied: urllib3<2.0,>=1.8 in ./venv/lib/python2.7/site-packages (from elasticsearch==5.3.0->Cuckoo==2.0.7) (1.25.8)
Requirement already satisfied: Werkzeug>=0.7 in ./venv/lib/python2.7/site-packages (from flask==0.12.2->Cuckoo==2.0.7) (1.0.0)
Requirement already satisfied: itsdangerous>=0.21 in ./venv/lib/python2.7/site-packages (from flask==0.12.2->Cuckoo==2.0.7) (1.1.0)
Requirement already satisfied: tlslite-ng==0.6.0 in ./venv/lib/python2.7/site-packages (from httpreplay<0.3,>=0.2.4->Cuckoo==2.0.7) (0.6.0)
Requirement already satisfied: greenlet>=0.4.10 in ./venv/lib/python2.7/site-packages (from gevent<1.3,>=1.2->Cuckoo==2.0.7) (0.4.15)
Requirement already satisfied: MarkupSafe>=0.23 in ./venv/lib/python2.7/site-packages (from jinja2==2.9.6->Cuckoo==2.0.7) (1.1.1)
Requirement already satisfied: pythonaes==1.0 in ./venv/lib/python2.7/site-packages (from peepdf<0.5,>=0.4.2->Cuckoo==2.0.7) (1.0)
Requirement already satisfied: future>=0.16.0 in ./venv/lib/python2.7/site-packages (from peepdf<0.5,>=0.4.2->Cuckoo==2.0.7) (0.18.2)
Requirement already satisfied: colorama==0.3.7 in ./venv/lib/python2.7/site-packages (from peepdf<0.5,>=0.4.2->Cuckoo==2.0.7) (0.3.7)
Requirement already satisfied: functools32; python_version < "3.0" in ./venv/lib/python2.7/site-packages (from pymisp==2.4.106->Cuckoo==2.0.7) (3.2.3.post2)
Requirement already satisfied: jsonschema in ./venv/lib/python2.7/site-packages (from pymisp==2.4.106->Cuckoo==2.0.7) (3.2.0)
Requirement already satisfied: enum34; python_version < "3.4" in ./venv/lib/python2.7/site-packages (from pymisp==2.4.106->Cuckoo==2.0.7) (1.1.10)
Requirement already satisfied: pycrypto in ./venv/lib/python2.7/site-packages (from roach<0.2,>=0.1.2->Cuckoo==2.0.7) (2.6.1)
Requirement already satisfied: cryptography>=2.1 in ./venv/lib/python2.7/site-packages (from roach<0.2,>=0.1.2->Cuckoo==2.0.7) (2.8)
Requirement already satisfied: olefile==0.43 in ./venv/lib/python2.7/site-packages (from sflock<0.4,>=0.3.10->Cuckoo==2.0.7) (0.43)
Requirement already satisfied: ecdsa in ./venv/lib/python2.7/site-packages (from tlslite-ng==0.6.0->httpreplay<0.3,>=0.2.4->Cuckoo==2.0.7) (0.15)
Requirement already satisfied: attrs>=17.4.0 in ./venv/lib/python2.7/site-packages (from jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (19.3.0)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in ./venv/lib/python2.7/site-packages (from jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (1.5.0)
Requirement already satisfied: pyrsistent>=0.14.0 in ./venv/lib/python2.7/site-packages (from jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (0.15.7)
Requirement already satisfied: setuptools in ./venv/lib/python2.7/site-packages (from jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (44.0.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in ./venv/lib/python2.7/site-packages (from cryptography>=2.1->roach<0.2,>=0.1.2->Cuckoo==2.0.7) (1.14.0)
Requirement already satisfied: contextlib2; python_version < "3" in ./venv/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in ./venv/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (1.2.0)
Requirement already satisfied: configparser>=3.5; python_version < "3" in ./venv/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (4.0.2)
Requirement already satisfied: pathlib2; python_version < "3" in ./venv/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (2.3.5)
Requirement already satisfied: pycparser in ./venv/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.1->roach<0.2,>=0.1.2->Cuckoo==2.0.7) (2.20)
Requirement already satisfied: scandir; python_version < "3.5" in ./venv/lib/python2.7/site-packages (from pathlib2; python_version < "3"->importlib-metadata; python_version < "3.8"->jsonschema->pymisp==2.4.106->Cuckoo==2.0.7) (1.10.0)
Building wheels for collected packages: Cuckoo
Building wheel for Cuckoo (setup.py) ... done
Created wheel for Cuckoo: filename=Cuckoo-2.0.7-py2-none-any.whl size=8895738 sha256=46a0511f510d43b61ad8c0a0cb45dd50a8e54b9114491f08a04d64fbdb2b44c2
Stored in directory: /Users/bebop/Library/Caches/pip/wheels/1b/eb/99/383277525069bc20411c6aa68a9e4e4cf620491dcf99fce6b0
Successfully built Cuckoo
Installing collected packages: Cuckoo
Attempting uninstall: Cuckoo
Found existing installation: Cuckoo 2.0.7
Uninstalling Cuckoo-2.0.7:
Successfully uninstalled Cuckoo-2.0.7
Successfully installed Cuckoo-2.0.7
$ cuckoo -d
_ _ _ _ _ _
/\ \ /\_\ /\ \ /\_\ /\ \ /\ \
/ \ \ / / / _ / \ \ / / / _ / \ \ / \ \
/ /\ \ \ \ \ \__ /\_\ / /\ \ \ / / / /\_\ / /\ \ \ / /\ \ \
/ / /\ \ \ \ \___\ / / // / /\ \ \ / / /__/ / / / / /\ \ \ / / /\ \ \
/ / / \ \_\ \__ / / / // / / \ \_\ / /\_____/ / / / / \ \_\ / / / \ \_\
/ / / \/_/ / / / / / // / / \/_/ / /\_______/ / / / / / // / / / / /
/ / / / / / / / // / / / / /\ \ \ / / / / / // / / / / /
/ / /________ / / /___/ / // / /________ / / / \ \ \ / / /___/ / // / /___/ / /
/ / /_________\/ / /____\/ // / /_________\/ / / \ \ \ / / /____\/ // / /____\/ /
\/____________/\/_________/ \/____________/\/_/ \_\_\\/_________/ \/_________/
Cuckoo Sandbox 2.0.7
www.cuckoosandbox.org
Copyright (c) 2010-2018
=======================================================================
Welcome to Cuckoo Sandbox, this appears to be your first run!
We will now set you up with our default configuration.
You will be able to see and modify the Cuckoo configuration,
Yara rules, Cuckoo Signatures, and much more to your likings
by exploring the /Users/bebop/.cuckoo directory.
Among other configurable items of most interest is the
new location for your Cuckoo configuration:
/Users/bebop/.cuckoo/conf
=======================================================================
Cuckoo has finished setting up the default configuration.
Please modify the default settings where required and
start Cuckoo again (by running `cuckoo` or `cuckoo -d`).


"I am not interested"
"Please stop contacting me"
"I don't want to get drinks"
"Thanks for the offer, but no"
"No no no no no no no no no no NO NO NO NO NO NO NO NO NO


Before Change
mevals = [None, ] * len(mf)
if mevecs is None:
mevecs = [np.eye(3) for i in range(len(mf))]
for j, f in enumerate(mf):
pdf += f * single_tensor_pdf(pdf_points,
After Change
else:
sticks = [sphere2cart(1, np.deg2rad(pair[0]), np.deg2rad(pair[1]))
for pair in angles]
sticks = np.array(sticks)
pdf = np.zeros(len(pdf_points))
mevecs = []
for s in sticks:
mevecs += [all_tensor_evecs(s).T]
for j, f in enumerate(mf):
pdf += f * single_tensor_pdf(pdf_points,
evals=mevals[j], evecs=mevecs[j], tau=tau)
return pdf


Home
Baby
Tips
Settings
Log Out
Are You Sure want to log Out?
Yes
No


- The goal is to turn all the lights off
- A pink square is a light that is ON
- A gray square is a light that is OFF
- When you click a light, it toggles the following lights in a cross pattern:
- The light you clicked
- The light above the light you clicked
- The light below the light you clicked
- The light to the left of the light you clicked
- The light to the right of the light you clicked
- If there is no light below the light you clicked, then
it wraps around (torus style) and toggles the light in the first row.
- And so on for each of the four directions


Dragonlord Oath
Second Trimester
Mana (27)
- 1 Black Lotus
- 1 Lion’s Eye Diamond
- 1 Mox Sapphire
- 1 Mox Ruby
- 1 Mox Pearl
- 1 Mox Jet
- 1 Mox Emerald
- 1 Mana Crypt
- 1 Sol Ring
- 1 Mana Vault
- 4 Forbidden Orchard
- 2 Mana Confluence
- 2 Flooded Strand
- 2 Polluted Delta
- 1 Tundra
- 1 Underground Sea
- 1 Tropical Island
- 1 Island
- 1 Cavern of Souls
- 1 Library of Alexandria
- 1 Tolarian Academy
Disruption, Removal (3)
- 1 Engineered Explosives
- 1 Ancient Grudge
- 1 Nihil Spellbomb
Countermagic (6)
- 2 Force of Will
- 2 Mental Misstep
- 2 Flusterstorm
Card Draw, Card Selection (13)
- 2 Sensei’s Divining Top
- 2 Gitaxian Probe
- 1 Ancestral Recall
- 1 Brainstorm
- 1 Repeal
- 1 Gifts Ungiven
- 1 Dig Through Time
- 1 Time Walk
- 1 Demonic Tutor
- 1 Ponder
- 1 Yawgmoth’s Will
Combo (5)
- 4 Oath of Druids
- 1 Pyrite Spellbomb
Creatures (4)
- 2 Auriok Salvagers
- 1 Dragonlord Dromoka
- 1 Griselbrand
Planeswalkers (2)
- 2 Jace, the Mind Sculptor
Sideboard
- 2 Nihil Spellbomb
- 2 Tormod’s Crypt
- 2 Nature’s Claim
- 1 Notion Thief
- 1 Ethersworn Canonist
- 1 Magus of the Moat
- 1 Steel Sabotage
- 1 Abrupt Decay
- 1 Ancient Grudge
- 1 Hurkyl’s Recall
- 1 Tinker
- 1 Sphinx of the Steel Wind


Attached are the results for
the first training race, but with some caveats.
We're also testing video
camera results and as a result were not able to pick
all the numbers. I also don't have the start list (I'm out of
town for the
week), so just numbers will
have to suffice for now.
We had approximately 65
starters for the handicap style race.
The "B" field
was given a five minute head
start and still finished well clear of the "A"
field. Same format next week but add an additional
lap (eight miles) for a
total of 43 miles.
Congratulations to our
Winners:
Jeff Cincoski - University
of Montana Cycling
Jennifer Nerison -
University of Montana Cycling
Thanks to our volunteers,
the race couldn't have happened without their help.
Thanks also to The Big
Dipper for the ice cream certificates.
1 - Jeff Cincoski UM Cycling
2 - MT Velo
3 - 5VV
4 - 5VV
5 - Woman, Bianchi
6 - 5VV
7 - 5VV
8 - 5VV
9 - Blue Jersey
10 - 5VV
11 - Green Sobe Jersey
12 - Lady, Blue Jersey
13 - Blue Jersey, may be NRO
14 - 5VV
15 - 5VV
16 - White Jersey, Red Sides
17 - NRO
A's: No idea where the A's ended and the C's begin. Number and name used
when I was sure of the finish order.
18 - Red Pants, Blue/white jersey
19 - Green Jersey, probably Flathead
20 - Red/Grey, possibly Tejay
21 - NRO
22 - Grey Jersey
23 - Green/White (probably Flathead)
24 - Red/Gray Jersey
25 - MT Velo (probably Scott Herzig)
26 - White/Red
27 - NRO (possibly 788, Longmire)
28 - MT Velo
29 - Red
30 - Blue (probably NRO)
31 - NRO
32 - 5VV
33 - 5VV
34 - Grey, Red bike
35 - Flathead
36 - Blue
37 - 786, Bob Presta, FVV
38 - 798, Tom Robertson, NRO
39 - 793, Keifer Hahn, NRO
40 - 473, Brink Kuchenbrod, MT Velo
41 - 324, Jason Hobson, FVV
42 - 481, John Gleaves, MT Velo
43 - 785, Steve Willis, Flathead
44 - 446, Brian Wirak, MT Velo
45 - 453, Ben Sullivan, MT Velo
46 - 471, Ely Wilkinson, FVV
47 - 466, Rankin Holmes, NRO
48 - 460, John Melvin, FVV
49 - 483, Stefan Farr, Unattached
50 - 787, Richard Allen, FVV
51 - 442, Martha Williamson, Unattached
52 - 449, Rob Aldredge, FVV
53 - 482, Brian Moreau, MT Velo
54 - 799, Steve Cohen, MT Velo
55 - 795, Scott Fisk, Great Divide
56 - 479, Fred Simpson, Unattached
The last of these riders were definitely C's, we just don't
know who they are
since they came in shortly after the A's. We show 57 finishers. I don't
think we missed any, but we may have since we were was
trying to get the camera to
function the way it was supposed to.


{#include Drift/insightsEmailBody} {#content-title}Baseline Drifts Daily Report - {action.context.start_time.toStringFormat()}{/content-title} {#content-body}
Systems with drift(s) from associated baselines
{action.context.unique_system_count}
Baselines with drifted systems
{action.context.drift.size()}
|
|
Hi,
|
|
|
|


class: center, middle # Python for GIS and Geoscience Doctoral schools of Ghent University
November 21, 23 and 25, 2022 Joris Van den Bossche, Stijn Van Hoey https://github.com/jorisvandenbossche/DS-python-geospatial --- class: center, middle # Who are you? Go to https://hackmd.io/MsRlxUUqQVKtyt_dNoInBg?both
--- ### Joris Van den Bossche
jorisvdbossche
jorisvandenbossche
* Open source software developer and teacher * Pandas, GeoPandas, scikit-learn, Apache Arrow .center[ ![:scale 90%](./static/img/work_joris_1.png)] --- ### Stijn Van Hoey
SVanHoey
stijnvanhoey
* Freelance developer and teacher * Research software engineer at [Fluves](https://www.fluves.com/) .center[ ![:scale 90%](./static/img/work_stijn_1.png)] --- class: middle, section_background # Setting up a working environment --- ## Setting up a working environment For the setup instructions, see the [setup page](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html). --- class: left, middle 0. Everyone has conda installed and the environment setup? If not, see [1-install-python-and-the-required-python-packages](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html#1-before-the-course-install-python-and-the-required-python-packages) 1. Make sure to (re)download ALL the course material, see [2-getting-the-course-materials](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html#2-first-day-of-the-course-getting-the-course-materials) also if you already did this before. 2. Next, also do section 3 and 4 of the [setup](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html#3-first-day-of-the-course-test-your-configuration) > If you succesfully done 1, 2 and 3, put up your `green sticky note` on your laptop screen.. Next: - Surf to and fill in [the questionnaire](https://hackmd.io/MsRlxUUqQVKtyt_dNoInBg?both) - In Jupyter Lab, start with the 'notebooks/00-jupyter_introduction.ipynb'. > Installation or setup issues? Put up your `red sticky note` on your laptop screen. --- class: center, middle When you see something like this... ![:scale 100%](./static/img/startup.png) ...relax, you're ready to start! --- class: center, middle ![:scale 100%](https://i.ytimg.com/vi/PlaYMh-u-2w/maxresdefault.jpg) --- class: middle, left ### Time is divided between - group sessions: we explain new concepts (aka 'theory') - practise sessions: you work on exercises or case studies In case of questions, remarks, suggestions, you can always interrupt us and just ask. During practise sessions, use the `red sticky note` on top of your laptop screen to let us know you have a question. ### Status check We will regularly ask for a check (ready with exercise, installation succesfull...). Use the `green sticky note` on top of your laptop screen to say 👍. ### Feel lost? Just ask either one of us, we are here to help you. --- class: middle, center ![:scale 80%](./static/img/issuetracker.png) Report bugs, typo's, suggestions... as issues ([New issue](https://github.com/jorisvandenbossche/course-python-geospatial/issues/new)) or see the [contributing guidelines](https://github.com/jorisvandenbossche/course-python-geospatial/blob/main/CONTRIBUTING.md) --- class: middle, section_background # Working with Python --- # Conda ### Why using conda? - Consistent package manager across Windows, Mac and Linux - Many precompiled packages available - Less problems with installation -- ### Why different environments? - Manage the dependencies of a specific project/paper/group/... - You can install different version of Python and other packages alongside on your computer - Easily share environments with other --- ## Small overview of conda commands Creating a new environment: ``` conda create -n my_env python=3.10 pandas # or from environment file conda env create -f environment.yml ``` Activating an environment: ``` conda activate my_env ``` Install a new package: ``` conda install matplotlib # if not working, try: pip install ... ``` List all installed packages: `conda list` List all your environments: `conda info -e` See the docs: https://docs.conda.io/projects/conda/en/latest/user-guide/index.html --- class: center, middle ### Keep track of your python ecosystem
with an `environment.yml`
``` conda env export > environment.yml ``` --- # Writing Python code ## IPython console
.center[![:scale 75%](./static/img/ipython.png)] --- ## Interactive Development Environment (IDE) * [**Spyder**](https://pythonhosted.org/spyder/) is shipped with Anaconda. The familiar environment for Matlab/Rstudio-users. * [**VS Code**](https://code.visualstudio.com/) is also shipped with Anaconda. General purpose editor with powerful plugin ecosystem. * [**PyCharm**](https://www.jetbrains.com/pycharm/): Popular for web-development and Django applications, powerful when doing 'real' development (packages, libraries, software) * [Eclipse + **pyDev plugin**](http://www.pydev.org/): If you already work in Eclipse, add the python environment * ... --- ## Jupyter Lab/Notebook
(*previously called IPython notebook*)
**Jupyter notebook** provides an **interactive** scripting environment,
ideal for exploration, prototyping,... .center[![:scale 70%](./static/img/notebook.png)] -- ...the stuff we're dealing with in this course! --- class: center, middle # How are you feeling? ![:scale 100%](http://esq.h-cdn.co/assets/15/51/980x490/landscape-1450137389-john-cleese.JPG) ### https://forms.gle/wYwY37f6z9j7J5iP8 Please fill in the questionnaire. --- class: center, middle # Closing notes --- class: center, middle # Python's scientific/GIS ecosystem #### ## Inspired by figure by Jake VanderPlas --- class: center, middle, bgheader background-image: url(./static/img/ecosystem-gis-overview_6.svg) background-size: cover --- class: center, middle, bgheader background-image: url(./static/img/ecosystem-gis-overview_5.svg) background-size: cover --- class: center, middle, bgheader background-image: url(./static/img/ecosystem-gis-overview_3.svg) background-size: cover --- class: center, middle, bgheader background-image: url(./static/img/ecosystem-gis-overview_2.svg) background-size: cover --- class: center, middle, bgheader background-image: url(./static/img/ecosystem-gis-overview_1.svg) background-size: cover --- class: center, middle, bgheader background-image: url(./static/img/ecosystem-gis-overview.svg) background-size: cover --- # A rich ecosystem of packages
![:scale 60%](https://pysal.org/pysal1.png) Python Spatial Analysis Library (https://pysal.org). Collection of packages for: * detection of spatial clusters, hot-spots, and outliers * construction of graphs from spatial data * spatial regression and statistical modeling on geographically embedded networks * spatial econometrics * exploratory spatio-temporal data analysis * ... --- # A rich ecosystem of packages
Building up on / integrating with GeoPandas: **MovingPandas** (https://anitagraser.github.io/movingpandas/): working with movement data **Momepy** (http://docs.momepy.org/en/latest/): quantitative analysis of urban morphology **geoplot** (https://residentmario.github.io/geoplot/): high-level Python geospatial plotting library (combining GeoPandas, matplotlib and cartopy) **...** Check out the [GeoPandas ecosystem](https://geopandas.org/en/stable/community/ecosystem.html#geopandas-ecosystem). --- # A rich ecosystem of packages
Building up on / integrating with xarray: **Satpy** (https://satpy.readthedocs.io/en/stable/): package for earth-observing satellite data processing **xesmf** (https://xesmf.readthedocs.io/): Universal Regridder for Geospatial Data **verde** (https://www.fatiando.org/verde/): interpolating spatial data on regular grids **rioxarray** (https://github.com/corteva/rioxarray): linking rasterio functionality with xarray **...** Check out the [Xarray ecosystem](https://docs.xarray.dev/en/stable/ecosystem.html). --- # A rich ecosystem of packages
Several more domain/application specific packages: **MetPy** (https://unidata.github.io/MetPy): working with weather data **xgcm** (https://xgcm.readthedocs.io/en/latest/index.html): working with output from General Circulation Models **osmnx** (https://github.com/gboeing/osmnx): street networks **...** --- # A rich ecosystem of packages
**Machine learning**: scikit-learn, tensorflow, pytorch, keras, chainer, ... **Performance**: Numba, Cython, Numexpr, Pythran, C/Fortran wrappers, ... **Visualisation**: Bokeh, Seaborn, Plotnine, Altair, Plotly, Mayavi, HoloViews, datashader, vaex ... **Data structures and parallel/distributed computation**: Xarray, Dask, Distributed, Cupy, ... Specialized packages in many **scientific fields**: astronomy, natural language processing, image processing, geospatial, ... **Packaging and distribution**: pip/wheels, conda, Anaconda, Canopy, ... --- class: center, middle ### Reading advice [Good Enough Practices in Scientific Computing](https://arxiv.org/pdf/1609.00037v1.pdf) > "*However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is*" --- class: center, middle # Thanks ### Joris Van den Bossche
jorisvdbossche
jorisvandenbossche
### Stijn Van Hoey
SVanHoey
stijnvanhoey


Symbol Index for "Timepoint"
Symbol Index for diagram
"Timepoint"
from (Association)
Location (Class)
Reference Point (Class)
RP/Dimension (Generalization Node)
Timepoint (Class)
Timepoint Association (Class)
to (Association)


Includes several statistical methods for the estimation of parameters and high quantiles of river flow distributions. The focus is on regional estimation based on homogeneity assumptions and computed from multivariate observations (multiple measurement stations). For details see Kinsvater et al. (2017) <arXiv:1701.06455>.
| Version: | 0.1.1 |
| Imports: | evd, TLMoments, magrittr, copula |
| Published: | 2017-01-30 |
| Author: | Friederike Deiters [aut, cre], Paul Kinsvater [aut] |
| Maintainer: | Friederike Deiters <deiters at statistik.tu-dortmund.de> |
| License: | GPL-2 | GPL-3 [expanded from: GPL (≥ 2)] |
| NeedsCompilation: | no |
| CRAN checks: | flood results |
| Reference manual: | flood.pdf |
| Package source: | flood_0.1.1.tar.gz |
| Windows binaries: | r-devel: flood_0.1.1.zip, r-release: flood_0.1.1.zip, r-oldrel: flood_0.1.1.zip |
| OS X binaries: | r-release: flood_0.1.1.tgz, r-oldrel: flood_0.1.1.tgz |
Please use the canonical form https://CRAN.R-project.org/package=flood to link to this page.


‘LDF govt. has limitations’
by Staff ReporterThe Left Democratic Front (LDF) government cannot implement everything that the CPI(M) wants, party Polit Bureau member S. Ramachandran Pillai has said.
Opening a seminar organised by the Democratic Youth Federation of India (DYFI) here on Sunday, he claimed that though the party was opposed to the Unlawful Activities (Prevention) Act, the LDF government could not shy away from implementing it as the law had been passed by the Parliament.
“There are many limitations posed by the Constitution and the socio-political situations. The government needs to take all these things into account. If there is any action from the government that is against the law of the land, the Centre will intervene. That should not happen,” he said.