Documentation for pyIAST¶
This Python package, pyIAST, takes pure-component gas adsorption isotherms in a nanoporous material and predicts mixture isotherms using the Ideal Adsorbed Solution Theory (IAST).
pyIAST has five options to characterize the pure-component adsorption isotherms from a set of simulated or experimental data points:
- Fit a Langmuir isotherm model
- Fit a quadratic isotherm model
- Fit a BET isotherm model
- Fit a Sips isotherm model
- Use linear interpolation (numerical quadrature for spreading pressures)
If you would like an additional model implemented, submit an issue on Github.
Installation¶
This code runs on Python 2.6 and 2.7. To install, clone the repository on Github. cd into the folder with the source code, /src, and run the setup.py script in the terminal:
python setup.py install
If on Windows, run from a command prompt (Start –> Accessories):
setup.py install
Use¶
Start up Python 2.6 or 2.7. I strongly recommend using IPython Notebook. As an example for use, see the /test directory. We test the IAST code with a binary mixture of Xe and Kr in IRMOF-1.
Simulated pure-component adsorption isotherms at 298 K (from single componenent grand-canonical Monte Carlo) are present in:
- IRMOF-1_clean_Xe_isotherm_298K.csv
- IRMOF-1_clean_Kr_isotherm_298K.csv
We simulated binary mixture isotherms of Xe/Kr in IRMOF-1 at 1 bar total pressure and 298 K at different mixture conditions using dual-component grand-canonical Monte Carlo. This data is present in mixture_Xe_Kr_IRMOF-1_clean_298K.csv. We use IAST to reproduce this result.
First, import the IAST package:
import IAST
Import the pure-component isotherms¶
Use the Pandas package (documentation for Pandas), to load the pure-component adsorption isotherms as DataFrames:
import pandas as pd
df_Xe = pd.read_csv("IRMOF-1_Xe_isotherm_298K.csv")
df_Kr = pd.read_csv("IRMOF-1_Kr_isotherm_298K.csv")
The units for pressure and loading in both DataFrames must be consistent; loading of gas must be in a molar quantity for IAST to apply (e.g. mmol/g or mmol/cm3). The IAST package will then work with these units throughout.
To load data into a Pandas DataFrame that is of a different format, see the documentation for Pandas.
Construct isotherm objects¶
We separate the process of characterizing the pure-component adsorption isotherms from performing IAST calculations. Construct the models by passing the DataFrame with the pure-component adsorption isotherm and the names of the columns that correspond to the loading and pressure.
- Langmuir isotherm model
xe_isotherm = IAST.LangmuirIsotherm(df_Xe,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
kr_isotherm = IAST.LangmuirIsotherm(df_Kr,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
You may access the model parameters identified by least squares fitting by:
xe_isotherm.M # saturation loading (units: mol/m3 here)
xe_isotherm.K # Langmuir constant (units : 1/bar here)
or print them automatically by:
xe_isotherm.print_params()
- Quadratic isotherm model
xe_isotherm = IAST.QuadraticIsotherm(df_Xe,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
kr_isotherm = IAST.QuadraticIsotherm(df_Kr,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
You may see the model parameters identified by least squares fitting by:
xe_isotherm.M # half of saturation loading (units: mol/m3 here)
xe_isotherm.Ka # isotherm constant (units : 1/bar here)
xe_isotherm.Kb # isotherm constant (units : 1/bar^2 here)
- BET isotherm model
xe_isotherm = IAST.BETIsotherm(df_Xe,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
kr_isotherm = IAST.BETIsotherm(df_Kr,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
- Sips isotherm model
xe_isotherm = IAST.SipsIsotherm(df_Xe,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
kr_isotherm = IAST.SipsIsotherm(df_Kr,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)")
- Linear interpolation model
The linear interpolation model has an additional, optional argument fill_value that tells us what loading to assume when we attempt to extrapolate beyond the data point with the highest pressure. In this example, we assume that the loading at the highest pressure point is equal to the saturation loading. By default, fill_value=None and an error is thrown when the IAST code tries to extrapolate loading beyond the highest pressure point in the pure-component adsorption isotherm.
xe_isotherm = IAST.InterpolatorIsotherm(df_Xe,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)",
fill_value=df_Xe["Loading(mol/m3)"].max())
kr_isotherm = IAST.InterpolatorIsotherm(df_Kr,
loading_key="Loading(mol/m3)",
pressure_key="Pressure(bar)",
fill_value=df_Kr["Loading(mol/m3)"].max())
Once an isotherm model is constructed, we can view the fit of the isotherm model to the data by:
IAST.plot_isotherm(xe_isotherm)
The plot can be rendered on the log-scale by passing options xlogscale=True and/or ylogscale=False.
Using the adsorption isotherm objects, we can calculate the predicted loading at a new pressure, for example 1.0, by:
xe_isotherm.loading(1.0)
or the reduced spreading pressure via:
xe_isotherm.spreading_pressure(1.0)
Peform IAST calculation¶
Given the pure-component isotherm models, we now illustrate how to use IAST to predict loading when the material is in equilibrium with a mixture of gases.
As an example, given the pure-component adsorption isotherm objects xe_isotherm and kr_isotherm above, we seek the loading [at the same temperature as the pure-component isotherms] in the presence of a 20/80 mol% Xe/Kr mixture at a pressure of 1.0 bar. To do this, we call:
# list/array of partial pressures
# (same units as in pure-component isotherms)
p = [.2, .8]
# Perform IAST calculation. returns component loadings.
q = IAST.IAST(p, [xe_isotherm, kr_isotherm], verboseflag=True)
which will return q, an array of component loadings at these mixture conditions predicted by IAST. Entry 0 will correspond to Xe; entry 1 will correspond to Kr.
Reverse IAST calculation¶
In reverse IAST, we specify the mole fractions of gas in the adsorbed phase and the total bulk gas pressure, then calculate the bulk gas composition that yields these adsorbed mole fractions. This is useful e.g. in catalysis, where one seeks to control the composition of gas adsorbed in the material.
As an example, given the pure-component adsorption isotherm objects xe_isotherm and kr_isotherm above, we seek the bulk gas composition [at the same temperature as the pure-component isotherms] that will yield a 20/80 mol% Xe/Kr mixture in the adsorbed phase at a total bulk gas pressure of 1.0. To do this, we call:
P_total = 1.0
z = [.2, .8] # list/array of desired mole fractions in adsorbed phase
y, q = IAST.reverse_IAST(z, P_total, [xe_isotherm, kr_isotherm])
which will return y, the required bulk gas phase mole fractions, and q, an array of component loadings at these mixture conditions predicted by IAST. Entry 0 will correspond to Xe; entry 1 will correspond to Kr.
Theory¶
Ideal Adsorbed Solution Theory was developed by Myers and Prausnitz:
Myers, A. L., & Prausnitz, J. M. (1965). Thermodynamics of mixed‐gas adsorption. AIChE Journal, 11(1), 121-127.
In our IAST calculations, we follow the method outlined in the more accessible reference:
Tarafder, A. and Mazzotti, M. A method for deriving explicit binary isotherms obeying ideal adsorbed solution theory. Chem. Eng. Technol. 2012, 35, No. 1, 102-108.
Tests¶
This code was tested using pure-component Xe and Kr adsorption isotherms in IRMOF-1 to predict the uptake of Xe and Kr at 1 bar in a variety of Xe mole fractions. The test is displayed in this IPython notebook, and the files for reproduction of this result are in the /test directory.
TL;DR The following plot shows the simulated Xe/Kr adsorption (points) using binary grand-canonical Monte Carlo simulations against the IAST prediction from pure-component adsorption isotherms (solid lines).
Class documentation and details¶
- class isotherms.LangmuirIsotherm(df, loading_key=None, pressure_key=None, K_guess=None, M_guess=None)¶
Langmuir isotherm object to store pure-component adsorption isotherm.
The Langmuir adsorption isotherm is:
\[L(P) = M\frac{KP}{1+KP},\]where \(L\) is the gas uptake, \(P\) is pressure (fugacity), \(M\) is the saturation loading, and \(K\) is the Langmuir constant.
- K = None¶
Langmuir constant K (units: 1 / pressure)
- M = None¶
Saturation loading (units: loading)
- RMSE = None¶
Root mean square error
- __init__(df, loading_key=None, pressure_key=None, K_guess=None, M_guess=None)¶
Instantiation. A LangmuirIsotherm object is instantiated by passing it the pure component adsorption isotherm in the form of a Pandas DataFrame. The least squares data fitting is done here.
Parameters: - df – DataFrame adsorption isotherm data
- loading_key – String key for loading column in df
- pressure_key – String key for pressure column in df
- K_guess – float guess Langmuir constant (units: 1/pressure)
- M_guess – float guess saturation loading (units: loading)
Returns: self
Return type: LangmuirIsotherm
- df = None¶
Pandas DataFrame on which isotherm was fit
- loading(P)¶
Given stored Langmuir parameters, compute loading at pressure P.
Parameters: P – Float or Array pressure (in corresponding units as df in instantiation) Returns: loading at pressure P (in corresponding units as df in instantiation) Return type: Float or Array
- loading_key = None¶
name of loading column
- pressure_key = None¶
name of pressure column
- print_params()¶
Print identified model parameters
- spreading_pressure(P)¶
Calculate reduced spreading pressure at a bulk gas pressure P. (see Tarafder eqn 4)
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: spreading pressure, \(\Pi\) Return type: Float
- class isotherms.QuadraticIsotherm(df, loading_key=None, pressure_key=None, Ka_guess=None, Kb_guess=None, M_guess=None)¶
Quadratic isotherm object to store pure-component isotherm.
The Quadratic adsorption isotherm is:
\[L(P) = M \frac{(K_a + 2 K_b P)P}{1+K_aP+K_bP^2}\]where \(L\) is the gas uptake, \(P\) is pressure (fugacity), \(M\) is half of the saturation loading, and constants \(K_a\) and \(K_b\) are model coefficients.
- Ka = None¶
isotherm coefficient (units: 1 / pressure)
- Kb = None¶
isotherm coefficient (units: 1 / pressure2)
- M = None¶
Half of saturation loading (units: loading)
- RMSE = None¶
Root mean square error
- __init__(df, loading_key=None, pressure_key=None, Ka_guess=None, Kb_guess=None, M_guess=None)¶
Instantiation. A QuadraticIsotherm object is instantiated by passing it the pure component adsorption isotherm in the form of a Pandas DataFrame. The least squares data fitting is done here.
Parameters: - df – DataFrame adsorption isotherm data
- loading_key – String key for loading column in df
- pressure_key – String key for pressure column in df
- Ka_guess – float guess for Quadratic isotherm constant (units: 1/pressure)
- Kb_guess – float guess for Quadratic isotherm constant (units: 1/pressure^2)
- M_guess – float guess for saturation loading (units: loading)
Returns: self
Return type: QuadraticIsotherm
- df = None¶
Pandas DataFrame on which isotherm was fit
- loading(P)¶
Given stored Quadratic isotherm parameters, compute loading at pressure P.
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: loading at pressure P (in corresponding units as df in instantiation) Return type: Float or Array
- loading_key = None¶
name of loading column
- pressure_key = None¶
name of pressure column
- print_params()¶
Print identified model parameters
- spreading_pressure(P)¶
Calculate reduced spreading pressure at a bulk gas pressure P. (see Tarafder eqn 4)
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: spreading pressure, \(\Pi\) Return type: Float
- class isotherms.BETIsotherm(df, loading_key=None, pressure_key=None, Ka_guess=None, Kb_guess=None, M_guess=None)¶
BET isotherm object to store pure-component adsorption isotherm.
The BET adsorption isotherm, a multi-layer model, is:
\[L(P) = M\frac{K_A P}{(1-K_B P)(1-K_B P+ K_A P)},\]where \(L\) is the gas uptake, \(P\) is pressure (fugacity), \(M\) is the number of adsorption sites on the bare surface, \(K_A\) is the Langmuir constant for the bare surface, and \(K_B\) is the Langmuir constant for the second and higher layers.
- Ka = None¶
Langmuir constant of first layer (units: 1 / pressure)
- Kb = None¶
Langmuir constant of second and higher layers (units: 1 / pressure)
- M = None¶
Saturation loading of first layer (units: loading)
- RMSE = None¶
Root mean square error
- __init__(df, loading_key=None, pressure_key=None, Ka_guess=None, Kb_guess=None, M_guess=None)¶
Instantiation. A BETIsotherm object is instantiated by passing it the pure component adsorption isotherm in the form of a Pandas DataFrame. The least squares data fitting is done here.
Parameters: - df – DataFrame adsorption isotherm data
- loading_key – String key for loading column in df
- pressure_key – String key for pressure column in df
Returns: self
Return type: BETIsotherm
- df = None¶
Pandas DataFrame on which isotherm was fit
- loading(P)¶
Given stored Langmuir parameters, compute loading at pressure P.
Parameters: P – Float or Array pressure (in corresponding units as df in instantiation) Returns: loading at pressure P (in corresponding units as df in instantiation) Return type: Float or Array
- loading_key = None¶
name of loading column
- pressure_key = None¶
name of pressure column
- print_params()¶
Print identified model parameters
- spreading_pressure(P)¶
Calculate reduced spreading pressure at a bulk gas pressure P. (see Tarafder eqn 4)
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: spreading pressure, \(\Pi\) Return type: Float
- class isotherms.SipsIsotherm(df, loading_key=None, pressure_key=None, K_guess=None, M_guess=None, n_guess=None)¶
Sips isotherm object to store pure-component adsorption isotherm.
The Sips adsorption isotherm is:
\[L(P) = M\frac{K^nP^n}{1+K^nP^n},\]where \(L\) is the gas uptake, \(P\) is pressure (fugacity), \(M\) is the saturation loading, \(K\) is the equilibrium constant, and \(n \in (0,1]\) is the index of heterogeneity.
- K = None¶
Langmuir constant K (units: 1 / pressure)
- M = None¶
Saturation loading (units: loading)
- RMSE = None¶
Root mean square error
- __init__(df, loading_key=None, pressure_key=None, K_guess=None, M_guess=None, n_guess=None)¶
Instantiation. A SipsIsotherm object is instantiated by passing it the pure component adsorption isotherm in the form of a Pandas DataFrame. The least squares data fitting is done here.
Parameters: - df – DataFrame adsorption isotherm data
- loading_key – String key for loading column in df
- pressure_key – String key for pressure column in df
- K_guess – float guess Langmuir constant (units: 1/pressure)
- M_guess – float guess saturation loading (units: loading)
- n_guess – float guess for index of heterogeneity
Returns: self
Return type: SipsIsotherm
- df = None¶
Pandas DataFrame on which isotherm was fit
- loading(P)¶
Given stored Sips parameters, compute loading at pressure P.
Parameters: P – Float or Array pressure (in corresponding units as df in instantiation) Returns: loading at pressure P (in corresponding units as df in instantiation) Return type: Float or Array
- loading_key = None¶
name of loading column
- n = None¶
index of heterogeneity (unitless)
- pressure_key = None¶
name of pressure column
- print_params()¶
Print identified model parameters
- spreading_pressure(P)¶
Calculate reduced spreading pressure at a bulk gas pressure P. (see Tarafder eqn 4)
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: spreading pressure, \(\Pi\) Return type: Float
- class isotherms.InterpolatorIsotherm(df, loading_key=None, pressure_key=None, fill_value=None)¶
Interpolator isotherm object to store pure-component adsorption isotherm.
Here, the isotherm is characterized by linear interpolation of data.
Loading = 0.0 at pressure = 0.0 is enforced here automatically for interpolation at low pressures.
Default for extrapolating isotherm beyond highest pressure in available data is to throw an exception. Pass a value for fill_value in instantiation to extrapolate loading as fill_value.
- __init__(df, loading_key=None, pressure_key=None, fill_value=None)¶
Instantiation. InterpolatorIsotherm is instantiated by passing it the pure component adsorption isotherm in the form of a Pandas DataFrame. Contructs linear interpolator from interp1d function in Scipy during instantiation.
e.g. to extrapolate loading beyond highest pressure point as 100.0, pass fill_value=100.0.
Parameters: - df – DataFrame adsorption isotherm data
- loading_key – String key for loading column in df
- pressure_key – String key for pressure column in df
- fill_value – Float value of loading to assume when an attempt is made to interpolate at a pressure greater than the largest pressure observed in the data
Returns: self
Return type: InterpolatorIsotherm
- df = None¶
Pandas DataFrame on which isotherm was fit
- loading(P)¶
Interpolate isotherm to compute loading at pressure P.
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: loading at pressure P (in corresponding units as df in instantiation) Return type: Float or Array
- loading_key = None¶
name of loading column
- pressure_key = None¶
name of pressure column
- spreading_pressure(P)¶
Calculate reduced spreading pressure at a bulk gas pressure P. (see Tarafder eqn 4)
Use trapezoid rule on isotherm data points to compute the reduced spreading pressure via the integral:
\[\Pi(p) = \int_0^p \frac{q(\hat{p})}{ \hat{p}} d\hat{p}\]Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: spreading pressure, \(\Pi\) Return type: Float
- isotherms.plot_isotherm(isotherm, withfit=True, xlogscale=False, ylogscale=False)¶
Plot isotherm data and fit using Matplotlib.
Parameters: - isotherm – LangmuirIsotherm,QuadraticIsotherm,InterpolatorIsotherm the adsorption isotherm object
- withfit – Bool plot fit as well
- xlogscale – Bool log-scale on x-axis
- ylogscale – Bool log-scale on y-axis
- class isotherms.SplineFitIsotherm(df, loading_key=None, pressure_key=None, fill_value=None, s=None, k=3)¶
Piecewise cubic splines are fit to the data. We use the scipy.interpolate.UnivariateSpline class
Loading = 0.0 at pressure = 0.0 is enforced here automatically for interpolation at low pressures.
Default for extrapolating isotherm beyond highest pressure is to fail. Pass a value for fill_value in instantiation to extrapolate loading as fill_value.
- UnivariateSpline = None¶
UnivariateSpline class from Scipy
- __init__(df, loading_key=None, pressure_key=None, fill_value=None, s=None, k=3)¶
Instantiation. InterpolatorIsotherm is instantiated by passing it the pure component adsorption isotherm in the form of a Pandas DataFrame. Contructs linear interpolator from interp1d function in Scipy during instantiation.
e.g. to extrapolate loading beyond highest pressure point as 100.0, pass fill_value=100.0.
Parameters: - df – DataFrame adsorption isotherm data
- loading_key – String key for loading column in df
- pressure_key – String key for pressure column in df
- s – Float smoothing parameter for determining the number of knots. See scipy.interpolate.UnivariateSpline
- k – Int degree of spline. See scipy.interpolate.UnivariateSpline
- fill_value – Float value of loading to assume when an attempt is made to interpolate at a pressure greater than the largest pressure observed in the data
Returns: self
Return type: InterpolatorIsotherm
- df = None¶
Pandas DataFrame on which isotherm was fit
- loading(P)¶
Interpolate isotherm to compute loading at pressure P.
Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: loading at pressure P (in corresponding units as df in instantiation) Return type: Float or Array
- loading_key = None¶
name of loading column
- pressure_key = None¶
name of pressure column
- spreading_pressure(P)¶
Calculate reduced spreading pressure at a bulk gas pressure P. (see Tarafder eqn 4)
Use trapezoid rule on isotherm data points to compute the reduced spreading pressure via the integral:
\[\Pi(p) = \int_0^p \frac{q(\hat{p})}{ \hat{p}} d\hat{p}\]Parameters: P – float pressure (in corresponding units as df in instantiation) Returns: spreading pressure, \(\Pi\) Return type: Float
- isotherms.plot_isotherm(isotherm, withfit=True, xlogscale=False, ylogscale=False)
Plot isotherm data and fit using Matplotlib.
Parameters: - isotherm – LangmuirIsotherm,QuadraticIsotherm,InterpolatorIsotherm the adsorption isotherm object
- withfit – Bool plot fit as well
- xlogscale – Bool log-scale on x-axis
- ylogscale – Bool log-scale on y-axis
- IAST.IAST(p, isotherms, verboseflag=False, warningoff=False)¶
Perform IAST calculation to predict multi-component adsorption isotherm from pure component adsorption isotherms.
The material is now in equilibrium with a mixture of gases with partial pressures in the array p.
Pass a list of pure-component adsorption isotherms isotherms.
Parameters: - p – Array or list partial pressures of gas components, e.g. [.5, .5]
- isotherms – list pure-component adsorption isotherms. e.g. [xe_isotherm, kr_isotherm]
- verboseflag – Bool print stuff
- warningoff – Bool when False, warnings will print when the IAST calculation result required extrapolation of the pure-component adsorption isotherm beyond the highest pressure in the data
Returns: q: predicted uptakes of each component
Return type: Array
- IAST.reverse_IAST(z, P_total, isotherms, verboseflag=False)¶
Perform reverse IAST to predict gas phase composition at total pressure P_total that will yield adsorbed mole fractions z.
Pass a list of pure-component adsorption isotherms isotherms.
Parameters: - z – Array desired adsorbed mole fractions, e.g. [.5, .5]
- P_total – Float total bulk gas pressure
- isotherms – list of pure-component adsorption isotherms. e.g. [xe_isotherm, kr_isotherm]
- verboseflag – Bool print stuff
Returns: y, q: bulk gas mole fractions that yield desired adsorbed mole fractions z at P_total, component loadings
Return type: Array, Array


Toggle navigation
Bob's Quest
About
This sample app is developed by
hwclass
for challengy aims. You can use it what type app you are working on. Keep calm ;)
Founders List
Photo
Founder
Location
Country
City
Postal Code
Address
Home Page
Garage Latitude
Garage Longitude
Show All
Latest Update :


Class: Seahorse::Client::Http::Headers
| Inherits: |
Object
|
| Includes: |
Enumerable
|
| Defined in: |
Overview
Provides a Hash-like interface for HTTP headers. Header names are treated indifferently as lower-cased strings. Header values are cast to strings.
headers = Http::Headers.new
headers['Content-Length'] = 100
headers[:Authorization] = 'Abc'
headers.keys
#=> ['content-length', 'authorization']
headers.values
#=> ['100', 'Abc']
You can get the header values as a vanilla hash by calling #to_h:
headers.to_h
#=> { 'content-length' => '100', 'authorization' => 'Abc' }
Instance Method Summary collapse
-
#[](key) ⇒ String
-
#[]=(key, value)
-
#clear
-
#delete(key)
-
#each {|key, value| ... } ⇒ nil
(also: #each_pair)
-
#each_pair {|key, value| ... } ⇒ nil
Alias for
Seahorse::Client::.
-
#has_key?(key) ⇒ Boolean
Alias for
Seahorse::Client::.
-
#include?(key) ⇒ Boolean
Alias for
Seahorse::Client::.
-
#key?(key) ⇒ Boolean
(also: #has_key?, #include?)
Returns
true
if the header is set. -
#keys ⇒ Array<String>
-
#to_h ⇒ Hash
Alias for
Seahorse::Client::.
-
#to_hash ⇒ Hash
(also: #to_h)
-
#update(headers) ⇒ Headers
-
#values ⇒ Array<String>
-
#values_at(*keys) ⇒ Array<String>
Constructor Details
.new(headers = {}) ⇒ Headers
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
29 30 31 32 33 34
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 29
def initialize(headers = {}) @data = {} headers.each_pair do |key, value| self[key] = value end end
Instance Method Details
#[](key) ⇒ String
38 39 40
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 38
def [](key) @data[key.to_s.downcase] end
#[]=(key, value)
44 45 46
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 44
def []=(key, value) @data[key.to_s.downcase] = value.to_s end
#clear
[ GitHub ]62 63 64
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 62
def clear @data = {} end
#delete(key)
58 59 60
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 58
def delete(key) @data.delete(key.to_s.downcase) end
#each {|key, value| ... } ⇒ nil Also known as: each_pair
85 86 87 88 89 90 91 92 93 94
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 85
def each(&block) if block_given? @data.each_pair do |key, value| yield(key, value) end nil else @data.enum_for(:each) end end
#each_pair {|key, value| ... } ⇒ nil
Alias for Seahorse::Client::.
95
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 95
alias each_pair each
#has_key?(key) ⇒ Boolean
Alias for Seahorse::Client::.
101
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 101
alias has_key? key?
#include?(key) ⇒ Boolean
Alias for Seahorse::Client::.
102
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 102
alias include? key?
#key?(key) ⇒ Boolean Also known as: has_key?, include?
Returns true
if the header is set.
98 99 100
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 98
def key?(key) @data.key?(key.to_s.downcase) end
#keys ⇒ Array<String>
67 68 69
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 67
def keys @data.keys end
#to_h ⇒ Hash
Alias for Seahorse::Client::.
108
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 108
alias to_h to_hash
#to_hash ⇒ Hash Also known as: to_h
105 106 107
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 105
def to_hash @data.dup end
#update(headers) ⇒ Headers
50 51 52 53 54 55
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 50
def update(headers) headers.each_pair do |k, v| self[k] = v end self end
#values ⇒ Array<String>
72 73 74
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 72
def values @data.values end
#values_at(*keys) ⇒ Array<String>
77 78 79
# File 'aws-sdk-core/lib/seahorse/client/http/headers.rb', line 77
def values_at(*keys) @data.values_at(*keys.map{ |key| key.to_s.downcase }) end


|
|
|
|
Dear evaluee,
before starting with your self-evaluation, you have the option to define who should give feedback to you. Enter name, email address and role of the people from who you wish to get feedback from to the table below. The "role" of a person defines the relationship the person has towards you.
In your feedback report you will see the results by role. Please take into consideration that there need to be answers of at least 3 colleagues, 3 subordinates, 3 customers and 3 others to be shown separately in the report. If there are answers from less than 3 respondents from these roles, the results will only be shown in the group "all".
Already nominated respondents (if any) are listed above the table.
Your respondents:
[]
| First name | Last name | Email | Role |
Push "Continue later", if you want to continue entering respondents at a later point of time.
Push "Finish nomination" to stop nominating respondents and to start filling in your self
evaluation. If you wish to enter further respondents at a later stage, you can return to this
page by using the "back" buttons within the survey form, or on the last page of the survey form
after you finished your self evaluation.
As soon as you push any of the buttons below, invitation messages will be sent to the
respondents that you have entered.
|
|
|
|
|
|
|
|


--- layout: post title: ! 'hrm, native instruments Komplete updater d/ls all of it''s installer files onto my desktop. #yesipaidover1000forthis' published: true ---


Data not transferred to BGarray. Run {{filename}}
Remove {{run_file}} before datatransfer can be restarted


|
Proton DotNet
|
Represents an AMQP Connection which is owned by a single engine instance More...
Public Member Functions | |
| T | Open () |
| Open the end point locally, sending the Open performative immediately if possible or holding it until SASL negotiations or the AMQP header exchange and other required performative exchanges has completed. The endpoint will signal any registered handler of the remote opening the endpoint once the remote performative that signals open completion arrives. More... | |
| T | Close () |
| Close the end point locally and send the closing performative immediately if possible or holds it until the Connection / Engine state allows it. If the engine encounters an error writing the performative or the engine is in a failed state from a previous error then this method will throw an exception. If the engine has been shutdown then this method will close out the local end of the endpoint and clean up any local resources before returning normally. More... | |
| T | LocalOpenHandler (Action< T > localOpenHandler) |
| Sets a Action for when an this endpoint is opened locally via a call to Open. Typically used by clients for logging or other state update event processing. Clients should not perform any blocking calls within this context. It is an error for the handler to throw an exception and the outcome of doing so is undefined. Typically used by clients as servers will typically listen to some parent resource event handler to determine if the remote is initiating a resource open. More... | |
| T | LocalCloseHandler (Action< T > localCloseHandler) |
| Sets a Action for when an this endpoint is closed locally via a call to Close. Typically used by clients for logging or other state update event processing. Clients should not perform any blocking calls within this context. It is an error for the handler to throw an exception and the outcome of doing so is undefined. More... | |
| T | OpenHandler (Action< T > localOpenHandler) |
| Sets a Action for when an AMQP Open frame is received from the remote peer. Used to process remotely initiated Connections. Locally initiated sessions have their own Action invoked instead. This method is typically used by servers to listen for the remote peer to open its endpoint, while a client would listen for the server to open its end of the endpoint once a local open has been performed. Typically used by clients as servers will typically listen to some parent resource event handler to determine if the remote is initiating a resource open. More... | |
| T | CloseHandler (Action< T > localCloseHandler) |
| Sets a EventHandler for when an AMQP closing frame is received from the remote peer. More... | |
| T | EngineShutdownHandler (Action< IEngine > shutdownHandler) |
| Sets an Action that is invoked when the engine that supports this endpoint is shutdown which indicates a desire to terminate all engine operations. Any endpoint that has been both locally and remotely closed will not receive this event as it will no longer be tracked by the parent its parent endpoint. A typical use of this event would be from a locally closed endpoint that is awaiting response from the remote. If this event fires then there will never be a remote response to any pending operations and the client or server instance should react accordingly to clean up any related resources etc. More... | |
Properties | |
| IEngine | Engine [get] |
| Provides access to the engine instance that owns the resources of this endpoint and its parent. More... | |
| IAttachments | Attachments [get] |
| Access the attachments instance that is associated with this resource where the application can store data relevant to the endpoint for later use. More... | |
| object | LinkedResource [get, set] |
| Allows the endpoint to have some user defined resource linked to it which can be used to store application state data or other associated object instances with this endpoint. More... | |
| ErrorCondition | ErrorCondition [get, set] |
| Provides access to the error condition that should be applied to the AMQP frame that closes or ends this endpoint when the close method is called be the user. Setting this value after closing the endpoint has no effect. More... | |
| ErrorCondition | RemoteErrorCondition [get] |
| If the remote has closed this endpoint and provided an ErrorCondition as part of the closing AMQP performative then this method will return it. More... | |
| bool | IsLocallyOpen [get] |
| Returns true if the endpoint open was previously called and the close method has not yet been invoked. This only reflects the state on the local end and the user should also check the remote state. More... | |
| bool | IsLocallyClosed [get] |
| Returns true if the endpoint having been previously locally opened is now closed due to a call the the close method. This does not reflect the state of the remote endpoint and that should be checked separately. More... | |
| bool | IsRemotelyOpen [get] |
| Returns true if this endpoint is currently remotely open meaning that the AMQP performative that completes the open phase of this endpoint's lifetime has arrived but the performative that closes it has not. More... | |
| bool | IsRemotelyClosed [get] |
| Returns true if this endpoint is currently remotely closed meaning that the AMQP performative that completes the close phase of this endpoint's lifetime has arrived. More... | |
| Symbol[] | OfferedCapabilities [get, set] |
| Access the capabilities to be offered on to the remote when this endpoint is opened. The offered capabilities value can only be modified prior to a call to open, once the endpoint has been opened locally an error will be thrown if this method is called. More... | |
| Symbol[] | DesiredCapabilities [get, set] |
| Access the capabilities that are desired on to the remote when this endpoint is opened. The desired capabilities value can only be modified prior to a call to open, once the endpoint has been opened locally an error will be thrown if this method is called. More... | |
| IReadOnlyDictionary< Symbol, object > | Properties [get, set] |
| Access the properties that are conveyed to the remote when this endpoint is opened. The properties value can only be modified prior to a call to open, once the endpoint has been opened locally an error will be thrown if this method is called. More... | |
| Symbol[] | RemoteOfferedCapabilities [get] |
| The capabilities offered by the remote when it opened its end of the endpoint. More... | |
| Symbol[] | RemoteDesiredCapabilities [get] |
| The capabilities desired by the remote when it opened its end of the endpoint. More... | |
| IReadOnlyDictionary< Symbol, object > | RemoteProperties [get] |
| The properties sent by the remote when it opened its end of this endpoint. More... | |
Represents an AMQP Connection which is owned by a single engine instance
| T | : | IEndpoint<T> |
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.Close | ( | ) |
Close the end point locally and send the closing performative immediately if possible or holds it until the Connection / Engine state allows it. If the engine encounters an error writing the performative or the engine is in a failed state from a previous error then this method will throw an exception. If the engine has been shutdown then this method will close out the local end of the endpoint and clean up any local resources before returning normally.
| EngineStateException | If the engine state is already failed or shutdown |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >, and Apache.Qpid.Proton.Engine.Implementation.ProtonLink< T >.
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.CloseHandler | ( | Action< T > | localCloseHandler | ) |
Sets a EventHandler for when an AMQP closing frame is received from the remote peer.
| localCloseHandler | The handler to invoke when the event occurs |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >.
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.EngineShutdownHandler | ( | Action< IEngine > | shutdownHandler | ) |
Sets an Action that is invoked when the engine that supports this endpoint is shutdown which indicates a desire to terminate all engine operations. Any endpoint that has been both locally and remotely closed will not receive this event as it will no longer be tracked by the parent its parent endpoint. A typical use of this event would be from a locally closed endpoint that is awaiting response from the remote. If this event fires then there will never be a remote response to any pending operations and the client or server instance should react accordingly to clean up any related resources etc.
| shutdownHandler | The handler to invoke when the event occurs |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >.
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.LocalCloseHandler | ( | Action< T > | localCloseHandler | ) |
Sets a Action for when an this endpoint is closed locally via a call to Close. Typically used by clients for logging or other state update event processing. Clients should not perform any blocking calls within this context. It is an error for the handler to throw an exception and the outcome of doing so is undefined.
| localCloseHandler | The handler to invoke when the event occurs |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >.
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.LocalOpenHandler | ( | Action< T > | localOpenHandler | ) |
Sets a Action for when an this endpoint is opened locally via a call to Open. Typically used by clients for logging or other state update event processing. Clients should not perform any blocking calls within this context. It is an error for the handler to throw an exception and the outcome of doing so is undefined. Typically used by clients as servers will typically listen to some parent resource event handler to determine if the remote is initiating a resource open.
| localOpenHandler | The handler to invoke when the event occurs |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >.
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.Open | ( | ) |
Open the end point locally, sending the Open performative immediately if possible or holding it until SASL negotiations or the AMQP header exchange and other required performative exchanges has completed. The endpoint will signal any registered handler of the remote opening the endpoint once the remote performative that signals open completion arrives.
| EngineStateException | If the engine state is already failed or shutdown |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >, and Apache.Qpid.Proton.Engine.Implementation.ProtonLink< T >.
| T Apache.Qpid.Proton.Engine.IEndpoint< T >.OpenHandler | ( | Action< T > | localOpenHandler | ) |
Sets a Action for when an AMQP Open frame is received from the remote peer. Used to process remotely initiated Connections. Locally initiated sessions have their own Action invoked instead. This method is typically used by servers to listen for the remote peer to open its endpoint, while a client would listen for the server to open its end of the endpoint once a local open has been performed. Typically used by clients as servers will typically listen to some parent resource event handler to determine if the remote is initiating a resource open.
| localOpenHandler | The handler to invoke when the event occurs |
Implemented in Apache.Qpid.Proton.Engine.Implementation.ProtonEndpoint< T >.
|
get |
Access the attachments instance that is associated with this resource where the application can store data relevant to the endpoint for later use.
|
getset |
Access the capabilities that are desired on to the remote when this endpoint is opened. The desired capabilities value can only be modified prior to a call to open, once the endpoint has been opened locally an error will be thrown if this method is called.
|
get |
Provides access to the engine instance that owns the resources of this endpoint and its parent.
Provides access to the error condition that should be applied to the AMQP frame that closes or ends this endpoint when the close method is called be the user. Setting this value after closing the endpoint has no effect.
|
get |
Returns true if the endpoint having been previously locally opened is now closed due to a call the the close method. This does not reflect the state of the remote endpoint and that should be checked separately.
|
get |
Returns true if the endpoint open was previously called and the close method has not yet been invoked. This only reflects the state on the local end and the user should also check the remote state.
|
get |
Returns true if this endpoint is currently remotely closed meaning that the AMQP performative that completes the close phase of this endpoint's lifetime has arrived.
|
get |
Returns true if this endpoint is currently remotely open meaning that the AMQP performative that completes the open phase of this endpoint's lifetime has arrived but the performative that closes it has not.
|
getset |
Allows the endpoint to have some user defined resource linked to it which can be used to store application state data or other associated object instances with this endpoint.
|
getset |
Access the capabilities to be offered on to the remote when this endpoint is opened. The offered capabilities value can only be modified prior to a call to open, once the endpoint has been opened locally an error will be thrown if this method is called.
|
getset |
Access the properties that are conveyed to the remote when this endpoint is opened. The properties value can only be modified prior to a call to open, once the endpoint has been opened locally an error will be thrown if this method is called.
|
get |
The capabilities desired by the remote when it opened its end of the endpoint.
|
get |
If the remote has closed this endpoint and provided an ErrorCondition as part of the closing AMQP performative then this method will return it.
|
get |
The capabilities offered by the remote when it opened its end of the endpoint.
|
get |
The properties sent by the remote when it opened its end of this endpoint.


Available Towers |
|
Selected Towers |
Note: If you are on a mobile device, you may need to tap on the lists to see the provider names.


The Avalon Component model is made up of a combination of classic interfaces that define the artifacts exchanged between a container and a component. The requirements presented by a component to container are expressed using meta-info descriptors that are colocated with component classes. Interfaces and default implementations are described under Avalon Framework package. The meta-info descriptors and related tools are included under the Avalon Meta package.
| Product | Description |
|---|---|
|
The Avalon Framework API and Implementation consists of interfaces that define relationships between commonly used application components, best-of-practice pattern enforcements, and several lightweight convenience implementations of the generic components. |
|
|
The Avalon Meta Model defines the functional criteria for component types and services. A Type definition contains information about deployment and runtime dependencies together with information about the services that a component type can provide. |
|
|
A set of utilities shared by framework components supporting default property management, environment variable resolution, exceptio reporting and a parameterized map implementation (using within the repositroy package). |
|
|
A joint initative by the Avalon Project and the Apache Directory Project in the development of an inteligent bootstrappig, resource replication, and artifact management framework. |
|
| Avalon Logkit |
Logkit is a complete logging framework (like log4j) as an Avalon component. Most Avalon containers use it as the default logging implementation. |


If you're new to Mesos
See the getting started page for more information about downloading, building, and deploying Mesos.
If you'd like to get involved or you're looking for support
See our community page for more details.
Mesos Fetcher
Mesos 0.23.0 introduced experimental support for the Mesos fetcher cache.
In this context we loosely regard the term “downloading” as to include copying from local file systems.
What is the Mesos fetcher?
The Mesos fetcher is a mechanism to download resources into the sandbox
directory of a task in preparation of running
the task. As part of a TaskInfo message, the framework ordering the task’s
execution provides a list of CommandInfo::URI
protobuf values, which becomes
the input to the Mesos fetcher.
The Mesos fetcher can copy files from a local filesytem and it also natively supports the HTTP, HTTPS, FTP and FTPS protocols. If the requested URI is based on some other protocol, then the fetcher tries to utilise a local Hadoop client and hence supports any protocol supported by the Hadoop client, e.g., HDFS, S3. See the agent configuration documentation for how to configure the agent with a path to the Hadoop client.
By default, each requested URI is downloaded directly into the sandbox directory and repeated requests for the same URI leads to downloading another copy of the same resource. Alternatively, the fetcher can be instructed to cache URI downloads in a dedicated directory for reuse by subsequent downloads.
The Mesos fetcher mechanism comprises of these two parts:
The agent-internal Fetcher Process (in terms of libprocess) that controls and coordinates all fetch actions. Every agent instance has exactly one internal fetcher instance that is used by every kind of containerizer.
The external program
mesos-fetcher
that is invoked by the former. It performs all network and disk operations except file deletions and file size queries for cache-internal bookkeeping. It is run as an external OS process in order to shield the agent process from I/O-related hazards. It takes instructions in form of an environment variable containing a JSON object with detailed fetch action descriptions.
The fetch procedure
Frameworks launch tasks by calling the scheduler driver method launchTasks()
,
passing CommandInfo
protobuf structures as arguments. This type of structure
specifies (among other things) a command and a list of URIs that need to be
“fetched” into the sandbox directory on the agent node as a precondition for
task execution. Hence, when the agent receives a request to launch a task, it
calls upon its fetcher, first, to provision the specified resources into the
sandbox directory. If fetching fails, the task is not started and the reported
task status is TASK_FAILED
.
All URIs requested for a given task are fetched sequentially in a single invocation of mesos-fetcher. Here, avoiding download concurrency reduces the risk of bandwidth issues somewhat. However, multiple fetch operations can be active concurrently due to multiple task launch requests.
The URI protobuf structure
Before mesos-fetcher is started, the specific fetch actions to be performed for
each URI are determined based on the following protobuf structure. (See
include/mesos/mesos.proto
for more details.)
message CommandInfo {
message URI {
required string value = 1;
optional bool executable = 2;
optional bool extract = 3 [default = true];
optional bool cache = 4;
optional string filename = 5;
}
...
optional string user = 5;
}
The field “value” contains the URI.
If the “executable” field is “true”, the “extract” field is ignored and has no effect.
If the “cache” field is true, the fetcher cache is to be used for the URI.
If the “output_file” field is set, the fetcher will use that name for the copy stored in the sandbox directory. “output_file” may contain a directory component, in which case the path described must be a relative path.
Specifying a user name
The framework may pass along a user name that becomes a fetch parameter. This causes its executors and tasks to run under a specific user. However, if the “user” field in the CommandInfo structure is specified, it takes precedence for the affected task.
If a user name is specified either way, the fetcher first validates that it is
in fact a valid user name on the agent. If it is not, fetching fails right here.
Otherwise, the sandbox directory is assigned to the specified user as owner
(using chown
) at the end of the fetch procedure, before task execution begins.
The user name in play has an important effect on caching. Caching is managed on a per-user base, i.e. the combination of user name and “uri” uniquely identifies a cacheable fetch result. If no user name has been specified, this counts for the cache as a separate user, too. Thus cache files for each valid user are segregated from all others, including those without a specified user.
This means that the exact same URI will be downloaded and cached multiple times if different users are indicated.
Executable fetch results
By default, fetched files are not executable.
If the field “executable” is set to “true”, the fetch result will be changed to be executable (by “chmod”) for every user. This happens at the end of the fetch procedure, in the sandbox directory only. It does not affect any cache file.
Archive extraction
If the “extract” field is “true”, which is the default, then files with a recognized extension that hints at packed or compressed archives are unpacked in the sandbox directory. These file extensions are recognized:
- .tar, .tar.gz, .tar.bz2, .tar.xz
- .gz, .tgz, .tbz2, .txz, .zip
In case the cache is bypassed, both the archive and the unpacked results will be found together in the sandbox. In case a cache file is unpacked, only the extraction result will be found in the sandbox.
The “output_file” field is useful here for cases where the URI ends with query parameters, since these will otherwise end up in the file copied to the sandbox and will subsequently fail to be recognized as archives.
Bypassing the cache
By default, the URI field “cache” is not present. If this is the case or its value is “false” the fetcher downloads directly into the sandbox directory.
The same also happens dynamically as a fallback strategy if anything goes wrong when preparing a fetch operation that involves the cache. In this case, a warning message is logged. Possible fallback conditions are:
- The server offering the URI does not respond or reports an error.
- The URI’s download size could not be determined.
- There is not enough space in the cache, even after attempting to evict files.
Fetching through the cache
If the URI’s “cache” field has the value “true”, then the fetcher cache is in effect. If a URI is encountered for the first time (for the same user), it is first downloaded into the cache, then copied to the sandbox directory from there. If the same URI is encountered again, and a corresponding cache file is resident in the cache or still en route into the cache, then downloading is omitted and the fetcher proceeds directly to copying from the cache. Competing requests for the same URI simply wait upon completion of the first request that occurs. Thus every URI is downloaded at most once (per user) as long as it is cached.
Every cache file stays resident for an unspecified amount of time and can be removed at the fetcher’s discretion at any moment, except while it is in direct use:
- It is still being downloaded by this fetch procedure.
- It is still being downloaded by a concurrent fetch procedure for a different task.
- It is being copied or extracted from the cache.
Once a cache file has been removed, the related URI will thereafter be treated as described above for the first encounter.
Unfortunately, there is no mechanism to refresh a cache entry in the current experimental version of the fetcher cache. A future feature may force updates based on checksum queries to the URI.
Recommended practice for now:
The framework should start using a fresh unique URI whenever the resource’s content has changed.
Determining resource sizes
Before downloading a resource to the cache, the fetcher first determines the size of the expected resource. It uses these methods depending on the nature of the URI.
- Local file sizes are probed with systems calls (that follow symbolic links).
- HTTP/HTTPS URIs are queried for the “content-length” field in the header. This
is performed by
curl
. The reported asset size must be greater than zero or the URI is deemed invalid. - FTP/FTPS is not supported at the time of writing.
- Everything else is queried by the local HDFS client.
If any of this reports an error, the fetcher then falls back on bypassing the cache as described above.
WARNING: Only URIs for which download sizes can be queried up front and for which accurate sizes are reported reliably are eligible for any fetcher cache involvement. If actual cache file sizes exceed the physical capacity of the cache directory in any way, all further agent behavior is completely unspecified. Do not use any cache feature with any URI for which you have any doubts!
To mitigate this problem, cache files that have been found to be larger than expected are deleted immediately after downloading and delivering the requested content to the sandbox. Thus exceeding total capacity at least does not accumulate over subsequent fetcher runs.
If you know for sure that size aberrations are within certain limits you can specify a cache directory size that is sufficiently smaller than your actual physical volume and fetching should work.
In case of cache files that are smaller then expected, the cache will dynamically adjust its own bookkeeping according to actual sizes.
Cache eviction
After determining the prospective size of a cache file and before downloading it, the cache attempts to ensure that at least as much space as is needed for this file is available and can be written into. If this is immediately the case, the requested amount of space is simply marked as reserved. Otherwise, missing space is freed up by “cache eviction”. This means that the cache removes files at its own discretion until the given space target is met or exceeded.
The eviction process fails if too many files are in use and therefore not evictable or if the cache is simply too small. Either way, the fetcher then falls back on bypassing the cache for the given URI as described above.
If multiple evictions happen concurrently, each of them is pursuing its own separate space goals. However, leftover freed up space from one effort is automatically awarded to others.
HTTP and SOCKS proxy settings
Sometimes it is desirable to use a proxy to download the file. The Mesos fetcher uses libcurl internally for downloading content from HTTP/HTTPS/FTP/FTPS servers, and libcurl can use a proxy automatically if certain environment variables are set.
The respective environment variable name is [protocol]_proxy
, where
protocol
can be one of socks4, socks5, http, https.
For example, the value of the http_proxy
environment variable would be used
as the proxy for fetching http contents, while https_proxy
would be used for
fetching https contents. Pay attention that these variable names must be
entirely in lower case.
The value of the proxy variable is of the format
[protocol://][user:password@]machine[:port]
, where protocol
can be one of
socks4, socks5, http, https.
FTP/FTPS requests with a proxy also make use of an HTTP/HTTPS proxy. Even though in general this constrains the available FTP protocol operations, everything the fetcher uses is supported.
Your proxy settings can be placed in /etc/default/mesos-slave
. Here is an
example:
export http_proxy=https://proxy.example.com:3128
export https_proxy=https://proxy.example.com:3128
The fetcher will pick up these environment variable settings since the utility
program mesos-fetcher
which it employs is a child of mesos-agent.
For more details, please check the libcurl manual.
Agent flags
It is highly recommended to set these flags explicitly to values other than their defaults or to not use the fetcher cache in production.
- “fetcher_cache_size”, default value: enough for testing.
- “fetcher_cache_dir”, default value: somewhere inside the directory specified by the “work_dir” flag, which is OK for testing.
Recommended practice:
- Use a separate volume as fetcher cache. Do not specify a directory as fetcher cache directory that competes with any other contributor for the underlying volume’s space.
- Set the cache directory size flag of the agent to less than your actual cache volume’s physical size. Use a safety margin, especially if you do not know for sure if all frameworks are going to be compliant.
Ultimate remedy:
You can disable the fetcher cache entirely on each agent by setting its “fetcher_cache_size” flag to zero bytes.
Future Features
The following features would be relatively easy to implement additionally.
- Perform cache updates based on resource check sums. For example, query the md5 field in HTTP headers to determine when a resource at a URL has changed.
- Respect HTTP cache-control directives.
- Enable caching for ftp/ftps.
- Use symbolic links or bind mounts to project cached resources into the sandbox, read-only.
- Have a choice whether to copy the extracted archive into the sandbox.
- Have a choice whether to delete the archive after extraction bypassing the cache.
- Make the segregation of cache files by user optional.
- Extract content while downloading when bypassing the cache.
- Prefetch resources for subsequent tasks. This can happen concurrently with running the present task, right after fetching its own resources.
Implementation Details
The Mesos Fetcher Cache Internals describes how the fetcher cache is implemented.


Client
Loan Details
Account Number : {{ LoanApplicant.strAccountNumber }} | Names : {{ LoanApplicant.strFullNames }}-{{ LoanApplicant.strSurname }}
Contact: {{ LoanApplicant.strCell }}
- Personal Information
- Contact
- Addresses
- Employment Details
- Income and Expenditure
- Credit Provider
- Advanced Amount
- Banking Details
- Notes
- Admin
Personal Files
- Names : {{ LoanApplicant.strFullNames }}
- Surname : {{ LoanApplicant.strSurname }}
- ID Number : {{ LoanApplicant.strIDNumber }}
- Date Of Birth : {{ LoanApplicant.strDateOfBirth }}
Next of Kin
- Names : {{ LoanApplicant.strNextOfKinNames }}
- Surname : {{ LoanApplicant.strNextOfKinAddress }}
- Cell : {{ LoanApplicant.strNextOfKinCell }}
Contact
- Tel : {{ LoanApplicant.strTel }}
- Cell : {{ LoanApplicant.strCell }}
- Email : {{ LoanApplicant.strEmail }}
Addresses
Postal Address
- Box Number : {{ LoanApplicant.strBoxNumber }}
- City/Town : {{ LoanApplicant.strPostalCityTown }}
- Province : {{ LoanApplicant.strPostalProvince }}
- Country : {{ LoanApplicant.strPostalCountry }}
- Postal Code : {{ LoanApplicant.strPostalPostalCode }}
Physical Address
- Stand Number : {{ LoanApplicant.strHouseNumber }}
- Street Name : {{ LoanApplicant.strStreetName }}
- City/Town : {{ LoanApplicant.strCityTown }}
- Province : {{ LoanApplicant.strProvince }}
- Country : {{ LoanApplicant.strCountry }}
- Postal Code : {{ LoanApplicant.strPostalCode }}
Employment Details
- Name of Employer : {{ Employment.strNameOfEmployer }}
- Employee Number : {{ Employment.strEmployeeNumber }}
- Employee Kind : {{ Employment.strEmployeeKind }}
- Department : {{ Employment.strDepartment }}
- Contract Type : {{ Employment.strContract }}
- Date Joined : {{ Employment.strDateJoined }}
- Stand Number : {{ Employment.strStandNumber }}
- Street Name : {{ Employment.strStreetName }}
- Town/City : {{ Employment.strTownCity }}
- Province : {{ Employment.strProvince }}
- Country : {{ Employment.strCountry }}
- Postal Code : {{ Employment.strPostalCode }}
Income & Expenditure
Income
- Income After Deduction : {{ IncomeExp.strIncomeAfterDeduction }}
- Overtime : {{ IncomeExp.strOverTime }}
- Commission : {{ IncomeExp.strCommission }}
- Other Income : {{ IncomeExp.strOtherIncome }}
- Total Income : {{ IncomeExp.strTotalIncome }}
Expenditure
- Bond Repayments : {{ IncomeExp.strBondRepayments }}
- Other Credit / Installments : {{ IncomeExp.strLoanInstallments }}
- Water/Electricity/Telephone : {{ IncomeExp.strWaterElectricityTel }}
- Insurance : {{ IncomeExp.strInsurance }}
- Children Maintenance : {{ IncomeExp.strChildrenMaintenance }}
- Vehicle Maintenance : {{ IncomeExp.strVehicleMaintenance }}
- Basic Necessities : {{ IncomeExp.strBasicNecessities }}
- Domestic Wages : {{ IncomeExp.strDomesticWages }}
- Education : {{ IncomeExp.strEducation }}
- Other Expenses : {{ IncomeExp.strOtherExpenses }}
- Total Expenses : {{ IncomeExp.strTotalExpenses }}
Affordability
- Affordability : {{ IncomeExp.strAffordability }}
Credit Provider
- Name of Credit Provider : SA Loans
- Physical Address : Mashapa Complex, 1st Floor, Office No B12 & 14, Thohoyandou, 0950
- Tel : +2715 962 0976
- NCR Registration : NCRCP5905
- Company Registration : 2007/036777/23
- Branch : {{ Employee.strBranchCode }}
- Loan Officer : {{ Employee.strFullNames }} {{ Employee.strSurname }}
Advanced Amount
- Credit Advanced Capital : {{ AmountAdvanced.strCreditAdvancedCapital }}
- Initiation Fee : {{ AmountAdvanced.strInitiationFee }}
- Monthly Service Fee : {{ AmountAdvanced.strMonthlyServiceFee }}
- Frequency : {{ AmountAdvanced.strFreequency }}
- Number Installments : {{ AmountAdvanced.strNumberInstallments }}
- Loan Term : {{ AmountAdvanced.strLoanTerm }}
- Amount Advanced to Client : {{ AmountAdvanced.strAmountAdvancedToClient }}
- Monthly Installments : {{ AmountAdvanced.strMonthlyInstallments }}
- Total Installments : {{ AmountAdvanced.strTotalInstallments }}
Banking Details
- Account Holder : {{ Banking.strAccountHolder }}
- Account Number : {{ Banking.strBankAccountNumber }}
- Banking Institution : {{ Banking.strBankingInstitution }}
- Branch Code : {{ Banking.strBranchCode }}
- Account Type : {{ Banking.strAccountType }}
- Date : {{ Banking.strDateCommencement }}
Admin
{% if UserRights.bolEmployeesLoanFormWriteAccess == True %}
{% endif %}
{% if UserRights.bolEmployeesLoanFormDeleteAccess == True %}
{% endif %}
{% if UserRights.bolEmployeesLoanFormReprintAccess == True %}
{% endif %}
{% if UserRights.bolEmployeeLoanFormProcessPayment == True %}
Process Payment
{% endif %}