Bayesian Regression Using NumPyro¶
In this tutorial, we will explore how to do bayesian regression in NumPyro, using a simple example adapted from Statistical Rethinking [1]. In particular, we would like to explore the following:
Write a simple model using the
sample
NumPyro primitive.Run inference using MCMC in NumPyro, in particular, using the No U-Turn Sampler (NUTS) to get a posterior distribution over our regression parameters of interest.
Learn about inference utilities such as
Predictive
andlog_likelihood
.Learn how we can use effect-handlers in NumPyro to generate execution traces from the model, condition on sample statements, seed models with RNG seeds, etc., and use this to implement various utilities that will be useful for MCMC. e.g. computing model log likelihood, generating empirical distribution over the posterior predictive, etc.
Tutorial Outline:¶
[ ]:
%reset -s -f
[2]:
import os
from IPython.display import set_matplotlib_formats
import jax.numpy as np
from jax import random, vmap
from jax.scipy.special import logsumexp
import matplotlib.pyplot as plt
import numpy as onp
import pandas as pd
import seaborn as sns
import numpyro
from numpyro.diagnostics import hpdi
import numpyro.distributions as dist
from numpyro import handlers
from numpyro.infer import MCMC, NUTS
plt.style.use('bmh')
if "NUMPYRO_SPHINXBUILD" in os.environ:
set_matplotlib_formats('svg')
assert numpyro.__version__.startswith('0.2.4')
Dataset¶
For this example, we will use the WaffleDivorce
dataset from Chapter 05, Statistical Rethinking [1]. The dataset contains divorce rates in each of the 50 states in the USA, along with predictors such as population, median age of marriage, whether it is a Southern state and, curiously, number of Waffle Houses.
[3]:
DATASET_URL = 'https://raw.githubusercontent.com/rmcelreath/rethinking/master/data/WaffleDivorce.csv'
dset = pd.read_csv(DATASET_URL, sep=';')
dset
[3]:
| Location | Loc | Population | MedianAgeMarriage | Marriage | Marriage SE | Divorce | Divorce SE | WaffleHouses | South | Slaves1860 | Population1860 | PropSlaves1860 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Alabama | AL | 4.78 | 25.3 | 20.2 | 1.27 | 12.7 | 0.79 | 128 | 1 | 435080 | 964201 | 0.450000 |
| 1 | Alaska | AK | 0.71 | 25.2 | 26.0 | 2.93 | 12.5 | 2.05 | 0 | 0 | 0 | 0 | 0.000000 |
| 2 | Arizona | AZ | 6.33 | 25.8 | 20.3 | 0.98 | 10.8 | 0.74 | 18 | 0 | 0 | 0 | 0.000000 |
| 3 | Arkansas | AR | 2.92 | 24.3 | 26.4 | 1.70 | 13.5 | 1.22 | 41 | 1 | 111115 | 435450 | 0.260000 |
| 4 | California | CA | 37.25 | 26.8 | 19.1 | 0.39 | 8.0 | 0.24 | 0 | 0 | 0 | 379994 | 0.000000 |
| 5 | Colorado | CO | 5.03 | 25.7 | 23.5 | 1.24 | 11.6 | 0.94 | 11 | 0 | 0 | 34277 | 0.000000 |
| 6 | Connecticut | CT | 3.57 | 27.6 | 17.1 | 1.06 | 6.7 | 0.77 | 0 | 0 | 0 | 460147 | 0.000000 |
| 7 | Delaware | DE | 0.90 | 26.6 | 23.1 | 2.89 | 8.9 | 1.39 | 3 | 0 | 1798 | 112216 | 0.016000 |
| 8 | District of Columbia | DC | 0.60 | 29.7 | 17.7 | 2.53 | 6.3 | 1.89 | 0 | 0 | 0 | 75080 | 0.000000 |
| 9 | Florida | FL | 18.80 | 26.4 | 17.0 | 0.58 | 8.5 | 0.32 | 133 | 1 | 61745 | 140424 | 0.440000 |
| 10 | Georgia | GA | 9.69 | 25.9 | 22.1 | 0.81 | 11.5 | 0.58 | 381 | 1 | 462198 | 1057286 | 0.440000 |
| 11 | Hawaii | HI | 1.36 | 26.9 | 24.9 | 2.54 | 8.3 | 1.27 | 0 | 0 | 0 | 0 | 0.000000 |
| 12 | Idaho | ID | 1.57 | 23.2 | 25.8 | 1.84 | 7.7 | 1.05 | 0 | 0 | 0 | 0 | 0.000000 |
| 13 | Illinois | IL | 12.83 | 27.0 | 17.9 | 0.58 | 8.0 | 0.45 | 2 | 0 | 0 | 1711951 | 0.000000 |
| 14 | Indiana | IN | 6.48 | 25.7 | 19.8 | 0.81 | 11.0 | 0.63 | 17 | 0 | 0 | 1350428 | 0.000000 |
| 15 | Iowa | IA | 3.05 | 25.4 | 21.5 | 1.46 | 10.2 | 0.91 | 0 | 0 | 0 | 674913 | 0.000000 |
| 16 | Kansas | KS | 2.85 | 25.0 | 22.1 | 1.48 | 10.6 | 1.09 | 6 | 0 | 2 | 107206 | 0.000019 |
| 17 | Kentucky | KY | 4.34 | 24.8 | 22.2 | 1.11 | 12.6 | 0.75 | 64 | 1 | 225483 | 1155684 | 0.000000 |
| 18 | Louisiana | LA | 4.53 | 25.9 | 20.6 | 1.19 | 11.0 | 0.89 | 66 | 1 | 331726 | 708002 | 0.470000 |
| 19 | Maine | ME | 1.33 | 26.4 | 13.5 | 1.40 | 13.0 | 1.48 | 0 | 0 | 0 | 628279 | 0.000000 |
| 20 | Maryland | MD | 5.77 | 27.3 | 18.3 | 1.02 | 8.8 | 0.69 | 11 | 0 | 87189 | 687049 | 0.130000 |
| 21 | Massachusetts | MA | 6.55 | 28.5 | 15.8 | 0.70 | 7.8 | 0.52 | 0 | 0 | 0 | 1231066 | 0.000000 |
| 22 | Michigan | MI | 9.88 | 26.4 | 16.5 | 0.69 | 9.2 | 0.53 | 0 | 0 | 0 | 749113 | 0.000000 |
| 23 | Minnesota | MN | 5.30 | 26.3 | 15.3 | 0.77 | 7.4 | 0.60 | 0 | 0 | 0 | 172023 | 0.000000 |
| 24 | Mississippi | MS | 2.97 | 25.8 | 19.3 | 1.54 | 11.1 | 1.01 | 72 | 1 | 436631 | 791305 | 0.550000 |
| 25 | Missouri | MO | 5.99 | 25.6 | 18.6 | 0.81 | 9.5 | 0.67 | 39 | 1 | 114931 | 1182012 | 0.097000 |
| 26 | Montana | MT | 0.99 | 25.7 | 18.5 | 2.31 | 9.1 | 1.71 | 0 | 0 | 0 | 0 | 0.000000 |
| 27 | Nebraska | NE | 1.83 | 25.4 | 19.6 | 1.44 | 8.8 | 0.94 | 0 | 0 | 15 | 28841 | 0.000520 |
| 28 | New Hampshire | NH | 1.32 | 26.8 | 16.7 | 1.76 | 10.1 | 1.61 | 0 | 0 | 0 | 326073 | 0.000000 |
| 29 | New Jersey | NJ | 8.79 | 27.7 | 14.8 | 0.59 | 6.1 | 0.46 | 0 | 0 | 18 | 672035 | 0.000027 |
| 30 | New Mexico | NM | 2.06 | 25.8 | 20.4 | 1.90 | 10.2 | 1.11 | 2 | 0 | 0 | 93516 | 0.000000 |
| 31 | New York | NY | 19.38 | 28.4 | 16.8 | 0.47 | 6.6 | 0.31 | 0 | 0 | 0 | 3880735 | 0.000000 |
| 32 | North Carolina | NC | 9.54 | 25.7 | 20.4 | 0.98 | 9.9 | 0.48 | 142 | 1 | 331059 | 992622 | 0.330000 |
| 33 | North Dakota | ND | 0.67 | 25.3 | 26.7 | 2.93 | 8.0 | 1.44 | 0 | 0 | 0 | 0 | 0.000000 |
| 34 | Ohio | OH | 11.54 | 26.3 | 16.9 | 0.61 | 9.5 | 0.45 | 64 | 0 | 0 | 2339511 | 0.000000 |
| 35 | Oklahoma | OK | 3.75 | 24.4 | 23.8 | 1.29 | 12.8 | 1.01 | 16 | 0 | 0 | 0 | 0.000000 |
| 36 | Oregon | OR | 3.83 | 26.0 | 18.9 | 1.10 | 10.4 | 0.80 | 0 | 0 | 0 | 52465 | 0.000000 |
| 37 | Pennsylvania | PA | 12.70 | 27.1 | 15.5 | 0.48 | 7.7 | 0.43 | 11 | 0 | 0 | 2906215 | 0.000000 |
| 38 | Rhode Island | RI | 1.05 | 28.2 | 15.0 | 2.11 | 9.4 | 1.79 | 0 | 0 | 0 | 174620 | 0.000000 |
| 39 | South Carolina | SC | 4.63 | 26.4 | 18.1 | 1.18 | 8.1 | 0.70 | 144 | 1 | 402406 | 703708 | 0.570000 |
| 40 | South Dakota | SD | 0.81 | 25.6 | 20.1 | 2.64 | 10.9 | 2.50 | 0 | 0 | 0 | 4837 | 0.000000 |
| 41 | Tennessee | TN | 6.35 | 25.2 | 19.4 | 0.85 | 11.4 | 0.75 | 103 | 1 | 275719 | 1109801 | 0.200000 |
| 42 | Texas | TX | 25.15 | 25.2 | 21.5 | 0.61 | 10.0 | 0.35 | 99 | 1 | 182566 | 604215 | 0.300000 |
| 43 | Utah | UT | 2.76 | 23.3 | 29.6 | 1.77 | 10.2 | 0.93 | 0 | 0 | 0 | 40273 | 0.000000 |
| 44 | Vermont | VT | 0.63 | 26.9 | 16.4 | 2.40 | 9.6 | 1.87 | 0 | 0 | 0 | 315098 | 0.000000 |
| 45 | Virginia | VA | 8.00 | 26.4 | 20.5 | 0.83 | 8.9 | 0.52 | 40 | 1 | 490865 | 1219630 | 0.400000 |
| 46 | Washington | WA | 6.72 | 25.9 | 21.4 | 1.00 | 10.0 | 0.65 | 0 | 0 | 0 | 11594 | 0.000000 |
| 47 | West Virginia | WV | 1.85 | 25.0 | 22.2 | 1.69 | 10.9 | 1.34 | 4 | 1 | 18371 | 376688 | 0.049000 |
| 48 | Wisconsin | WI | 5.69 | 26.3 | 17.2 | 0.79 | 8.3 | 0.57 | 0 | 0 | 0 | 775881 | 0.000000 |
| 49 | Wyoming | WY | 0.56 | 24.2 | 30.7 | 3.92 | 10.3 | 1.90 | 0 | 0 | 0 | 0 | 0.000000 |
Let us plot the pair-wise relationship amongst the main variables in the dataset, using seaborn.pairplot
.
[4]:
vars = ['Population', 'MedianAgeMarriage', 'Marriage', 'WaffleHouses', 'South', 'Divorce']
sns.pairplot(dset, x_vars=vars, y_vars=vars, palette='husl');
From the plots above, we can clearly observe that there is a relationship between divorce rates and marriage rates in a state (as might be expected), and also between divorce rates and median age of marriage.
There is also a weak relationship between number of Waffle Houses and divorce rates, which is not obvious from the plot above, but will be clearer if we regress Divorce
against WaffleHouse
and plot the results.
[5]:
sns.regplot('WaffleHouses', 'Divorce', dset);
This is an example of a spurious association. We do not expect the number of Waffle Houses in a state to affect the divorce rate, but it is likely correlated with other factors that have an effect on the divorce rate. We will not delve into this spurious association in this tutorial, but the interested reader is encouraged to read Chapters 5 and 6 of [1] which explores the problem of causal association in the presence of multiple predictors.
For simplicity, we will primarily focus on marriage rate and the median age of marriage as our predictors for divorce rate throughout the remaining tutorial.
Regression Model to Predict Divorce Rate¶
Let us now write a regressionn model in NumPyro to predict the divorce rate as a linear function of marriage rate and median age of marriage in each of the states.
First, note that our predictor variables have somewhat different scales. It is a good practice to standardize our predictors and response variables to mean 0
and standard deviation 1
, which should result in faster inference.
[6]:
standardize = lambda x: (x - x.mean()) / x.std()
dset['AgeScaled'] = dset.MedianAgeMarriage.pipe(standardize)
dset['MarriageScaled'] = dset.Marriage.pipe(standardize)
dset['DivorceScaled'] = dset.Divorce.pipe(standardize)
We write the NumPyro model as follows. While the code should largely be self-explanatory, take note of the following:
In NumPyro, model code is any Python callable which can optionally accept additional arguments and keywords. For HMC which we will be using for this tutorial, these arguments and keywords remain static during inference, but we can reuse the same model to generate predictions on new data.
In addition to regular Python statements, the model code also contains primitives like
sample
. These primitives can be interpreted with various side-effects using effect handlers. For more on effect handlers, refer to [3], [4]. For now, just remember that asample
statement makes this a stochastic function that samples some latent parameters from a prior distribution. Our goal is to infer the posterior distribution of these parameters conditioned on observed data.The reason why we have kept our predictors as optional keyword arguments is to be able to reuse the same model as we vary the set of predictors. Likewise, the reason why the response variable is optional is that we would like to reuse this model to sample from the posterior predictive distribution. See the section on plotting the posterior predictive distribution, as an example.
[7]:
def model(marriage=None, age=None, divorce=None):
a = numpyro.sample('a', dist.Normal(0., 0.2))
M, A = 0., 0.
if marriage is not None:
bM = numpyro.sample('bM', dist.Normal(0., 0.5))
M = bM * marriage
if age is not None:
bA = numpyro.sample('bA', dist.Normal(0., 0.5))
A = bA * age
sigma = numpyro.sample('sigma', dist.Exponential(1.))
mu = a + M + A
numpyro.sample('obs', dist.Normal(mu, sigma), obs=divorce)
Model 1: Predictor - Marriage Rate¶
We first try to model the divorce rate as depending on a single variable, marriage rate. As mentioned above, we can use the same model
code as earlier, but only pass values for marriage
and divorce
keyword arguments. We will use the No U-Turn Sampler (see [5] for more details on the NUTS algorithm) to run inference on this simple model.
The Hamiltonian Monte Carlo (or, the NUTS) implementation in NumPyro takes in a potential energy function. This is the negative log joint density for the model. Therefore, for our model description above, we need to construct a function which given the parameter values returns the potential energy (or negative log joint density). Additionally, the verlet integrator in HMC (or, NUTS) returns sample values simulated using Hamiltonian dynamics in the unconstrained space. As such, continuous variables with bounded support need to be transformed into unconstrained space using bijective transforms. We also need to transform these samples back to their constrained support before returning these values to the user. Thankfully, this is handled on the backend for us, within a convenience class for doing MCMC inference that has the following methods:
run(...)
: runs warmup, adapts steps size and mass matrix, and does sampling using the sample from the warmup phase.print_summary()
: print diagnostic information like quantiles, effective sample size, and the Gelman-Rubin diagnostic.get_samples()
: gets samples from the posterior distribution.
Note the following:
JAX uses functional PRNGs. Unlike other languages / frameworks which maintain a global random state, in JAX, every call to a sampler requires an explicit PRNGKey. We will split our initial random seed for subsequent operations, so that we do not accidentally reuse the same seed.
We run inference with the
NUTS
sampler. To run vanilla HMC, we can instead use the HMC class.
[8]:
# Start from this source of randomness. We will split keys for subsequent operations.
rng_key = random.PRNGKey(0)
rng_key, rng_key_ = random.split(rng_key)
num_warmup, num_samples = 1000, 2000
# Run NUTS.
kernel = NUTS(model)
mcmc = MCMC(kernel, num_warmup, num_samples)
mcmc.run(rng_key_, marriage=dset.MarriageScaled.values, divorce=dset.DivorceScaled.values)
mcmc.print_summary()
samples_1 = mcmc.get_samples()
sample: 100%|██████████| 3000/3000 [00:06<00:00, 487.44it/s, 3 steps of size 7.35e-01. acc. prob=0.92]
mean std median 5.0% 95.0% n_eff r_hat
a -0.00 0.11 -0.00 -0.20 0.17 1737.87 1.00
bM 0.35 0.13 0.35 0.14 0.56 1812.08 1.00
sigma 0.95 0.10 0.94 0.78 1.09 1951.25 1.00
Number of divergences: 0
Posterior Distribution over the Regression Parameters¶
We notice that the progress bar gives us online statistics on the acceptance probability, step size and number of steps taken per sample while running NUTS. In particular, during warmup, we adapt the step size and mass matrix to achieve a certain target acceptance probability which is 0.8, by default. We were able to successfully adapt our step size to achieve this target in the warmup phase.
During warmup, the aim is to adapt hyper-parameters such as step size and mass matrix (the HMC algorithm is very sensitive to these hyper-parameters), and to reach the typical set (see [6] for more details). If there are any issues in the model specification, the first signal to notice would be low acceptance probabilities or very high number of steps. We use the sample from the end of the warmup phase to seed the MCMC chain (denoted by the second sample
progress bar) from
which we generate the desired number of samples from our target distribution.
At the end of inference, NumPyro prints the mean, std and 90% CI values for each of the latent parameters. Note that since we standardized our predictors and response variable, we would expect the intercept to have mean 0, as can be seen here. It also prints other convergence diagnostics on the latent parameters in the model, including effective sample size and the gelman rubin
diagnostic (\(\hat{R}\)). The value for these diagnostics indicates that the chain has converged to the target distribution. In our case, the “target distribution” is the posterior distribution over the latent parameters that we are interested in. Note that this is often worth verifying with multiple chains for more complicated models. In the end, samples_1
is a collection (in our case, a
dict
since init_samples
was a dict
) containing samples from the posterior distribution for each of the latent parameters in the model.
To look at our regression fit, let us plot the regression line using our posterior estimates for the regression parameters, along with the 90% Credibility Interval (CI). Note that the hpdi function in NumPyro’s diagnostics module can be used to compute CI. In the functions below, note that the collected samples from the posterior are all along the leading axis.
[9]:
def plot_regression(x, y_mean, y_hpdi):
# Sort values for plotting by x axis
idx = np.argsort(x)
marriage = x[idx]
mean = y_mean[idx]
hpdi = y_hpdi[:, idx]
divorce = dset.DivorceScaled.values[idx]
# Plot
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))
ax.plot(marriage, mean)
ax.plot(marriage, divorce, 'o')
ax.fill_between(marriage, hpdi[0], hpdi[1], alpha=0.3, interpolate=True)
return ax
# Compute empirical posterior distribution over mu
posterior_mu = np.expand_dims(samples_1['a'], -1) + \
np.expand_dims(samples_1['bM'], -1) * dset.MarriageScaled.values
mean_mu = np.mean(posterior_mu, axis=0)
hpdi_mu = hpdi(posterior_mu, 0.9)
ax = plot_regression(dset.MarriageScaled.values, mean_mu, hpdi_mu)
ax.set(xlabel='Marriage rate', ylabel='Divorce rate', title='Regression line with 90% CI');
We can see from the plot, that the CI broadens towards the tails where the data is relatively sparse, as can be expected.
Posterior Predictive Distribution¶
Let us now look at the posterior predictive distribution to see how our predictive distribution looks with respect to the observed divorce rates. To get samples from the posterior predictive distribution, we need to run the model by substituting the latent parameters with samples from the posterior. NumPyro provides a handy Predictive utility for this purpose. Note that by default we generate a single prediction for
each sample from the joint posterior distribution, but this can be controlled using the num_samples
argument.
[10]:
from numpyro.infer import Predictive
rng_key, rng_key_ = random.split(rng_key)
predictive = Predictive(model, samples_1)
predictions = predictive(rng_key_, marriage=dset.MarriageScaled.values)['obs']
df = dset.filter(['Location'])
df['Mean Predictions'] = np.mean(predictions, axis=0)
df.head()
[10]:
| Location | Mean Predictions | |
|---|---|---|
| 0 | Alabama | -0.016397 |
| 1 | Alaska | 0.577286 |
| 2 | Arizona | 0.010056 |
| 3 | Arkansas | 0.585147 |
| 4 | California | -0.092986 |
Predictive Utility With Effect Handlers¶
To remove the magic behind Predictive
, let us see how we can combine effect handlers with the vmap JAX primitive to implement our own simplified predictive utility function that can do vectorized predictions.
[11]:
def predict(rng_key, post_samples, model, *args, **kwargs):
model = handlers.condition(handlers.seed(model, rng_key), post_samples)
model_trace = handlers.trace(model).get_trace(*args, **kwargs)
return model_trace['obs']['value']
# vectorize predictions via vmap
predict_fn = vmap(lambda rng_key, samples: predict(rng_key, samples, model, marriage=dset.MarriageScaled.values))
Note the use of the condition
, seed
and trace
effect handlers in the predict
function.
The
seed
effect-handler is used to wrap a stochastic function with an initialPRNGKey
seed. When a sample statement inside the model is called, it uses the existing seed to sample from a distribution but this effect-handler also splits the existing key to ensure that futuresample
calls in the model use the newly split key instead. This is to prevent us from having to explicitly pass in aPRNGKey
to eachsample
statement in the model.The
condition
effect handler conditions the latent sample sites to certain values. In our case, we are conditioning on values from the posterior distribution returned by MCMC.The
trace
effect handler runs the model and records the execution trace within anOrderedDict
. This trace object contains execution metadata that is useful for computing quantities such as the log joint density.
It should be clear now that the predict
function simply runs the model by substituting the latent parameters with samples from the posterior (generated by the mcmc
function) to generate predictions. Note the use of JAX’s auto-vectorization transform called vmap to vectorize predictions. Note that if we didn’t use vmap
, we would have to use a native for loop which for each sample which is much slower. Each draw from the
posterior can be used to get predictions over all the 50 states. When we vectorize this over all the samples from the posterior using vmap
, we will get a predictions_1
array of shape (num_samples, 50)
. We can then compute the mean and 90% CI of these samples to plot the posterior predictive distribution. We note that our mean predictions match those obtained from the Predictive
utility class.
[12]:
# Using the same key as we used for Predictive - note that the results are identical.
predictions_1 = predict_fn(random.split(rng_key_, num_samples), samples_1)
mean_pred = np.mean(predictions_1, axis=0)
df = dset.filter(['Location'])
df['Mean Predictions'] = np.mean(predictions, axis=0)
df.head()
[12]:
| Location | Mean Predictions | |
|---|---|---|
| 0 | Alabama | -0.016397 |
| 1 | Alaska | 0.577286 |
| 2 | Arizona | 0.010056 |
| 3 | Arkansas | 0.585147 |
| 4 | California | -0.092986 |
[13]:
hpdi_pred = hpdi(predictions_1, 0.9)
ax = plot_regression(dset.MarriageScaled.values, mean_pred, hpdi_pred)
ax.set(xlabel='Marriage rate', ylabel='Divorce rate', title='Predictions with 90% CI');
We have used the same plot_regression
function as earlier. We notice that our CI for the predictive distribution is much broader as compared to the last plot due to the additional noise introduced by the sigma
parameter. Most data points lie well within the 90% CI, which indicates a good fit.
Posterior Predictive Density¶
Likewise, making use of effect-handlers and vmap
, we can also compute the log likelihood for this model given the dataset, and the log posterior predictive density [6] which is given by
.
Here, \(i\) indexes the observed data points \(y\) and \(s\) indexes the posterior samples over the latent parameters \(\theta\). If the posterior predictive density for a model has a comparatively high value, it indicates that the observed data-points have higher probability under the given model.
[14]:
def log_likelihood(rng_key, params, model, *args, **kwargs):
model = handlers.condition(handlers.seed(model, rng_key), params)
model_trace = handlers.trace(model).get_trace(*args, **kwargs)
obs_node = model_trace['obs']
return obs_node['fn'].log_prob(obs_node['value'])
def log_pred_density(rng_key, params, model, *args, **kwargs):
n = list(params.values())[0].shape[0]
log_lk_fn = vmap(lambda rng_key, params: log_likelihood(rng_key, params, model, *args, **kwargs))
log_lk_vals = log_lk_fn(random.split(rng_key, n), params)
return (logsumexp(log_lk_vals, 0) - np.log(n)).sum()
Note that NumPyro provides the log_likelihood utility function that can be used directly for computing log likelihood
as in the first function for any general model. In this tutorial, we would like to emphasize that there is nothing magical about such utility functions, and you can roll out your own inference utilities using NumPyro’s effect handling stack.
[15]:
rng_key, rng_key_ = random.split(rng_key)
print('Log posterior predictive density: {}'.format(log_pred_density(rng_key_,
samples_1,
model,
marriage=dset.MarriageScaled.values,
divorce=dset.DivorceScaled.values)))
Log posterior predictive density: -66.70353698730469
Model 2: Predictor - Median Age of Marriage¶
We will now model the divorce rate as a function of the median age of marriage. The computations are mostly a reproduction of what we did for Model 1. Notice the following:
Divorce rate is inversely related to the age of marriage. Hence states where the median age of marriage is low will likely have a higher divorce rate.
We get a higher log likelihood as compared to Model 2, indicating that median age of marriage is likely a much better predictor of divorce rate.
[16]:
rng_key, rng_key_ = random.split(rng_key)
mcmc.run(rng_key_, age=dset.AgeScaled.values, divorce=dset.DivorceScaled.values)
mcmc.print_summary()
samples_2 = mcmc.get_samples()
sample: 100%|██████████| 3000/3000 [00:05<00:00, 534.66it/s, 7 steps of size 7.99e-01. acc. prob=0.92]
mean std median 5.0% 95.0% n_eff r_hat
a -0.00 0.10 -0.00 -0.17 0.16 1897.70 1.00
bA -0.56 0.11 -0.56 -0.74 -0.38 1827.11 1.00
sigma 0.82 0.09 0.81 0.67 0.95 1813.78 1.00
Number of divergences: 0
[17]:
posterior_mu = np.expand_dims(samples_2['a'], -1) + \
np.expand_dims(samples_2['bA'], -1) * dset.AgeScaled.values
mean_mu = np.mean(posterior_mu, axis=0)
hpdi_mu = hpdi(posterior_mu, 0.9)
ax = plot_regression(dset.AgeScaled.values, mean_mu, hpdi_mu)
ax.set(xlabel='Median marriage age', ylabel='Divorce rate', title='Regression line with 90% CI');
[18]:
rng_key, rng_key_ = random.split(rng_key)
predictions_2 = Predictive(model, samples_2)(rng_key_,
age=dset.AgeScaled.values)['obs']
mean_pred = np.mean(predictions_2, axis=0)
hpdi_pred = hpdi(predictions_2, 0.9)
ax = plot_regression(dset.AgeScaled.values, mean_pred, hpdi_pred)
ax.set(xlabel='Median Age', ylabel='Divorce rate', title='Predictions with 90% CI');
[19]:
rng_key, rng_key_ = random.split(rng_key)
print('Log posterior predictive density: {}'.format(log_pred_density(rng_key_,
samples_2,
model,
age=dset.AgeScaled.values,
divorce=dset.DivorceScaled.values)))
Log posterior predictive density: -59.22672653198242
Model 3: Predictor - Marriage Rate and Median Age of Marriage¶
Finally, we will also model divorce rate as depending on both marriage rate as well as the median age of marriage. Note that the model’s posterior predictive density is similar to Model 2 which likely indicates that the marginal information from marriage rate in predicting divorce rate is low when the median age of marriage is already known.
[20]:
rng_key, rng_key_ = random.split(rng_key)
mcmc.run(rng_key_, marriage=dset.MarriageScaled.values,
age=dset.AgeScaled.values, divorce=dset.DivorceScaled.values)
mcmc.print_summary()
samples_3 = mcmc.get_samples()
sample: 100%|██████████| 3000/3000 [00:06<00:00, 472.77it/s, 3 steps of size 6.11e-01. acc. prob=0.89]
mean std median 5.0% 95.0% n_eff r_hat
a -0.00 0.10 0.00 -0.16 0.15 1946.02 1.00
bA -0.61 0.16 -0.61 -0.87 -0.35 1480.89 1.00
bM -0.06 0.16 -0.07 -0.33 0.21 1416.08 1.00
sigma 0.83 0.09 0.82 0.69 0.96 1559.51 1.00
Number of divergences: 0
[21]:
rng_key, rng_key_ = random.split(rng_key)
print('Log posterior predictive density: {}'.format(
log_pred_density(rng_key_,
samples_3,
model,
marriage=dset.MarriageScaled.values,
age=dset.AgeScaled.values,
divorce=dset.DivorceScaled.values)
))
Log posterior predictive density: -59.12065505981445
Divorce Rate Residuals by State¶
The regression plots above shows that the observed divorce rates for many states differs considerably from the mean regression line. To dig deeper into how the last model (Model 3) under-predicts or over-predicts for each of the states, we will plot the posterior predictive and residuals (Observed divorce rate - Predicted divorce rate
) for each of the states.
[22]:
# Predictions for Model 3.
rng_key, rng_key_ = random.split(rng_key)
predictions_3 = Predictive(model, samples_3)(rng_key_,
marriage=dset.MarriageScaled.values,
age=dset.AgeScaled.values)['obs']
y = np.arange(50)
fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(12, 16))
pred_mean = np.mean(predictions_3, axis=0)
pred_hpdi = hpdi(predictions_3, 0.9)
residuals_3 = dset.DivorceScaled.values - predictions_3
residuals_mean = np.mean(residuals_3, axis=0)
residuals_hpdi = hpdi(residuals_3, 0.9)
idx = np.argsort(residuals_mean)
# Plot posterior predictive
ax[0].plot(np.zeros(50), y, '--')
ax[0].errorbar(pred_mean[idx], y, xerr=pred_hpdi[1, idx] - pred_mean[idx],
marker='o', ms=5, mew=4, ls='none', alpha=0.8)
ax[0].plot(dset.DivorceScaled.values[idx], y, marker='o',
ls='none', color='gray')
ax[0].set(xlabel='Posterior Predictive (red) vs. Actuals (gray)', ylabel='State',
title='Posterior Predictive with 90% CI')
ax[0].set_yticks(y)
ax[0].set_yticklabels(dset.Loc.values[idx], fontsize=10);
# Plot residuals
residuals_3 = dset.DivorceScaled.values - predictions_3
residuals_mean = np.mean(residuals_3, axis=0)
residuals_hpdi = hpdi(residuals_3, 0.9)
err = residuals_hpdi[1] - residuals_mean
ax[1].plot(np.zeros(50), y, '--')
ax[1].errorbar(residuals_mean[idx], y, xerr=err[idx],
marker='o', ms=5, mew=4, ls='none', alpha=0.8)
ax[1].set(xlabel='Residuals', ylabel='State', title='Residuals with 90% CI')
ax[1].set_yticks(y)
ax[1].set_yticklabels(dset.Loc.values[idx], fontsize=10);
The plot on the left shows the mean predictions with 90% CI for each of the states using Model 3. The gray markers indicate the actual observed divorce rates. The right plot shows the residuals for each of the states, and both these plots are sorted by the residuals, i.e. at the bottom, we are looking at states where the model predictions are higher than the observed rates, whereas at the top, the reverse is true.
Overall, the model fit seems good because most observed data points like within a 90% CI around the mean predictions. However, notice how the model over-predicts by a large margin for states like Idaho (bottom left), and on the other end under-predicts for states like Maine (top right). This is likely indicative of other factors that we are missing out in our model that affect divorce rate across different states. Even ignoring other socio-political variables, one such factor that we have not
yet modeled is the measurement noise given by Divorce SE
in the dataset. We will explore this in the next section.
Regression Model with Measurement Error¶
Note that in our previous models, each data point influences the regression line equally. Is this well justified? We will build on the previous model to incorporate measurement error given by Divorce SE
variable in the dataset. Incorporating measurement noise will be useful in ensuring that observations that have higher confidence (i.e. lower measurement noise) have a greater impact on the regression line. On the other hand, this will also help us better model outliers with high measurement
errors. For more details on modeling errors due to measurement noise, refer to Chapter 15 of [1].
To do this, we will reuse Model 3, with the only change that the final observed value has a measurement error given by divorce_sd
(notice that this has to be standardized since the divorce
variable itself has been standardized to mean 0 and std 1).
[23]:
def model_se(marriage, age, divorce_sd, divorce=None):
a = numpyro.sample('a', dist.Normal(0., 0.2))
bM = numpyro.sample('bM', dist.Normal(0., 0.5))
M = bM * marriage
bA = numpyro.sample('bA', dist.Normal(0., 0.5))
A = bA * age
sigma = numpyro.sample('sigma', dist.Exponential(1.))
mu = a + M + A
divorce_rate = numpyro.sample('divorce_rate', dist.Normal(mu, sigma))
numpyro.sample('obs', dist.Normal(divorce_rate, divorce_sd), obs=divorce)
[24]:
# Standardize
dset['DivorceScaledSD'] = dset['Divorce SE'] / np.std(dset.Divorce.values)
[25]:
rng_key, rng_key_ = random.split(rng_key)
kernel = NUTS(model_se, target_accept_prob=0.9)
mcmc = MCMC(kernel, num_warmup=1000, num_samples=3000)
mcmc.run(rng_key_, marriage=dset.MarriageScaled.values, age=dset.AgeScaled.values,
divorce_sd=dset.DivorceScaledSD.values, divorce=dset.DivorceScaled.values)
mcmc.print_summary()
samples_4 = mcmc.get_samples()
sample: 100%|██████████| 4000/4000 [00:10<00:00, 379.65it/s, 15 steps of size 2.93e-01. acc. prob=0.92]
mean std median 5.0% 95.0% n_eff r_hat
a -0.06 0.10 -0.06 -0.22 0.10 3966.50 1.00
bA -0.61 0.16 -0.61 -0.87 -0.36 2557.72 1.00
bM 0.05 0.17 0.05 -0.23 0.31 2492.16 1.00
divorce_rate[0] 1.15 0.37 1.15 0.54 1.75 3605.29 1.00
divorce_rate[1] 0.69 0.54 0.69 -0.16 1.64 5458.77 1.00
divorce_rate[2] 0.42 0.33 0.42 -0.09 0.97 5768.17 1.00
divorce_rate[3] 1.41 0.46 1.40 0.68 2.18 4829.59 1.00
divorce_rate[4] -0.90 0.13 -0.90 -1.13 -0.70 7228.24 1.00
divorce_rate[5] 0.65 0.40 0.64 -0.02 1.28 6164.41 1.00
divorce_rate[6] -1.36 0.35 -1.36 -1.98 -0.80 6419.94 1.00
divorce_rate[7] -0.34 0.49 -0.33 -1.14 0.45 5691.21 1.00
divorce_rate[8] -1.89 0.58 -1.89 -2.77 -0.88 3791.78 1.00
divorce_rate[9] -0.62 0.17 -0.61 -0.90 -0.35 6524.70 1.00
divorce_rate[10] 0.76 0.29 0.76 0.26 1.22 6696.11 1.00
divorce_rate[11] -0.56 0.49 -0.55 -1.38 0.23 4890.48 1.00
divorce_rate[12] 0.19 0.50 0.20 -0.68 0.96 2576.29 1.00
divorce_rate[13] -0.87 0.23 -0.87 -1.22 -0.48 8447.07 1.00
divorce_rate[14] 0.55 0.30 0.55 0.07 1.05 5253.27 1.00
divorce_rate[15] 0.28 0.39 0.28 -0.34 0.94 7013.39 1.00
divorce_rate[16] 0.50 0.43 0.50 -0.21 1.21 8324.40 1.00
divorce_rate[17] 1.25 0.35 1.25 0.71 1.86 4808.26 1.00
divorce_rate[18] 0.43 0.39 0.43 -0.27 1.01 6052.16 1.00
divorce_rate[19] 0.39 0.55 0.36 -0.47 1.32 3211.61 1.00
divorce_rate[20] -0.56 0.33 -0.56 -1.13 -0.06 6399.09 1.00
divorce_rate[21] -1.11 0.27 -1.11 -1.54 -0.67 5767.24 1.00
divorce_rate[22] -0.27 0.26 -0.27 -0.68 0.15 6910.95 1.00
divorce_rate[23] -1.00 0.31 -1.00 -1.53 -0.50 5810.23 1.00
divorce_rate[24] 0.43 0.41 0.43 -0.22 1.12 6717.94 1.00
divorce_rate[25] -0.03 0.31 -0.03 -0.53 0.49 6905.40 1.00
divorce_rate[26] -0.03 0.51 -0.02 -0.89 0.77 6514.63 1.00
divorce_rate[27] -0.15 0.39 -0.14 -0.78 0.50 4559.53 1.00
divorce_rate[28] -0.26 0.50 -0.28 -1.05 0.62 5159.18 1.00
divorce_rate[29] -1.80 0.25 -1.80 -2.19 -1.38 6357.90 1.00
divorce_rate[30] 0.17 0.42 0.17 -0.49 0.85 5562.08 1.00
divorce_rate[31] -1.66 0.16 -1.66 -1.93 -1.41 7311.42 1.00
divorce_rate[32] 0.11 0.24 0.11 -0.30 0.49 6410.47 1.00
divorce_rate[33] -0.05 0.50 -0.05 -0.80 0.83 4267.04 1.00
divorce_rate[34] -0.13 0.23 -0.13 -0.47 0.27 6629.46 1.00
divorce_rate[35] 1.27 0.40 1.27 0.65 1.94 4871.94 1.00
divorce_rate[36] 0.22 0.35 0.22 -0.34 0.83 5629.33 1.00
divorce_rate[37] -1.02 0.22 -1.02 -1.40 -0.68 6758.30 1.00
divorce_rate[38] -0.92 0.53 -0.93 -1.80 -0.07 5132.88 1.00
divorce_rate[39] -0.68 0.32 -0.66 -1.16 -0.12 6307.37 1.00
divorce_rate[40] 0.25 0.56 0.24 -0.73 1.11 6443.72 1.00
divorce_rate[41] 0.74 0.34 0.74 0.16 1.28 7850.45 1.00
divorce_rate[42] 0.20 0.18 0.20 -0.10 0.50 6700.29 1.00
divorce_rate[43] 0.80 0.45 0.81 0.04 1.53 3717.15 1.00
divorce_rate[44] -0.41 0.53 -0.42 -1.32 0.41 4688.78 1.00
divorce_rate[45] -0.39 0.25 -0.39 -0.81 -0.00 6562.05 1.00
divorce_rate[46] 0.13 0.31 0.12 -0.34 0.67 6398.53 1.00
divorce_rate[47] 0.56 0.45 0.57 -0.22 1.30 5244.04 1.00
divorce_rate[48] -0.63 0.27 -0.63 -1.10 -0.20 7119.43 1.00
divorce_rate[49] 0.86 0.60 0.88 -0.11 1.83 4082.14 1.00
sigma 0.58 0.11 0.58 0.40 0.76 1188.92 1.00
Number of divergences: 0
Effect of Incorporating Measurement Noise on Residuals¶
Notice that our values for the regression coefficients is very similar to Model 3. However, introducing measurement noise allows us to more closely match our predictive distribution to the observed values. We can see this if we plot the residuals as earlier.
[26]:
rng_key, rng_key_ = random.split(rng_key)
predictions_4 = Predictive(model_se, samples_4)(rng_key_,
marriage=dset.MarriageScaled.values,
age=dset.AgeScaled.values,
divorce_sd=dset.DivorceScaledSD.values)['obs']
[27]:
sd = dset.DivorceScaledSD.values
residuals_4 = dset.DivorceScaled.values - predictions_4
residuals_mean = np.mean(residuals_4, axis=0)
residuals_hpdi = hpdi(residuals_4, 0.9)
err = residuals_hpdi[1] - residuals_mean
idx = np.argsort(residuals_mean)
y = np.arange(50)
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(6, 16))
# Plot Residuals
ax.plot(np.zeros(50), y, '--')
ax.errorbar(residuals_mean[idx], y, xerr=err[idx],
marker='o', ms=5, mew=4, ls='none', alpha=0.8)
# Plot SD
ax.errorbar(residuals_mean[idx], y, xerr=sd[idx],
ls='none', color='orange', alpha=0.9)
# Plot earlier mean residual
ax.plot(np.mean(dset.DivorceScaled.values - predictions_3, 0)[idx], y,
ls='none', marker='o', ms=6, color='black', alpha=0.6)
ax.set(xlabel='Residuals', ylabel='State', title='Residuals with 90% CI')
ax.set_yticks(y)
ax.set_yticklabels(dset.Loc.values[idx], fontsize=10);
ax.text(-2.8, -7, 'Residuals (with error-bars) from current model (in red). '
'Black marker \nshows residuals from the previous model (Model 3). '
'Measurement \nerror is indicated by orange bar.');
The plot above shows the residuals for each of the states, along with the measurement noise given by inner error bar. The gray dots are the mean residuals from our earlier Model 3. Notice how having an additional degree of freedom to model the measurement noise has shrunk the residuals. In particular, for Idaho and Maine, our predictions are now much closer to the observed values after incorporating measurement noise in the model.
To better see how measurement noise affects the movement of the regression line, let us plot the residuals with respect to the measurement noise.
[28]:
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(10, 6))
x = dset.DivorceScaledSD.values
y1 = np.mean(residuals_3, 0)
y2 = np.mean(residuals_4, 0)
ax.plot(x, y1, ls='none', marker='o')
ax.plot(x, y2, ls='none', marker='o')
for i, (j, k) in enumerate(zip(y1, y2)):
ax.plot([x[i], x[i]], [j, k], '--', color='gray');
ax.set(xlabel='Measurement Noise', ylabel='Residual', title='Mean residuals (Model 4: red, Model 3: blue)');
The plot above shows what has happend in more detail - the regression line itself has moved to ensure a better fit for observations with low measurement noise (left of the plot) where the residuals have shrunk very close to 0. That is to say that data points with low measurement error have a concomitantly higher contribution in determining the regression line. On the other hand, for states with high measurement error (right of the plot), incorporating measurement noise allows us to move our posterior distribution mass closer to the observations resulting in a shrinkage of residuals as well.
References¶
McElreath, R. (2016). Statistical Rethinking: A Bayesian Course with Examples in R and Stan CRC Press.
Stan Development Team. Stan User’s Guide
Goodman, N.D., and StuhlMueller, A. (2014). The Design and Implementation of Probabilistic Programming Languages
Pyro Development Team. Poutine: A Guide to Programming with Effect Handlers in Pyro
Hoffman, M.D., Gelman, A. (2011). The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.
Betancourt, M. (2017). A Conceptual Introduction to Hamiltonian Monte Carlo.
JAX Development Team (2018). Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
Gelman, A., Hwang, J., and Vehtari A. Understanding predictive information criteria for Bayesian models


Python Sample scripts
The following are sample scripts intended to give an idea how to use the GDAL's Python interface. Please feel free to use them in your applications.
From GDAL 3.2, Python utility scripts Programs are located inside the osgeo_utils module. From GDAL 3.3, Python sample scripts are located inside the osgeo_utils.samples sub-module.
Python Raster Sample scripts
assemblepoly: Script demonstrates how to assemble polygons from arcs. Demonstrates various aspects of OGR Python API.
fft: Script to perform forward and inverse two-dimensional fast Fourier transform.
gdal2grd: Script to write out ASCII GRD rasters (used in Golden Software Surfer). from any source supported by GDAL.
gdal_vrtmerge: Similar to gdal_merge, but produces a VRT file.
gdalcopyproj: Duplicate the geotransform, projection and/or GCPs from one raster dataset to another, which can be useful after performing image manipulations with other software that ignores or discards georeferencing metadata.
gdalfilter: Example script for applying kernel based filters to an image using GDAL. Demonstrates use of virtual files as an intermediate representation.
get_soundg: Script to copy the SOUNDG layer from an S-57 file to a Shapefile, splitting up features with MULTIPOINT geometries into many POINT features, and appending the point elevations as an attribute.
histrep: Module to extract data from many rasters into one output.
load2odbc: Load ODBC table to an ODBC datastore. Uses direct SQL since the ODBC driver is read-only for OGR.
rel: Script to produce a shaded relief image from the elevation data. (similar functionality in gdaldem now)
tigerpoly: Script demonstrating how to assemble polygons from arcs in TIGER/Line datasource, writing results to a newly created shapefile.
tolatlong: Script to read coordinate system and geotransformation matrix from input file and report latitude/longitude coordinates for the specified pixel.
val_repl: Script to replace specified values from the input raster file with the new ones. May be useful in cases when you don't like value, used for NoData indication and want replace it with other value. Input file remains unchanged, results stored in other file.
vec_tr: Example of applying some algorithm to all the geometries in the file, such as a fixed offset.
vec_tr_spat: Example of using Intersect() to filter based on only those features that truly intersect a given rectangle. Easily extended to general polygon!
classify: Demonstrates using numpy for simple range based classification of an image. This is only an example that has stuff hardcoded.
gdal_lut: Read a LUT from a text file, and apply it to an image. Sort of a '1 band' version of pct2rgb.
magphase: Example script computing magnitude and phase images from a complex image.
hsv_merge: Merge greyscale image into RGB image as intensity in HSV space.
gdal_ls: Display the list of files in a virtual directory, like /vsicurl or /vsizip
gdal_cp: Copy a virtual file
Python Vector Sample scripts
ogrupdate: Update a target datasource with the features of a source datasource. Contrary to ogr2ogr, this script tries to match features between the datasources, to decide whether to create a new feature, or to update an existing one.
ogr_layer_algebra: Application for executing OGR layer algebra operations.
ogr_dispatch: Dispatch features into layers according to the value of some fields or the geometry type.
wcs_virtds_params: Generates MapServer WCS layer definition from a tileindex with mixed SRS
ogr_build_junction_table: Create junction tables for layers coming from GML datasources that reference other objects in _href fields
gcps2ogr: Outputs GDAL GCPs as OGR points
Python Coordinate Reference System Sample scripts
crs2crs2grid: A script to produce PROJ.4 grid shift files from HTDP program.
Python direct ports of c++ programs
gdalinfo: A direct port of apps/gdalinfo.c
ogrinfo: A direct port of apps/ogrinfo.cpp
ogr2ogr: A direct port of apps/ogr2ogr.cpp
gdallocationinfo: A direct port of apps/gdallocationinfo.cpp
Python sample scripts that are now programs
Sample scripts might be upgraded to proper gdal utilities (programs) in next versions with added functionality and documentation. The following samples from previous versions are now programs.
gdal2xyz.py: Translates a raster file into xyz format.
gdal_retile.py: Script for restructuring data in a tree of regular tiles.
val_at_coord: see gdallocationinfo


Grab the hottest Bayern Munich outfit in the market now! Support the german giant's winning streak by grabbing the new genuine official Bayern Munich Windbreaker for the year 2015-16
Following on from the Grünwald Stadium and later the Olympic Stadium, the stunning ground in the north of Munich is the third home venue in Bayern’s long Bundesliga history, and certainly the most spectacular. Europe’s most modern stadium, with a capacity of more than 70,000, has already established itself in the collective conscience of football fans all over the world within a short timescale. Visitors continue to be particularly impressed by the facade. 2,760 diamond-shaped cushions form the world’s biggest membrane cladding covering an area of 66,500 m². Even if it is not your first visit, it’s hard to believe the evidence of your own eyes at night matches when the whole stadium appears to be red when viewed from the outside. The three-tier interior of the Allianz Arena has extraordinary acoustics that rapidly turn the stadium into a cauldron when hosting thrilling encounters. It is no surprise that the Allianz Arena is nearly always full to the rafters. Around two million fans visit the stadium every season with all 71,000 available seats and standing places being sold out for all 17 Bundesliga home games. Bayern’s incredible attendance record invariably includes 68,000 full houses for European fixtures (the capacity is reduced by 3,000 in the all-seat configuration). As a rule, shortly after the Bundesliga fixtures are announced at the beginning of July, more than half of Bayern’s league games in the Allianz Arena sell out at once.
SFA Footballism Offer - $200.00 $89.95!!


--- title: "Google Cloud Build Sequential Build Numbers" date: "2018-12-12" categories: - "Google Cloud Build" - "Sequential Build Numbers" - "Build Numbers" - "GCB" - "CI" ---
Sequential build numbers are useful for communication between humans. It's much easier to say "build 845 went to production today" versus "cecce6da-3646-41c4-93ce-98069443d3c1 went to production today." Unfortunately, while Google Cloud Build is a great product, it doesn't yet have to ability to use seqential build numbers. As of this writing (December 2018), it is using UUIDs instead.
Below is a workaround to get sequential build numbers. It uses Google's Datastore product, which is priced per operation as well as data stored, and supports transactions. That makes it cheap (probably less than $1 per month) and safe for this use case. A program called `build_number` takes your Git repo name as an argument, and will atomically increment a counter in Google Datastore, then print the result to standard output.


Interested in connecting further? Drop a line!
gradeWebsite developed by Wassim Khemici using HTML5, CSS, Javascript, and Materialize. All photos and text are original content created by Wassim Khemici. grade


Last night, I read the science questions for the Indiana Academic Bowl our school hosted. The city schools all sent teams of students to participate in an evening of trivia and brain-power boxing. It was fun and it was great to see some of our own students putting their heads together to answer science questions.
I have a small problem with the name “Academic Bowl.” We all love our little bits of trivia, but labeling a trivia night as “academic” seems a little off to me. Academics should be pushing real-life problem solving and innovative thinking, and no question came close to that. Sure, our students can determine the initial velocity of an object with lightning speed, but what good is that in life? What are they going to do with that object once it is thrown?
Or better yet, why are they throwing that object at all?
I think we are sending a mixed message. We ask for creative and original thought in class, but when it “matters” (i.e. “tests”), we are still only asking for basic, rote-memorization, recall. This must change.
My Academic Bowl would look much more like Science Olympiad (do they still do that?). Teams are given a problem prior to the competition. Instead of drilling physics equations, they spend the time leading up to the event problem solving, planning, testing, and designing a solution. It could be something as simple as:
Design a system to keep an egg from breaking when dropped from a height of 50 feet.”
Or, something more complex:
“Design, sketch, and propose a location for a new power station in your city. Include resources needed, civic impacts, environmental concerns, and other pertinent information.
We need to remember that everything we expose our learners to sends a message and leaves some mark on their life.
Maybe I am feeling a little snarky this morning, but I do not want to let my kids think that “academics” equals “trivia.”
Written on March 21st , 2012 by Brian Bennett Categorized in: Science Teaching
Leave a comment


Beauty professional and also renowned make-up artist Sonia Kashuk, who has dealt with the globe’s leading models, answers your concerns and also offers pointers, techniques as well as devices to help you look your best.
Q: My make-up constantly looks fantastic right after I have actually used it yet after a couple of hrs it discolors. Just how do you make your makeup last throughout the day?
A: The response could stun you…… whether the item claims “long-wearing” or not, you can keep your make-up looking fresh from sunset ’til dawn using proper application techniques. The greatest trick I have actually discovered over the years: Loosened powder is your skin tone’s friend!
• • Concealer-Gently dot concealer as well as mix utilizing your finger or concealer brush around the under-eye area. Seal protection by lightly sweeping the location with a small powder brush cleaned with sheer, loose powder.
To attempt: Sonia Kashuk Hardly There Loose Powder in Translucent Matte.
• • Lotion Blush-I am a company follower that less is much more…… specifically with cream products. As a result of its uniformity, it is the nature of cream blush to proceed the face. One method to prevent this from happening is to apply a dusting of large loosened powder over the cheeks after you have actually applied the flush to keep it where it belongs-on your cheeks.
To try: Sonia Kashuk Crème Blush Duo in Color My Globe or Brighten My Day.
• • Eyeliner-Eyeliner smudges by nature, however tends to look sexier when slightly imperfect. Instead of utilizing black, which can look rough, I typically recommend attempting navy blue.
To attempt: Sonia Kashuk Eye Definer in Midnight. Pointer: For a strong, solid line, use a liquid lining for accuracy and stay-put surface.
• • Lipstick/Lip Gloss-There are items on the marketplace made particularly to prevent feathering, and also a secret trick of mine that I always use if I do not have my Lip Glossing/No Plume Pencil on me is to dab structure on your fingertip and smooth it over your lip line. This ravels any type of lines around the mouth and supplies an also surface for lip color to stick…… as well as stick to.
Q: Just how do I recognize what make-up colors work best for my coloring?
A: Here’s a shocking idea: To recognize shades that will certainly help you, utilize your hair as an overview, not your eyes! If you’re blonde, opt for creams as well as taupes. Brunettes look excellent in mochas and also delicious chocolate browns. If you’re auburn or a redhead, pick tones of copper as well as peach, or awesome tones like pinks and also lavenders. Gray hair looks gorgeous with tones of grays, soft purples and also blues.
If you’re unsure concerning shades, stick to neutrals. You could still achieve drama and also variety with neutral tones-they’re actually my favored to collaborate with. Or, making it really easy, grab among my brand-new Face Palettes, which are developed to correspond to your certain hair tone for adeptly coordinated, corresponding colors.
Sonia Kashuk Elegance is readily available exclusively at Target shops.
Do you recognize your best make-up shades? Appeal Specialist Sonia Kashuk could aid.


{% extends 'messenger/panel.html' %}
{% block column_contains %}
{% include "messenger/article.html" with zone_comment=True %}
{% include 'messenger/articles.html' with article=comment zone_comment=None articles=comments is_article=0 %}
{% endblock %}


{% macro deprecated(new_component_link, component_name) %}
This component has been deprecated.
You should remove this component and replace it with {{ component_name }}


I am the code of the team and I have done the systems of the project. My first task was to pass the base code to STL beacuse we had to use that library. After that, I did the two types of movement of the player:
Quest Movement: The player moves tile to tile
Lobby Movement: The player is free to move everywhere
Once I have finished that, I did the collisions of each movement:
Quest Collisions:
Lobby Collisions:
After the collisions, I did the pathfinding. Enemies move to the closest available tile
Now enemies can go to you, so what is next? Time to fight! My next step was to make the player basic attack and enemy basic attack.
As it can be seen in the video, when someone deals damage (or takes money, use mana, gain mana or hp...) a label appears showing the exact number of points, this was my work a few days later.
When I finished the basics attacks, I did the other player attacks:
Flash: Common for the 3 characters
Strong Basic Attack: Common for the 3 characters. This attack has life-steal, it steals 30%-80% of the damage.
Warrior ability: Deals damage in the 8 tile close to him.
Archer ability: Throw an arrow that go through enemies.

My next work was to do breakable objects such as rocks, barrels and snowmen.
Once, I finished this, I did what I think is my best work, The room System:
It remembers the order of rooms
Places the player in the right position after changing the room
It spawns the enemies, objects and drops. And It remembers if an enemy was killed and then it does not spawn again, same with objects and drops.
Change the music between rooms
Manages the map of the game while playing the quest
It plays a cutscene if the room has one.
It can close the doors and reopen when all enemies have been killed
Also, I did a small drop system: Each enemy has their % of droping diferents item (hp potion, mana potion or 4 diferent types of gems).
When our designer created the lobby map, I did the change scenes between lobby-home, home-lobby, shop-lobby, lobby-shop.
I have done interactable objects, they have a x number tile of range and if you are in you become able to interact.
As my research was about splines, I did a system that organizes splines.
We wanted that our player could choose the name he wanted, so I did an input text.
I also did the logic in the map when playing a quest
I did the shop of the game.
Finally, I did swapable controls.


Feature Definition
The features
feature_share
Teachers can share resources. For example: links, documents, essays, extracts, citations, abstracts, taxonomies, curricula, primary sources, blog posts, annotated web pages, video, photos, conversation logs. This feature can be expressed in a number of ways across a variety of platforms, including social media sharing but also more general wiki write-ups, or streamlined resources bundling and sharing.
feature_translation
This feature includes all aspects of managing an incoming queue of translation requests: job prioritization, translator reputation, job queuing and locking, content versioning, translation ranking, translation permissioning, translator payments. This feature is expressed with a translation
feature_discussion
Teachers can message and comment to each other in translated dialogue about various objects in the system; to gain support for
feature_enrichment
Teachers and administrators can enrich existing data with semi-automated metadata.
feature_curriculum
The curriculum development feature includes all aspects of managing coursework over a class or semester. This feature could be represented as a "curriculum mapping tool" with calendaring, Gantt-style charts,
feature_translation
feature_profile
Profile Creation and Management including avatar uploading, background and summary of interests, links to existing pages about each user of the site; for insight into teacher context.
feature_video
Support for video uploading and management for engaging students; to simplify a task that would otherwise be a major technical hurdle.
feature_rating
The teachers are able to rate content as being superior or inferior, probably on a simple scale of 0-3 stars; to help filtering of diverse information to capture what is relevant for busy teachers.


| < Day Day Up > |
8.4 High AvailabilitySo far we've concerned ourselves with the slaves. Using a proper heartbeat setup and load balancer, you can achieve a high degree of availability and transparency for MySQL-based applications. In its current state, MySQL doesn't offer a lot in the way of high availability support on the master, but that doesn't mean all hope is lost. In this section, we'll look at several high-availability solutions (both commercial and free). Each of the options considered has pros and cons, which we've done our best to document. 8.4.1 Dual-Master ReplicationWe looked at dual-master replication back in Chapter 7. While it doesn't help in scaling an application (both servers must handle the full write load), you can achieve much improved availability and transparency by putting a load balancer in the mix. Figure 8-5 illustrates this arrangement. Figure 8-5. Dual-master replication for high availabilityAside from the downsides mentioned in Chapter 7 (mostly a lack of conflict resolution), there isn't a lot that can go wrong with this setup. The worst problem is the potential for data loss, but that's really no different from master/slave replication. After a query writes a record to master 1, MySQL records the query in the binary log, and the other master has a chance to read it. If master 1 happens to crash between the time that the record is written and when the binary log is updated, however, the other master (and any slaves) will never know about the query. As far as master 2 is concerned, the query never happened. The solution would be for MySQL to provide synchronous replication with two-phase commit, but it doesn't. On the plus side, this solution is relatively easy to set up and understand. If you already know how to configure replication and have a working load balancer set up with good health checks, dual-master replication isn't much extra work. If you need to perform maintenance on the masters, you can simply take master 1 offline, do the work, bring it back online, and repeat the process on the other as soon as the first has caught up. Of course, it's best to do this gracefully. Set the health check to fail, and wait until clients are no longer accessing the master before shutting it down. Otherwise you risk interrupting in-progress transactions. If your load balancer is sophisticated enough, you can virtually eliminate the problem of conflict resolution. Here's how it works: rather than having both masters active, configure the load balancer so that master 1 is active, and master 2 is on standby.  Most load balancers provide a mechanism for doing this. However, a wrinkle occurs when master 1 comes back online. What should the load balancer do? If it begins sending connections to master 1 again, you'll have a situation in which writes could be occurring to both masters at the same time. That's a recipe for conflict. Remember, MySQL connections can be long-running, so the load balancer can't assume that clients will suddenly disconnect from master 2. The load balancer needs to be configured so that the notion of the "live master" changes only when the current live master goes down. 8.4.2 Shared Storage with StandbyBy increasing the cost and complexity of your infrastructure, you can eliminate the problem of lost updates described previously. Instead of two servers with their own copies of the data using replication to stay in sync, you can configure the active and standby masters to use shared storage.[3] It's very important to realize that the standby master shouldn't mount the filesystem or start MySQL until the first is offline.
Figure 8-6 shows one implementation of shared storage. It's worth pointing out that a load balancer isn't strictly necessary in this scenario. All you really need is an agent running on each node to monitor the other. If the agent running on master 2 finds that master 1 is unavailable, it takes over master 1's IP address and starts up MySQL with an identical configuration (same data directory, log filenames, etc.). If the configuration is truly identical, starting up MySQL on master 2 is logically no different from fixing master 1 and bringing MySQL up there. However, in reality there is an important difference: time. Master 2 is already booted and ready to go. Starting up MySQL takes a matter of seconds. The only delay is imposed by consistency checks on the data. Shared storage means the possibility of share corruption if you're not using InnoDB or BDB tables. Figure 8-6.  We don't recommend you try it unless you have a lot of time available for testing all the possible edge cases you're likely to encounter with flaky network equipment. Instead, spend some time looking at existing tools. There are numerous open source projects that can be adapted to do this for MySQL. The best candidate is keepalived (http://keepalived.sourceforge.net/), a keep-alive health check facility written to work in conjunction with LVS. There are also two commercial solutions on the market today, described in the next section. 8.4.3 Commercial SolutionsAs of this writing, there are two commercial products worthy of consideration for high availability. Each takes a completely different approach to solving the problem, so different sites may find one or the other suitable, or neither. Keep an eye on this market: we expect to see a lot of new development in this area in the next year or so. <IP_ADDRESS> Veritas cluster a gentVeritas has a well established reputation for providing the technology necessary to build many sorts of clusters. Their MySQL offering builds on the shared storage with standby model we just looked at. The cluster agent runs on both the active and standby nodes, monitoring the health of the primary master. When the agent detects a problem on the master, it brings the standby instance online and takes over the primary master's functionality. <IP_ADDRESS> EMIC NetworksEMIC Networks provides a full-blown clustering solution for MySQL. By combining a number of relatively inexpensive servers running EMIC's version of MySQL, you can create incredibly robust MySQL clusters without needing to worry about the single point of failure most other architectures have. |
| < Day Day Up > |


{{ define "main" }} {{/* Identify the section parameter for event lookup */}} {{$section := ".Params.program" }} {{/* Define any events that are related to this page by the lookup above */}} {{ $events := where (where (.Site.RegularPages) ".Section" "event") $section .File.Path }} {{/* If there are events, show the layout with the events listing */}} {{ if $events }}


Customers
Rentals
RV
Log out
View All
Create New
Create a brand new, fresh Rental Contract with a shiny and magnificent Customer
Date Start
Date End
Drop-Off Address
Pick-Up Address
Customer
Pick Customer
Motorhome
Pick Motorhome
Sales Assistant
Pick Sales Assistant
Extras
Child Seat
Bed Linen
Chairs
Picnic Table
Bike Rack
Cancel
Create


LogReporter
A great graphical tool to analyze traces
Static Public Member Functions
|
List of all members
reporter.Strings Class Reference
Static Public Member Functions
static String
getString
(String key)
Detailed Description
Definition at line
6
of file
Strings.java
.
Member Function Documentation
◆
getString()
static String reporter.Strings.getString
(
String
key
)
static
Definition at line
12
of file
Strings.java
.
Here is the caller graph for this function:
The documentation for this class was generated from the following file:
LogReporterTool/com/reporter/
Strings.java
reporter
Strings
Generated by
1.8.17


A confirmation box pops up when you click one of the buttons or hover over the blue boxes. If you click no, nothing happens. If you click yes, the provided handler in .confirmOn() is called.
| Normal button | |
| Button with confirmOn text options set | |
| Using mouseover event |
Mouseover me
|
| Using mouseover event with data argument |
Mouseover me
|
| Button with confirmOn and another class prefix |
Invetek - Loran Kloeze | https://github.com/invetek/jquery-confirmon


Mini-Game word chain [pokemon edition]
Discussion in 'Community Games' started by Indiboy, Feb 27, 2016.
Page 1 of 214
Go to First Unread
Page 1 of 214
Go to First Unread
Discussion in 'Community Games' started by Indiboy, Feb 27, 2016.


Functions | |
| template<typename Point1 , typename Point2 > | |
| void | boost::geometry::add_point (Point1 &p1, Point2 const &p2) |
| Adds a point to another. | |
| template<typename Point > | |
| void | boost::geometry::add_value (Point &p, typename detail::param< Point >::type value) |
| Adds a value to each coordinate of a point. | |
| template<typename P1 , typename P2 > | |
| P1 | boost::geometry::cross_product (P1 const &p1, P2 const &p2) |
| Computes the cross product of two vector. | |
| template<typename Point1 , typename Point2 > | |
| void | boost::geometry::divide_point (Point1 &p1, Point2 const &p2) |
| Divides a point by another. | |
| template<typename Point > | |
| void | boost::geometry::divide_value (Point &p, typename detail::param< Point >::type value) |
| Divides each coordinate of a point by a value. | |
| template<typename P1 , typename P2 > | |
| select_coordinate_type< P1, P2 > ::type | boost::geometry::dot_product (P1 const &p1, P2 const &p2) |
| Computes the dot product (or scalar product) of 2 vectors (points). | |
| template<typename Point1 , typename Point2 > | |
| void | boost::geometry::multiply_point (Point1 &p1, Point2 const &p2) |
| Multiplies a point by another. | |
| template<typename Point > | |
| void | boost::geometry::multiply_value (Point &p, typename detail::param< Point >::type value) |
| Multiplies each coordinate of a point by a value. | |
| template<typename Point1 , typename Point2 > | |
| void | boost::geometry::subtract_point (Point1 &p1, Point2 const &p2) |
| Subtracts a point to another. | |
| template<typename Point > | |
| void | boost::geometry::subtract_value (Point &p, typename detail::param< Point >::type value) |
| Subtracts a value to each coordinate of a point. |
| void boost::geometry::add_value | ( | Point & | p, | |
| typename detail::param< Point >::type | value | |||
| ) |
Adds a value to each coordinate of a point.
| p | point | |
| value | value to add |
Definition at line 86 of file arithmetic.hpp.
| void boost::geometry::add_point | ( | Point1 & | p1, | |
| Point2 const & | p2 | |||
| ) |
| void boost::geometry::subtract_value | ( | Point & | p, | |
| typename detail::param< Point >::type | value | |||
| ) |
Subtracts a value to each coordinate of a point.
| p | point | |
| value | value to subtract |
Definition at line 118 of file arithmetic.hpp.
| void boost::geometry::subtract_point | ( | Point1 & | p1, | |
| Point2 const & | p2 | |||
| ) |
| void boost::geometry::multiply_value | ( | Point & | p, | |
| typename detail::param< Point >::type | value | |||
| ) |
Multiplies each coordinate of a point by a value.
| p | point | |
| value | value to multiply by |
Definition at line 150 of file arithmetic.hpp.
| void boost::geometry::multiply_point | ( | Point1 & | p1, | |
| Point2 const & | p2 | |||
| ) |
| void boost::geometry::divide_value | ( | Point & | p, | |
| typename detail::param< Point >::type | value | |||
| ) |
Divides each coordinate of a point by a value.
| p | point | |
| value | value to divide by |
Definition at line 183 of file arithmetic.hpp.
| void boost::geometry::divide_point | ( | Point1 & | p1, | |
| Point2 const & | p2 | |||
| ) |
| P1 boost::geometry::cross_product | ( | P1 const & | p1, | |
| P2 const & | p2 | |||
| ) |
Computes the cross product of two vector.
Both vectors shall be of the same type. This type also determines type of result vector.
| p1 | first vector | |
| p2 | second vector |
Definition at line 91 of file cross_product.hpp.
| select_coordinate_type<P1, P2>::type boost::geometry::dot_product | ( | P1 const & | p1, | |
| P2 const & | p2 | |||
| ) |
Computes the dot product (or scalar product) of 2 vectors (points).
Definition at line 59 of file dot_product.hpp.
|
December 1, 2009 |
Copyright © 1995-2009 Barend Gehrels, Geodan, Amsterdam Copyright © 2008-2009 Bruno Lalande, Paris Copyright © 2009 Mateusz Loskot, Cadcorp, London |


{include="includes"}{$timezone_js}
KT-Shin
- A
Shinterest
/
Shaarli
fork
It looks like it's the first time you run KT-Shin. Please configure it:
Login
Password
{$timezone_html}
Page title
{include="page.footer"} {include="includesfooter"}


--- layout: index-layout subject: Free Gift With Purchase ---
Free Tote Bag over $60!
$29.00
Shop Now$42.00
Shop Now$28.00
Shop NowNo longer want to receive these emails? Unsubscribe.
Elate Beauty - US 18B 26 Bastion Square Victoria, British Columbia V8W 1H9


In class-based object oriented programming, classes act as templates for the creation of object instances. They describe the what members each object will have, how these members will be initialized, where they will be visible from. In strictly typed languages it is also possible to define the type of each member. All this information, is essentially meta data for the objects, because it describes the object itself. Therefore, a class contains the meta data of its objects.
Yet, with the use of the few available keywords (public, private, static, abstract etc., we can only describe just a few aspects of an object. It would be nice if we could describe more details. Many languages, including C# and Java, offer this option through the use of compile time attributes on each member of a class. So, putting attributes on a class and its members gives a way to further describe the behaviour of the objects.
In PHP, things are much more limited. First of all, it is not possible to define the type of each field. In addition, there is no standard way to put attributes. Many frameworks provide this by using special tags in documentation comments, but this is not supported directly by the language and therefore it is non-standard and limited. In order to provide this functionality, Oxygen relies on standard PHP features like abstraction, inheritance, and late static binding.
The ability to describe with meta data an object is very useful when it come to the central business objects of an application, which is usually called "The Model". In Oxygen the basic abstract class XItem provides this ability to all the objects that extend it. Every XItem class is linked with an XMeta object that contains all the extra information about the objects. All one has to do is to extend the XItem class and fill the extra information in the XMeta object.
Imagine a simple bookstore application, that has a class Book. To keep things simple, let's say that each book has only one field witht the name "Title". An implementation of this concept in PHP could be like this:
class Book { public $Title; }
That's pretty much everything we can say about the book in the class definition. However there are a lot of extra information that we would like to put here. For example, all books are stored in the table "book" of a database. The title of the book is actually a string, which can be empty but not null. When print in the user interface, the title should go under the label "Title" in English but "Titre" in French. I can continue forever with such requirements. My point is that all this info does not change during the application and describe all book instances. Therefore, we are talking about meta-data of a book which have to be coded in the class of book.
Here is the code to do this in Oxygen:
class Book extends XItem { public $Title; public static function FillMeta(XMeta $m){ $m->SetDBTableName('book'); $m->Title = XMeta::String()->WithLabel('en','Title','fr','Titre'); } }
By overriding the static function FillMeta (with late static binding), we put all the meta-data in an XMeta object. This object is used by Oxygen for many things, including loading from a database, exporting to XML, reading from an HTTP POST request etc.. However, the XMeta object can be used for a variety of other things, so here is an easy way to retrieve it:
$meta = Book::Meta(); echo $meta->Title->GetName() // output: Title echo $meta->Title->GetLabel() // output: Title in English or Titre in French
Each instance of an XItem has a unique identifier under the field id. This identifier is used by Oxygen as a key to the objects. Essentially, if two objects of the same type have the same id, they are considered equal. In addition, there is an id-based central repository of all objects that make it easy to keep just one instance of each object in memory. In other words, if two objects of the same type have the same id, they should be loaded only once.
The id field is also used as the primary key for database operations. For more info about this see Object Relational Mapping.
Most of meta information regard the fields of an object. In a meta object, all fields are represented by instances of the class XMetaField. Every meta field is based on an XType which is responsible for the all the conversions, ie. importing a value into a field of an object and exporting the value from the field (see the chapter The type system). The easiest way to create an XMetaField is from its XType, but for the most common types, this can be also done width convenience functions from XMeta:
$field = MetaString::Field(); // or $field = XMeta::String();
However, the class XMetaField is designed to work in cooperation with the class XMeta. This is because, apart from the type, each field must also have a name. The name is automatically assigned by the XMeta class, after the call to FillMeta, and it is equal to the name of the PHP field inside the final filled meta object.
$field = XMeta::String(); $name = $field->GetName(); // $name is null, because it is not filled in by any XMeta object.
In addition, every field is automatically assigned a label, which is equal to the lemma with the same name in the dictionary (see the chapter on Internationalization). The value of the label can be changed with the method WithLabel which is overloaded.
// No call $m->Title = XMeta::String(); // label: oxy::txtTitle() // Call with one string argument $m->Title = XMeta::String->WithLabel('BookTitle'); // label: oxy::txtBookTitle() // Call with one Lemma argument $m->Title = XMeta::String()->WithLabel( oxy::txtBookTitle() ); // label: oxy::txtBookTitle() $m->Title = XMeta::String()->WithLabel( new Lemma('en','Title','fr','Titre') ); // label: Lemma('en','Title','fr','Titre') // Call with two or more string arguments $m->Title = XMeta::String()->WithLabel('en','Title','fr','Titre'); // label: Lemma('en','Title','fr','Titre')
Apart from these general properties, the field carries also specific information regarding the way it is going to be automatically loaded from a database of from an XML document. Here is an example of a field with database and XML mapping:
$m->Title = XMeta::String()->WithLabel('BookTitle')->WithDBAlias('title')->WithXmlAlias('title')->WithXmlBehaviour(Xml::Element);
These extra properties are presented in the chapters Object Relational Mapping and XML mapping respectively.
Apart from the class XItem, Oxygen has also the class XList which can be seen as an array of such items. This array, however, is aware of the extra meta information. This fact will be used by the ORM module to automatically retrieve lists of items for example.
Each list is based on a meta object. For example, a list of books can be created from the book meta object of book:
// either $a = Book::Meta()->MakeList(); // or $a = Book::MakeList();
The list object is used similar to a PHP array:
$a = Book::MakeList(); $a[] = new Book(); echo count($a); // output: 1;
In addition, the XList implements the LINQ interface (see the chapter on LINQ) which means that all LINQ methods are directly available without wrapping with the from() function.
$a = Book::MakeList(); $x = new Book(); $x->Title = 'Lord of the rings'; $a[] = $x; $x = new Book(); $x->Title = 'Alice in wonderland'; $a[] = $x; foreach ($a->Skip(1) as $x) echo $x->Title; // output: Alice in wonderland
It is now evident that there is a direct analogy between the field of an object and the fields of its meta object. the value of a field is a PHP variable while the value of the meta field is an XMetaField object.
$x = new Book(); $x->Title = 'Alice in wonderland'; $m = Book::Meta(); echo gettype($x->Title); // output: string echo gettype($m->Title); // output: object echo get_class($m->Title); // output: XMetaField
When working with the values of a field, we perform operation on the PHP variable. For example, to check whether the title is equal to 'Alice in wonderland', we have to go through an equality operation between the value of the variable and another string, like this:
$compare = $x->Title == 'Alice in wonderland';
This operation compares the value of the specific book with a string. Shifting this operation to the meta object level would imply the comparison of the title any given book with a value. This can be expressed in PHP with a closure:
$compare = function($x){ return $x->Title == 'Alice in wonderland'; };
This type of operation that returns a boolean value is called a predicate. In general, the predicates that operate on a field and a value are very useful. However, it is very common to be hard coded in another language, such as SQL:
SELECT * FROM Book WHERE Title = 'Alice in wonderland' -- a predicate in SQL
One of the main goals of Oxygen is to reduce the development time, which includes not only the time spent on the first version of an application but also the required time to make changes on top of the initial code. This time is directly linked to the modularity and the reusability of the code. In the above example, the query is not at all reusable. the predicate is encoded in SQL and rests inside a PHP string. This gives limited flexibility: there is no direct way to perform operations on predicates. So, if there is a query to find the books published in 2011 and another query to find the books written in english, then there is no way to produce a query to find the english books of 2011. In order to to this, one have to extract the predicates from the initial queries and then perform operations on them. Yet, the predicates would be still strings, and there is no predefined operation to combine the two string predicates. So, the operation is defined ad-hoc: concatenate the two strings with an 'AND' in between.
The other problem with the SQL statement is that it has to be executed in a database server. Once the data is retrieved from the server, the predicate is no longer valid. So, the code is required to check if the title is equal to 'Alice in wonderland' in the database level is different from the code in PHP level and probably the code in Javascript level. Yet, all three pieces do the same thing.
From the above example, it becomes evident that it would be better to work with structures that support all the necessary operations and can be produce the code in any language. For this reason, Oxygen uses the class XPred, which encapsulates an abstract predicate. Much emphasis is given to the predicates based on fields. These predicates can be created directly from the meta fields:
$m = Book::Meta(); $pred = $m->Title->Eq( 'Alice in wonderland' );
Apart from the equality, there are many other predicates based on fields that can be retrieved the same way. These include NotEq (=not equal), IsNull, IsNotNull, Lt (=less than), Le (=less than or equal to), Gt (=greater than), Ge (=greater than or equal to), In (=exists in), NotIn, Like and NotLike.
$m = Book::Meta(); $pred = $m->Year->Ge( 2000 ); // where the year is greater or equal to 2000
It is possible to combine predicates with the boolean operations (AND, OR and NOT) in order to produce other more complex ones.
$m = Book::Meta(); $pred_is_english = $m->Language == 'en'; $pred_is_published_in_2010 = $m->Year->Eq(2010); $pred_is_published_in_2011 = $m->Year->Eq(2011); $pred_is_not_english = $pred_is_english->IsNotTrue(); // NOT $pred_is_english_published_in_2011 = XPred::All( $pred_is_english , $pred_is_published_in_2011 ); // AND $pred_is_published_in_2010_or_2011 = XPred::Any( $pred_is_published_in_2010 , $pred_is_published_in_2011 ); // OR
In addition, it is possible to have predicates with operations on two fields. This is not very often, but it is used in certain cases:
$m = Book::Meta(); $pred_is_a_translation = $m->Language->NotEq( $m->OriginalLanguage );
Predicates can be used directly on an XList, instead of closures. For instance, the following two lists are equivalent:
$list1 = $list->Where( function($x){ return $x->Title == 'Alice in wonderland'; } ); $list2 = $list->Where( Book::Meta()->Title->Eq( 'Alice in wonderland' ) );
However, the first version is limited to be used only in PHP. It does not carry information about how to be converted to SQL or to Javascript code. On the other hand, the second version is linked with the meta object which provides all the information needed for the translation. This will be analysed further in the chapter on Object Relational Mapping.
Much like the predicates, sorting is another operation that is very often coded multiple times in the same application (in SQL, in PHP and in Javascript). Oxygen offers a similar solution with the class XOrderBy. This can also be created from the meta field:
$m = Book::Meta(); $order_by_title_asc = $m->Title->Asc(); $order_by_year_desc = $m->Year->Desc();
Ordering with more than one fields can be done with the method ThenBy
$order_by_year_desc_and_title_asc = $order_by_year_desc->ThenBy( $order_by_title_asc );
The XOrder objects can be used in XList instead of closures. The following two clauses are equivalent, but once again the first one can only be used in PHP, while the second is translatable to other languages.
$list1 = $list->OrderBy( function($x){ return $x->Title; } ); $list2 = $list->OrderBy( Book::Meta()->Title->Asc() );
Finally, it is possible to define a default sorting for items inside the meta class.
class Book extends XItem { ... public static function FillMeta(XMeta $m){ $m->Title = XMeta::String(); $m->OrderBy( $m->Title->Asc() ); } ... }


Press n or j to go to the next uncovered block, b, p or k for the previous block.
| File | Statements | Branches | Functions | Lines | |||||
|---|---|---|---|---|---|---|---|---|---|
| backend |
|
80% | 8/10 | 33.33% | 1/3 | 100% | 1/1 | 80% | 8/10 |
| backend/src/controller |
|
72.13% | 220/305 | 33.33% | 6/18 | 79.17% | 38/48 | 71.53% | 201/281 |
| backend/src/entity |
|
100% | 101/101 | 100% | 0/0 | 100% | 14/14 | 100% | 83/83 |
| backend/src/enums |
|
96.55% | 28/29 | 91.67% | 11/12 | 100% | 3/3 | 96.55% | 28/29 |
| backend/src/middleware |
|
81.82% | 18/22 | 50% | 2/4 | 100% | 5/5 | 81.82% | 18/22 |
| backend/src/router |
|
93.55% | 58/62 | 100% | 0/0 | 0% | 0/4 | 95.08% | 58/61 |
| backend/test |
|
96.97% | 32/33 | 100% | 0/0 | 88.89% | 8/9 | 96.88% | 31/32 |