Chapter 9 Messy and Missing Data
You have to love the nice, complete datasets I have served up to you in this and other statistics texts. Indeed, some trials will work out this way – in general, the simpler the treatment (and the technology used to apply it) the greater your chances of a complete dataset. Planters will jam, nozzles will plug, and if a trial has 12 treatments a couple of them may end up in the wrong plot. Best to avoid those if possible.
As well, the smaller the experiment, or the more controlled the environment, the greater your odds of a complete dataset. For that reason, decades of university and industry research has been performed in 10- x 40- plots stacked closely together on manicured, table-top-flat ground. If you were a seed-breeder, you had dibs on these fields.
If you can farm in your head or your basement, drop me a note and I will exempt you, with envy from this unit. For those of us who farm by the acre or section, however, the issue of data quality is an agonizing and sometimes subjective topic – but critical. Remember, most of our statistics are models, and the saying goes: “junk in, junk out.” Your models are only as good as the data you use to build them.
Part of your job as a researcher or end-user of any data, before you conduct or read the results from any test, is to ask yourself – are the data reliable enough to support the inferences? A trial with wildly-different experimental units may bias results – if all the replicates of one treatment end up in the better units and others are concentrated in the poorer ones. You may falsely recommend a product if you don’t catch this.
At the other extreme, the failure conduct research on similar experimental units will introduce background variance (or noise) that prevents a statistical test from concluding a difference is not the result of chance, even though the treatments are, in fact, different. In that case, you may fail to introduce – or adopt – a product or technology with real promise.
In this unit, we will first learn ways to inspect datasets for extreme values which, even given the inherent variability of data, may be suspect. Boxplots, histograms, and probability plots will be our tools for these exercises.
We will then address the uncomfortable question of what to do when we have missing data, either because a plot was compromised during the trial, or because we rejected that plot because its extreme value.
9.1 Inspecting data for Normal Distributions
I know, I know, it is so exciting to have data! The hard physical work of the research is done, the data is painstakingly entered into Excel. Let’s run the ANOVAs and regressions now – if we hurry we can still make it to happy hour!
It is so tempting to jump right into deep analyses as the data roll in. But it is important to realize these analyses are based on assumptions:
- That the observations for each treatment level are normally distributed around the treatment mean
- That the variance or spread of observations around each level of treatment is roughly equal.
- That experimental units are comparable among treatment levels, so that the treatment and error effects are appropriately separated.
It is very possible, based on the nature of trials, that one or more of these assumptions may be violated. If you have ever counted weeds in a herbicide trial, you have noted that well-treated plots have weed counts that are consistently near zero – but that weedy checks have wildly variable counts (100, 200, 300 weeds). Growth analysis (where plants of different sizes are measured) is also prone to messy data issues, because variance in measurements increases numerically as plants grow. Experimental units (plots) in both commercial and research farm fields can vary because of prior management unknown to the researcher.
9.1.1 Histograms
In the very first unit of this course Unit 2, you were introduced to the histogram. Recall the histogram is a vertical bar chart in which the width of bars defines the different ranges into which observations are grouped, and the height represents the count or proportion of observations falling into each range.
In the data below, we have a dataset with 500 observations of corn yield. The mean is approximately 180. We can see the data distribution is approximately normal.
library(tidyverse)
# library(fGarch)
#
set.seed(082720)
= data.frame(obs_value = rnorm(500, mean=180, sd=5)) %>%
norm_data mutate(dataset="normal")
# set.seed(5)
# skewed_data = data.frame(obs_value = rsnorm(1000, mean = 20, sd = 10, xi = 100)) %>%
# mutate(dataset="skewed")
#
# norm_and_skewed = rbind(norm_data, skewed_data) %>%
# as.data.frame() %>%
# dplyr::filter(obs_value>0)
#
#
# write.csv(norm_and_skewed, "data/norm_and_skewed.csv", row.names = FALSE)
# norm_and_skewed = read.csv("data/norm_and_skewed.csv")
%>%
norm_data # filter(dataset=="normal") %>%
ggplot(aes(x=obs_value)) +
geom_histogram(breaks = seq(160,200,2), fill="darkolivegreen", color="black")
The summary data are shown below. We can see that the median and mean are both approximately 180 bushels per acre. We can also see the 1st and 3rd quantiles (equal to the 25th and 75th percentiles) are a little over three bushels from the median. The minimum and maximum observations are also similarly spaced from the median.
summary(norm_data$obs_value)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 161.5 176.9 180.0 180.2 183.4 194.1
When we are dealing with data such as pest counts, our data may be non-normal. Rather than being symmetrical, the data may be skewed to one side or another. For example, in the dataset below, total velvetleaf dry weight in grams per square meter was measured. If you have worked like me with weed populations, you realize weed competitiveness is all about outracing the crop to the sun. If the weed loses, which it will in most cases, it will be small. But the proud few weeds who beat the crop will be huge. That is reflected in the data below.
library(fGarch)
## Loading required package: timeDate
##
## Attaching package: 'timeDate'
## The following objects are masked from 'package:agricolae':
##
## kurtosis, skewness
## Loading required package: timeSeries
## Loading required package: fBasics
##
## Attaching package: 'fBasics'
## The following object is masked from 'package:muStat':
##
## stdev
set.seed(5)
= data.frame(obs_value = rsnorm(1000, mean = 6, sd = 4, xi = 50))
velvetleaf_skewed
%>%
velvetleaf_skewed ggplot(aes(x=obs_value)) +
geom_histogram(fill="darkolivegreen", color="black", binwidth = 1)
When we look at the histogram, the data are skewed to the right. The histogram is not symmetrical.
summary(velvetleaf_skewed$obs_value)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.7237 2.7993 5.1859 6.0021 8.5883 23.7734
When we look at the summary data, we first notice the mean and median are different. For a dataset this size (1000 observations, we would expect them to be more similar.) We notice that the first quantile is closer to the median than the third quantile. The greatest indication the data is skewed, however, is is that the minimum is about 4 plants less than the median, while the maximum is about 18 plants greater.
Data like this may be transformed (mathematically re-scaled) so that it is more normal for analyses. We will cover this below.
9.1.2 Rank Percentile Plots
Another way to inspect the normality of datasets is to use a rank percentile plot. This plot uses the percentile rank of each observation, from lowest to highest, as the x-value of each point. The y-value of the point is its observed value.
The data for our normally-distributed corn yield dataset are plotted in the rank percentile plot below. Normally-distributed data tend to be strongly linear in the middle of the plot. If we draw a regression line through the plot, you can see most of the data are close to that line. The lowest percentile points fall below the line. That means they are a little lower in value than the normal distribution function might predict. The opposite is true of the highest percentile points. This indicates our distribution is a little bit wider than normal, but not enough that we cannot use it for analysis.
%>%
norm_data mutate(rank = percent_rank(obs_value)) %>%
ggplot(aes(x=rank, y=obs_value)) +
geom_point()
Our skewed data, however, shows up quite differently in the rank percentile plot. We can see that most of the data closely fit a line. But starting around the 75th percentile, the observed values are much greater than the predicted values – almost twice as much. This means the distribution is much wider to the right of the distribution curve than to the left, and that the data are non-normal
%>%
velvetleaf_skewed mutate(rank = percent_rank(obs_value)) %>%
ggplot(aes(x=rank, y=obs_value)) +
geom_point()
9.1.3 Box Plots
The first two methods I have shown you, the histogram and rank percentile plots, are useful if you have a few treatments with a large number of replicates. They are taught in every statistics course and you should know about them. But, in my experience, they are not useful if you have a trial with fewer replications. A normal distribution is not going to appear in a histogram if you only have four replicates – instead you will just see the four individual measurements.
Box plots, on the other hand, are very useful for inspecting multiple treatments. In the plot below, boxplots for four corn treatments are shown. The treatments are labeled A, B, C, and D. The data are plotted so their treatments are listed along the vertical axis, and their values are listed along the y-axis.
= data.frame(trt = rep(c("A", "B", "C", "D"), each=4),
norm_data_mult_trts mean = rep(c(160,170,180,190), each=4))
set.seed(082720)
= norm_data_mult_trts %>%
norm_data_mult_trts mutate(error = rnorm(16, 0, 5)) %>%
mutate(yield = mean + error) %>%
# create outlier
mutate(yield = if_else(row_number()==8, 195, yield))
%>%
norm_data_mult_trts ggplot(aes(x=trt, y=yield)) +
geom_boxplot(outlier.colour="black", outlier.shape=16,
outlier.size=2, notch=FALSE, fill="lightblue") +
coord_flip() +
stat_summary(fun=mean, geom="point", shape=23, size=2)
The boxplots help us understand the distribution of the data. Lets start with the box, which tells about the spread of the data. The left side of the box is the 25th percentile, the line in the middle is the 50th percentile (or median), and the right side of the box is the 75th percentile. So the box shows us the spread of the middle half of the observations for each treatment.
The diamond shown within the box is the mean. In a normal distribution, the median and mean should be close.
The lines extending from the left and right side of the box are called whiskers. The whiskers extend to the lowest and highest observations for a treatment. The whiskers extend no more more than 1.5 times the inter-quartile range, which for the lower whisker is the the difference between the 25th and 50th percentiles, and for the upper whisker is the difference between the 50th and 75th percentiles.
In treatment B, we can see the upper whisker is missing, and instead there is a point to the right of the bar. If an observation is beyond 1.5 times the interquartile range, the whisker is not shown and the observation is instead represented by a point. This observation is called an outlier, meaning that it outside the range of values expected in a normal distribution. We will talk more about outliers in the next section.
The boxplot tells us something beyond the distribution of the individual treatments. If the boxes are markedly different in their width, the data may have substantially different variances. We should investigate these further using a mean-variance plot, and perhaps a statistical test of heterogeneity.
9.2 Inspecting Data for Equal Variances
So far, we have learned to use the t-test and analysis of variance to test named treatments (that is, hybrids, management practices, and other products that can be described by name). These tests generally assume not only that observed values are normally distributed, but that the variances are approximately equal among the different treatments in our experiment. If the variances are unequal, we may calculate least significant differences (LSDs) or honest significant differences (HSDs) that are inappropriate. Among treatments that have smaller variances, our LSD or HSD may be overestimated; among treatments that have larger variances, the LSD or HSD may be underestimated.
The visual inspection of individual treatment distributions in the box plot above, followed by a scatter plot of the treatment variances versus their means, can give us a visual sense of unequal variances. These suspicions can then be tested using a Test for Homogeneity that calculates the probability of differences in variances as greater as those observed.
9.2.1 Mean-Variance Plot
In a mean-variance plot, the treatment means are plotted along the horizontal axis and the variances are plotted along the vertical axis. The plot for the corn yield dataset we have used so far is shown below.
= norm_data_mult_trts %>%
norm_stats group_by(trt) %>%
summarise(mean=mean(yield),
var=var(yield)) %>%
ungroup()
ggplot(data=norm_stats, aes(x=mean, y=var)) +
geom_point(aes(color=trt), size=3)
We can see the variance of treatment B is many times greater than that of the other treatments. In general, we like to see the variances differ by no more than a factor of 2.
In cases where we are dealing with populations that either “thrive or die” based on environment – particularly pest populations – we may see relationships between the mean and variance. Pest count data is often like this. In our velvetleaf counts, for example, we might find that our greater treatment means are also associated with greater variations in counts between plots.
<- function(n,mean,sd) { mean+sd*scale(rnorm(n)) }
rnorm2 <- rnorm2(100,4,1)
r mean(r) ## 4
## [1] 4
sd(r)
## [1] 1
set.seed(3)
= data.frame(A = rnorm2(4, 5, sqrt(2)),
norm_data_prop_var B = rnorm2(4, 10, sqrt(6)),
C = rnorm2(4, 15, sqrt(10)),
D = rnorm2(4, 20, sqrt(14))) %>%
gather(trt, yield)
%>%
norm_data_prop_var ggplot(aes(x=trt, y=yield)) +
geom_boxplot(outlier.colour="black", outlier.shape=16,
outlier.size=2, notch=FALSE, fill="lightblue") +
coord_flip() +
stat_summary(fun=mean, geom="point", shape=23, size=2)
In this case, the mean-variance plot may show a linear relationship between variance and mean.
= norm_data_prop_var %>%
norm_data_prop_var_stats group_by(trt) %>%
summarise(mean=mean(yield),
var=var(yield)) %>%
ungroup()
%>%
norm_data_prop_var_stats ggplot(aes(x=mean, y=var)) +
geom_point(aes(color=trt), size=3)
Finally, we may observe a dataset in which the distributions not only increase with means, but seem to do so exponentially.
<- function(n,mean,sd) { mean+sd*scale(rnorm(n)) }
rnorm2 <- rnorm2(100,4,1)
r mean(r) ## 4
## [1] 4
sd(r)
## [1] 1
set.seed(3)
= data.frame(A = rnorm2(4, 5, 0.3),
norm_data_prop_sd B = rnorm2(4, 10, 2.5),
C = rnorm2(4, 15, 4.7),
D = rnorm2(4, 20, 6.9)) %>%
gather(trt, yield)
%>%
norm_data_prop_sd ggplot(aes(x=trt, y=yield)) +
geom_boxplot(outlier.colour="black", outlier.shape=16,
outlier.size=2, notch=FALSE, fill="lightblue") +
coord_flip() +
stat_summary(fun=mean, geom="point", shape=23, size=2)
In this case, the mean-variance plot may show a curved relationship between variance and mean.
= norm_data_prop_sd %>%
s_norm_stats group_by(trt) %>%
summarise(mean=mean(yield),
var=var(yield),
sd=sd(yield)) %>%
ungroup()
ggplot(data=s_norm_stats, aes(x=mean, y=var)) +
geom_point(aes(color=trt), size=3)
We may want to check whether the standard deviation has a more linear relationship to mean.
ggplot(data=s_norm_stats, aes(x=mean, y=sd)) +
geom_point(aes(color=trt), size=3)
The largest mean has a significant difference of 6.9, while the smallest mean has a significant difference of 0.3. In other words, the largest significant difference is 23 times the smallest significant difference.
9.2.2 Homogeneity of Variance Tests
In previous units, we learned how to compare two variances – using the F-test. In the Analysis of Variance, we tested whether the variance among treatment means was greater than the variance within treatments. If the treatment variance was sufficiently greater than the error variance, we concluded the treatment effect explained a significant amount of the variation in observed values.
In this unit, we want to do something similar – we want to compare the variances associated with multiple treatments to see if they are significantly different. When data are normally-distributed, the method for comparing multiple variances is Bartlett’s Test. (If you continue your statistical travels, you may come across Levene’s Test, but that is for non-normal data.)
Bartlett’s test, as best as I can tell (the formula is awful and no one seems willing to explain it), acts like a sum of squares for variances, comparing the variances of the individual treatments to their mean when pooled together. This is an incomplete explanation, but I hope it will satisfy the curious. Fortunately for us, the test is easy to implement in R and the output simple.
If we run Bartlett’s test on our corn data above, we get the following results.
bartlett.test(yield~trt, norm_data_mult_trts)
##
## Bartlett test of homogeneity of variances
##
## data: yield by trt
## Bartlett's K-squared = 6.0302, df = 3, p-value = 0.1102
Let’s go through the output. “Bartlett’s K-squared” is the statistic produced by the nasty formula I referenced above. Don’t worry about that. The degrees of freedom refers to the four treatments whose variances we are comparing. Most important, of course, is our p-value. There are many opinions on when to transform data – but I would recommend against transforming data unless the –value is less than 0.01. I would also recommend running your data on both the transformed and untransformed data and comparing results. If transformation does not change your inferences, then skip it.
Here is the Bartlett’s test on our velevleaf data where the mean and standard deviation were linearly related:
bartlett.test(yield~trt, norm_data_prop_sd)
##
## Bartlett test of homogeneity of variances
##
## data: yield by trt
## Bartlett's K-squared = 14.217, df = 3, p-value = 0.002625
In this case, we will want to analyze both the transformed and untransformed data before deciding which to us for our final inferences.
9.3 Dealing with Messy Data
Dealing with messy data is one of the more uncomfortable aspects of statistics, but also one of the most important. Our tests and summary statistics are based on assumptions. For tests, we assume the data are from a populations that have approximately normal distributions. We also assume they have variances that are equal – otherwise our mean separation tests will fail.
And finally, and this is a concept that I learned just while writing this: the validity of our inferences is based on the assumption that our samples represent the population that we are studying. Which brings us back to outliers.
9.3.1 Outliers
Outliers can have a powerful effect in skewing data. Particularly in smaller datasets (i.e. fewer than 10 replicates per treatment), an outlier can have noticeable effects on a distribution’s normality and its variance. In regression analyses, one outlier can significantly change the slope of the model.
Does this mean that outliers should be omitted from the dataset? Not necessarily – first we should inspect the data more closely. The outlier might be a mistake in recording a measurement. It could be an inconsistent scale in a combine. These would be experimental errors that mean the outlier is an artifact of our methods, rather than a representative sample from our population. In that case, we may want to remove that observation from our dataset.
But investigating the outlier may also include examining the location where it was taken. This can be difficult if you are not the primary researcher and on-site where the data were gathered. But if you can overlay your plot map with a soils map, or work with aerial imagery, or examine as-applied maps, you may be able to identify differences in environment or management that caused a dramatic difference in the observed value.
In such a case, you may decide that plot did not represent the environment about which you were trying to draw inferences, and choose to omit it from the dataset. At the same time, however, knowing that the outlier’s environment or management had a dramatic effect on its performance, you may generate new hypotheses about that product. In fact, you may learn more from your outlier, through the new research it inspires, than you do from the original experiment.
Also, before removing an outlier, it is a good idea to run your tests with and without it to see whether it changes your conclusions. When you run your model, look at the standardized residuals. How many standard errors is the outlier from the predicted value? As a rule, if an observed value is more than two standard deviations from the predicted value, I scrutinize it before allowing it into the final analysis.
If you are comparing different treatments, does it change the significance of tests or differences among treatments? If you are generating a regression model, does the change in slope have a dramatic effect on the values you will predict? Will the change in slope have a dramatic effect on grower inputs and cost, or will the effect be more modest?
These are important questions, because as uncomfortable as it is to identify and/or remove outliers, working with incomplete datasets can be even nastier. If the statistical significance of tests or means separations are not affected by the outlier, it is best to leave it in the dataset if possible, especially if treatment replications are limited.
9.3.2 Non-normal Data and Unequal Variances
Above, we discussed two other problems with data: data that were skewed (or non-normal) and therefore could not be modelled based on the normal distribution, and data were treatment variances were not equal – they were, in statistical terminology, they suffered from heterogeneity of variances.
Both issues can arise out of trials where observation values vary widely: counts that include rare events or where one treatment (like a check plot) can “blow up” and have a huge value. Growth studies, where size increases occur at exponential rates, are another.
These two issues may be similarly addressed by transforming the data. When we transform data, we use a different measuring system to rescale it. The easiest example of rescaling data is pH. Recall pH is the concentration of hydrogen atoms in a solution. This concentration can range from \(0\) to \(10^{-14}\)
So when is the last time you read that your soil pH was \(6.5 \times 10^{-6}\) ? You wouldnt. We commonly speak of pH as ranging from 1 (highly acidic) through 7 (neutral) to 14 (highly basic). Your are used to using the logarithmic scale(10, 1, 0.10, 0.010), rather than the arithmatic scale (1,2,3,4,5). The decibel scale for sound and the Richter scale for earthquakes also use the logarithmic scale.
<IP_ADDRESS> Natural Logarithm
There are several ways to transform data, but the one I have most-often used is the natural logarithm The natural logaritm transformation is often used when we are working with data that have a wide range of values. What constitutes a wide range of values? Think growth analysis, or counts of common events (for example, weed counts in a herbicide trial that includes treatments that vary widely in effectiveness). In these trials, it is not uncommon for observed values to vary by two or more orders of magnitude (powers of 10).
Our process for working with transformed data is as follows:
- Transform the original observations to their natural logs.
- Calculate the ANOVA
- Cacluate treatment means using transformed data
- Back-transform the treatment means to the original measurement scale so they are more intuitive to users
Lets work with our velvetleaf data above. Below is the analysis of variance and means separation using the least significant difference test.
library(agricolae)
= aov(yield~trt, norm_data_prop_sd)
vleaf_model_before summary(vleaf_model_before)
## Df Sum Sq Mean Sq F value Pr(>F)
## trt 3 500.0 166.67 8.767 0.00237 **
## Residuals 12 228.1 19.01
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
= LSD.test(vleaf_model_before, "trt")
lsd_before $groups lsd_before
## yield groups
## D 20 a
## C 15 ab
## B 10 bc
## A 5 c
The treatment effect is significant, and some of the treatments are significantly different from each other, but there are also noticeable overlaps.
Now let’s transform the data using the natural log. The original and transformed data are show below.
= norm_data_prop_sd %>%
vleaf_log mutate(log_yield = log(yield))
vleaf_log
## trt yield log_yield
## 1 A 5.164436 1.641796
## 2 A 5.046640 1.618723
## 3 A 4.563925 1.518183
## 4 A 5.224999 1.653455
## 5 B 9.236621 2.223176
## 6 B 10.956095 2.393896
## 7 B 12.839163 2.552500
## 8 B 6.968121 1.941346
## 9 C 8.564738 2.147654
## 10 C 15.867385 2.764266
## 11 C 19.858115 2.988613
## 12 C 15.709762 2.754282
## 13 D 10.384946 2.340357
## 14 D 23.843831 3.171526
## 15 D 19.807285 2.986050
## 16 D 25.963938 3.256709
Now when we run our Bartlett’s test, we see the p-value is 0.09 – there is no longer a significant difference among the treatment variances.
bartlett.test(log_yield~trt, vleaf_log)
##
## Bartlett test of homogeneity of variances
##
## data: log_yield by trt
## Bartlett's K-squared = 6.7935, df = 3, p-value = 0.07878
The largest standard deviation is still 6.5 times the smallest standard deviation – but the difference has decreased dramatically. When we run our analysis of variance, we see that our p-value has decreased by several orders of magniture.
= aov(log_yield~trt, vleaf_log)
vleaf_model_after summary(vleaf_model_after)
## Df Sum Sq Mean Sq F value Pr(>F)
## trt 3 3.995 1.3316 14.24 0.000294 ***
## Residuals 12 1.122 0.0935
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
When we run our LSD test, we notice more significant differences among the means, especially treatments A and B, which were associated with lower treatment means.
= LSD.test(vleaf_model_after, "trt")
lsd_after $groups lsd_after
## log_yield groups
## D 2.938660 a
## C 2.663704 ab
## B 2.277729 b
## A 1.608039 c
Our last step is to back-transform the means from our LSD test.
= lsd_after$groups %>%
back_transformed_means rownames_to_column(var="treatment") %>%
mutate(yield = exp(log_yield))
%>%
back_transformed_means ::select(treatment, yield, groups) dplyr
## treatment yield groups
## 1 D 18.890521 a
## 2 C 14.349335 ab
## 3 B 9.754507 b
## 4 A 4.993011 c
In the above table, we have back-transformed the means in our LSD table to their original scale.
9.4 Dealing with Missing Data
Missing data can be very problematic. Whether missing data are the result of deleting outliers, or plots lost to weather or human damage, there are three options:
- Drop the treatment entirely from the dataset
- Drop one observation from each of the other treatments; if a Randomized Complete Block Design was used, delete and entire block
- Predict the value for the plot that was omitted or lost
As you see, these are ugly choices to make. If you drop the treatment entirely from the dataset, you lose all ability to test it agaist the other treatments. If you drop other replicates or the remainder of a block, you retain all treatments but reduce the degrees of freedom for statistical tests, rendering them less sensitive.
The third option, predicting the value that is missing, comes with its own challenges. The missing value for a plot is generally calculated using the linear additive model. For a completely randomized design, the linear model is:
\[Y_{ij} = \mu + T_i + \epsilon_{(i)j} \]
So the missing value would be equal to \(\mu + Ti\), where \(i\) would be whatever treatment level the missing plot received.
In a randomized complete block design, the the linear additive model is:
\[Y_{ij} = \mu + B_i + T_j + BT_{ij} \]
The missing value would be equal to \(\mu + B_i + T_j\), where \(i\) is the block to which the missing plot occurred, and \(j\) is the treatment the treatment the missing plot received.
Note that we did not include \(\epsilon_{(i)j}\) or \(BT_{ij}\) in estimating the missing values. Although this approach is widely used, this is a shortcome. When we predict a missing value from the effects of treatment or treatment within block, we are using mean effects. So the predicted value will be exactly the mean for a given treatment or treatment within block. Because the predicted value is closer to the treatment and block means than it would be otherwise, it will contribute less to the treatment variance than it would normally.
We can demonstrate this with a normally-distributed dataset.
set.seed(083020)
= data.frame(yield = rnorm(4, mean=165, sd=5.2)) %>%
more_norm_data_1 mutate(trt="A")
set.seed(083021)
= data.frame(yield = rnorm(4, mean=169, sd=5.2)) %>%
more_norm_data_2 mutate(trt="B")
set.seed(083022)
= data.frame(yield = rnorm(4, mean=170, sd=5.2)) %>%
more_norm_data_3 mutate(trt="C")
set.seed(083023)
= data.frame(yield = rnorm(4, mean=172, sd=5.2)) %>%
more_norm_data_4 mutate(trt="D")
= rbind(more_norm_data_1, more_norm_data_2, more_norm_data_3, more_norm_data_4) %>%
more_norm_data mutate(trt = as.factor(trt)) %>%
mutate(random_no = rnorm(16,0,1)) %>%
arrange(random_no) %>%
mutate(plot=row_number()) %>%
::select(plot, trt, yield) %>%
dplyrmutate(yield = round(yield,1))
more_norm_data
## plot trt yield
## 1 1 A 165.0
## 2 2 C 172.1
## 3 3 C 169.3
## 4 4 D 170.1
## 5 5 B 171.6
## 6 6 C 172.3
## 7 7 B 161.3
## 8 8 A 175.3
## 9 9 B 162.1
## 10 10 D 166.3
## 11 11 C 167.1
## 12 12 D 174.2
## 13 13 D 166.7
## 14 14 A 158.3
## 15 15 B 169.7
## 16 16 A 160.3
Here is the anova for the original data:
summary.aov(lm(yield~trt, data=more_norm_data))
## Df Sum Sq Mean Sq F value Pr(>F)
## trt 3 80.13 26.71 1.021 0.418
## Residuals 12 313.84 26.15
And here is the boxplot:
%>%
more_norm_data ggplot(aes(x=trt, y=yield)) +
geom_boxplot(outlier.colour="black", outlier.shape=16,
outlier.size=2, notch=FALSE, fill="lightblue") +
coord_flip() +
stat_summary(fun=mean, geom="point", shape=23, size=2)
Here is the table with the treatment and error effects broken out. Unlike previous effects tables, I have also added the within treatment variance.
= more_norm_data %>%
more_norm_data_effects mutate(mu = mean(yield)) %>%
group_by(trt) %>%
mutate(trt_mean = mean(yield),
trt_var = var(yield)) %>%
ungroup() %>%
mutate(trt_effect = trt_mean - mu) %>%
mutate(error_effect = yield - trt_effect - mu) %>%
::select(plot, trt, yield, mu, trt_var, trt_effect, error_effect)
dplyr
::kable(more_norm_data_effects) knitr
| plot | trt | yield | mu | trt_var | trt_effect | error_effect |
|---|---|---|---|---|---|---|
| 1 | A | 165.0 | 167.6062 | 57.589167 | -2.88125 | 0.275 |
| 2 | C | 172.1 | 167.6062 | 6.146667 | 2.59375 | 1.900 |
| 3 | C | 169.3 | 167.6062 | 6.146667 | 2.59375 | -0.900 |
| 4 | D | 170.1 | 167.6062 | 13.469167 | 1.71875 | 0.775 |
| 5 | B | 171.6 | 167.6062 | 27.409167 | -1.43125 | 5.425 |
| 6 | C | 172.3 | 167.6062 | 6.146667 | 2.59375 | 2.100 |
| 7 | B | 161.3 | 167.6062 | 27.409167 | -1.43125 | -4.875 |
| 8 | A | 175.3 | 167.6062 | 57.589167 | -2.88125 | 10.575 |
| 9 | B | 162.1 | 167.6062 | 27.409167 | -1.43125 | -4.075 |
| 10 | D | 166.3 | 167.6062 | 13.469167 | 1.71875 | -3.025 |
| 11 | C | 167.1 | 167.6062 | 6.146667 | 2.59375 | -3.100 |
| 12 | D | 174.2 | 167.6062 | 13.469167 | 1.71875 | 4.875 |
| 13 | D | 166.7 | 167.6062 | 13.469167 | 1.71875 | -2.625 |
| 14 | A | 158.3 | 167.6062 | 57.589167 | -2.88125 | -6.425 |
| 15 | B | 169.7 | 167.6062 | 27.409167 | -1.43125 | 3.525 |
| 16 | A | 160.3 | 167.6062 | 57.589167 | -2.88125 | -4.425 |
Plot 8 has a greater error effect than most other plots. Let’s treat it as an outlier, delete it, and recalculate the treatment means. Let’s delete it and see how that changes the treatment effect for treatment C.
= data.frame(plot=8 ,trt="C", yield=NA)
plot_8
= more_norm_data %>%
more_norm_data_interp ::filter(!plot==8) %>%
dplyrrbind(plot_8) %>%
arrange(plot) %>%
mutate(mu = mean(yield, na.rm = TRUE)) %>%
group_by(trt) %>%
mutate(trt_mean = mean(yield, na.rm = TRUE),
trt_var = var(yield, na.rm = TRUE)) %>%
ungroup() %>%
mutate(trt_effect = trt_mean - mu) %>%
mutate(error_effect = yield - trt_effect - mu) %>%
::select(plot, trt, yield, mu, trt_var, trt_effect, error_effect)
dplyr
::kable(more_norm_data_interp) knitr
| plot | trt | yield | mu | trt_var | trt_effect | error_effect |
|---|---|---|---|---|---|---|
| 1 | A | 165.0 | 167.0933 | 11.830000 | -5.8933333 | 3.800 |
| 2 | C | 172.1 | 167.0933 | 6.146667 | 3.1066667 | 1.900 |
| 3 | C | 169.3 | 167.0933 | 6.146667 | 3.1066667 | -0.900 |
| 4 | D | 170.1 | 167.0933 | 13.469167 | 2.2316667 | 0.775 |
| 5 | B | 171.6 | 167.0933 | 27.409167 | -0.9183333 | 5.425 |
| 6 | C | 172.3 | 167.0933 | 6.146667 | 3.1066667 | 2.100 |
| 7 | B | 161.3 | 167.0933 | 27.409167 | -0.9183333 | -4.875 |
| 8 | C | NA | 167.0933 | 6.146667 | 3.1066667 | NA |
| 9 | B | 162.1 | 167.0933 | 27.409167 | -0.9183333 | -4.075 |
| 10 | D | 166.3 | 167.0933 | 13.469167 | 2.2316667 | -3.025 |
| 11 | C | 167.1 | 167.0933 | 6.146667 | 3.1066667 | -3.100 |
| 12 | D | 174.2 | 167.0933 | 13.469167 | 2.2316667 | 4.875 |
| 13 | D | 166.7 | 167.0933 | 13.469167 | 2.2316667 | -2.625 |
| 14 | A | 158.3 | 167.0933 | 11.830000 | -5.8933333 | -2.900 |
| 15 | B | 169.7 | 167.0933 | 27.409167 | -0.9183333 | 3.525 |
| 16 | A | 160.3 | 167.0933 | 11.830000 | -5.8933333 | -0.900 |
We can see that removing the yield data from plot 4 causes the treatment effect of treatment C to change – in fact, it has gone from negative to positive. The other treatment effects have also changed. The within-treatment variance for treatment C has also decreased by about one-third. When we re-run our analysis of variance, we see the treatment effect is 0.062 – almost significant at the P=0.05 level.
summary.aov(lm(yield ~ trt, data=more_norm_data_interp))
## Df Sum Sq Mean Sq F value Pr(>F)
## trt 3 166.1 55.36 3.697 0.0463 *
## Residuals 11 164.7 14.98
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 1 observation deleted due to missingness
What would happen if we estimated the yield for plot 8 using the population mean, mu, and the treatment effect?
\[ Y_{3,4} = \mu + T_i = 168.96 + 1.37 = 170.33 \]
We see that mu, treatment variance, treatment effect, and error have again changed. The variance within Treatment 3 has again decreased by about one-third.
= data.frame(plot=8 ,trt="C", yield=170.33)
plot_8_interp
= more_norm_data %>%
more_norm_data_interp_8 ::filter(!plot==8) %>%
dplyrrbind(plot_8_interp) %>%
arrange(plot) %>%
mutate(mu = mean(yield, na.rm = TRUE)) %>%
group_by(trt) %>%
mutate(trt_mean = mean(yield, na.rm = TRUE),
trt_var = var(yield, na.rm = TRUE)) %>%
ungroup() %>%
mutate(trt_effect = trt_mean - mu) %>%
mutate(error_effect = yield - trt_effect - mu) %>%
::select(plot, trt, yield, mu, trt_var, trt_effect, error_effect)
dplyr
::kable(more_norm_data_interp_8) knitr
| plot | trt | yield | mu | trt_var | trt_effect | error_effect |
|---|---|---|---|---|---|---|
| 1 | A | 165.00 | 167.2956 | 11.83000 | -6.095625 | 3.800 |
| 2 | C | 172.10 | 167.2956 | 4.61338 | 2.930375 | 1.874 |
| 3 | C | 169.30 | 167.2956 | 4.61338 | 2.930375 | -0.926 |
| 4 | D | 170.10 | 167.2956 | 13.46917 | 2.029375 | 0.775 |
| 5 | B | 171.60 | 167.2956 | 27.40917 | -1.120625 | 5.425 |
| 6 | C | 172.30 | 167.2956 | 4.61338 | 2.930375 | 2.074 |
| 7 | B | 161.30 | 167.2956 | 27.40917 | -1.120625 | -4.875 |
| 8 | C | 170.33 | 167.2956 | 4.61338 | 2.930375 | 0.104 |
| 9 | B | 162.10 | 167.2956 | 27.40917 | -1.120625 | -4.075 |
| 10 | D | 166.30 | 167.2956 | 13.46917 | 2.029375 | -3.025 |
| 11 | C | 167.10 | 167.2956 | 4.61338 | 2.930375 | -3.126 |
| 12 | D | 174.20 | 167.2956 | 13.46917 | 2.029375 | 4.875 |
| 13 | D | 166.70 | 167.2956 | 13.46917 | 2.029375 | -2.625 |
| 14 | A | 158.30 | 167.2956 | 11.83000 | -6.095625 | -2.900 |
| 15 | B | 169.70 | 167.2956 | 27.40917 | -1.120625 | 3.525 |
| 16 | A | 160.30 | 167.2956 | 11.83000 | -6.095625 | -0.900 |
And here is the kicker, wait for it…
summary.aov(lm(yield ~ trt, data=more_norm_data_interp_8))
## Df Sum Sq Mean Sq F value Pr(>F)
## trt 3 175.9 58.63 4.271 0.0287 *
## Residuals 12 164.8 13.73
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Our treatment differences are now significant. Why? Because when we estimate a missing value using only the population mean and treatment effect, we decrease the overall variance. And why does that happen? Because we have now created that is almost exactly equal to the treatment mean. Was there a change in the originally observed values associated with this change in significance? No. And this is problem. But there is a way to reduce it.
The problem with the model we have used so far is we did not include the error effect in our yield estimate. If we added it in, our yield estimate for plot 8 would be more appropriate. Of course, we cannot calculate the error effect because it is random and changes among plots. But, knowing that error effects are normally distributed around the treatment mean, we can model that distribution and draw an individual from it at random, to use as the error effect in our estimate.
The error distribution has its own mean, which should be close to zero:
= mean(more_norm_data_interp$error_effect, na.rm=TRUE)
err_mean err_mean
## [1] 0
And its own standard deviation:
= sd(more_norm_data_interp$error_effect, na.rm=TRUE)
err_sd err_sd
## [1] 3.430275
Knowing these two parameters, we can select a value for our error effect from that distribution.
set.seed(12)
= rnorm(1, err_mean, err_sd)
err_pred err_pred
## [1] -4.213954
Let’s plug that into our yield estimate and see how our statistics change.
\[ Y_{3,4} = \mu + T_i = 168.96 + 1.37 - 5.37 = 164.96 \]
We see that mu, treatment variance, treatment effect, and error have again changed. The variance within Treatment 3 has again decreased by about one-third.
= data.frame(plot=8 ,trt="C", yield=164.96)
plot_8_interp_sd
= more_norm_data %>%
more_norm_data_interp_8_sd ::filter(!plot==8) %>%
dplyrrbind(plot_8_interp_sd) %>%
arrange(plot) %>%
mutate(mu = mean(yield, na.rm = TRUE)) %>%
group_by(trt) %>%
mutate(trt_mean = mean(yield, na.rm = TRUE),
trt_var = var(yield, na.rm = TRUE)) %>%
ungroup() %>%
mutate(trt_effect = trt_mean - mu) %>%
mutate(error_effect = yield - trt_effect - mu) %>%
::select(plot, trt, yield, mu, trt_var, trt_effect, error_effect)
dplyr
::kable(more_norm_data_interp_8_sd) knitr
| plot | trt | yield | mu | trt_var | trt_effect | error_effect |
|---|---|---|---|---|---|---|
| 1 | A | 165.00 | 166.96 | 11.83000 | -5.760 | 3.800 |
| 2 | C | 172.10 | 166.96 | 10.10152 | 2.192 | 2.948 |
| 3 | C | 169.30 | 166.96 | 10.10152 | 2.192 | 0.148 |
| 4 | D | 170.10 | 166.96 | 13.46917 | 2.365 | 0.775 |
| 5 | B | 171.60 | 166.96 | 27.40917 | -0.785 | 5.425 |
| 6 | C | 172.30 | 166.96 | 10.10152 | 2.192 | 3.148 |
| 7 | B | 161.30 | 166.96 | 27.40917 | -0.785 | -4.875 |
| 8 | C | 164.96 | 166.96 | 10.10152 | 2.192 | -4.192 |
| 9 | B | 162.10 | 166.96 | 27.40917 | -0.785 | -4.075 |
| 10 | D | 166.30 | 166.96 | 13.46917 | 2.365 | -3.025 |
| 11 | C | 167.10 | 166.96 | 10.10152 | 2.192 | -2.052 |
| 12 | D | 174.20 | 166.96 | 13.46917 | 2.365 | 4.875 |
| 13 | D | 166.70 | 166.96 | 13.46917 | 2.365 | -2.625 |
| 14 | A | 158.30 | 166.96 | 11.83000 | -5.760 | -2.900 |
| 15 | B | 169.70 | 166.96 | 27.40917 | -0.785 | 3.525 |
| 16 | A | 160.30 | 166.96 | 11.83000 | -5.760 | -0.900 |
When we looks at the ANOVA, we see that the mean square and p-value are approximately the same as they were before the missing value was interpolated.
summary.aov(lm(yield ~ trt, data=more_norm_data_interp_8_sd))
## Df Sum Sq Mean Sq F value Pr(>F)
## trt 3 148.4 49.46 3.179 0.0633 .
## Residuals 12 186.7 15.56
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
In R, there is a nice package called mice that does this for us. We will learn about it in an exercise this week.
One final note, however: we should not interpolate missing values if 5% or more of the data are missing. Why? Because, as we have seen above, that interpolated value can markedly change our interpretation of the data. Restricting interpolation to datasets where a small percentage of data are missing reduces the leverage one observation has on conclusions from the data. It also increases the accuracy of the interpolated values.
In smaller datasets, then, we may to use the approaches above. How important is it to have that treatment in the trial, versus losing a replicaton of the other treatments? For the latter option, you may want to test whether including or omitting that replicate changes your conclusion from the data. If not, it may be easiest to drop the replication.
9.5 Summary
Agronomic data are rarely nice and neat. The scale in which we work (acres and hectares), plus the variations in soil types, equipment performance, weather, weeds and other pests, make it virtually impossible to ensure our experimental units are near-equivalent. Above all, our subjects are alive and integrate every aspect of their environment into their growth. Unlike human subjects, corn plants cannot answer a history questionnaire. Months or years of a trial can be erased by one plugged nozzle, one broken singulator, one strong wind, one “white combine”. It’s a nasty business.
It is important that we inspect our data and consider its shortcomings. We have ways to address these shortcomings. Outliers may be trimmed, or we may use other techniques to overcome them. We have also learned how to re-scale data (using logarithms or square roots) so that variances are closer to equal, and how to “fill in” missing values using imputation so that our datasets can be balanced.
If stuck, consult with other data scientists. There are “robust” statistics that are based on using the median, rather than the mean, for summaries and tests. There are also non-parametric tests, some of which we will be introduced to towards the end of this course. Non-parametric methods don’t use linear models – therefore, they are more insulated from the problems discussed above. We will learn about these in a future unit.


We buy services!
- deploy plug-and-play functionalities
- target front-end vortals
- unleash bricks-and-clicks networks
- embrace distributed synergies
- enable transparent e-business
- visualize visionary methodologies
- innovate wireless functionalities
- revolutionize cutting-edge solutions
- empower sticky ROI
- mesh extensible convergence
- iterate sticky deliverables
- reinvent intuitive models
- implement best-of-breed metrics
- benchmark frictionless niches


| 9 | (A) | a pelare pyghte was thare down appon the plate erthe |
| 9 | (C) | ¶ A pyler was downn pygħt . on þe pleyn erþe |
| 9 | (D) | A piler doun was pigħt | on þe playn erthe |
| 9 | (E) | ¶ A pyler doun was pyȝt . on the playn erthe . |
| (Ex) | ---- no text ---- | |
| 9 | (L) | a pyler pyȝt was doun vpon þe playn erþe |
| 9 | (P) | A pilar phytgh was doune on þe playne erthe |
| 9 | (U) | ¶ A piler doun was ypigħt / al of grey marble |
| (V) | ---- no text ---- |


Round 1
Waiting for other players. Page will refresh itself ...
Restaurant Name
$$$$
Davis, CA 95616
Restaurant Name
$$$$
Davis, CA 95616


| 847 | (A) | and this towrede town fułł tenefułł to wyn |
| 851 | (C) | And þis towred town . is trowbelous to wynne |
| 837 | (D) | And this towrid toun is teneful to wynne |
| 814 | (E) | & þis touryd town teenful to wynne |
| (Ex) | ---- no text ---- | |
| 868 | (L) | & þis toured toun is tenful to wynne |
| 790 | (P) | & þis turred toune es tenefull to wyn |
| 823 | (U) | And this toured toun / is teneful to wynne |
| (V) | ---- no text ---- |


SEA Games: ‘Super fast’ outing for Malaysian water polo team
NEW CLARK CITY, Nov 29 — The 30th SEA Games in the Philippines will only officially open tomorrow but the national men’s water polo team will be the first (team) to return home early.
They lost 7-14 to Indonesia in their third preliminary match at the New Clark City Aquatics Centre here today.
This latest defeat has put Malaysia out of contention of at least retaining the bronze medal they won in 2017 Kuala Lumpur SEA Games.
In their earlier two matches, they had lost 5-19 to Thailand and 5-17 to defending champions Singapore.
Speaking to the media after today’s match, national men’s water polo head coach, Voon Yong Hui said that Indonesia, who currently lead the standings in the preliminary round, are at the top of their game.
“We brought eight new players who are mostly still studying and only retained four players from the 2017 Kuala Lumpur SEA Games squad, and generally speaking, our chances of retaining the bronze medal in 2017 is very difficult because we lost all three matches.
“Facing the hosts on Sunday will certainly be a tough job, considering their players are bigger in size compared to our players who are quite small,” Voon said.
He said the national players would return home on Dec 2 (Monday) after the match against the Philippines on Sunday (December 1).
Only five teams are taking part in men’s water polo and according to the format, following the round-robin stage, the gold medal will be awarded to the top finisher, silver to the runner-up and bronze to the team that finishes in third place.
Malaysia are not taking part in the women’s competition, which features only three countries — defending champions Thailand, Singapore and the Philippines. — Bernama


Graphics Notes
- Graphics APIs like OpenGL and webGL are designed to draw points, lines and triangles. That’s it.
- Graphics APIs like OpenGL and webGL use a series of shaders to do this, mainly a vertex shader and a fragment shader.
- One series of shaders is called a “program”.
- Shaders are written in a language called GLSL.
- Vertex shaders compute vertex positions, fragment shaders compute colors for each pixel.
- Buffers are arrays of binary data uploaded to the GPU through a shader.
- Attributes are used to specify how to pull data out of your buffers and provide them to the shader.
- Vertex array objects (VAOs) are used to hold the state of attributes.
- Uniforms are global variables you set before executing your shader program.
- Textures are arrays of data you can randomly access in your shader program.
- Varyings are a way for a vertex shader to pass data onwards to a fragment shader.
- Fragment shader output is defined with “out”.
- To pass data into the vertex shader you must find the attribute location for a variable in GLSL, then create a buffer, then bind the buffer, then do the same for a vertex array.
- Different primitives are used for drawing, but the most common is gl.TRIANGLES.
- “Clip space” is what GLSL understands coordinates to be. It’s always on a plane of -1 to 1 for both x and y axis regardless of height and width. To pass in “normal” vertices you have to convert them to clip space.
- Translation, rotation, and scale are types of “transformations”. Order matters for them.
- Translation is for moving things. Simply add it to the position: a_position + u_translation
- Rotation is for rotating things alone a given axis.
- Create a new rotated position: rotatedPosition = vec2(a_position.x * u_rotation.y + a_position.y * u_rotation.x, a_position.y * u_rotation.y - a_position.x * u_rotation.x)
- Basically you multiply the X coordinate by the Y position on the unit circle, and the Y coordinate by the X position on the unit circle and add them together for the final X.
- Multiply the Y by Y on the circle and subtract x times the x on the circle for the final Y.
- The points on the unit circle can be looked up using sine and cosine of an angle.
- Scale is for making things bigger or smaller. Simply multiply it to the position: a_position * u_scale. Negative values flip the geometry.
- You can run any transformation with matrix math.
-
Left column transforms X, next transforms Y, next transforms Z. (Sometimes it goes top to bottom depending on the API).
- Identity matrix is the equivalent of “1” it multiplies all rows by 1 in a matrix
-
[
1, 0, 0, 0
0, 1, 0, 0
0, 0, 1, 0
0, 0, 0, 1
]
- Scaling matrix multiples each row by the scale you want.
-
[
s1, 0, 0, 0
0, s2, 0, 0
0, 0, s3, 0
0, 0, 0, 1
]
- Translation matrix puts values in the last unused column, it’s also used to move the origin of what you’re rotating.
-
[
1, 0, 0, 0
0, 1, 0, 0
0, 0, 1, 0
t1, t2, t3, 1
]
- Rotation has different matrices for the three different axes.
-
X, rotate the other two Y and Z axes (c = cos(angle), s = sin(angle)):
-
[
1, 0, 0, 0
0, c, s, 0
0, -s, c, 0
0, 0, 0, 1
]
-
Y, rotate X and Z:
-
[
c, 0, -s, 0
0, 1, 0, 0
s, 0, c, 0
0, 0, 0, 1
]
-
Z, rotate X and Y:
-
[
c, -s, 0, 0
-s, c, 0, 0
0, 0, 1, 0
0, 0, 0, 1
]
- Matrices can be multiplied together, allowing us to do all transformations at once in whatever order we want.
- Typical order is translation -> rotation -> scale.
- A “projection” matrix can be made to go from clip space to pixels and back. Involves 3 scale matrices and a translation matrix.


What is it?
SnapMotion is a free, open source (MIT License) jQuery plugin. It is easily integrated into your website.
Compatibility
Firefox 3+, Chrome 5+, Safari 3+, Opera 9+, IE6+.
Demo
View our demonstrations:
Features
This plugin was built to make creating a slideshow on your website extremely simple and painless. By including the few files provided in this package, you'll be up and running within minutes.
- Auto-sliding
- Image titles & captions
- Flexible configuration
Documentation
Firstly, you'll need to reference the SnapMotion JavaScript and CSS files within your <head> tag.
<script src="js/jquery.snapmotion.js"></script> <link rel="stylesheet" href="css/snapmotion.css" type="text/css" />
Create your slides using a <ul> with a <li> for each slide. Each <li> should include the slide <img /> and a <span> tag containing any text you'd like to display on the slide.
For example:
<ul id="slideshow">
Heading 1
Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet.
Heading 2
Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet.
Finally, call the SnapMotion() method on your slideshow. For example:
$("ul#slideshow").SnapMotion();
Optional parameters include:
- slideWidth (int, the width of a slide)
- slideHeight (int, the height of a slide)
- showCaption (boolean, whether to show a caption)
- captionWidth (int, the width of the caption area)
- captionHeight (int, the height of the caption area)
- showControls (boolean, whether to show next/previous controls)
- controlIndentX (int, the indentation of the 'controls' for the slideshow, in pixels, from the left of each slide)
- controlIndentY (int, the indentation of the 'controls' for the slideshow, in pixels, from the top of each slide)
- animationLength (int, the length, in milliseconds, of each slide transition)
- delay (int, the length, in milliseconds, between each slide transition)
- transition (string, slideX (default), slideY, fade)
<script type="text/javascript"> $(document).ready(function(){ $("ul#slideshow").SnapMotion({ 'slideWidth' : 960, 'controlIndent' : 640, 'animationLength' : 375, 'delay' : 7500, 'transition' : 'slideY' }); }); </script>


Easy Batch requires a Java 1.6+ runtime.
Easy Batch uses Git as source control manager and is hosted at Github. To download Easy Batch source code, clone it's git repository as follows :
git clone https://github.com/benas/easy-batch.git
You can also get the source code in a zip file from the home page of the project at Github : https://github.com/benas/easy-batch
Easy Batch uses maven as build tool. To build Easy Batch with maven, use the following command from the root directory of the project :
mvn install
After the build is finished, the easybatch-core-{version}.jar file will be added to your local maven repository in the following directory :
io/github/benas/easybatch/easybatch-core/{version}/easybatch-core-{version}.jar
To use Easy Batch, you have to include easybatch-core-{version}.jar in your application's classpath.
If you use maven, add the following dependency to your project pom :
<dependency> <groupId>io.github.benas</groupId> <artifactId>easybatch-core</artifactId> <version>${version}</version> </dependency>


Sprint 7 Cultural Blog - Motivation
How does life look through the 'pleasure, passion and purpose' lens?
Tony Hsieh offers the following definitions of pleasure, passion and higher purpose:
- Pleasure: this type of happiness is about always chasing the next high. It is
the rock-star type of happiness because it is very hard to maintain unless you are
living the lifestyle of a rock star.
- Passion: also known as "flow," where peak performance meets peak engagement, and
time flies by.
- Higher purpose: this is about being part of something bigger than yourself that
has meaning to you.
With my first sceptical glance, I was worried that this would set up a cycle of disatisfaction
and unfulfilment. What if all of your expectations are built on a hubris that star, model
employees can get into a perfect cadence of pleasure, passion and higher purpose as offered by
Tony? My glum and gloomy thought was, "All power to you if you can find a work place that can
set all of this up for you in a nice package." I don't look like it, but I can be a cynical and
negative person (which I must improve!)
Of course, I have an open mind, flexible to new ideas that challenge and break my assumptions
:)
Reading through the chapter, I realised that you have to make your own cadence for yourself. No
one else is going to meet the needs of your pleasure, passion, and higher purpose if you don't
do some introspection and define for yourself what gives you fulfilment and enjoyment. And,
you have to be willing to stand by what these things are for you, and keep true to them. Build
your vision for what makes you happy, and people will be drawn to your passion and will help
you follow your dreams.
From the 'discovering values and higher purpose exercise:
What are your core values?
My core values are:
- Integrity - being honest, and having strong principles.
- Courage - being able to speak up for what is right, no matter the personal or
reputational cost.
- Respect, loyalty, and fidelity - following through on my promises, being respectful and
loyal to those around me, and knowing I have to work hard to earn and protect others'
trust.
- Persistence and resilience - we can get to our goal, we just have to keep at it, no
matter how hard it feels right now.
- Kindness and compassion - assume positive intent from all, and treat everyone with the
kindness and care you would need yourself to thrive.
What do you stand for?
I stand for:
- Equal opportunity, everyone having a fair go.
- Open-minded kindness and empathy.
- Building the right goal for the right reasons for the right people.
- Being transparent and accountable.
- Self-care and continuous improvement - you have to learn to put the oxygen mask for
yourself first before those around you, so you can support them with confidence in an
emergency.
How did you find the resilience ideas and meditation exercise?
Chade-Meng Tan offered the following ideas on resilience:
- Inner calm: once we can consistently access the inner calm in the mind, it
becomes the foundation of all optimism and resilience.
- Emotional resilience: success and failure are emotional experiences. By working
at this level, we can increase our capacity for them.
- Cognitive resilience: understanding how we explain our setbacks to ourselves and
creating useful mental habits to help us develop optimism.
My first reading of this was a little challenging to my current thought patterns. I personally
feel no capacity to celebrate success for myself on an individual level. This feeling was
confirmed when I tried the meditation exercise: I could easily conjure my physical and
emotional responses to memories of failure, but when I tried to reflect on memories of success,
I struggled to find anything that I could
own as my own success. My thought patterns
continuously shifted those successes to being other peoples, hand-in-hand with my imposter
syndrome. And when I found memories of success, my thought patterns shut down those emotions of
success, as if I didn't have any right to feel good about myself.
Uh, did I mention I've got issues when I started this blog?
So, on one hand, it feels good that I can articulate and explore my issues objectively, with a
goal to incorporate Meng's insights and wisdom into my work on self-improvement.
On the other hand - yikes, it's hard doing this introspection, but so worthwhile. I'm thinking
I'll have to read this book again when I've finished my EDA journey!
Any other musings?
I might take a nap to shrug off this minor existential crisis, and then try some more coding
exercises!


| 983 | (A) | Fayne als fewle of þe daye was the freke than |
| 988 | (C) | ¶ Fayne as folowe of þe daye . was þe kynge þanne |
| 970 | (D) | Fayn as foul of þe day was þe freke thenne |
| (E) | ---- no text ---- | |
| 30 | (Ex) | Fayne as Foule wi th t. . . |
| 1008 | (L) | fayn as þe foul of day was þe freke þanne |
| 923 | (P) | ????? als þe foule of þe daye was þe freke þen |
| 961 | (U) | Feyn os foule of the day / was the freke thanne |
| 48 | (V) | Fayn as foule of þe day . was þe kyng þanne |


To use this application, you must be a premium subscriber to radioreference.com.
If you are a premium subscriber, enter your Radio Reference credentials to continue.


Congratulations, FIRSTNAME!
You have done an exceptional job, especially considering this was your first assignment. We've caught ourselves some dangerous subversives and have proven the necessity of the work we are doing :).
We'll have to find time to meet up and exchange recipes some day soon. Now, considering your fondness for spatulas, I think I have just the recipe for you!
Remember how I told you that crêpes can be so much more than dessert? This is your chance to take advantage of this fact! Procure some butter and buckwheat flour, and you will cooking up delicious crêpes for everyone in the company in no time! I am sure this will make you lots of friends among your colleagues :).
Just take whatever other foodstuff you can find in the kitchen and use it for fillings! For special occasions, you may be able to convince the staff to fetch some extra ingredients for you. I will put in a good word. Enjoy!
Salvinu Manduca
Senior Consultant


--- layout: post status: publish published: true title: Strong Names and Open Source author: display_name: Tomas Restrepo login: tomasr email<EMAIL_ADDRESS>url: http://winterdom.com/ author_login: tomasr author_email<EMAIL_ADDRESS>author_url: http://winterdom.com/ wordpress_id: 398 wordpress_url: http://winterdom.com/2002/05/strongnamesandopensource date: '2002-05-18 16:10:15 +0000' date_gmt: '2002-05-18 16:10:15 +0000' categories: - ".NET" tags: [] comments: [] ---
I've been thinking about just how Strong Names in .NET (as generated by SN.EXE) and Open Source libraries should be mixed,
and I must admit I haven't quite made up my mind, yet. I started thinking about this after Simon's
and Peter's recent discussion
on the security merits of strong named assemblies.
Peter quite correctly states that "...the public key in the component assembly manifest lets us confirm that the signature could only have come from someone who has the matching private key, which is protected (right?) with the appropriate degree of pomp and circumstance.". So here's the rub: If your library is open sourced, and it should also be strongly signed (as I believe in most cases should be), what happens to the private key used to sign it?
Many .NET open source projects seem to be distributing the full keyfile generated by sn.exe containing both the private and public keys. This is sensible in that it allows anyone else to easily rebuild the assembly as needed, but it also defeats part of the security stated above. After all, once you have the private key, you could easily resign a modified assembly that did whatever it wanted. The obvious solution would be not to distribute the key at all, and instead retain control of it with the project's core members/leaders. This makes it more annoying for a third part to rebuild the library from the sources, as they'd need to generate a new private key to sign it with, but I'm starting to think this makes an awful lot of sense. After all, if someone else rebuilt the library, it obviously doesn't come from the same team that built it originally, does it?
So, what does anyone else think? What would be the right thing to do here? And, since we're on the topic of strongly signed assemblies, how many people are actually using delayed signing and tightly kept private keys in their work environment, and how well is it working out for you?


Collected below are a series of notes and/or prompts to asking better questions while doing documentary/ethnographic research. It assumes the interviewer has at least a notepad in hand and, perhaps, basic recording technology.
An interview is not a dialogue. The whole point of the interview is to get the narrator to tell her story. Limit your own remarks to a few pleasantries to break the ice, then brief questions to guide her along. It is not necessary to give her the details of your great-grandmother’s life growing up in Abbeville in order to get her to tell you about her grandfather’s trip to Texas. Just say, “I understand your grandfather went to Texas during the oil boom. What did he tell you about his time there?”
Ask questions that require more of an answer than “yes” or “no.” Start with “why,” “how,” “where,” “what kind of …” instead of “Was Henry Miller a good boss?” ask “What did the drilling crew say about Henry Miller?”
Ask one question at a time. Sometimes interviewers ask a series of questions all at once. Probably the narrator will answer only the first or last one. You will catch this kind of questioning when you listen through the recording after the session, and you can avoid it the next time.
Ask brief questions. We all know the irrepressible speech-maker who, when questions are called for at the end of a lecture, gets up and asks five-minute questions. It is unlikely that the narrator is so dull that it takes more than a sentence or two for her to understand the question.
Start with questions that are not controversial; save the delicate questions, if there are any, until you have become better acquainted. A good place to begin is with the narrator’s youth and background.
Don’t let periods of silence fluster you. Give your narrator a chance to think of what she wants to add before you hustle her along with the next question. Relax, write a few words on your notepad. The sure sign of a beginning interviewer is a tape where every brief pause signals the next question
Don’t worry if your questions are not as beautifully phrased as you would like them to be for posterity. A few fumbled questions will help put your narrator at ease as she realizes that you are not perfect and she need not worry if she isn’t either. It is not necessary to practice fumbling a few questions; most of us are nervous enough to do that naturally.
Don’t interrupt a good story because you have thought of a question, or because your narrator is straying from the planned outline. If the information is pertinent, let her go on, but jot down your question on your notepad so you will remember to ask it later.
If your narrator does stray into subjects that are not pertinent (the most common problems are to follow some family member’s children or to get into a series of family medical problems), try to pull her back as quickly as possible. “Before we move on, I’d like to find out how the closing of the mine in 1935 affected your family’s finances. Do you remember that?”
It is often hard for a narrator to describe people. An easy way to begin is to ask her to describe the person’s appearance. From there, the narrator is more likely to move into character description.
Don’t switch the recorder off and on. It is much better to waste a little tape on irrelevant material than to call attention to the tape recorder by a constant on-off operation. Of course you can turn off the recorder if the telephone rings or if someone interrupts your session.
Interviews, for beginning interviewers, usually work out better if there is no one present except the narrator and the interviewer. Sometimes two or more narrators can be successfully recorded, but usually each one of them would have been better alone.
End the interview at a reasonable time. An hour and a half is probably the maximum. First, you must protect your narrator against over-fatigue; second, you will be tired even if she isn’t. Some narrators tell you very frankly if they are tired, or their spouses will. Otherwise, you must plead fatigue, another appointment, or no more tape.
Don’t use the interview to show off your knowledge, vocabulary, charm, or other abilities. Good interviewers do not shine; only their interviews do.
The next few tips are more focused on oral history work, but they apply to other kinds of research as well:
Try to establish at every important point in the story where the narrator was or what her role was in this event, in order to indicate how much is eye-witness information and how much based on reports of others. “Where were you at the time of the mine disaster?” “Did you talk to any of the survivors later?” Work around these questions carefully, so that you will not appear to be doubting the accuracy of the narrator’s account.
Do not challenge accounts you think might be inaccurate. Instead, try to develop as much information as possible that can be used by later researchers in establishing what probably happened.  As Walter Lord explained when describing his interviews with survivors of the Titanic, “Every lady I interviewed had left the sinking ship in the last lifeboat. As I later found out from studying the placement of the lifeboats, no group of lifeboats was in view of another and each lady probably was in the last lifeboat she could see leaving the ship.”
Try to avoid “off the record” information—the times when your narrator asks you to turn off the recorder while she tells you a good story. Ask her to let you record the whole things and promise that you will erase that portion if she asks you to after further consideration. You may have to erase it later, or she may not tell you the story at all, but once you allow “off the record” stories, she may continue with more and more, and you will end up with almost no recorded interview at all. “Off the record” information is only useful if you yourself are researching a subject and this is the only way you can get the information. It has no value if your purpose is to collect information for later use by other researchers.


Referendum: A ‘No’ vote stands for patriotism – Ofosu Ampofo
“A ‘No’ vote is a victory for a good decision for Ghana. We are better off with the current system at the Assemblies because partisanship at the Assemblies would bring confusion,” Ofosu- Ampofo said in an interview monitored by MyNewsGh.com.
The National Chairman of the National Democratic Congress (NDC) Speaking in an interview on Asempa FM’s Ekosii Sen, explained a ‘YES’ vote will further restrict the nation, particularly local communities along the lines of New Patriotic Party (NPP) and NDC.
“The same violent situation at the national level would happen at the local level if the Assemblies’ election becomes partisanship and if we don’t take care we would have NPP and NDC sanitation days. We don’t need partisanship at the local level,” he said.
According to Chairman Ofosu-Ampofo, the party’s position on the referendum has been one and already communicated.


{{alert.msg}}
Calling all users! All users get over here!!
Show a user list:
Show user list
{{key}}:{{value}}
ºل͜º ༽ºل͜º ༽ºل͜º ༽ ＥＶＥＲＹＯＮＥ，ＧＥＴ ＩＮ ＨＥＲＥ ༼ ºل͜º༼ ºل͜º༼ ºل͜º ༽
Create a user:
Open user creation form
I will find you..  ̿̿ ̿̿ ̿'̿'̵͇̿̿з=༼ ▀̿̿Ĺ̯̿̿▀̿ ̿ ༽=ε/̵͇̿̿/’̿’̿ ̿ ̿̿[} ̿ ̿ ̿ ̿^
Search by ID
{{errorData}}
{{key}}:{{value}}
Open user info update form
Delete user
Are you an admin?
YES I AM!!!!
Totally!(Not really, actually no...)
Update user info
{{userUpdateResponce}}
{{userDeleteResponce}}
Add Alert


---
DEPRECATION WARNING
This project has been deprecated and will no longer receive updates.
Please consider using Watchman and the watch function from webcompiler instead.
---
simple-recursive-watch
A simple cross platform recursive directory watcher for NodeJS, based on fs.watch, but not relying on its recursive option.
I just got tired of gulp.watch and gulp-watch. Because they rely on a hopelessly broken gaze library, which besides being too resource intensive for my liking, also crashes all the time.
Errors happen all the time, we cannot completely prevent them from happening. A good application is the one that handles those errors gracefully instead of dying all the time, producing a long and unreadable error stack that prevents you from seeing what really matters.
Fortunately, NodeJS contains a very handy utility called fs.watch, which is much more performant and less resource intensive, because it uses native operating system events to do it's job.
Unfortunately, though it cannot be reliably used to spy on a whole tree of folders recursively. Which is what this library is for.
Installation
npm i simple-recursive-watch --save
API Documentation
To get better acquainted with the available tools feel free to skim through the auto-generated API Docs.
Exposes 1 class
DirectoryWatcher
- recursively watches for file changes in a directory
interface DirectoryWatcher {
constructor(dir: string, type: RegExp, ...exclude: Array<string>);
start();
stop();
static watch(dir: string, extension: string, callback: Function, ...exclude: Array<string>): DirectoryWatcher;
}
Arguments
dir
- a full system path to a directorytype
- a regular expression to match filesexclude
- files/directories to exclude (not full paths, just file/directory names)extension
- a file extension to watch forcallback
- a callback function to execute whenever a file system change occurs
Example usage
import {DirectoryWatcher} from 'simple-recursive-watch';
import {join} from 'path';
var libDir = join(__dirname, 'lib');
DirectoryWatcher.watch(libDir, 'js', function () {
// some JavaScript file, not named "ignoreMe.js" and not residing
// in a "__tests__" directory, was changed in the "lib" directory
}, '__tests__', 'ignoreMe.js');
Explanation
The provided callback is invoked, immediately and only once, when somewhere in the src directory a ".js" file is created, edited, renamed, moved around or deleted, excluding those residing in a "__tests__" directory or having a name "ignoreMe.js".
You can specify any number of file or directory names to ignore.


_y = get_dict_values(x_dict, self.cond_var)
if len(_y) == 0:
x = self.inverse(_z[0])
else:
x = self.inverse(_z[0], y=_y[0])
output_dict = {self.var[0]: x}
p(x=f^{-1}_{flow}(z)),
where :math:`z \sim p_{prior}(z)`.
Once initializing, it can be handled as a distribution module.
Moreover, this distribution can take a conditional variable.
.. math::
p(x=f^{-1}_{flow}(z, y)),
where :math:`z \sim p_{prior}(z)` and :math:`y` is given.


Gaming laptops may be getting Nvidia's Super GPUs early next year
by Mark KnappGaming laptops may be getting Nvidia's Super GPUs early next year
Nvidia looks to turn up the heat on gaming laptops
(Image credit: Future)
Nvidia recently shook up the graphics card market with its line of Super RTX and GTX cards, and now it appears the company may be preparing to deliver that same performance boost to the mobile computing market. Mobile Max-Q versions of Nvidia's GTX 1650 Super up through the RTX 2080 Super have shown in a leak from NotebookCheck translated and shared by Hothardware .
The refresh is expect in March 2020, and the leaked graphics cards are believed to be the following:
- Nvidia GeForce RTX 2080 Super Max-Q with 8GB GDDR6 (80W TDP)
- Nvidia GeForce RTX 2070 Super Max-Q with 8GB GDDR6 (80W TDP)
- Nvidia GeForce RTX 2060 Super Max-Q with 8GB GDDR6 (80W TDP)
- Nvidia GeForce GTX 1660 Super Max-Q with 4GB GDDR6 (35W TDP)
- Nvidia GeForce GTX 1650 Super Max-Q with 4GB GDDR6 (35W TDP)
When the first Super RTX graphics cards launched, they boosted the performance of existing cards. But, they didn't quite manage to outshine what AMD had done with the introduction of the Radeon RX 5700 on a price-to-performance level. AMD's high performance, value, and efficiency thanks to its 7-nanometer architecture helped it get an edge, even if it's not competing at the highest-end.
- See the MSI Alpha 15 with AMD Navi graphics
- These are the best gaming laptops
- Here's our review of the AMD Radeon RX 5700XT
A mobile battle
Now, AMD is geared to gain even more ground in the mobile space with the Radeon RX 5500M. That's where Nvidia's new graphics processors could come in to keep its footing.
These new Super Max-Q graphics processors would effectively take all the recent GPU offerings from Nvidia (excluding the MX models) and upgrade them for better performance. And, that comes at a time when we're seeing more machines that can use the extra performance, like the Asus gaming laptop with a 300Hz display.
AMD's RX 5500M may be a strong low-to-mid-range options, but it's just that. As of yet, AMD doesn't have the anticipated RX 5300M or RX 5700M available in laptops to round out its performance options.
Nvidia's launch of Super Max-Q GPUs in early 2020 could help it cement its dominance in the mobile gaming space, making it harder for AMD to break in even if its Navi mobile GPUs are competitive.
- Learn about Intel's Xe graphics card


DataStoreManagerDelegate
@objc
public protocol DataStoreManagerDelegate : AnyObject
Methods for performing actions in a data store manager.
-
Tells the delegate that the specified storage type of a data store needs to perform migration.
Handle the logic to perform the migration. For example:
switch version { case 0: migrateFromVersionZeroToOne() fallthrough case 1: migrateFromVersionOneToTwo() default: break }
After the delegate method is called, the schema version will be updated to the version defined in dataStoreManager(_:currentSchemaVersionForType:)
Declaration
Swift
@objc optional func dataStoreManager(_ manager: DataStoreManager, performMigrationFromOldVersion oldVersion: Int, forType type: DataStoreStorageType)
Parameters
manager
The data store manager object informing the delegate of this event.
oldVersion
The old schema version for reference to do migration.
type
A storage type constant.
-
Asks the delegate for the cost with which associate to the object.
When memory is limited or when the total cost of the cache eclipses the maximum allowed total cost, the cache could begin an eviction process to remove some of its elements. However, this eviction process is not in a guaranteed order. As a consequence, if you try to manipulate the cost values to achieve some specific behavior, the consequences could be detrimental to your program. Typically, the obvious cost is the size of the value in bytes. If that information is not readily available, you should not go through the trouble of trying to compute it, as doing so will drive up the cost of using the cache. Pass in 0 for the cost value if you otherwise have nothing useful to pass.
Unlike an
NSMutableDictionary
object, a cache does not copy the key objects that are put into it.Declaration
Swift
@objc optional func dataStoreManager(_ manager: DataStoreManager, cacheCostLimitForObject object: Any) -> Int
Parameters
manager
An object representing the data store manager requesting this information.
object
The object to be cached.
Return Value
The
cost
value is used to compute a sum encompassing the costs of all the objects in the cache.
-
Asks the delegate for the cloud kit container record type of the data store manager.
Use this string to differentiate between different record types in your app. The string is primarily for your benefit, so choose type names that reflect the data in the corresponding records.
Declaration
Swift
@available(watchOSApplicationExtension 3.0, *) @objc optional func dataStoreManager(_ manager: DataStoreManager, cloudKitContainerRecordIDForKey key: String) -> CKRecord.ID
Parameters
manager
An object representing the data store manager requesting this information.
key
The key to identify the data store manager object.
Return Value
The app-defined string that identifies the type of the record.
-
Tells the delegate that the specified storage type of a data store needs to handle when the value of one or more keys in the local key-value store changed due to incoming data pushed from iCloud.
The user info dictionary can contain the reason for the notification as well as a list of which values changed, as follows:
- The value of the NSUbiquitousKeyValueStoreChangeReasonKey key, when present, indicates why the key-value store changed. Its value is one of the constants in Change Reason Values.
The value of the [NSUbiquitousKeyValueStoreChangedKeysKey(apple-reference-documentation://hsxKbLUIAR), when present, is an array of strings, each the name of a key whose value changed.
Declaration
Swift
@objc optional func dataStoreManager(_ manager: DataStoreManager, ubiquitousCloudStoreDidChangeExternallyWithUserInfo userInfo: [AnyHashable : Any]?)
Parameters
manager
An object representing the data store manager requesting this information.
userInfo
The user info dictionary.


Recipe 4.19. Assigning and Testing with One Statement
Credit: Alex Martelli, Martin Miller
Problem
You are transliterating C or Perl code
to Python, and to keep close to the original's
structure, you'd like an
expression's result to be both assigned and tested
(as in if((x=foo( )) or while((x=foo(
)) in such other languages).
Solution
In Python, you can't code if x=foo(): . . . . Assignment is a statement, so it cannot fit into
an expression, and you can only use expressions as conditions of
if and while statements. This
isn't a problem, it just means you have to structure
your code Pythonically! For example, to process a file object
f line by line, instead of the following
C-like (and syntactically incorrect, in Python) approach:
while (line=f.readline( )) != '':
process(line)
you can code a highly Pythonic (readable, clean, fast) approach:
for line in f:
process(line)
But sometimes, you're transliterating from C, Perl,
or another language, and you'd like your
transliteration to be structurally close to the original. One simple
utility class makes it easy:
class DataHolder(object):
def _ _init_ _(self, value=None):
self.value = value
def set(self, value):
self.value = value
return value
def get(self):
return self.value
# optional and strongly discouraged, but nevertheless handy at times:
import _ _builtin_ _
_ _builtin_ _.DataHolder = DataHolder
_ _builtin_ _.data = data = DataHolder( )
With the help of the DataHolder class and its
instance data, you can keep your C-like code
structure intact in transliteration:
while data.set(file.readline( )) != '':
process(data.get( ))
Discussion
In Python, assignment is a statement, not an expression. Thus, you
cannot assign the result that you are also testing, for example, in
the condition of an if, elif,
or while statement. This is usually fine: just
structure your code to avoid the need to assign while testing (in
fact, your code will often become clearer as a result). In
particular, whenever you feel the need to assign-and-test within the
condition of a while loop, that's
a good hint that your loop's structure probably
wants to be refactored into a generator (or other iterator). Once you
have refactored in this way, your loops become plain and simple
for statements. The example given in the recipe,
looping over each line read from a text file, is one where the
refactoring has already been done on your behalf by Python itself,
since a file object is an iterator whose items are
the file's lines.
However, sometimes you may be writing Python code that is the
transliteration of code originally written in C, Perl, or some other
language that supports assignment-as-expression. Such
transliterations often occur in the first Python version of an
algorithm for which a reference implementation is supplied, an
algorithm taken from a book, and so on. In such cases,
it's often preferable to have the structure of your
initial transliteration be close to that of the code
you're transcribing. You can refactor later and make
your code more Pythonicclearer, faster, and so on. But first,
you want to get working code as soon as possible, and specifically
you want code that is easy to check for compliance to the original it
has been transliterated from. Fortunately, Python offers enough power
to make it quite easy for you to satisfy this requirement.
Python doesn't let us redefine the meaning of
assignment, but we can have a method (or function) that saves its
argument somewhere and also returns that
argument so it can be tested. That somewhere is
most naturally an attribute of an object, so a method is a more
natural choice than a function. Of course, we could just retrieve the
attribute directly (i.e., the get method is
redundant), but it looks nicer to me to have symmetry between
data.set and data.get.
data.set(whatever) can be seen as little more than
syntactic sugar around data.value=whatever, with
the added value of being acceptable as an expression. Therefore,
it's the one obviously right way to satisfy the
requirement for a reasonably faithful transliteration. The only
difference between the resulting Python code and the original (say) C
or Perl code, is at the syntactic sugar levelthe overall
structure is the same, and that's the key issue.
Importing _ _builtin_ _ and assigning to its
attributes is a trick that basically defines a new built-in object at
runtime. You can use that trick in your
application's start-up code, and then all other
modules will automatically be able to access your new built-ins
without having to do an import.
It's not good Python practice,
though; on the contrary, it's pushing the boundaries
of Pythonic good taste, since the readers of all those other modules
should not have to know about the strange side effects performed in
your application's startup code. But since this
recipe is meant to offer a quick-and-dirty approach for a first
transliteration that will soon be refactored to make it better, it
may be acceptable in this specific context to cut more corners than
one would in production-level code.
On the other hand, one trick you should definitely
not use is the following abuse of a currently
existing wart in list comprehensions:
while [line for line in [f.readline( )] if line!='']:
process(line)
This trick currently works, since both Python 2.3 and 2.4 still
"leak" the list comprehension
control variable (here, line) into the
surrounding scope. However, besides being obscure and unreadable,
this trick is specifically deprecated: list comprehension control
variable leakage will be fixed in some future
version of Python, and this trick will then stop working at all.
See Also
The Tutorial section on classes; the
documentation for the _ _builtin_ _ module in the
Library Reference and Python in a
Nutshell; Language Reference and
Python in a Nutshell documentation on list
comprehensions.
|


If you came here from a download link, please try again or contact me if it still doesn't work. If you just came here randomly, you might find the links below useful


Thank you for watching!
I hope to see you next time.
Sole Signal - Super Mario 64 "See You Next Time" OC ReMix


5 Facts about Me:
Mouse over to see facts!
- Born in North Dakota
- Hometown is Yakima, WA
- Played the clarinet in highschool
- Stationed for 5 years in Minot, ND serving in the Air Force
- Has a cat named Max
Click here to take a quiz!!


[Editor's Note: 60 grams of fresh mushroom is roughly equivalent to 6 grams of dried mushrooms.]
Slipping through the Coil of Space and Time
*The Holy Trinity: Psilocybin, Harmine, DimethylTrypamine*
By DMT 333 (-Fu- 11/6/01)
After completing another successful extraction of N,N DimethylTryptamine Freebase from mimosa rootbark, I was again ready to experiment with a synergistic combination I have dubbed 'The Holy Trinity'. This combination consists of 60 grams of fresh Psilocybe Cubensis (with veils still intact); 1 gram of Harmala alkaloid extract (equivalent to 4g Syrian Rue seeds), and to top it all off 60 milligrams of DMT freebase.
Keep in mind that the Harmala alkaloids are Mono Amino Oxidase Inhibitors which causes these alkaloids to inhibit the production of the MAO enzyme which breaks down serotonin as well as other tryptamines i.e. DMT/Psilocybin therefore potentiating the effects achieved by the psychoactive tryptamines ingested.
Having fasted for 8 hours, at 5:55 PM I ingested 1 gram of Harmala Extract in a 'OOO' sized gelcap; promptly following at 6:36 PM I thoroughly chewed and ingested the 60 grams of mushrooms and chased it down with a small glass of freshly squeezed orange juice with lime.
Immediately after consumption of the fungi I felt a distinct hazy-dreamlike attribute to my environment, something I always noticed upon the onset of an MAOI + DMT/Psilocybin journey. From 7:00-7:45 I began to progressively watch my ego disintegrate itself into the aethyr. This process of ego dissolution started out as a delicate web-like structure that appeared to be made of silver illuminating threads of silk emmanating from the center of my field of vision. This web continued to increase in detail and otherworldiness as multi-colored translucent tentacles began to spiral around each silver thread of this 'web'.
As this web-like structure continued to become more intricate I could feel my ego being overshadowed by what I can only describe as an unstoppable mechanistic, emotionally void state of consciousness; in addition there is a crackling noise that progressively emcompasses your awareness as this state of consciousness unfolds in your mind - it is quite similar to tinnitus (musician's curse). I have come to refer to this state of consciousness that develops as 'mycelial consiousness' or 'plant consciousness', others I know refer to it as 'insectoid' or even 'extraterrestrial' awareness. (Or hell you can even call it the 'void') It soon appeared as if I penetrated as what I can only describe as a membrane or bubble, for when that happened this web-like structure became very animated.
I lost all my bearings; I knew I was once again in that familiar place known as hyperspace but I forgot who I was and how I got there. I was in absolutely awe as this 'web' morphed into what appeared as a plasma octopoid with flailing appendages similar to tentacles that were moving VERY fast and appeared to be tending to some task quite dilligently. In addition when these tentacles would move they would emit a reverberating set of tones each movement produced a unique set of harmonic audio expressions. Soon I became aware that this 'octopoid' was actually redirecting my awareness to these *incredible* structures that were made with his appendages/harmonic manipulations.
Each time my attention would come to rest I would suddenly feel my awareness redirected to another structure being built that was more spectacular than the last. Before long I noticed a pressure in my center and I realized once again I was contained in a physical vehicle and that I was perceiving the weight of my hands interlaced on my solar plexus. My ego was reassembling itself and the web/octopoid simultaneously was losing it's animate life-like qualities.
I looked at the clock, it read 7:38 PM, 'perfect!', I thought to myself, 'Now's the time to load up on DMT while the MAOI is still coursing through my veins.' Immediately I retrieved my DMT pipe along with the premeasured dose of 60 mg I had weighed out. I started playing a CD of a Japanese Shakuhama Flute player and sat in full lotus position with the DMT pipe pressed up against my 3rd eye contemplating the journey about to take place and what I wanted to gain from it.
As the music in the background began to play I began to torch the bottom of my glass DMT oil pipe and watched as the DMT Crystals first liquified then vaporized into the chamber of the pipe. I took 4 huge lungfuls and as I exhaled the fourth lungful of smoke I was washed away into hyperspace once again. Upon entering hyperspace I percieved myself falling through a tunnel in zero gravity at light speed then once again I penetrated a 'bubble/membrane' and was in what I refer to as 'The Dome' or 'The Control Panel' only this time instead of a percieved 'octopoid' redirecting my awareness to various structures there was this huge gelatinous-hexagonal-rubix-cube type machine that would reform itself into structures according to these progressive harmonic tones that permeated my reality causing various emotions to emerge in addition it was redirecting my attention to various intersections of it's restructured embodiment.
Each time my attention was pointed to one of these intersections/nodes a vision followed by a revelation, would envelope me along with an emotion I can only describe as pure elation and awe. The only vision/relevation that I vividly remember was one of humanity's history. I recall seeing many people in positions of power from ancient civilizations from all around the globe. I could see a distinct spectrum of colors for each spirit that belonged to each of these kings/warriors/ thieves/martyrs/prophets and shamans.
I saw their entire lifespan from birth to death and then saw how each of these kings/warriors etc. subconsiously chose the exact moment they were going to die as well as how they died and as result of their choices their spirits were reborn into the bodies of the leaders/warriors/shamans of the next generation in order to continue their spiritual development were it left off in addition to being catalysts for events of novelty that were yet to unfold (ie some warriors reborn as leaders etc). This process of reincarnation continued generation after generation up until the present day.
As I was coming out of hyperspace I realized I was emitting these very strange noises from my mouth that were actually driving and changing the visual content of my experience, what struck me as even more strange/amazing was that in addition to every noticeable change in the visual content of my trip and vocal emissions coincided wih changes in my physical environment such as windblowing/crickets chirping. Feeling like a newborn baby fresh from the womb I shed a tear in absolute reverence for life and being.
Note: Do not assume because of the way much of this was presented that I believe in mushroom entities or DMT entities... I do not come to such premature conclusions without proper empirical evidence. For me, everything is one spirit. One Life, One Love.
May all of your experiences be fruitful -Fu-
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


Heat map
Actual use
Predicted need
Crisis
Select the crisis type
Benefit changes
Benefit delays
Child holiday meals
Debt
Delayed wages
Domestic violence
Homeless
Low income
Other
Sickness
Unemployed
Opacity
Foodbanks
Visible
Reach
(At the moment we only have actual customers)
Actual customers
Drive time
Walking
Transit
Legend
Explanation goes here


Functions
The following functions are available globally.
-
This is a factory for IP addresses (as Strings).
Declaration
Swift
public func RVS_IPAddressExtractIPAddress(_ inString: String, isPadded inIsPadded: Bool = false) -> RVS_IPAddress!
Parameters
inString
The string to be parsed.
isPadded
Optional IPv6 padding variable. If true (default is false), then IPv6 addresses will be fully padded. Ignored for IPv4.
Return Value
a valid IPv4 or IPv6 address object. nil, if the String cannot produce a valid IP address.
-
This is a factory for IP addresses (as Int Arrays).
Declaration
Swift
public func RVS_IPAddressExtractIPAddress(array inArray: [Int], port inPort: Int = 0, isPadded inIsPadded: Bool = false) -> RVS_IPAddress!
Parameters
array
The Array to be used.
port
An optional (default is 0 -no port) Int, with the TCP port.
isPadded
Optional IPv6 padding variable. If true (default is false), then IPv6 addresses will be fully 0-padded (no shortcuts). Ignored for IPv4.
Return Value
a valid IPv4 or IPv6 address object. nil, if the Array cannot produce a valid IP address.


Upload descriptor, this contains meta information regarding the
uploaded file, including error codes in a consistent
| Constant | Description |
| ERR_CANT_WRITE | Upload error constant - Cannot write (or move the uploaded file) |
| ERR_CUSTOM | Upload error constant - Custom (caused by event callbacks) |
| ERR_MIME_FINFO | Upload error constant - Fileinfo failed (extension not available, or unable to resolve) |
| ERR_NAMING | Upload error constant - Naming (could not determine name if filename or extension was missing in the... |
| ERR_NONE | Upload error constant - None |
| ERR_OVERRIDE | Upload error constant - File override (if 'allow_override' is false and naming matches) |
| ERR_SIZE | Upload error constant - Size exceeded |
| ERR_UNKNOWN | Upload error constant - Unknown (like no input etc) |
| Property | Description |
| $backend | The backend type for this descriptor |
| $error | Error code (defaults to none) |
| $extension | File extension (may be empty) and does not contain the initial dot |
| $filename | Filename without the extension |
| $mime | File MIME type, this may be fake |
| $real_mime | Real MIME type, this is false by default and will only have a value if |
| $rsrc | External resource, some backends, like the 'url' may supply the socket |
Generated by Tuxxedo Engine 1.3.0 "Intergalactic" Alpha 1 (experimental).


{% extends "base.html" %}
{% macro api_param_doc(param, operationId, read_direction=True) -%}
{% if param.type %}
{% if param.type == 'object' %}
{% for attr_name, attr in param.properties.items() %}
{% if (read_direction and attr_name|not_key) or (not read_direction and not attr.readOnly) %}
{% endif %}
{% endfor %}
{{attr_name}}
null
{{param.description}}
Authenticate your account by including your API key in API requests. You can manage your API keys on your profile page. Your API keys carry many privileges, so be sure to keep them secure! Do not share your secret API keys in publicly accessible areas such GitHub, client-side code, and so forth.
Authentication to the API is performed via HTTP Basic Auth. Provide your API key as the basic auth username value. You do not need to provide a password.
All API requests must be made over HTTPS. Calls made over plain HTTP will fail.
curl {{api_base_url}} \ -u BQokikJOvBiI2HlWgH4olfQ24ret3awZ:
curl uses the -u flag to pass basic auth credentials. (Adding a colon after your API key prevents curl from asking for a password.)
{{resp_code}}


The C&S Connect fields for committee selection under the Staff Tab and VCC Tab will include this list at the top so you can find them more quickly.
Begin typing the committee name into the input and it will autocomplete. If you can't find your committee by typing the name, please check the full committee list provided on this page. You can copy and paste the name.


{% block content %} {% if interfaceinfo %}
| Operational status: | {{ interfaceinfo['operStatus'][5:] }} |
| Port name: | {{ interfaceinfo['name'] }} |
| Link speed (Mbps): | {{ interfaceinfo['port_speed_mbps'] }} |
| Configuration mode: | {{ interfaceinfo['config_mode'][4:] }} |
| Unicast packets received: | {{ interfaceinfo['packets_rx'] }} |
| Bytes received: | {{ interfaceinfo['bytes_rx'] }} |
| Unicast packets transmitted: | {{ interfaceinfo['packets_tx'] }} |
| Bytes transmitted: | {{ interfaceinfo['bytes_tx'] }} |
| Receive throughput (bps): | {{ interfaceinfo['throughput_rx_bps'] }} |
| Transmit throughput (bps): | {{ interfaceinfo['throughput_tx_bps'] }} |
| Errors received: | {{ interfaceinfo['error_rx'] }} |
| Error transmitted: | {{ interfaceinfo['error_tx'] }} |
| Packets dropped: | {{ interfaceinfo['drop_tx'] }} |
| Remote system name: | {{ lldpinfo['system_name'] }} |
| Remote system description: | {{ lldpinfo['system_description'][:60] }} |
| Remote port: | {{ lldpinfo['port_description'] }} |
| Requested PoE power: | {{ lldpinfo['poe_plus_info']['requested_power_in_watts'] }} Watt |
| PoE power provided: | {{ lldpinfo['poe_plus_info']['actual_power_in_watts'] }} Watt |


---
layout: post
title: Deploying grails app to Jetty
tags:
- ! '*nix'
- apache
- debian
- grails
- jetty
status: publish
type: post
published: true
meta:
aktt_notify_twitter: 'no'
_edit_last: '2'
---
Short and sweet, step by step guide for creating a grails war and deploying it to your Jetty server (includes apache2 proxy steps)
- grails war (your app), in this case epic.war
- Copy to server [code]scp epic.war user@server:/location[/code]
- On Debian the location for jetty webapps is: /var/lib/jetty/webapps
- Copy from upload location to the above folder (I used sudo)
- Change permissions: [code]sudo chown jetty:adm epic-0-0.1.war[/code]
- created a *.xml context file in /etc/jetty/contexts, with something like this [note case is important!!]:
[code]
/epic
/webapps/epic-0-0.1.war
[/code]
- restart jetty : [code]sudo /etc/init.d/jetty restart[/code] (try stop/start as well)
- test with : [code]lynx http://localhost:8080/epic/[/code]
- Nice but I'd like http://localhost/epic/ so enable mod_proxy in apache if you haven't already : [code]a2enmod proxy[/code]
- edit /etc/apache2/mods-enabled/proxy.load and if not present add at end: [code]LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so[/code]
- edit your virtualhost conf file I am using 000-default and add:[code]ProxyRequests Off
Order deny,allow
Allow from all
ProxyPass /epic http://localhost:8080/epic
ProxyPassReverse /epic http://localhost:8080/epic
ProxyPreserveHost On[/code]
- restart apache : [code]sudo /etc/init.d/apache2 restart[/code]
- now you can [code]lynx http://localhost/epic/[/code]
Update: I had omitted the ProxyPreserveHost On
from the Apache configuration, which resulted in css, images and external javascripts not loading. It also caused an issue with accessing controllers.
Sources: