1 How doth the city sit solitary, that was full of people! how is she become as a widow! she that was great among the nations, and princess among the provinces, how is she become tributary!
2 She weepeth sore in the night, and her tears are on her cheeks: among all her lovers she hath none to comfort her: all her friends have dealt treacherously with her, they are become her enemies.
3 Judah is gone into captivity because of affliction, and because of great servitude: she dwelleth among the heathen, she findeth no rest: all her persecutors overtook her between the straits.
4 The ways of Zion do mourn, because none come to the solemn feasts: all her gates are desolate: her priests sigh, her virgins are afflicted, and she is in bitterness.
5 Her adversaries are the chief, her enemies prosper; for the LORD hath afflicted her for the multitude of her transgressions: her children are gone into captivity before the enemy.
6 And from the daughter of Zion all her beauty is departed: her princes are become like harts that find no pasture, and they are gone without strength before the pursuer.
7 Jerusalem remembered in the days of her affliction and of her miseries all her pleasant things that she had in the days of old, when her people fell into the hand of the enemy, and none did help her: the adversaries saw her, and did mock at her sabbaths.
8 Jerusalem hath grievously sinned; therefore she is removed: all that honoured her despise her, because they have seen her nakedness: yea, she sigheth, and turneth backward.
9 Her filthiness is in her skirts; she remembereth not her last end; therefore she came down wonderfully: she had no comforter. O LORD, behold my affliction: for the enemy hath magnified himself.
10 The adversary hath spread out his hand upon all her pleasant things: for she hath seen that the heathen entered into her sanctuary, whom thou didst command that they should not enter into thy congregation.
11 All her people sigh, they seek bread; they have given their pleasant things for meat to relieve the soul: see, O LORD, and consider; for I am become vile.
12 Is it nothing to you, all ye that pass by? behold, and see if there be any sorrow like unto my sorrow, which is done unto me, wherewith the LORD hath afflicted me in the day of his fierce anger.
13 From above hath he sent fire into my bones, and it prevaileth against them: he hath spread a net for my feet, he hath turned me back: he hath made me desolate and faint all the day.
14 The yoke of my transgressions is bound by his hand: they are wreathed, and come up upon my neck: he hath made my strength to fall, the Lord hath delivered me into their hands, from whom I am not able to rise up.
15 The Lord hath trodden under foot all my mighty men in the midst of me: he hath called an assembly against me to crush my young men: the Lord hath trodden the virgin, the daughter of Judah, as in a winepress.
16 For these things I weep; mine eye, mine eye runneth down with water, because the comforter that should relieve my soul is far from me: my children are desolate, because the enemy prevailed.
17 Zion spreadeth forth her hands, and there is none to comfort her: the LORD hath commanded concerning Jacob, that his adversaries should be round about him: Jerusalem is as a menstruous woman among them.
18 The LORD is righteous; for I have rebelled against his commandment: hear, I pray you, all people, and behold my sorrow: my virgins and my young men are gone into captivity.
19 I called for my lovers, but they deceived me: my priests and mine elders gave up the ghost in the city, while they sought their meat to relieve their souls.
20 Behold, O LORD; for I am in distress: my bowels are troubled; mine heart is turned within me; for I have grievously rebelled: abroad the sword bereaveth, at home there is as death.
21 They have heard that I sigh: there is none to comfort me: all mine enemies have heard of my trouble; they are glad that thou hast done it: thou wilt bring the day that thou hast called, and they shall be like unto me.
22 Let all their wickedness come before thee; and do unto them, as thou hast done unto me for all my transgressions: for my sighs are many, and my heart is faint.
1 How hath the Lord covered the daughter of Zion with a cloud in his anger, and cast down from heaven unto the earth the beauty of Israel, and remembered not his footstool in the day of his anger!
2 The Lord hath swallowed up all the habitations of Jacob, and hath not pitied: he hath thrown down in his wrath the strong holds of the daughter of Judah; he hath brought them down to the ground: he hath polluted the kingdom and the princes thereof.
3 He hath cut off in his fierce anger all the horn of Israel: he hath drawn back his right hand from before the enemy, and he burned against Jacob like a flaming fire, which devoureth round about.
4 He hath bent his bow like an enemy: he stood with his right hand as an adversary, and slew all that were pleasant to the eye in the tabernacle of the daughter of Zion: he poured out his fury like fire.
5 The Lord was as an enemy: he hath swallowed up Israel, he hath swallowed up all her palaces: he hath destroyed his strong holds, and hath increased in the daughter of Judah mourning and lamentation.
6 And he hath violently taken away his tabernacle, as if it were of a garden: he hath destroyed his places of the assembly: the LORD hath caused the solemn feasts and sabbaths to be forgotten in Zion, and hath despised in the indignation of his anger the king and the priest.
7 The Lord hath cast off his altar, he hath abhorred his sanctuary, he hath given up into the hand of the enemy the walls of her palaces; they have made a noise in the house of the LORD, as in the day of a solemn feast.
8 The LORD hath purposed to destroy the wall of the daughter of Zion: he hath stretched out a line, he hath not withdrawn his hand from destroying: therefore he made the rampart and the wall to lament; they languished together.
9 Her gates are sunk into the ground; he hath destroyed and broken her bars: her king and her princes are among the Gentiles: the law is no more; her prophets also find no vision from the LORD.
10 The elders of the daughter of Zion sit upon the ground, and keep silence: they have cast up dust upon their heads; they have girded themselves with sackcloth: the virgins of Jerusalem hang down their heads to the ground.
11 Mine eyes do fail with tears, my bowels are troubled, my liver is poured upon the earth, for the destruction of the daughter of my people; because the children and the sucklings swoon in the streets of the city.
12 They say to their mothers, Where is corn and wine? when they swooned as the wounded in the streets of the city, when their soul was poured out into their mothers' bosom.
13 What thing shall I take to witness for thee? what thing shall I liken to thee, O daughter of Jerusalem? what shall I equal to thee, that I may comfort thee, O virgin daughter of Zion? for thy breach is great like the sea: who can heal thee?
14 Thy prophets have seen vain and foolish things for thee: and they have not discovered thine iniquity, to turn away thy captivity; but have seen for thee false burdens and causes of banishment.
15 All that pass by clap their hands at thee; they hiss and wag their head at the daughter of Jerusalem, saying, Is this the city that men call The perfection of beauty, The joy of the whole earth?
16 All thine enemies have opened their mouth against thee: they hiss and gnash the teeth: they say, We have swallowed her up: certainly this is the day that we looked for; we have found, we have seen it.
17 The LORD hath done that which he had devised; he hath fulfilled his word that he had commanded in the days of old: he hath thrown down, and hath not pitied: and he hath caused thine enemy to rejoice over thee, he hath set up the horn of thine adversaries.
18 Their heart cried unto the Lord, O wall of the daughter of Zion, let tears run down like a river day and night: give thyself no rest; let not the apple of thine eye cease.
19 Arise, cry out in the night: in the beginning of the watches pour out thine heart like water before the face of the Lord: lift up thy hands toward him for the life of thy young children, that faint for hunger in the top of every street.
20 Behold, O LORD, and consider to whom thou hast done this. Shall the women eat their fruit, and children of a span long? shall the priest and the prophet be slain in the sanctuary of the Lord?
21 The young and the old lie on the ground in the streets: my virgins and my young men are fallen by the sword; thou hast slain them in the day of thine anger; thou hast killed, and not pitied.
22 Thou hast called as in a solemn day my terrors round about, so that in the day of the LORD's anger none escaped nor remained: those that I have swaddled and brought up hath mine enemy consumed.
1 I am the man that hath seen affliction by the rod of his wrath.
2 He hath led me, and brought me into darkness, but not into light.
3 Surely against me is he turned; he turneth his hand against me all the day.
4 My flesh and my skin hath he made old; he hath broken my bones.
5 He hath builded against me, and compassed me with gall and travail.
6 He hath set me in dark places, as they that be dead of old.
7 He hath hedged me about, that I cannot get out: he hath made my chain heavy.
8 Also when I cry and shout, he shutteth out my prayer.
9 He hath inclosed my ways with hewn stone, he hath made my paths crooked.
10 He was unto me as a bear lying in wait, and as a lion in secret places.
11 He hath turned aside my ways, and pulled me in pieces: he hath made me desolate.
12 He hath bent his bow, and set me as a mark for the arrow.
13 He hath caused the arrows of his quiver to enter into my reins.
14 I was a derision to all my people; and their song all the day.
15 He hath filled me with bitterness, he hath made me drunken with wormwood.
16 He hath also broken my teeth with gravel stones, he hath covered me with ashes.
17 And thou hast removed my soul far off from peace: I forgat prosperity.
18 And I said, My strength and my hope is perished from the LORD:
19 Remembering mine affliction and my misery, the wormwood and the gall.
20 My soul hath them still in remembrance, and is humbled in me.
21 This I recall to my mind, therefore have I hope.
22 It is of the LORD's mercies that we are not consumed, because his compassions fail not.
23 They are new every morning: great is thy faithfulness.
24 The LORD is my portion, saith my soul; therefore will I hope in him.
25 The LORD is good unto them that wait for him, to the soul that seeketh him.
26 It is good that a man should both hope and quietly wait for the salvation of the LORD.
27 It is good for a man that he bear the yoke in his youth.
28 He sitteth alone and keepeth silence, because he hath borne it upon him.
29 He putteth his mouth in the dust; if so be there may be hope.
30 He giveth his cheek to him that smiteth him: he is filled full with reproach.
31 For the Lord will not cast off for ever:
32 But though he cause grief, yet will he have compassion according to the multitude of his mercies.
33 For he doth not afflict willingly nor grieve the children of men.
34 To crush under his feet all the prisoners of the earth,
35 To turn aside the right of a man before the face of the most High,
36 To subvert a man in his cause, the Lord approveth not.
37 Who is he that saith, and it cometh to pass, when the Lord commandeth it not?
38 Out of the mouth of the most High proceedeth not evil and good?
39 Wherefore doth a living man complain, a man for the punishment of his sins?
40 Let us search and try our ways, and turn again to the LORD.
41 Let us lift up our heart with our hands unto God in the heavens.
42 We have transgressed and have rebelled: thou hast not pardoned.
43 Thou hast covered with anger, and persecuted us: thou hast slain, thou hast not pitied.
44 Thou hast covered thyself with a cloud, that our prayer should not pass through.
45 Thou hast made us as the offscouring and refuse in the midst of the people.
46 All our enemies have opened their mouths against us.
47 Fear and a snare is come upon us, desolation and destruction.
48 Mine eye runneth down with rivers of water for the destruction of the daughter of my people.
49 Mine eye trickleth down, and ceaseth not, without any intermission,
50 Till the LORD look down, and behold from heaven.
51 Mine eye affecteth mine heart because of all the daughters of my city.
52 Mine enemies chased me sore, like a bird, without cause.
53 They have cut off my life in the dungeon, and cast a stone upon me.
54 Waters flowed over mine head; then I said, I am cut off.
55 I called upon thy name, O LORD, out of the low dungeon.
56 Thou hast heard my voice: hide not thine ear at my breathing, at my cry.
57 Thou drewest near in the day that I called upon thee: thou saidst, Fear not.
58 O Lord, thou hast pleaded the causes of my soul; thou hast redeemed my life.
59 O LORD, thou hast seen my wrong: judge thou my cause.
60 Thou hast seen all their vengeance and all their imaginations against me.
61 Thou hast heard their reproach, O LORD, and all their imaginations against me;
62 The lips of those that rose up against me, and their device against me all the day.
63 Behold their sitting down, and their rising up; I am their musick.
64 Render unto them a recompence, O LORD, according to the work of their hands.
65 Give them sorrow of heart, thy curse unto them.
66 Persecute and destroy them in anger from under the heavens of the LORD.
1 How is the gold become dim! how is the most fine gold changed! the stones of the sanctuary are poured out in the top of every street.
2 The precious sons of Zion, comparable to fine gold, how are they esteemed as earthen pitchers, the work of the hands of the potter!
3 Even the sea monsters draw out the breast, they give suck to their young ones: the daughter of my people is become cruel, like the ostriches in the wilderness.
4 The tongue of the sucking child cleaveth to the roof of his mouth for thirst: the young children ask bread, and no man breaketh it unto them.
5 They that did feed delicately are desolate in the streets: they that were brought up in scarlet embrace dunghills.
6 For the punishment of the iniquity of the daughter of my people is greater than the punishment of the sin of Sodom, that was overthrown as in a moment, and no hands stayed on her.
7 Her Nazarites were purer than snow, they were whiter than milk, they were more ruddy in body than rubies, their polishing was of sapphire:
8 Their visage is blacker than a coal; they are not known in the streets: their skin cleaveth to their bones; it is withered, it is become like a stick.
9 They that be slain with the sword are better than they that be slain with hunger: for these pine away, stricken through for want of the fruits of the field.
10 The hands of the pitiful women have sodden their own children: they were their meat in the destruction of the daughter of my people.
11 The LORD hath accomplished his fury; he hath poured out his fierce anger, and hath kindled a fire in Zion, and it hath devoured the foundations thereof.
12 The kings of the earth, and all the inhabitants of the world, would not have believed that the adversary and the enemy should have entered into the gates of Jerusalem.
13 For the sins of her prophets, and the iniquities of her priests, that have shed the blood of the just in the midst of her,
14 They have wandered as blind men in the streets, they have polluted themselves with blood, so that men could not touch their garments.
15 They cried unto them, Depart ye; it is unclean; depart, depart, touch not: when they fled away and wandered, they said among the heathen, They shall no more sojourn there.
16 The anger of the LORD hath divided them; he will no more regard them: they respected not the persons of the priests, they favoured not the elders.
17 As for us, our eyes as yet failed for our vain help: in our watching we have watched for a nation that could not save us.
18 They hunt our steps, that we cannot go in our streets: our end is near, our days are fulfilled; for our end is come.
19 Our persecutors are swifter than the eagles of the heaven: they pursued us upon the mountains, they laid wait for us in the wilderness.
20 The breath of our nostrils, the anointed of the LORD, was taken in their pits, of whom we said, Under his shadow we shall live among the heathen.
21 Rejoice and be glad, O daughter of Edom, that dwellest in the land of Uz; the cup also shall pass through unto thee: thou shalt be drunken, and shalt make thyself naked.
22 The punishment of thine iniquity is accomplished, O daughter of Zion; he will no more carry thee away into captivity: he will visit thine iniquity, O daughter of Edom; he will discover thy sins.
1 Remember, O LORD, what is come upon us: consider, and behold our reproach.
2 Our inheritance is turned to strangers, our houses to aliens.
3 We are orphans and fatherless, our mothers are as widows.
4 We have drunken our water for money; our wood is sold unto us.
5 Our necks are under persecution: we labour, and have no rest.
6 We have given the hand to the Egyptians, and to the Assyrians, to be satisfied with bread.
7 Our fathers have sinned, and are not; and we have borne their iniquities.
8 Servants have ruled over us: there is none that doth deliver us out of their hand.
9 We gat our bread with the peril of our lives because of the sword of the wilderness.
10 Our skin was black like an oven because of the terrible famine.
11 They ravished the women in Zion, and the maids in the cities of Judah.
12 Princes are hanged up by their hand: the faces of elders were not honoured.
13 They took the young men to grind, and the children fell under the wood.
14 The elders have ceased from the gate, the young men from their musick.
15 The joy of our heart is ceased; our dance is turned into mourning.
16 The crown is fallen from our head: woe unto us, that we have sinned!
17 For this our heart is faint; for these things our eyes are dim.
18 Because of the mountain of Zion, which is desolate, the foxes walk upon it.
19 Thou, O LORD, remainest for ever; thy throne from generation to generation.
20 Wherefore dost thou forget us for ever, and forsake us so long time?
21 Turn thou us unto thee, O LORD, and we shall be turned; renew our days as of old.
22 But thou hast utterly rejected us; thou art very wroth against us.
| << Jeremiah | Contents | Ezekiel >> |


Enhancements in Java SE 6
for the java.lang.instrument Package
The following functionality has been added to the
java.lang.instrument
package between its initial
release in JDK 5.0 and Java™ SE 6.
- Support for retransformation of class files - To
facilitate dynamic transformation of classes that have already been
loaded, the following methods have been added.
They are suited to operations such as adding instrumentation to
methods whose classes have already been loaded. Support for
retransformation means that access to the original class file is no
longer required in order to instrument loaded classes. They enable
the easy removal of applied transformations and are designed to
work in multi-agent environments.
- Support for instrumenting native methods - The following
methods allow native methods to be instrumented by providing a
JVM-aware mechanism for wrapping the native method.
- Support for appending to class loader search - The
following methods allow instrumentation support classes to be
defined in the appropriate class loader.


pyshgp.gp.evaluation module¶
The evaluation
module defines classes to evaluate program CodeBlocks.
-
class
pyshgp.gp.evaluation.
DatasetEvaluator
(X, y, interpreter: pyshgp.push.interpreter.PushInterpreter = 'default', penalty: float = inf, last_str_from_stdout: bool = False, verbosity_config: Optional[pyshgp.monitoring.VerbosityConfig] = None)[source]¶ Bases:
pyshgp.gp.evaluation.Evaluator
Evaluator driven by a labeled dataset.
-
class
pyshgp.gp.evaluation.
Evaluator
(interpreter: pyshgp.push.interpreter.PushInterpreter = 'default', penalty: float = inf, verbosity_config: pyshgp.monitoring.VerbosityConfig = None)[source]¶ Bases:
abc.ABC
Base class or evaluators.
Parameters: - interpreter (PushInterpreter, optional) – PushInterpreter used to run program and get their output. Default is an interpreter with the default configuration and all core instructions registered.
- penalty (float, optional) – When a program’s output cannot be evaluated on a particular case, the penalty error is assigned. Default is 5e5.
- verbosity_config (Optional[VerbosityConfig] (default = None)) – A VerbosityConfig controling what is logged during evaluation. Default is no verbosity.
-
default_error_function
(actuals, expecteds) → numpy.array[source]¶ Produce errors of actual program output given expected program output.
The default error function is intented to be a universal error function for Push programs which only output a subset of the standard data types.
Parameters: - actuals (list) – The values produced by running a Push program on a sequences of cases.
- expecteds (list) – The ground truth values for the sequence of cases used to produce the actuals.
Returns: An array of error values describing the program’s performance.
Return type: np.array
-
class
pyshgp.gp.evaluation.
FunctionEvaluator
(error_function: Callable)[source]¶ Bases:
pyshgp.gp.evaluation.Evaluator
Evaluator driven by an error function.
-
pyshgp.gp.evaluation.
damerau_levenshtein_distance
(a: Sequence, b: Sequence) → int[source]¶ Damerau Levenshtein Distance that works for both strings and lists.
https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance. This implemenation is heavily inspired by the implementation in the jellyfish package. https://github.com/jamesturk/jellyfish


The Model-Based Meta-Analysis sub-Special Interest Group (MBMA sub-SIG) of the Statistics and Pharmacometrics Special Interest group (SxP SIG) held an “Introduction to MBMA” webinar on January 16th 2021.
Abstract
As landscapes for the treatment of many diseases have grown increasingly competitive and costs to conduct clinical trials have soared, it is critical to understand the landscape of available treatments, as well as the probability that a specific compound could compete successfully in this market.
Model-based meta-analysis (MBMA) is a tool for integrating prior knowledge, both public and proprietary, and for using it to inform discovery and development decisions. As in traditional meta-analysis, MBMA allows one to more precisely quantify a treatment effect by incorporating data from multiple studies. Inclusion of model parameters further enables quantification and explanation of variability observed across trials by accounting for differences in effect modifiers, such as treatments, doses, time, population characteristics, etc. Thus, MBMA is a powerful tool for predicting new, potentially interesting clinical scenarios through clinical trial simulation. Additionally, applying the MBMA to perform simulations can improve strategic decision-making by projecting the likelihood of achieving differentiable efficacy and/or safety.
The webinar will enable you to understand what MBMA is and will present some examples showing the impact it can have in drug development and other critical business decisions.


Veronika Decides to Die
Veronika Decides to Die is a movie starring Sarah Michelle Gellar, Jonathan Tucker, and Erika Christensen.  Under the supervision of an unorthodox...


|
Statistics with R Basel R Bootcamp |
from rottentomatoes.com
In this practical you’ll practice “mixed effects modeling” with the lme4
, sjstats
, stats
, and pbkrtest
packages.
By the end of this practical you will know how to:
BaselRBootcamp
R project. It should already have the folders 1_Data
and 2_Code
. Make sure that the data files listed in the Datasets
section above are in your 1_Data
folder# Done!
MixedModels_practical.R
in the 2_Code
folder.# Done!
library()
load the tidyverse
, lme4
, and sjstats
packages (if you don’t have them, you’ll need to install them with install.packages()
)!# Load packages necessary for this script
library(tidyverse)
library(lme4)
library(sjstats)
tomatometer.csv
and schools.csv
data into R and store it as a new object called tom
and schools
, respectively (Hint: Don’t type the path directly! Use the “tab” completion!).# Load tomatometer.csv from the 1_Data folder
XX <- read_csv(file = "XX/XX")
# Load school.csv from the 1_Data folder
XX <- read_csv(file = "XX/XX")
# Load tomatometer.csv from the 1_Data folder
tom <- read_csv(file = "1_Data/tomatometer.csv")
Parsed with column specification:
cols(
ID = col_double(),
Movie = col_character(),
State = col_character(),
Tomatometer = col_double()
)
# Load school.csv from the 1_Data folder
schools <- read_csv(file = "1_Data/schools.csv")
Parsed with column specification:
cols(
id = col_double(),
extra = col_double(),
open = col_double(),
agree = col_double(),
social = col_double(),
class = col_character(),
school = col_character()
)
# Print the object(s)
XXX
XXX
# Print the tom object
tom
# A tibble: 5,995 x 4
ID Movie State Tomatometer
<dbl> <chr> <chr> <dbl>
1 1 M1 Sober 28.3
2 1 M2 Sober 27.5
3 1 M3 Sober 11.5
4 1 M4 Sober 17.2
5 1 M5 Sober 31.0
6 1 M6 Sober 36.1
7 1 M7 Sober 31.1
8 1 M8 Sober 32.7
9 1 M9 Sober 44.1
10 1 M10 Sober 22.6
# … with 5,985 more rows
schools
# A tibble: 1,200 x 7
id extra open agree social class school
<dbl> <dbl> <dbl> <dbl> <dbl> <chr> <chr>
1 1 63.7 43.4 38.0 75.1 d IV
2 2 69.5 46.9 31.5 98.1 a VI
3 3 79.7 32.3 40.2 116. d VI
4 4 63.0 44.4 30.5 90.5 c IV
5 5 64.2 36.9 37.4 98.5 d IV
6 6 51.0 46.3 38.8 75.2 d I
7 7 60.1 37.0 38.6 95.9 d III
8 8 64.2 42.2 34.9 91.5 d IV
9 9 56.7 32.8 31.7 115. a III
10 10 47.2 44.3 25.0 123. b I
# … with 1,190 more rows
summary()
function to print more details on the columns of the datasets.summary(XXX)
summary(XXX)
summary(tom)
ID Movie State Tomatometer
Min. : 1.0 Length:5995 Length:5995 Min. : 0.0
1st Qu.: 50.5 Class :character Class :character 1st Qu.:29.1
Median :100.0 Mode :character Mode :character Median :40.6
Mean :100.5 Mean :42.5
3rd Qu.:150.5 3rd Qu.:55.4
Max. :200.0 Max. :98.1
summary(schools)
id extra open agree
Min. : 1 Min. :30.2 Min. :22.3 Min. :18.5
1st Qu.: 301 1st Qu.:54.2 1st Qu.:36.2 1st Qu.:31.9
Median : 600 Median :60.2 Median :40.0 Median :35.1
Mean : 600 Mean :60.3 Mean :40.1 Mean :35.1
3rd Qu.: 900 3rd Qu.:66.5 3rd Qu.:43.9 3rd Qu.:38.4
Max. :1200 Max. :90.8 Max. :57.9 Max. :58.4
social class school
Min. : 46.3 Length:1200 Length:1200
1st Qu.: 89.3 Class :character Class :character
Median : 99.2 Mode :character Mode :character
Mean : 99.5
3rd Qu.:109.8
Max. :152.0
View()
function to view the entire dataframe(s) in a new window.View(XXX)
View(XXX)
View(tom)
View(schools)
In the first part of this practical we will work with the tom
data example from the slides and test the effect of a person’s State
(“Sober” vs. “Drunk”) on his or her Tomatometer
rating.
Tomatometer
with State
and save the result as FE_mod
. Then inspect the results.# Use lm, as lmer only works if at least one random effect is specified
FE_mod <- glm(formula = XXX ~ XXX,
data = XXX)
# Inspect the results
summary(XXX)
# Use lm, as lmer only works if at least one random effect is specified
FE_mod <- glm(formula = Tomatometer ~ State,
data = tom)
# look at the resuts
summary(FE_mod)
Call:
glm(formula = Tomatometer ~ State, data = tom)
Deviance Residuals:
Min 1Q Median 3Q Max
-43.26 -7.61 -0.03 7.65 42.71
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 55.443 0.212 261 <2e-16 ***
StateSober -25.806 0.300 -86 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for gaussian family taken to be 135)
Null deviance: 1806949 on 5994 degrees of freedom
Residual deviance: 808883 on 5993 degrees of freedom
AIC: 46423
Number of Fisher Scoring iterations: 2
State == "Sober"
are. To obtain a more intuitive model with “Sober” as the base level, coerce the State
variable into a factor and set the levels. Do this by running the following code.# Coerce the State variable into a factor
tom <- tom %>%
mutate(State = factor(State, levels = c("Sober", "Drunk")))
# Use lm, as lmer only works if at least one random effect is specified
FE_mod <- glm(formula = Tomatometer ~ State,
data = tom)
# look at the resuts
summary(FE_mod)
Call:
glm(formula = Tomatometer ~ State, data = tom)
Deviance Residuals:
Min 1Q Median 3Q Max
-43.26 -7.61 -0.03 7.65 42.71
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 29.637 0.212 140 <2e-16 ***
StateDrunk 25.806 0.300 86 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for gaussian family taken to be 135)
Null deviance: 1806949 on 5994 degrees of freedom
Residual deviance: 808883 on 5993 degrees of freedom
AIC: 46423
Number of Fisher Scoring iterations: 2
Compare the model output of B3 with the one you obtained in B1. How did the coefficients change? Why?
Now run a model with by-subjects random intercepts (subjects identifiers are stored in the ID
variable). Hint: Random effects are specified in parenthesis in the formula in the following way (
design_matrix|grouping_variable)
. Note: The REML = FALSE
in the model specification tells R to fit the model using maximum likelihood (ML), rather than restricted maximal likelihood (REML; for more information on this in a technical approach see here and here, for a less technical approach see here). This will later be important for certain model comparisons that only work if the model was fitted using ML.
# Mixed effects model with by-subject random intercepts
subj_RI_mod <- lmer(formula = XXX ~ XXX + # These are the fixed effects
(1|XXX), # These are the random effects
data = XXX, # Specify the data used
REML = FALSE)
# Mixed effects model with by-subject random intercepts
subj_RI_mod <- lmer(formula = Tomatometer ~ State + # These are the fixed effects
(1|ID), # These are the random effects
data = tom, # Specify the data used
REML = FALSE)
summary()
, inspect the results of the mixed effects model.summary(subj_RI_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ State + (1 | ID)
Data: tom
AIC BIC logLik deviance df.resid
45279 45305 -22635 45271 5991
Scaled residuals:
Min 1Q Median 3Q Max
-3.670 -0.662 0.005 0.667 4.014
Random effects:
Groups Name Variance Std.Dev.
ID (Intercept) 31.8 5.64
Residual 103.1 10.15
Number of obs: 5995, groups: ID, 200
Fixed effects:
Estimate Std. Error t value
(Intercept) 29.634 0.440 67.3
StateDrunk 25.815 0.262 98.4
Correlation of Fixed Effects:
(Intr)
StateDrunk -0.298
summary(FE_mod)
Call:
glm(formula = Tomatometer ~ State, data = tom)
Deviance Residuals:
Min 1Q Median 3Q Max
-43.26 -7.61 -0.03 7.65 42.71
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 29.637 0.212 140 <2e-16 ***
StateDrunk 25.806 0.300 86 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for gaussian family taken to be 135)
Null deviance: 1806949 on 5994 degrees of freedom
Residual deviance: 808883 on 5993 degrees of freedom
AIC: 46423
Number of Fisher Scoring iterations: 2
State
change, now that you incorporated the random effects? Can you find out what did change from the model outputs?# While the estimates of the fixed effects did not change, the t-values did change
# substantially (the t-value for StateDrunk in FE_mod is lower than the one in
# subj_RI_mod, and the one for the intercept dropped by 50%).
subj_RI_mod
using the fixef()
function.# Print fixed effects from subj_RI_mod
fixef(XXX)
fixef(subj_RI_mod)
(Intercept) StateDrunk
29.6 25.8
subj_RI_mod
using the ranef()
function.# Print random effects from subj_RI_mod
ranef(XXX)
# Print random effects from subj_RI_mod
ranef(subj_RI_mod)
$ID
(Intercept)
1 -3.72667
2 -2.70725
3 -9.30260
4 9.34072
5 0.53670
6 -10.29137
7 -6.84484
8 3.88843
9 0.34551
10 2.71830
11 2.13535
12 8.48446
13 -10.96901
14 6.26632
15 -6.61537
16 -6.12848
17 1.48922
18 6.63510
19 -9.08709
20 -7.72281
21 2.43045
22 3.40350
23 2.74789
24 -5.33134
25 0.16750
26 2.68319
27 -4.58569
28 6.68480
29 -5.98544
30 1.80638
31 -5.51928
32 -5.85655
33 1.01406
34 -0.88831
35 -2.69176
36 2.52656
37 -2.00271
38 3.57485
39 -5.52901
40 -3.45215
41 4.54121
42 -10.03567
43 5.07659
44 -2.44412
45 -5.86155
46 11.14202
47 -1.91923
48 1.73472
49 0.25704
50 -4.65665
51 -9.76560
52 -3.26407
53 0.13204
54 -3.51976
55 -4.09580
56 4.27692
57 -2.98161
58 -1.51631
59 10.44240
60 -4.56785
61 4.67294
62 -4.90100
63 8.46020
64 6.64420
65 -4.24220
66 -1.58203
67 -4.55004
68 2.48010
69 -1.20119
70 1.00701
71 6.57440
72 -4.93223
73 -3.91590
74 3.65264
75 -1.82554
76 2.50970
77 -2.43900
78 -0.34239
79 0.96237
80 -4.56023
81 2.59965
82 -2.98027
83 -2.23420
84 -2.35557
85 -3.98495
86 6.09909
87 2.03265
88 -0.16937
89 5.94204
90 -1.14358
91 2.58765
92 0.87057
93 8.42346
94 -2.20985
95 2.06736
96 12.15186
97 7.10375
98 0.35536
99 7.60744
100 -12.25240
101 -8.94774
102 -5.53636
103 8.16086
104 2.72944
105 -3.24956
106 -2.67552
107 9.08858
108 9.66545
109 -2.83804
110 6.13601
111 6.86645
112 -5.61401
113 -2.28125
114 0.34448
115 0.41903
116 -8.49295
117 -2.87699
118 1.37610
119 5.49082
120 3.15692
121 14.25065
122 -1.08276
123 2.39225
124 -2.88023
125 0.96372
126 -9.49349
127 1.53127
128 0.55844
129 3.10027
130 6.52566
131 -1.27722
132 -3.75534
133 -3.78681
134 -12.96350
135 4.21634
136 0.00956
137 -5.24179
138 -7.55851
139 5.66366
140 -1.44616
141 -7.79828
142 0.10655
143 -7.51961
144 16.93483
145 6.49490
146 -4.23681
147 7.16867
148 -0.82098
149 -1.32887
150 0.95348
151 -4.09563
152 4.41867
153 -4.01217
154 -0.37468
155 0.77177
156 -5.56181
157 -1.84094
158 8.57867
159 -0.62826
160 0.34012
161 5.15346
162 -4.89625
163 1.04244
164 4.08565
165 -1.41477
166 -3.89184
167 3.92631
168 7.13471
169 6.76708
170 0.80036
171 0.77873
172 3.37548
173 -2.07455
174 6.98781
175 -1.37140
176 5.88747
177 3.24726
178 -2.03141
179 5.09084
180 -5.30449
181 0.17091
182 1.14726
183 -0.25429
184 4.67817
185 -1.39411
186 0.96866
187 -5.14298
188 -0.60384
189 4.03981
190 0.26823
191 -5.37581
192 -8.78411
193 -11.40835
194 6.90097
195 4.83762
196 -4.47838
197 -5.63746
198 -3.13302
199 -0.39908
200 12.51086
with conditional variances for "ID"
State
variable to the left side of the bar |
in the random effects part of the formula.# Mixed effects model with by-subject random intercepts and slopes
subj_RI_RS_mod <- lmer(formula = XXX ~ XXX + # These are the fixed effects
(XXX|XXX), # These are the random effects
data = XXX, # Specify the data used
REML = FALSE)
# Mixed effects model with by-subject random intercepts and slopes
subj_RI_RS_mod <- lmer(formula = Tomatometer ~ State + # These are the fixed effects
(State|ID), # These are the random effects
data = tom, # Specify the data used
REML = FALSE)
summary()
function to each of the objects. Did the coefficients change? Why not? What did change?summary(subj_RI_RS_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ State + (State | ID)
Data: tom
AIC BIC logLik deviance df.resid
44906 44947 -22447 44894 5989
Scaled residuals:
Min 1Q Median 3Q Max
-3.539 -0.659 0.020 0.669 3.977
Random effects:
Groups Name Variance Std.Dev. Corr
ID (Intercept) 16.6 4.07
StateDrunk 37.0 6.08 0.26
Residual 93.5 9.67
Number of obs: 5995, groups: ID, 200
Fixed effects:
Estimate Std. Error t value
(Intercept) 29.635 0.338 87.8
StateDrunk 25.818 0.498 51.9
Correlation of Fixed Effects:
(Intr)
StateDrunk 0.004
summary(subj_RI_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ State + (1 | ID)
Data: tom
AIC BIC logLik deviance df.resid
45279 45305 -22635 45271 5991
Scaled residuals:
Min 1Q Median 3Q Max
-3.670 -0.662 0.005 0.667 4.014
Random effects:
Groups Name Variance Std.Dev.
ID (Intercept) 31.8 5.64
Residual 103.1 10.15
Number of obs: 5995, groups: ID, 200
Fixed effects:
Estimate Std. Error t value
(Intercept) 29.634 0.440 67.3
StateDrunk 25.815 0.262 98.4
Correlation of Fixed Effects:
(Intr)
StateDrunk -0.298
summary(FE_mod)
Call:
glm(formula = Tomatometer ~ State, data = tom)
Deviance Residuals:
Min 1Q Median 3Q Max
-43.26 -7.61 -0.03 7.65 42.71
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 29.637 0.212 140 <2e-16 ***
StateDrunk 25.806 0.300 86 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for gaussian family taken to be 135)
Null deviance: 1806949 on 5994 degrees of freedom
Residual deviance: 808883 on 5993 degrees of freedom
AIC: 46423
Number of Fisher Scoring iterations: 2
# For FE_mod and subj_RI_mod the answer is the same as in task B5. The coefficients
# again didn't change, as the group means obviously also didn't change. But again
# the t-values changed considerably (e.g. the StateDrunk t-value in subj_RI_mod
# is 98.31, the on in subj_RI_RS_mod is 51.81).
# Mixed effects model with by-subject and by-movie random intercepts and slopes
max_mod <- lmer(formula = XXX ~ XXX + # These are the fixed effects
(XXX|XXX) + (XXX|XXX), # These are the random effects
data = XXX, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
# Mixed effects model with by-subject and by-movie random intercepts and slopes
max_mod <- lmer(formula = Tomatometer ~ State + # These are the fixed effects
(State|ID) + (State|Movie), # These are the random effects
data = tom, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
converge_ok()
function from the sjstats
package let’s you check whether the model converged or not. If the functions returns TRUE
you’re good. Test whether max_mod
converged.converge_ok(max_mod)
1.35601713944997e-06
TRUE
max_mod
is now the maximal model justified by the design. What does this mean? Why is it a good thing to specify this model?# It means you specified all the possible random effects, including their correlations.
# Barr and colleagues found that Type I error rates are inflated in case of failure to
# specify the maximal structure. The results of the next task indeed confirm this. However,
# there is a tradeoff, as we will learn later on.
summary()
.summary(max_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ State + (State | ID) + (State | Movie)
Data: tom
Control: lmerControl(optimizer = "bobyqa")
AIC BIC logLik deviance df.resid
41815 41876 -20899 41797 5986
Scaled residuals:
Min 1Q Median 3Q Max
-3.656 -0.666 0.004 0.676 4.322
Random effects:
Groups Name Variance Std.Dev. Corr
ID (Intercept) 19.4 4.40
StateDrunk 42.9 6.55 0.13
Movie (Intercept) 24.9 4.99
StateDrunk 29.9 5.47 -0.05
Residual 52.5 7.24
Number of obs: 5995, groups: ID, 200; Movie, 15
Fixed effects:
Estimate Std. Error t value
(Intercept) 29.63 1.33 22.2
StateDrunk 25.83 1.50 17.2
Correlation of Fixed Effects:
(Intr)
StateDrunk -0.042
summary(subj_RI_RS_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ State + (State | ID)
Data: tom
AIC BIC logLik deviance df.resid
44906 44947 -22447 44894 5989
Scaled residuals:
Min 1Q Median 3Q Max
-3.539 -0.659 0.020 0.669 3.977
Random effects:
Groups Name Variance Std.Dev. Corr
ID (Intercept) 16.6 4.07
StateDrunk 37.0 6.08 0.26
Residual 93.5 9.67
Number of obs: 5995, groups: ID, 200
Fixed effects:
Estimate Std. Error t value
(Intercept) 29.635 0.338 87.8
StateDrunk 25.818 0.498 51.9
Correlation of Fixed Effects:
(Intr)
StateDrunk 0.004
summary(subj_RI_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ State + (1 | ID)
Data: tom
AIC BIC logLik deviance df.resid
45279 45305 -22635 45271 5991
Scaled residuals:
Min 1Q Median 3Q Max
-3.670 -0.662 0.005 0.667 4.014
Random effects:
Groups Name Variance Std.Dev.
ID (Intercept) 31.8 5.64
Residual 103.1 10.15
Number of obs: 5995, groups: ID, 200
Fixed effects:
Estimate Std. Error t value
(Intercept) 29.634 0.440 67.3
StateDrunk 25.815 0.262 98.4
Correlation of Fixed Effects:
(Intr)
StateDrunk -0.298
summary(FE_mod)
Call:
glm(formula = Tomatometer ~ State, data = tom)
Deviance Residuals:
Min 1Q Median 3Q Max
-43.26 -7.61 -0.03 7.65 42.71
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 29.637 0.212 140 <2e-16 ***
StateDrunk 25.806 0.300 86 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for gaussian family taken to be 135)
Null deviance: 1806949 on 5994 degrees of freedom
Residual deviance: 808883 on 5993 degrees of freedom
AIC: 46423
Number of Fisher Scoring iterations: 2
# The t-value for the State effect changed dramatically. Now it is "only" 17.28.
# While still substantial, this illustrates the point Barr et al. 2013 make in their
# paper where they argue that for confirmatory hypothesis testing one should always
# specify the maximal random effects structure justified by the design, as otherwise
# the Type I error rate is inflated (i.e., the fixed effects are too often classified
# as being significant, even though they are in truth not related to the outcome
# variable).
When fitting statistical models, we are often interested in how much systematic variation they can capture. In linear (mixed effects) models this is the \(R^2\) value (the coefficient of determination); for generalized (mixed effects) models, we can compute pseudo \(R^2\) values.
r2()
function from the sjstats
package to compute \(R^2\) values. Look at the help menu of the function to get an overview of what models you can use it for.?r2
r2(max_mod)
R-Squared for (Generalized) Linear (Mixed) Model
Family : gaussian (identity)
Formula: list(~State | ID, ~State | Movie) Tomatometer ~ State NA
Marginal R2: 0.552
Conditional R2: 0.826
# The marginal r-squared only considers the variance of the fixed effects. In this
# case this means that the fixed in our maximal model can account for 55% of the variation
# in the data.
# The conditional r-squared takes both the fixed and random effects into account. Our maximal
# model can thus account for 83% of the variance in the data.
max_mod
with those of less complex models. Are the changes large? Which \(R^2\) values changed?r2(max_mod)
R-Squared for (Generalized) Linear (Mixed) Model
Family : gaussian (identity)
Formula: list(~State | ID, ~State | Movie) Tomatometer ~ State NA
Marginal R2: 0.552
Conditional R2: 0.826
r2(subj_RI_RS_mod)
R-Squared for (Generalized) Linear (Mixed) Model
Family : gaussian (identity)
Formula: ~State | ID Tomatometer ~ State NA
Marginal R2: 0.552
Conditional R2: 0.690
r2(subj_RI_mod)
R-Squared for (Generalized) Linear (Mixed) Model
Family : gaussian (identity)
Formula: ~1 | ID Tomatometer ~ State NA
Marginal R2: 0.553
Conditional R2: 0.658
It is often useful to visualize your model’s output. Run the code below to extract coefficients from max_mod
and plot the data along with a few sample lines to visualize the variability in the slopes and intercepts. As plotting is not the topic of this bootcamp we don’t have time to go into details here. If you are interested in learning more about plotting, there are many books and tutorials about it, and we also cover it in the R for Data Science bootcamp.
max_mod
data.# extract fixed effects
m_line <- fixef(max_mod)
# extract random effects
ranefs <- ranef(max_mod)
predicted <- tibble(
intercept = ranefs$ID[,1] + fixef(max_mod)[1],
slope = ranefs$ID[,2] + fixef(max_mod)[2])
# randomly draw 15 subjects to plot the fitted lines
rand15 <- sample(1:nrow(predicted), 15)
LMM_plot <- ggplot(tom, aes(State, Tomatometer)) +
geom_point(colour= "#606061", alpha = .15, size = 2.5)+
geom_segment(aes(x = 1, y = intercept, xend = 2, yend = intercept + slope),
data = predicted %>% slice(rand15), colour = "#EA4B68", size = 1.5,
alpha = .8) +
geom_segment(aes(x = 1, y = m_line[1], xend = 2, yend = sum(m_line)),
colour = "black", size = 2, alpha = 1) +
theme(axis.title.x = element_text(vjust = -1),
axis.title.y = element_text(vjust = 1)) +
theme_bw() +
ylim(0, 100) +
theme(
strip.text = element_text(size = 12, face = "bold"),
axis.text = element_text(size = 16),
axis.title = element_text(size = 18,face = "bold")
)
LMM_plot
As you probably noticed, the lmer()
summary output does not include p-values. This is not because the authors of lme4
were lazy, but because how to best compute p-values for mixed effects models is a still ongoing discussion. However, several possibilities exist of how to test whether a variable is a significant predictor (i.e., a significant fixed effect). We will look at a couple of them.
One possibility to obtain p-values is by conducting a likelihood ratio test (LRT). In an LRT, the model is fitted once with and once without the fixed effect of interest, all else being equal. These two models are then compared in using the LRT. This test only works if the models were fit using maximum likelihood (ML) estimation, rather than restricted maximum likelihood (REML) estimation, which is why we used REML = FALSE
when fitting the mixed effects models.
IO_mod
) where you predict Tomatometer
with only the grand mean, and the by-subjects and by-movies random intercepts.# Intercept only mixed effects model with by-subject and by-movie random intercepts
IO_mod <- lmer(formula = XXX ~ 1 + # There are no fixed effects so add 1 to fit the intercept
(1|XXX) + (1|XXX), # These are the random effects
data = XXX, # Specify the data used
REML = FALSE)
# Intercept only mixed effects model with by-subject and by-movie random intercepts
IO_mod <- lmer(formula = Tomatometer ~ 1 + # There are no fixed effects so add 1 to fit the intercept
(1|ID) + (1|Movie), # These are the random effects
data = tom, # Specify the data used
REML = FALSE)
summary()
.summary(IO_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: Tomatometer ~ 1 + (1 | ID) + (1 | Movie)
Data: tom
AIC BIC logLik deviance df.resid
50332 50359 -25162 50324 5991
Scaled residuals:
Min 1Q Median 3Q Max
-2.416 -0.819 -0.034 0.805 3.153
Random effects:
Groups Name Variance Std.Dev.
ID (Intercept) 27.2 5.22
Movie (Intercept) 30.4 5.51
Residual 244.2 15.63
Number of obs: 5995, groups: ID, 200; Movie, 15
Fixed effects:
Estimate Std. Error t value
(Intercept) 42.54 1.48 28.7
State
variables. Save the output as RI_mod
. Note that we again need to use another optimizer to avoid convergence problems (You don’t need to worry about this now, you can use the code template below).# Intercept only mixed effects model with by-subject and by-movie random intercepts
RI_mod <- lmer(formula = XXX ~ XXX + # The fixed effects
(1|XXX) + (1|XXX), # These are the random effects
data = XXX, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
# Intercept only mixed effects model with by-subject and by-movie random intercepts
RI_mod <- lmer(formula = Tomatometer ~ State + # The fixed effects
(1|ID) + (1|Movie), # These are the random effects
data = tom, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
RI_mod
converged using the converge_ok()
function.converge_ok(RI_mod)
9.19191550083649e-08
TRUE
anova()
function, by entering both model outputs (IO_mod
, and RI_mod
) as arguments. Is the difference significant? What does this mean?# Perform an LRT using the anova() function
anova(XXX, XXX)
# Perform an LRT using the anova() function
anova(IO_mod, RI_mod)
Data: tom
Models:
IO_mod: Tomatometer ~ 1 + (1 | ID) + (1 | Movie)
RI_mod: Tomatometer ~ State + (1 | ID) + (1 | Movie)
Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
IO_mod 4 50332 50359 -25162 50324
RI_mod 5 43214 43247 -21602 43204 7121 1 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
This method is not a formal way of testing significance, as it does not produce a p-value. In this method, confidence intervals are obtained and checked if they include zero. If they don’t, this is interpreted as a significant result.
confint()
function of the lme4
package. Do this by entering max_mod
into the confint()
function.# Compute confidence intervals for the fixed effects
ci_mod <- confint(XXX)
# Compute confidence intervals for the fixed effects
ci_mod <- confint(max_mod)
Computing profile confidence intervals ...
ci_mod
object. Do the confidence intervals you obtained match the conclusion you drew from inspecting the t-values and the LRT?p_value()
function of the sjstats
package. Enter max_mod
to the p_value()
function.# Compute p-values for the fixed effects
p_value(fit = XXX)
# Compute p-values for the fixed effects
p_value(fit = max_mod)
Computing p-values via Wald-statistics approximation (treating t as Wald z).
term p.value std.error
1 (Intercept) 0 1.33
2 StateDrunk 0 1.50
p_value()
function with an additional argument, p.kr = TRUE
. Then the function will compute p-values based on conditional F-tests with Kenward-Roger approximation for the degrees of freedom. Repeat the previous task by extending the function with p.kr = TRUE
.# Compute p-values for the fixed effects with Kenward-Roger approximation
p_value(fit = max_mod, p.kr = TRUE)
Computing p-values via Kenward-Roger approximation. Use `p.kr = FALSE` if computation takes too long.
term p.value std.error
1 (Intercept) 0 1.37
2 StateDrunk 0 1.54
You may remember the keep it maximal principle, which states that you should always specify the maximal random effects structure justified by the design to guard against type I error inflation (Barr, Levy, Scheepers, & Tily, 2013). However, the maximal model may come with a loss of power to detect fixed effects and it may make sense to select which random effects structure to use (Matuschek, Kliegl, Vasishth, Baayen, & Bates, 2017). To express it with the words of Matuschek and Colleagues: “[A] parsimonious mixed model […] containing only variance components supported by the data improves the balance between Type I error and power” (p. 305). To select which model to use, we will use LRTs.
When performing this LRT, we cannot use the usual alpha limit of .05. To see why, read the following quote from Matuschek and colleagues (2017; p. 308)
Within the context of model selection, it is important to resist the reflex of choosing \(\alpha_{LRT} = 0.05\). The \(\alpha_{LRT}\) cannot be interpreted as the ‘‘expected model-selection Type I error-rate” but rather as the relative weight of model complexity and goodness-of-fit. For example, choosing \(\alpha_{LRT} = 0\), an infinite penalty on the model complexity is implied and consequently the minimal model is always chosen as the best, irrespective of the evidence provided by the data. Choosing \(\alpha_{LRT} = 1\) implies an infinite penalty on the goodness-of-fit, and the maximal model is always chosen as the best. Therefore, choosing \(\alpha_{LRT} = 0.05\) may imply an overly strong penalty on the model complexity and hence select a reduced model even if data favor a more complex one.
We will follow their example and use \(\alpha_{LRT} = 0.2\). As the anova()
function uses \(\alpha = 0.05\) we will have to implement this procedure ourselves.
# This is just a placeholder to increase the space to the next task to make it
# easier to not cheat.
# There is nothing to see here...
# Just a random sidenote (which is actually very interesting so you may want to
# to read on, even if this kind of defies the purpose of this part to prevent you from
# looking at the answers first; so consider first completing the task and then reading
# the random but somehow terribly interesting sidenote):
# Did you know that the reason why R uses the arrow "<-" as assignment
# operator is because it is based on S, which in turn is based on APL?
# Now apparently, APL was designed on a specific keyboard that had a
# "<-" key and there was no "==" implemented to test equality. So equality
# was tested using "=" and "<-" was chosen as assignment operator
# (info obtained from this blogpost: https://colinfay.me/r-assignment/).
||
in the random effects structure, that is (design_matrix||grouping_variable)
to do this. Fit the maximal model but constrain the correlations to zero.# Constrained mixed effects model with by-subject and by-movie random intercepts and slopes
con_mod <- lmer(formula = XXX ~ XXX + # These are the fixed effects
(XXX||XXX) + (XXX||XXX), # These are the random effects
data = XXX, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
# Constrained mixed effects model with by-subject and by-movie random intercepts and slopes
con_mod <- lmer(formula = Tomatometer ~ State + # These are the fixed effects
(State||ID) + (State||Movie), # These are the random effects
data = tom, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
converge_ok()
.converge_ok(con_mod)
0.00528726235046066
FALSE
This model didn’t converge. This can happen and it’s not always easy to find out how to get the model to converge. One way would be to use different Bayesian methods with the rstanarm
package. This is rather advanced and we will ignore it for now, but should this happen to you in your work you may not want to interpret this model.
Now let’s prepare the setup for the LRT. The difference in the deviances of two models (defined as negative two times the log-likelihood) approximately follows a \(\chi^2\) distribution. Therefore we can test the significance by testing whether the \(\chi^2\) value is larger than a threshold value. To do this we set up our \(\alpha_{LRT}\) and derive the threshold value. Do this by running the following code.
alpha <- .2 # set up the alpha value
st <- qchisq(1 - alpha, df = 2) # derive the critical value above which the difference
# is significant. We have two degrees of freedom because
# the constrained model has two more degrees of freedom
deviance()
function from the stats
package and save the difference as d_diff
.d_diff <- deviance(XXX) - deviance(XXX)
d_diff <- deviance(con_mod) - deviance(max_mod)
d_diff
is larger than the significance threshold st
.# Test whether the difference in the deviance between the two models is large enough
# to be considered significant
d_diff > st
[1] FALSE
FALSE
. What does that mean? Do you keep the maximal model in this case? What would it mean if the result were TRUE
, how would your next steps look?# FALSE as output in this case means that the difference between the models was
# not large enough that we can consider the maximal model to fit the data better.
re_var()
function from the sjstats
package.re_var(max_mod)
Within-group-variance: 52.474
Between-group-variance: 19.393 (ID)
Between-group-variance: 24.874 (Movie)
Random-slope-variance: 42.903 (ID.StateDrunk)
Random-slope-variance: 29.893 (Movie.StateDrunk)
Slope-Intercept-covariance: 3.710 (ID.(Intercept))
Slope-Intercept-covariance: -1.270 (Movie.(Intercept))
Slope-Intercept-correlation: 0.129 (ID)
Slope-Intercept-correlation: -0.047 (Movie)
Another test of the significance of fixed effects, and the one you should prefer in an uncertain case (i.e., if p-values are on the border of your alpha-level), is to use parametric bootstrap, where, many times, data is resampled with replacement and then the model is fitted again, to obtain an empirical distribution of the effect or parameter of interest. The drawback of this technique is that it can take rather long.
lmer()
) you can use the PBmodcomp()
function from the pbkrtest
package. Check out the help function of PBmodcomp()
like this.?PBmodcomp
PBmodcomp()
function to get p-values for the model. You will have to specify a less complex model against which to test. Use IO_mod
for this. Also, because this procedure can take rather long, only use 100 simulations.# perform parametric bootstrap
pb_mod <- PBmodcomp(largeModel = XXX,
smallModel = XXX,
nsim = XXX)
# perform parametric bootstrap
pb_mod <- PBmodcomp(largeModel = max_mod,
smallModel = IO_mod,
nsim = 100)
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
pb_mod
object and inspect the results. Are the results different from the other tests you’ve performed before? How does the p-value from the parametric bootstrap test compare to the one obtained from the likelihood test?The intra-class correlation (ICC) is “the proportion of the variance explained by the grouping structure in the population” (Hox 2002, p.15). It is calculated by dividing the between-group variance by the sum of the within-group and the between-group variance (i.e., the total variance).
icc()
function from the sjstats
package.icc(max_mod)
Caution! ICC for random-slope-intercept models usually not meaningful. Use `adjusted = TRUE` to use the mean random effect variance to calculate the ICC. See 'Note' in `?icc`.
Intraclass Correlation Coefficient for Linear mixed model
Family : gaussian (identity)
Formula: Tomatometer ~ State + (State | ID) + (State | Movie)
ICC (ID): 0.2005
ICC (Movie): 0.2571
adjusted = TRUE
to your ICC call to take care of the warning.icc(max_mod, adjusted = TRUE)
Intraclass Correlation Coefficient for Generalized Linear Mixed Model
Family : gaussian (identity)
Formula: list(~State | ID, ~State | Movie) Tomatometer ~ State NA
Adjusted ICC: 0.6130
Conditional ICC: 0.2748
Check the icc()
help page to understand what happened now that you added the adjusted = TRUE
argument.
Often, the ICC is computed on the intercept only model with random intercepts. In task C1. you have already fitted this model and saved it as IO_mod
. Compute the ICC form this model. Hint: no need to use the adjusted
argument, as no random slopes are involved.
icc(IO_mod)
Intraclass Correlation Coefficient for Linear mixed model
Family : gaussian (identity)
Formula: Tomatometer ~ 1 + (1 | ID) + (1 | Movie)
ICC (ID): 0.0902
ICC (Movie): 0.1006
RI_mod
, that differs from the IO_mod
you’ve just used only in that it also contains the fixed effect.icc(RI_mod)
Intraclass Correlation Coefficient for Linear mixed model
Family : gaussian (identity)
Formula: Tomatometer ~ State + (1 | ID) + (1 | Movie)
ICC (ID): 0.2448
ICC (Movie): 0.2289
So far we only considered crossed random effects. Now we will also look at data with nested random effects, where every level of a nested factor only appears within a single level of a higher order factor. A popular example is the case of classes within schools. Every class is only part of a single school and thus class is nested within schools.
To know whether random effects are crossed or nested, you usually need to know the design, as the structure is often not obvious from the factor levels. For example, even though class may be nested in schools, the classes may be numbered from 1 to the number of classes in that school. So school 1 may have classes 1 to 10, and school 2 may have classes from 1 to 6. Obviously class 1 of school 1 is not the same as class 1 of school 2. However, what is obvious for you is not obvious for R. It doesn’t know about the concept of schools and classes, so you have to tell it whether these factors are nested or not, using the appropriate syntax in the formula. Now consider the case where the classes are nested in schools, but numbered from 1 to total number of classes, that is, such that every class has a unique label. In this case it does not matter what structure you specify.
school
data set you have already loaded in section A. Using the table()
function, create a cross table of school and class. (Hint: You’ll have to enter the variables separately.)table(XXX, XXX)
table(schools$school, schools$class)
a b c d
I 50 50 50 50
II 50 50 50 50
III 50 50 50 50
IV 50 50 50 50
V 50 50 50 50
VI 50 50 50 50
extra
) with openness (open
) and the social score (social
) with random intercepts for classes nested in school and save it as hier_mod
. Hint: The nested random effects structure is specified with the following syntax (1|higher_level_variable/lower_level_variable)
.# Hierarchical mixed effects model
hier_mod <- lmer(formula = extra ~ open + social + # These are the fixed effects
(1|school/class), # These are the nested random effects
data = schools, # Specify the data used
REML = FALSE)
converge_ok()
converge_ok(hier_mod)
0.00147383412711221
FALSE
summary()
.summary(hier_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: extra ~ open + social + (1 | school/class)
Data: schools
AIC BIC logLik deviance df.resid
3545 3575 -1766 3533 1194
Scaled residuals:
Min 1Q Median 3Q Max
-10.015 -0.334 0.015 0.341 10.580
Random effects:
Groups Name Variance Std.Dev.
class:school (Intercept) 8.186 2.861
school (Intercept) 77.838 8.823
Residual 0.967 0.984
Number of obs: 1200, groups: class:school, 24; school, 6
Fixed effects:
Estimate Std. Error t value
(Intercept) 6.00e+01 3.66e+00 16.39
open 6.03e-03 4.96e-03 1.22
social 5.18e-04 1.85e-03 0.28
Correlation of Fixed Effects:
(Intr) open
open -0.054
social -0.050 -0.006
cross_mod
.# Hierarchical mixed effects model
cross_mod <- lmer(formula = extra ~ open + agree + # These are the fixed effects
(1|school) + (1|class), # These are the crossed random effects
data = schools, # Specify the data used
REML = FALSE)
converge_ok(cross_mod)
6.91482090087394e-06
TRUE
summary()
function.summary(cross_mod)
Linear mixed model fit by maximum likelihood ['lmerMod']
Formula: extra ~ open + agree + (1 | school) + (1 | class)
Data: schools
AIC BIC logLik deviance df.resid
4716 4746 -2352 4704 1194
Scaled residuals:
Min 1Q Median 3Q Max
-7.898 -0.544 0.012 0.531 8.237
Random effects:
Groups Name Variance Std.Dev.
school (Intercept) 81.17 9.01
class (Intercept) 5.61 2.37
Residual 2.78 1.67
Number of obs: 1200, groups: school, 6; class, 4
Fixed effects:
Estimate Std. Error t value
(Intercept) 60.02428 3.89319 15.42
open 0.01084 0.00834 1.30
agree -0.00545 0.00959 -0.57
Correlation of Fixed Effects:
(Intr) open
open -0.085
agree -0.086 -0.008
# There are changes in t-values, and estimates of the fixed effects. Furthermore,
# the variance estimates of the random effects changed, as well as the goodness of fit,
# that is the model with the crossed random effects structure fits way worse.
# Constrained model with by-subjects random intercepts and slopes, without their
# correlation, and with by-movie random intercepts
con_mod2 <- lmer(formula = Tomatometer ~ State + # These are the fixed effects
(State||ID) + (1|Movie), # These are the random effects
data = tom, # Specify the data used
REML = FALSE,
control = lmerControl(optimizer = "bobyqa")) # use a different optimizer
# to avoid non convergence
df
argument in the qchisq()
function to df = 1
because these two models only differ in one degree of freedom.# alpha is still the same
alpha <- .2 # set up the alpha value
st1 <- qchisq(1 - alpha, df = 1) # derive the critical value above which the difference
# is significant. This time we have only one degree of
# freedom because the more complex model only has one
# parameter more than the the simpler model
# get the difference in deviance between the two models
d_diff <- deviance(con_mod2) - deviance(con_mod)
# Test whether this difference is large enough to be considered significant
d_diff > st1
[1] TRUE
# con_mod, with correlations constrained to zero would be selected as the maximal model
# does not perform siginificantly better, and the less complex models perform significantly
# worse. As this model did not converge and the maximal model did, it may in this case still
# make sense to go with the maximal model.
cancer_remission.csv
data into R and save it as cr
.cr <- read_csv(file = "1_Data/cancer_remission.csv")
Parsed with column specification:
cols(
remission = col_double(),
CancerStage = col_character(),
LengthofStay = col_double(),
Experience = col_double(),
DID = col_double()
)
cr
summary(cr)
View(cr)
remission
) by the cancer stage (CancerStage
), the length of the patients stay (LengthofStay
), the doctors experience (Experience
), and by specifying by doctor (DID
) random intercepts. Use the glmer
function with family = binomial
to do this. Also, use the “bobyqa” optimizer as in some examples before.cr_mod <- glmer(formula = XXX ~ XXX + XXX + XXX +
(1 | XXX),
data = XXX,
family = binomial, # with this you tell glmer to run a logistic regression
control = glmerControl(optimizer = "bobyqa"))
cr_mod <- glmer(formula = remission ~ CancerStage + LengthofStay + Experience +
(1 | DID),
data = cr,
family = binomial, # with this you tell glmer to run a logistic regression
control = glmerControl(optimizer = "bobyqa"))
converge_ok(cr_mod)
2.42537482394112e-06
TRUE
summary(cr_mod)
Generalized linear mixed model fit by maximum likelihood (Laplace
Approximation) [glmerMod]
Family: binomial ( logit )
Formula: remission ~ CancerStage + LengthofStay + Experience + (1 | DID)
Data: cr
Control: glmerControl(optimizer = "bobyqa")
AIC BIC logLik deviance df.resid
7436 7485 -3711 7422 8518
Scaled residuals:
Min 1Q Median 3Q Max
-3.589 -0.447 -0.203 0.406 6.963
Random effects:
Groups Name Variance Std.Dev.
DID (Intercept) 3.87 1.97
Number of obs: 8525, groups: DID, 407
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.3738 0.5094 -4.66 3.2e-06 ***
CancerStageII -0.4126 0.0737 -5.60 2.2e-08 ***
CancerStageIII -0.9991 0.0958 -10.43 < 2e-16 ***
CancerStageIV -2.3143 0.1550 -14.93 < 2e-16 ***
LengthofStay -0.1207 0.0328 -3.68 0.00023 ***
Experience 0.1196 0.0265 4.50 6.7e-06 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Correlation of Fixed Effects:
(Intr) CncSII CnSIII CncSIV LngthS
CancerStgII 0.015
CancrStgIII 0.059 0.493
CancerStgIV 0.070 0.334 0.318
LengthofSty -0.303 -0.265 -0.337 -0.290
Experience -0.923 -0.004 -0.008 -0.013 -0.011
r2()
and once using the cod()
function. Are the results the same? Find out what the differences are by checking the help pages.r2(cr_mod)
R-Squared for (Generalized) Linear (Mixed) Model
Family : binomial (logit)
Formula: ~1 | DID remission ~ CancerStage + LengthofStay + Experience NA
Marginal R2: 0.097
Conditional R2: 0.585
cod(cr_mod)
R-Squared for (Generalized) Linear (Mixed) Model
Tjur's D: 0.409
Note: Running and interpreting generalized linear mixed effects models can be quite challenging and we don’t have the time here to cover them in more detail. You can check out this tutorial or the references listed under resources if you want to learn more about this.
# load packages
library(lme4)
library(sjstats)
# look at sleepstudy data contained in lme4
head(sleepstudy)
# look at describtion of sleepstudy
?sleepstudy
# fixed effects only model predicting the reaction time
# from the number of days with sleep deprivation
sleep_FE <- glm(formula = Reaction ~ Days, data = sleepstudy)
# inspect model
summary(sleep_FE)
# intercept only model mixed effects model
sleep_IO <- lmer(formula = Reaction ~ 1 + (1|Subject),
data = sleepstudy,
REML = FALSE)
# inspect model
summary(sleep_IO)
# run random interceptes model
sleep_RI <- lmer(formula = Reaction ~ Days + (1|Subject),
data = sleepstudy,
REML = FALSE)
# inspect output
summary(sleep_RI)
# check whether model converged
converge_ok(sleep_RI)
# get p_values for fixed effects using Kenward-Roger approximation
p_value(sleep_RI, p.kr = TRUE)
# alternatively, get p-value using anova() function
anova(sleep_IO, sleep_RI)
# run mixed effects model without correlations between random effects parameters
sleep_nc <- lmer(formula = Reaction ~ Days + (Days||Subject),
data = sleepstudy,
REML = FALSE)
# check whether the model converged
converge_ok(sleep_nc)
# check model output
summary(sleep_nc)
# run maximal model (by-subjects random slopes and intercepts)
sleep_max <- lmer(formula = Reaction ~ Days + (Days|Subject),
data = sleepstudy,
REML = FALSE)
# check whether the model converged
converge_ok(sleep_max)
# check model output
summary(sleep_max)
### select the random effects structure
# set alpha to .2
alpha <- .2
# derive the critical value above which the difference is significant.
st1 <- qchisq(1 - alpha, df = 1)
# get the difference in deviance between the two models
d_diff <- deviance(sleep_nc) - deviance(sleep_max)
# Test whether this difference is large enough to be considered significant
d_diff > st1
# correlation is not necessary. Now test whether the random slopes are necessary
d_diff <- deviance(sleep_RI) - deviance(sleep_nc)
# Test whether this difference is large enough to be considered significant
d_diff > st1 # -> keep sleep_nc model
# compute r-squared
r2(sleep_nc)
# compute icc
icc(sleep_nc, adjusted = TRUE)
### Visualize Model
# extract fixed effects
m_line <- fixef(sleep_nc)
# extract random effects
ranefs <- ranef(sleep_nc)
predicted <- tibble(
intercept = ranefs$Subject[,1] + fixef(sleep_nc)[1],
slope = ranefs$Subject[,2] + fixef(sleep_nc)[2])
# randomly draw 10 subjects to plot the fitted lines
rand10 <- sample(1:nrow(predicted), 10)
LMM_plot <- ggplot(sleepstudy, aes(Days, Reaction)) +
geom_point(colour= "#606061", alpha = .15, size = 2.5)+
geom_segment(aes(x = 0, y = intercept, xend = 9, yend = intercept + slope * 9),
data = predicted %>% slice(rand10), colour = "#EA4B68", size = 1.5,
alpha = .8) +
geom_segment(aes(x = 0, y = m_line[1], xend = 9, yend = m_line[1] + 9 * m_line[2]),
colour = "black", size = 2, alpha = 1) +
theme(axis.title.x = element_text(vjust = -1),
axis.title.y = element_text(vjust = 1)) +
theme_bw() +
theme(
strip.text = element_text(size = 12, face = "bold"),
axis.text = element_text(size = 16),
axis.title = element_text(size = 18,face = "bold")
)
LMM_plot
| File | Rows | Columns |
|---|---|---|
| tomatometer.csv | 5995 | 4 |
| schools.csv | 1200 | 7 |
| cancer_remission.csv | 8525 | 5 |
The tomatometer.csv dataset contains tomatometer ratings from 200 raters who each rated 15 movies, once while they were sober and once while they were drunk.
The schools.csv dataset contains ratings of extraversion, openness to experience, agreeableness, and a social score of 1200 students from different classes of 6 different schools.
The cancer_remission.csv dataset contains data on lung cancer remission from patients nested within doctors
| Name | Description |
|---|---|
| ID | Participant id code |
| Movie | Movie ID from M1 to M15 |
| State | the state in which the rating was done (“sober”, or “drunk”) |
| Tomatometer | tomatometer rating from 0 to 100 |
| Name | Description |
|---|---|
| id | pupil id |
| extra | extraversion rating of pupil from 0 to 100 |
| open | openness to experience rating of pupil from 0 to 100 |
| agree | agreeableness rating of pupil from 0 to 100 |
| social | social score of pupil |
| class | the class a pupil is in (nested under school; levels “a”, “b”, “c”, and “d”) |
| school | the school a pupil / class is in (levels “I”, “II”, “III”, “IV”) |
| Name | Description |
|---|---|
| remission | whether the cancer is in remission (0 = No, 1 = Yes) |
| CancerStage | four different cancer stages (levels “I”, “II”, “III”, “IV”) |
| LengthofStay | how long a participant stayed in hospital (score ranging from 1 to 10) |
| Experience | experience of the doctor (ranging from 7 to 29, probably years of experience) |
| DID | doctor id |
| Package | Installation |
|---|---|
tidyverse |
install.packages("tidyverse") |
lme4 |
install.packages("lme4") |
sjstats |
install.packages("sjstats") |
pbkrtest |
install.packages("pbkrtest") |
| Function | Package | Description |
|---|---|---|
lmer |
lme4 |
Fit a linear mixed effects model |
glmer |
lme4 |
Fit a generalized linear mixed effects model |
fixef |
lme4 |
Extract fixed effects coefficients from lmer or glmer output |
ranef |
lme4 |
Extract random effects coefficients from lmer or glmer output |
anova |
stats |
Generic function to run (in this case) a likelihood ratio test |
confint |
stats |
Compute confidence intervals for various statistical outputs |
deviance |
stats |
Extract the deviance of various statistical outputs |
qchisq |
stats |
Quantile function of the \(\chi^2\) distribution to get critical \(\chi^2\) values |
PBmodcomp |
pbkrtest |
Perform parametric bootstrap to obtain p-values from linear mixed effects models |
converge_ok |
sjstats |
Test whether a model converged or not (some warnings produced by lmer and glmer may be informative but not necessarily mean you need to worry, so this function is a good proxy to use. If it yields FALSE you may want to investigate things and try to take measures to improve your model) |
p_value |
sjstats |
Compute p-values for fixed effects of mixed effects models using Wald’s test or conditional F-tests with Kenward-Roger approximation for the degrees of freedom |
r2 |
sjstats |
Extract \(R^2\) value from (generalized) linear (mixed effects) model outputs |
icc |
sjstats |
Compute ICCs from (generalized) linear mixed effects model outputs |
re_var |
sjstats |
Extract random effects and residual variances from (generalized) linear mixed effects model outputs |
A nice and non-technical introduction to mixed effects models can be found in a chapter titled An introduction to linear mixed modeling in experimental psychology by Henrik Singmann and David Kellen.
A more technical introduction can be found in the mixed effects models chapter of Analysis of Longitudinal and Cluster-Correlated Data by Nan Laird.
A freely available tutorial on mixed effects models can be found here.
An introduction to \(R^2\) and ICC in mixed effects model can be found in the article The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded by Shinichi Nakagawa, Paul Johnson, and Holger Schielzeth.
A rather technical introduction to mixed effects model with lme4
is given in Fitting Linear Mixed-Effects Models Using lme4 by Douglas Bates, Martin Mächler, Benjamin Bolker, and Steven Walker.
One of the classic introductions to regression models, including mixed effects models, is Data Analysis Using Regression and Multilevel/Hierarchical Models by Andrew Gelman and Jennifer Hill.
Another great introduction to regression, including mixed effects models, is Statistical Rethinking by Richard McElreath.


--- layout: default title: Page not found description: "Page not found" permalink: 404.html sitemap: exclude: "yes" ---
Sorry, the page you are looking for does not exist or it has been moved.
If you are sure that it actually exists you can try to contact me


Variable
| Static Public Summary | ||
| public |
BOOLEAN: {"type": *, "toBe": *} This object combines all validation functions related to booleans |
|
| public |
NUMBER: {"min": *, "max": *, "equal": *, "type": *} This object combines all validation functions related to numbers |
|
| public |
STRING: {"minLength": *, "maxLength": *, "equal": *, "match": *, "notEmpty": *, "type": *} This object combines all validation functions related to strings |
|
| public |
TEST_FUNCTIONS: {...types: Object} |
|
| public |
Regex that validates if a string is a valid url |
|
| public |
array: {"of": *, "notEmpty": *, "length": *, "type": *} |
|
| public |
|
|
| public |
date: {"after": *, "before": *, "between": *, "equal": *, "type": *} |
|
| public |
|
|
| public |
|
|
| public |
fn: {"type": *, "result": *} |
|
| public |
|
|
| public |
|
|
| public |
password: {...STRING: Object, "numbers": *, "uppercase": *, "specialChars": *, "matchesOneOf": *, "matchesAllOf": *} |
|
| public |
|
|
| public |
|
Static Public
public BOOLEAN: {"type": *, "toBe": *} source
import {BOOLEAN} from<EMAIL_ADDRESS>This object combines all validation functions related to booleans
public NUMBER: {"min": *, "max": *, "equal": *, "type": *} source
import {NUMBER} from<EMAIL_ADDRESS>This object combines all validation functions related to numbers
public STRING: {"minLength": *, "maxLength": *, "equal": *, "match": *, "notEmpty": *, "type": *} source
import {STRING} from<EMAIL_ADDRESS>This object combines all validation functions related to strings
public TEST_FUNCTIONS: {...types: Object} source
import {TEST_FUNCTIONS} from<EMAIL_ADDRESS>public URL_REGEX: undefined source
import {URL_REGEX} from<EMAIL_ADDRESS>Regex that validates if a string is a valid url
public array: {"of": *, "notEmpty": *, "length": *, "type": *} source
import array from<EMAIL_ADDRESS>public boolean: {...BOOLEAN: Object} source
import boolean from<EMAIL_ADDRESS>public date: {"after": *, "before": *, "between": *, "equal": *, "type": *} source
import date from<EMAIL_ADDRESS>public email: {...STRING: Object, "type": *, "user": *, "domain": *} source
import email from<EMAIL_ADDRESS>public float: {...NUMBER: Object, "type": *} source
import float from<EMAIL_ADDRESS>public fn: {"type": *, "result": *} source
import fn from<EMAIL_ADDRESS>public int: {...NUMBER: Object, "type": *} source
import int from<EMAIL_ADDRESS>public number: {...NUMBER: Object} source
import number from<EMAIL_ADDRESS>public password: {...STRING: Object, "numbers": *, "uppercase": *, "specialChars": *, "matchesOneOf": *, "matchesAllOf": *} source
import password from<EMAIL_ADDRESS>public string: {...STRING: Object} source
import string from<EMAIL_ADDRESS>public url: {...STRING: Object, "type": *, "domain": *, "protocol": *} source
import url from '@cesium133/forgjs/src/testFunctions/types/url.js'


; ; ------------------------------------------------------------ ; ; PureBasic - 2D Drawing example file ; ; (c) 2005 - Fantaisie Software ; ; ------------------------------------------------------------ ; If OpenWindow(0, 100, 200, 300, 200, "2D Drawing Test") ; Create an offscreen image, with a green circle in it. ; It will be displayed later ; If CreateImage(0, 300, 200) If StartDrawing(ImageOutput(0)) Circle(100,100,50,RGB(0,0,255)) ; a nice blue circle... Box(150,20,20,20, RGB(0,255,0)) ; and a green box FrontColor(RGB(255,0,0)) ; Finally, red lines.. For k=0 To 20 LineXY(10,10+k*8,200, 0) Next DrawingMode(#PB_2DDrawing_Transparent) BackColor(RGB(0,155,155)) ; Change the text back and front colour FrontColor(RGB(255,255,255)) DrawText(10,50,"Hello, this is a test") StopDrawing() EndIf EndIf ; Create a gadget to display our nice image ; ImageGadget(0, 0, 0, 0, 0, ImageID(0)) ; ; This is the 'event loop'. All the user actions are processed here. ; It's very easy to understand: when an action occurs, the Event ; isn't 0 and we just have to see what have happened... ; Repeat Event = WaitWindowEvent() Until Event = #PB_Event_CloseWindow ; If the user has pressed on the window close button EndIf End ; All the opened windows are closed automatically by PureBasic


Christina Moulton, Teak Mobile Inc.
Session 1: Design Patterns & Obj-C Features, Apple's Tools, SDK & Core Classes
Exercise 1: Interface Builder, IBOutlets & IBActions
Lunch
Session 2: iOS App Ecosystem, 3rd Party Tools & Testing, Key Resources
Exercise 2: JSON REST APIs, Storyboards & Tableviews
Session 3: Code Signing, App Store Submission, What Should Be a Native App vs. Web App
Code Signing: Local & Beta Testing
App Store Submission
What Should Be a Native App vs. Web App
Over USB
Sign up for iOS Developer Program ($99/yr)
Create & install iOS Development certificate
Add device to Provisioning Portal
Create & install Development Provisioning Profile
Build & run with Development Provisioning Profile
Create App ID (if using Push, GameKit, ...)
Create & install Ad Hoc Distribution Provisioning Profile
Build & archive with Ad Hoc Distribution Provisioning Profile
Share from Xcode Organizer (e.g., TestFlight)
They either drag the .ipa in to iTunes or install over-the-air
Create App ID (if using Push, GameKit, ...)
Create & install App Store Distribution Provisioning Profile
Remove debug code (NSLogs are slow, check analytics IDs)
Build & archive with App Store Distribution Provisioning Profile
Create app (or update) in iTunesConnect
Fill in iTunesConnect data
Set iTunesConnect app to "Ready to Upload"
Verify & submit from Xcode Organizer
Wait (usually 5-8 business days)
Speed, memory
Easier to create native-feeling UI
Immediate access to new features
Persistence & offline operation
Easier immersive or touch-drive UIs (reduced lag)
Easier security
Reuse existing pages
Multi-platform ("write once, test everywhere")
Can extend with native wrapper
Also available within browser
Webviews are often buggy or lag other browsers
Immediate app updates
Standards fragmentation
Deep, rich touch-driven UX
Intense in-memory or processing requirements
Users are often offline
Want to use iOS native components
Bleeding edge or deeply detailed device features (e.g., camera flash control)
App Store marketing channel
Existing web-based assets
Need multi-platform, fast
At most a few non-core native features (e.g., camera, location)
Rapid prototyping
Skills availability
Control over distribution & updates
Target new platforms: Cars, etc.
Christina Moulton, Teak Mobile Inc.


HOME
MAIN
RACE
ABOUT
Hakka Cuisine
Cantonese Cuisine
MENU
LUNCH
DINNER
GALLERY
Images
Video
HONESTY POLICY
FAQs
CONTACT US
CONTACT US
FEEDBACK
Foodie Racer
Results
Chef Henry
Chef Suzy


I felt the blog was getting clogged with dogs, so here are some cogs (for balance). It also gives way to the opportunity for an ill-fitting brain-cog image on a blog post about EQ - enjoy :)
Emotional Intelligence or EQ (emotional quotient) is about the ability to identify and manage your own emotions, as well as those of others. To have high emotional intelligence, you need to be self-aware and from that have the ability to self-regulate, while also needing social skills and empathy to interact with others effectively.
While IQ assesses your academic ability, your EQ is about being aware of your emotions and how you control and express them. The other major part of EQ is your ability to empathise and connect with other people, which affects how good a leader or team player you can be. This is why EQ is considered as - if not more - important as IQ in a workplace.
While someone with a high IQ might be able to complete a task well on their own, they will be incredibly difficult to work with if they don't also have good social skills. Unless you can understand yourself and others and communicate that well, it's hard to work well in a team environment. Furthermore, the soft skills of a high EQ become more and more in demand as you move into leadership positions later in your career. You can't be a good leader if you don't understand the needs of the people you are leading, or aren't approachable and trustworthy.


Counting Children
Using
eval("")
in CSS to Count Elements
A minimum order quantity of five (5) items is required to check out.


| Created: | 4/29/2013 12:00:00 AM |
| Modified: | 4/30/2013 4:14:35 PM |
| Project: | |
| Advanced: |
| Attribute | ||
|
Public Identifier. Type UBLVersionID |
||
Details:
|
||
|
Public Identifier. Type CustomizationID |
||
Details:
|
||
|
Public Identifier. Type ProfileID |
||
Details:
|
||
|
Public Identifier. Type ProfileExecutionID |
||
Details:
|
||
|
Public Identifier. Type ID |
||
Details:
|
||
|
Public Indicator. Type CopyIndicator |
||
Details:
|
||
|
Public Identifier. Type UUID |
||
Details:
|
||
|
Public Date. Type IssueDate |
||
Details:
|
||
|
Public Time. Type IssueTime |
||
Details:
|
||
|
Public Text. Type Note |
||
Details:
|
| Element | Source Role | Target Role |
|
«ABIE» Customer Party Class |
Name: |
Name: Retailer Customer Party |
|
Details:
The retailer.
|
||
|
«ABIE» Instruction For Returns Line Class |
Name: |
Name: |
|
Details:
A line providing details about one type of article to be returned.
|
||
|
«ABIE» Document Reference Class |
Name: |
Name: |
|
Details:
A reference to another document associated with this document.
|
||
|
«ABIE» Party Class |
Name: |
Name: Manufacturer Party |
|
Details:
The manufacturer.
|
||
|
«ABIE» Shipment Class |
Name: |
Name: |
|
Details:
The shipment.
|
||
|
«ABIE» Signature Class |
Name: |
Name: |
|
Details:
A signature applied to this document.
|
||
|
«ABIE» Supplier Party Class |
Name: |
Name: Seller Supplier Party |
|
Details:
The seller.
|


Bonjour! i'm blessed dharly
Be that independent girl, Be that strong girl that everyone knew would make it through the worst,be that girl who never backed down.
#Girls can do anything
About me
I am a young enterprenuer and software developper, business and technology has been always my interest. as the world moving so fast i really want to do my party by solving problems using E-commerce in technology in business by buying and selling of goods and services over the Internet. and aparty from that i like fashion and beauty and i enjoyed music in my free time


This is an experiment to create printable certificates using only client-side technologies. Personally, I get writers cramp after a few hours of filling out advancement cards for my Pack. Since it is B&G season, It's time to get this out for everyone to try.
This webapp is compatible with a Purchase Order CSV export from Scoutbook. Open it in your favorite spreadsheet program, add "Pack Number", "Cubmaster" and "Den Leader" columns, and you should be good to go. If you don't, you"ll get lines you can sign by hand. Extra columns will be ignored.
We'll call this an early alpha release. It's very rough around the edges, but it might be of use to you, so we're releasing it now. If you'd like to help improve this, all code is on GitHub. Please fork away to your heart's content.
Note: Since this is an early-stage project, it WILL NOT work in anything but the latest browsers. Google Chrome is my development browser, so things should work the best there. IE11 is also pretty good. Firefox squashes the award images.
Good luck, and I hope this helps you!


Welcome PolyXpress Administrator!
Home
Logout
User Applications
Administration
PolyXpress User Application
Quick access to PolyXpress tools.
Mobile Designer
Player
Point Finder
Administration
Status
/status
Provide status of web application server.
Data Management
Data Management
Perform CRUD operations on PolyXpress Users, Stories, Chapters, and Pages.


As of 05/15/2014 the MFK is:
Fund
.
Org
.
Dept
.
SubDept
.
Grant
.
Program
.
I/A
.
O/A
.
DeptAcct
.
FN
.
CCTR
.
Comment
.
Grant Program Numbers for this Protocol
Grant Program Numbers for selected Protocol
Sequence #
GPN
B083200-G
1125200
F472400-G
1245400


@package Gaclib.API
@{
SelectNavigationButton(navDemo);
}
Source code of all demos
Source code of all demos are in the "Tutorial" folder of the GacUI for Windows repo:
-
Hello, world! : Every possible ways to create a hello world GacUI project.
- Cpp : Create a GacUI project purely in C++.
- WorkflowScript : Still a C++ project but use script to create the window.
- CppXml : Create a GacUI project from an XML resource, with some C++ code generated from it.
- MVVM : Using the MVVM pattern with XML resource.
-
Layout : Demonstrates all layout features.
- Alignment : Align controls.
- Stack : Stack layout.
- Flow : Flow layout with configurable axis directions.
- Table : Table layout.
- RichTextEmbedding : Embed controls in a rich text document.
-
Controls : Demonstrates all controls and control templates. This demo is under development.
-
XML : Demonstrates all features in the XML resource file. This demo is under development.
- etc ...
Running a demo
Using Visual Studio 2013 or higher version to open the solution, and then run the demo by pressing F5.
If you want to modify the demo for practicing, you may notice that there are a Codegen.bat file in the UI folder in every demo projects. Modify the xml and run the bat file, the corresponding binary resource in the UIRes folder will be updated. After these steps are done, press F5 to see the result!


--- title: Presentation on State Educational Twitter Hashtags at #siteconf link: http://jmichaelrosenberg.com/presentation-on-state-educational-twitter-hashtags-at-siteconf/ author: joshuarosenberg description: post_id: 4600 date: 2016-03-22 23:09:48 created_gmt: 2016/03/23 03:09:48 comment_status: closed post_name: presentation-on-state-educational-twitter-hashtags-at-siteconf status: publish post_type: post categories: [presentation] tags: [SITE] ---


A delight for veggie lovers! Choose from our wide range of delicious vegetarian pizzas, it's softer and tastier
Choose your favourite non-veg pizzas from our menu. Get fresh non-veg pizza with your choice of crusts & toppings
Indulge into mouth-watering taste of Pizza mania range, perfect answer to all your food cravings


Use the list below to specify special annotations. Methods annotated with one of these annotations will be ignored by this inspection.
Use the checkbox below to ignore methods of which the containing class does not implement a non-library interface.
Powered by InspectionGadgets


@{if user.desktop !== 1 && !mobile}
@{fi}
@(300x300 pixels)
@{if CONF.allownickname}
@(Nick name)
@{fi}
@(Email address)
@(Phone number)
@{if CONF.allowtheme}
@{fi} @{if CONF.allowbackground}
@(Background image:)
@(Upload image)
@{fi}
@{if CONF.allowappearance}
@(Appearance):
@(You can choose light or dark mode. Not all apps support appearance.)
@(Dark)
@(Light)
@{fi}
@(Enable notifications)
@(Enable email notifications)
@(Enable phone notifications)
@(Enable sounds)
@(Enable 2FA one-time password)
@(Change password)
@{if CONF.allowdesktop}
@(Desktop mode)
@(IMPORTANT)
: @(if you save this settings then the platform will be reloaded automatically because you have changed a layout.)
@{fi}
@(Enter the code from the app here:)
@(New password)
@(Generate)
@(Passsword will be encrypted.)
@(Date format)
@(Time)
@(Number format)
@(Language)
@(low volume)
@(medium volume)
@(loud volume)
@(Screen locking)
@(Screen will be locked only on desktop computers. )
@(Idle time)
@(Must be defined in minutes.)
@(PIN code)
@(Only 4 numbers.)
@(SAVE SETTINGS)


{% extends "base.html" %} {% block body_content %} {% include "_logout.html" %} {% from "_macros.html" import render_field, render_hidden %}
Use the form to update your details


include "partials/_head.html"
include "partials/_header.html"CONTACT US
OUR STORE
500 Terry Francois Street
San Francisco, CA 94158
<EMAIL_ADDRESS>I Tel.<PHONE_NUMBER>
View map >OPENING HOURS
Monday - Friday: 11.00 - 18.30
Saturday: 11.00 - 17.00
Sunday: 12.30 - 16.30


% content_for :title do %> The page you were looking for doesn't exist (404)<% end %>
You may have mistyped the address or the page may have moved.


JavaScript vs Ruby
READY! Fight!
JavaScript was very tedious when writing a program. I found it more difficult to write code because it would not compile the errors so easily and I felt lost in why it would not work. I think it's mostly because of practice, but it does not take the tedious part of it.
Some of the differences that I noticed between ruby and JavaScript was that the way ruby is organized in classes it allows for a better structure and readability to the parameters, but in the JavaScript functions it became tedious having to loop around functions. I felt much more comfortable looping if statements inside a while statement without having so many semi-colons in ruby than JavaScript. I feel ruby has many more ways of coming across a solution than JavaScript just because the compiler reads differently and assumes differently what you want it to do.
-Richard Santin


The length of time a system takes to go through one cycle of its motion.
Period has dimensions T, and SI unit s.


About 100 fans brawl in Ukraine ahead of Europa League game
LVIV, Ukraine (AP) - Ukrainian police say they have detained 12 people involved in a brawl between about 100 soccer fans ahead of a Europa League game.
The incident happened late Wednesday, a day before French club Saint-Etienne was to play Olexandriya in Lviv on Thursday.
Police say Ukrainian, French and German fans were involved in the fight. It was stopped after various police units were rushed to the scene.
Police detained four Lviv locals, said to be between the ages of 20 and 22, and eight foreigners. Two police officers were injured.
More AP soccer: https://apnews.com/Soccer and https://twitter.com/AP_Sports
Sign up for Daily Newsletters
Copyright © 2019 The Washington Times, LLC.
Please read our comment policy before commenting.


Introduction to T-Watch¶
Background¶
With the development of smart homes and IOTs in the past two years, not only computers, the Internet of Things also involves smart phones and household appliances, as well as smart bracelets, watches and items and equipment that are closely related to our lives. At present, the functions of these smart watches and wristbands are similar. The smart bracelets are mainly used to record sports data, message push, call reminder, and heart rate monitoring. For smart watches, the functions may be more abundant, such as dial change, support call, GPS positioning, photo recording, offline payment and other smart phone functions. And our product is different, T-Watch is a programmable smart watch!
Why should I program?¶
Why does the watch support programmable? Why are many of us going to learn programming? Why children from Europe and the United States are more creative and imaginative.
The father of Scratch once commented on modern students: “There are two kinds of students in the world, one is “A-type students”, that is, they have been learning very well from elementary school to high school, and although the A-type students have good test scores, they are step by step, not Having the creativity and innovation necessary to succeed in today’s society.
The other is “X-type students.” “X-type students” are willing to take risks and try to try new things; they are eager to ask their own questions instead of simply solving the problems in the textbooks. ”
With the development of technology, the demand for X-type talents will become larger and larger. So learning programming is a top priority. Interest is the best teacher, we use T-Wathch programmable smart watch as a platform and medium, with Scratch, micropython and other popular programming languages. Greatly reduced the threshold for learning programming.
Product Features¶
Main chip: ESP32, dual core MCU (integrated dual mode Bluetooth/wifi), PMU power management
Display: 1.54 inch LCD capacitive touch screen
Sensor: BMA423 three-axis accelerometer, built-in step counting algorithm, activity recognition / tracking, advanced gesture recognition, etc.
GPS module: Quickly locate the current position using the Sony CXD5603GF module
Lora module: a new generation of popular wireless communication technology
Combination kit: lithium battery, design open mold, and thick strap, and black and white
Development platform: ESP-IDF (native SDK), Arduino, Lua, MicroPython, Scratch
After sales service¶
Quality technical support
Fully open source hardware software
Independent community communication platform
Detailed authoritative learning materials


From the first lesson, I have learned the various components of the internet and I've begun the journey of understanding how to write my own web page (of which this is a basic example).
I've learned about formatting text, adding links, embedding videos and pictures and how to produce a webpage using scratchpad and save it to a file on my computer using CodePen. I've learned of two types of HTML tags,'box'
tags (I can't remember the official name for these tags).


×
Are you sure to delete all protocols?
×
Delete the simulation
Toggle navigation
NEST Desktop
NEST Simulator
Github of NEST Desktop
NEST Documentation
Synchronize simulations
Configuration
Close app


Sorry we can not connect to the bank , for their hearts are made of stone but trust us we will do our best to make your donation go to the right place no matter what