the " stuff " sticks to itself better than it sticks to the cookie. now if you pull the cookies apart, you create a region of local stress, and one of the two interfaces will begin to unstick. at that point, you get something called " stress concentration " at the tip of the crack ( red arrow ) - where the tensile force concentrates : to get the stuff to start separating at a different part of the cookie, you need to tear the stuffing ( which is quite good at sticking to itself ) and initiate a delamination at a new point ( where there is no stress concentration ). those two things together explain your observation. cookie picture credit ( also explanation about manufacturing process introducing a bias ) update a plausible explanation was given in this article describing work by cannarella et al : nabisco won ’ t divulge its oreo secrets, but in 2010, newman ’ s own — which makes a very similar “ newman - o ” — let the discovery channel into its factory to see how their version of cookies are made. the key aspect for twist - off purposes : a pump applies the cream onto one wafer, which is then sent along the line until a robotic arm places a second wafer on top of the cream shortly after. the cream always adheres better to one of these wafers — and all of the cookies in a single box end up oriented in the same direction. which side is the stronger wafer - to - cream interface? “ we think we know, ” says spechler. the key is that fluids flow better at high temperatures. so the hot cream flows easily over the first wafer, filling in the tiny cracks of the cookie and sticking to it like hot glue, whereas the cooler cream just kind of sits on the edges of those crevices.


you are right that the two algorithms of dijkstra ( shortest paths from a single start node ) and prim ( minimal weight spanning tree starting from a given node ) have a very similar structure. they are both greedy ( take the best edge from the present point of view ) and build a tree spanning the graph. the value they minimize however is different. dijkstra selects as next edge the one that leads out from the tree to a node not yet chosen closest to the starting node. ( then with this choice, distances are recalculated. ) prim choses as edge the shortest one leading out of the tree constructed so far. so, both algorithms chose a " minimal edge ". the main difference is the value chosen to be minimal. for dijkstra it is the length of the complete path from start node to the candidate node, for prim it is just the weight of that single edge. to see the difference you should try to construct a few examples to see what happens, that is really instructive. the simplest example that shows different behaviour is a triangle $ x, y, z $ with edges $ \ { x, y \ } $ and $ \ { x, z \ } $ of length 2, while $ \ { y, z \ } $ has length 1. starting in $ x $ dijkstra will choose $ \ { x, y \ } $ and $ \ { x, z \ } $ ( giving two paths of length 2 ) while prim chooses $ \ { x, y \ } $ and $ \ { y, z \ } $ ( giving spanning tree of weight 3 ). as for kruskal, that is slightly different. it solves the minimal spanning tree, but during execution it chooses edge that may not form a tree, they just avoid cycles. so the partial solutions may be disconnected. in the end you get a tree.


the power spectral density describes the density of power in a stationary random process $ x ( t ) $ per unit of frequency. by the wiener - khinchin theorem, it can be calculated as follows for a wide - sense stationary random process : $ $ s _ { xx } ( f ) = \ int _ { - \ infty } ^ { \ infty } r _ { xx } ( \ tau ) e ^ { - j2 \ pi f \ tau } d \ tau $ $ where $ r _ { xx } ( \ tau ) $ is the autocorrelation function of the process $ x ( t ) $ : $ $ r _ { xx } ( \ tau ) = \ mathbb { e } \ left ( x ( t ) x ( t - \ tau ) \ right ) $ $ this is only valid for a wide - sense stationary process because its autocorrelation function is only a function of the time lag $ \ tau $ and not the absolute time $ t $ ; stated differently, this means that its second - order statistics don't change as a function of time. with that said, if you have a sufficiently - detailed and accurate statistical model for your signal, then you can calculate its power spectral density using the relationship above. as an example, this can be used to calculate the power spectral density of communications signals, given the statistics of the information symbols carried by the signal and any pulse shaping employed during transmission. in most practical situations, this level of information is not available, however, and one must resort to estimating a given signal's power spectral density. one very straightforward approach is to take the squared magnitude of its fourier transform ( or, perhaps, the squared magnitude of several short - time fourier transforms and average them ) as the estimate of the psd. however, assuming that the signal you're observing contains some stochastic component ( which is often the case ), this is again just an estimate of what the true underlying psd is based upon a single realization ( i. e. a single observation ) of the random process. whether the power spectrum that you calculate bears any meaningful resemblance to the actual psd of the process is situation - dependent. as this previous post notes, there are many methods for psd estimation ; which is most suitable depends upon the character of the random process, any a priori information that you might have, and what features of the signal you're most interested in.


it does. you would find the average percentage of the atmosphere that is argon is very slightly higher at the floor of valleys. however, bear in mind first of all it wouldn't be anywhere near a complete stratification - - a layer of pure argon, then another of pure n2, and so on. a mixture of nearly ideal gases doesn't do that, at least at equilibrium, because it would eliminate the considerable entropy of mixing. ( it can happen in liquids because liquids have strong intermolecular forces that normally favor separation and oppose the entropy of mixing. ) another way to think about it is that since the atoms and molecules in gases don't ( much ) interact, there's nothing stopping an individual argon atom going slightly faster than nearby nitrogen and oxygen molecules from bouncing up higher than they do. what you would get in a theoretical ideal ( uniform gravitational field, complete stillness - - no wind - - and uniform temperature ) would be an exponential fall of pressure with altitude, and the exponential for heavier gases would be steeper than for lighter gases. that would result in enrichment of the heavier gases at lower altitudes. a little work starting from the boltzmann distribution of gravitational potential energies of each type of atom and molecule would get you an ideal estimate of the argon excess as a function of altitude. in practice the lower atmosphere has so much mixing due to wind and big thermal gradients that i doubt you could even measure the mild excess of argon and other heavy gases. there is one fascinating short - term exception, which bears directly on your question. sometimes volcanoes will belch out a considerable quantity of co2, which is significantly denser than air, and this co2 can accumulate briefly in a thick layer at the bottom of a valley or over a lake, if there isn't much wind. it can persist for some hours, perhaps days, before it diffuses away and is mixed with the rest of the atmosphere. then indeed the valley bottom becomes an invisible death trap for humans and animals : walk into the valley, or be unable to exit fast enough when it happens, and you will suffocate for no reason you can see. the most famous example of this is the lake nyos disaster in 1986 which killed thousands of humans and animals. i think the government now has mixing devices installed in that lake to prevent any future sudden release of co2.


here's a video of physicist richard feynman discussing this question. imagine a blue dot and a red dot. they are in front of you, and the blue dot is on the right. behind them is a mirror, and you can see their image in the mirror. the image of the blue dot is still on the right in the mirror. what's different is that in the mirror, there's also a reflection of you. from that reflection's point of view, the blue dot is on the left. what the mirror really does is flip the order of things in the direction perpendicular to its surface. going on a line from behind you to in front of you, the order in real space is your back your front dots mirror the order in the image space is mirror dots your front your back although left and right are not reversed, the blue dot, which in reality is lined up with your right eye, is lined up with your left eye in the image. the key is that you are roughly left / right symmetric. the eye the blue dot is lined up with is still your right eye, even in the image. imagine instead that two - face was looking in the mirror. ( this is a fictional character whose left and right side of his face look different. his image on wikipedia looks like this : ) if two - face looked in the mirror, he would instantly see that it was not himself looking back! if he had an identical twin and looked right at the identical twin, the " normal " sides of their face would be opposite each other. two - face's good side is the right. when he looked at his twin, the twin's good side would be to the original two - face's left. instead, the mirror two - face's good side is also to the right. here is an illustration : so two - face would not be confused by the dots. if the blue dot is lined up with two - face's good side, it is still lined up with his good side in the mirror. here it is with the dots : two - face would recognize that left and right haven't been flipped so much as forward and backward, creating a different version of himself that cannot be rotated around to fit on top the original.


you can think of the dct as a compression step. typically with mfccs, you will take the dct and then keep only the first few coefficients. this is basically the same reason that the dct is used in jpeg compression. dcts are chosen because their boundary conditions work better on these types of signals. let's contrast the dct with the fourier transform. the fourier transform is made up of sinusoids that have an integer number of cycles. this means, all of the fourier basis functions start and end at the same value - - they do not do a good job of representing signals that start and end at different values. remember that the fourier transform assumes a periodic extension : if you imagine your signal on a sheet of paper, the fourier transform wants to roll that sheet into a cylinder so that the left and right sides meet. think of a spectrum that is shaped roughly like a line with negative slope ( which is pretty typical ). the fourier transform will have to use a lot of different coefficients to fit this shape. on the other hand, the dct has cosines with half - integer numbers of cycles. there is, for example, a dct basis function that looks vaguely like that line with negative slope. it does not assume a period extension ( instead, an even extension ), so it will do a better job of fitting that shape. so, let's put this together. once you've computed the mel - frequency spectrum, you have a representation of the spectrum that is sensitive in a way similar to how human hearing works. some aspects of this shape are more relevant than others. usually, the larger more overarching spectral shape is more important than the noisy fine details in the spectrum. you can imagine drawing a smooth line to follow the spectral shape, and that the smooth line you draw might tell you just about as much about the signal. when you take the dct and discard the higher coefficients, you are taking this spectral shape, and only keeping the parts that are more important for representing this smooth shape. if you used the fourier transform, it wouldn't do such a good job of keeping the important information in the low coefficients. if you think about feeding the mfccs as features to a machine learning algorithm, these lower - order coefficients will make good features, since they represent some simple aspects of the spectral shape, while the higher - order coefficients that you discard are more noise - like and are not important to train on. additionally, training on the mel spectrum


magnitudes themselves would probably not be as good because the particular amplitude at different frequencies are less important than the general shape of the spectrum.


here is my favourite " wow " proof. theorem there exist two positive irrational numbers $ s, t $ such that $ s ^ t $ is rational. proof if $ \ sqrt2 ^ \ sqrt 2 $ is rational, we may take $ s = t = \ sqrt 2 $. if $ \ sqrt 2 ^ \ sqrt 2 $ is irrational, we may take $ s = \ sqrt 2 ^ \ sqrt 2 $ and $ t = \ sqrt 2 $ since $ ( \ sqrt 2 ^ \ sqrt 2 ) ^ \ sqrt 2 = ( \ sqrt 2 ) ^ 2 = 2 $.


edit : i now think that this list is long enough that i shall be maintaining it over time - - updating it whenever i use a new book / learn a new subject. while every suggestion below should be taken with a grain of salt - - i will say that i spend a huge amount of time sifting through books to find the ones that conform best to my ( and hopefully your! ) learning style. here is my two cents ( for whatever that's worth ). i tried to include all the topics i could imagine you could want to know at this point. i hope i picked the right level of difficult. feel absolutely free to ask my specific opinion about any book. basic analysis : rudin - - apostol measure theory : royden ( only if you get the newest fourth edition ) - - folland general algebra : d & f - - rotman - - lang - - grillet finite group theory : isaacs - - kurzweil general group theory : robinson - - rotman ring theory : t. y. lam - - times two commutative algebra : eisenbud - - a & m - - reid homological algebra : weibel - - rotman - - vermani category theory : mac lane - - adamek et. al - - berrick et. al - - awodey - - mitchell linear algebra : roman - - hoffman and kunze - - golan field theory : morandi - - roman complex analysis : ahlfors - - cartan - - freitag riemann surfaces : varolin ( great first read, can be a little sloppy though ) - - freitag ( overall great book for a second course in complex analysis! ) - - forster ( a little more old school, and with a slightly more algebraic bend then a differential geometric one ) - - donaldson scv : gunning et. al - - ebeling point - set topology : munkres - - steen et. al - - kelley differential topology : pollack et. al - - milnor - - lee algebraic topology : bredon - - may - - bott and tu ( great, great book ) - - rotman - - massey - - tom dieck differential geometry : do carmo - - spivak - - jost - - lee representation theory of finite groups : serre - - steinberg - - liebeck - - isaacs general representation theory : fulton and harris - - humphreys - - hall representation theory of compact groups :


tom dieck et. al - - sepanski ( linear ) algebraic groups : springer - - humphreys " elementary " number theory : niven et. al - - ireland et. al algebraic number theory : ash - - lorenzini - - neukirch - - marcus - - washington fourier analysis - - katznelson modular forms : diamond and shurman - - stein local fields : lorenz and levy - - read chapters 23, 24, 25. this is by far my favorite quick reference, as well as " learning text " for the basics of local fields one needs to break into other topics ( e. g. class field theory ). serre - - this is the classic book. it is definitely low on the readability side, especially notationally. it also has a tendency to consider things in more generality than is needed at a first go. this isn't bad, but is not good if you're trying to " brush up " or quickly learn local fields for another subject. fesenko et. al - - a balance between 1. and 2. definitely more readable than 2., but more comprehensive than 1. if you are wondering whether or not so - and - so needs henselian, this is the place i'd check. iwasawa - - a great place to learn the bare - bones of what one might need to learn class field theory. i am referencing, in particular, the first three chapters. if you are dead - set on just learning what you need to, this is a pretty good reference, but if you're likely to wonder about why so - and - so theorem is true, or get a broader understanding of the basics of local fields, i recommend 1. class field theory : lorenz and levy - - read chapters 28 - 32, second only to iwasawa, but with a different flavor ( cohomological vs. formal group laws ) tate and artin - - the classic book. a little less readable then any of the alternatives here. childress - - focused mostly on the global theory opposed to the local. actually deduces local at the end as a result of global. thus, very old school. iwasawa ( read the rest of it! ) milne - - where i first started learning it. very good, but definitely roughly hewn. a lot of details are left out, and he sometimes forgets to tell you where you are going. metric groups : markley algebraic geometry : reid - -


shafarevich - - hartshorne - - griffiths and harris - - mumford


ok, this question appears to have generated some controversy. on the one hand is the answer by niels nielsen ( currently accepted ), which implies that the orange color is from sodium. on the other hand is the answer by stessenj, which implies that the orange is normal black body radiation from the soot. plus there are lots of commentators arguing about rightness or wrongness of the sodium answer. the only good way to settle the matter is an experiment. i did it, with some modifications. first, instead of gas stove i used a jet lighter ( zl - 3 zengaz ). second, instead of humidifier i used a simple barber water spray. the third necessary component is a diffraction grating, a cheap one i had bought on aliexpress. i inserted it into colorless safety goggles to avoid necessity for a third hand. when i lit the lighter i saw a set of images in the first diffraction order : violet, blue, green, yellow and some blurred dim red. so far consistent with the spectrum of blue flame given on wikipedia. then i sprayed water in the air, simultaneously moving the lighter trying to find the place where the flame will change color. as the flame got orange jets instead of initial blue, i noticed orange image of the flame appear between red and yellow images in the diffraction grating. below is a photo i could take with the grating attached to a photo camera's lens, having mounted the camera on a tripod and holding the lighter and spray in both hands while 10s exposure was in progress ( sorry for bad quality ). notice the yellow / orange ( colors are not calibrated ) tall spike at the rhs : that is the part only present in the orange flame. ( the jet indeed became visibly taller when it changed its color to orange. ) from this follows that the orange color indeed comes from sodium, otherwise the orange flame's image would be much wider and spread into multiple colors like the flame from a candle or a non - jet lighter. the readers are welcome to replicate this experiment. edit ok, i've managed to measure some spectra using my amadeus spectrometer with custom driver. i used 15 s integration time with the flame about 3 - 5 cm from the sma905 connector on the spectrometer body. below the two spectra are superimposed, with the blue curve corresponding to the blue flame, and the orange one corresponds to the flame with some orange. i've filtered the data


with 5 - point moving average before plotting. the spectrometer has lower sensitivity near uv and ir, so disregard the noise there. ( click the image for a larger version. ) what's worth noting is that not only the sodium 590 nm line is present in the orange flame, but also two potassium lines – 766 nm and 770 nm. edit2 just tried the same with a humidifier instead of the spray. the result with filtered tap water is the same : orange flame with sodium peak. with distilled water, although the experiment with the spray still resulted in orange flame ( basically the same as with tap water ), with the humidifier i got no orange at all. anyway, in no one case was i able to make the lighter emit continuous spectrum. whenever i got orange flame, it always appeared to be sodium d doublet, not continuous spectrum.


as an extension to moyner's answer, the on - chip sqrt is usually an rsqrt, i. e. a reciprocal square root that computes $ a \ rightarrow 1 / \ sqrt { a } $. so if in your code you're only going to use $ 1 / r $ ( if you're doing molecular dynamics, you are ), you can compute r = rsqrt ( r2 ) directly and save yourself the division. the reason why rsqrt is computed instead of sqrt is that its newton iteration has no divisions, only additions and multiplications. as a side - note, divisions are also computed iteratively and are almost just as slow as rsqrt in hardware. if you're looking for efficiency, you're better off trying to remove superfluous divisions. some more modern architectures such as ibm's power architectures do not provide rsqrt per - se, but an estimate accurate to a few bits, e. g. frsqrte. when a user calls rsqrt, this generates an estimate and then one or two ( as many as required ) iterations of newton's or goldschmidt's algorithm using regular multiplications and additions. the advantage of this approach is that the iteration steps may be pipelined and interleaved with other instructions without blocking the fpu ( for a very nice overview of this concept, albeit on older architectures, see rolf strebel's phd thesis ). for interaction potentials, the sqrt operation can be avoided entirely by using a polynomial interpolant of the potential function, but my own work ( implemented in mdcore ) in this area show that, at least on x86 - type architectures, the sqrt instruction is fast enough. update since this answer seems to be getting quite a bit of attention, i would also like to address the second part of your question, i. e. is it really worth it to try to improve / eliminate basic operations such as sqrt? in the context of molecular dynamics simulations, or any particle - based simulation with cutoff - limited interactions, there is a lot to be gained from better algorithms for neighbour finding. if you're using cell lists, or anything similar, to find neighbours or create a verlet list, you will be computing a large number of spurious pairwise distances. in the naive case, only 16 % of particle pairs inspected will actually be within the cutoff distance of


each other. although no interaction is computed for such pairs, accessing the particle data and computing the spurious pairwise distance carries a large cost. my own work in this area ( here, here, and here ), as well as that of others ( e. g. here ), show how these spurious computations can be avoided. these neighbour - finding algorithms even out - perform verlet lists, as described here. the point i want to emphasize is that although there may be some improvements to gain from better knowing / exploiting the underlying hardware architecture, there are also potentially larger gains to be had in re - thinking the higher - level algorithms.


edit : i am rewriting the answer in response to updates to the original question. tl ; dr : use cram background 1 : quality binning and fastq compression in the old days, base callers outputted base quality at full resolution – you could see quality from q2 to q40 in full range. as a result, quality strings were like semi - random strings and very difficult to compress. later people gradually realized that keeping base quality in low resolution wouldn't affect downstream analysis. the illumina basecaller started to output quality in 8 distinct values and later changed that to 4 bins. this change greatly simplified quality string and made them compressed better. for example, in old days, a 30x bam would take ~ 100 gb. with quality binning, it would only take ~ 60 gb. background 2 : gatk base quality recalibration in early 2010s, illumina base quality was not calibrated well. gatk people introduced bqsr to correct that and observed noticeable improvement in snp accuracy. nonetheless, with improved illumina base caller, their base quality became more accurate. meanwhile, the world moved to 30x deep sequencing. the depth overwhelms slight inaccuracy in quality. i would say around 2015, bqsr was already unnecessary for data produced at the time. does it hurt to apply bqsr? yes. first, bqsr introduces subtle biases towards the reference and towards known snps. second, bqsr distorts the data. at least for some datasets, i observed that snp accuracy dropped with variant quality after bqsr ; i didn't observe this with raw quality. third, bqsr is slow. fourth, for new sequencers producing data at higher quality, bqsr is likely to decrease data quality. last, related to the question, bqsr added another semi - random quality string and made compression even harder. nowadays, running bqsr is a waste of resource for worse results. the official gatk best practice no longer uses bqsr according to their wdl file. cram these days a 30x human cram only takes ~ 15 gb ( see this file ). this is a huge contrast to ~ 100 gb bam in early 2010s. op only saw ~ 10 % saving probably due to a ) bqsr and / or b ) old data with full quality resolution. on encoding / decoding speed, cram was much


slower than bam. not any more. the latest htslib implementation of cram is faster than bam on encoding and only slightly slower on decoding. the poor performance of igv on cram could be that the java cram decoder is not as optimized. it is true that cram is not as widely supported as bam. however, all the other alternatives are much worse. petagene said they had igv - pg ( pdf ) for their format. that is not official igv and i couldn't find more recent update beyond the 2019 press release. i don't see other viable options. note that the common practice is to keep all raw reads, mapped or not, in bam or cram such that you can get raw reads back later. bam / cram additionally keeps metadata like read group, sample name, run information etc and is actually more popular than fastq in large sequencing centers. also note that you don't need to sort cram by coordinate. unsorted cram is only a little larger than sorted cram. cram and its competitors the core cram developer, james bonfield, is one of the most knowledgeable researchers on compression ( and one of the best c programmers ) in this field. he has done a lot of compression evaluation over the years. the conclusion is that on a fair benchmark, cram is comparable to the best tools so far in terms of compression ratio. petagene could compress better in the plot @ terdon showed mostly because it has a special treatment of the oq tag generated by gatk bqsr. it is a typical trick marketing people use to make their methods look better. with bqsr phased out, this plot is no longer relevant. on commercial software in general, i welcome commercial tools and think they are invaluable to users. i also have huge respect to dragen developers. however, on fastq storage, i would strongly recommend against closed - source compressors. if those tools go under, you may lose your data. not worth it.


since general relativity is a local theory just like any good classical field theory, the earth will respond to the local curvature which can change only once the information about the disappearance of the sun has been communicated to the earth's position ( through the propagation of gravitational waves ). so yes, the earth would continue to orbit what should've been the position of the sun for 8 minutes before flying off tangentially. but i should add that such a disappearance of mass is unphysical anyway since you can't have mass - energy just poofing away or even disappearing and instantaneously appearing somewhere else. ( in the second case, mass - energy would be conserved only in the frame of reference in which the disappearance and appearance are simultaneous - this is all a consequence of gr being a classical field theory ). a more realistic situation would be some mass configuration shifting its shape non - spherically in which case the orbits of satellites would be perturbed but only once there has been enough time for gravitational waves to reach the satellite.


pca computes eigenvectors of the covariance matrix ( " principal axes " ) and sorts them by their eigenvalues ( amount of explained variance ). the centered data can then be projected onto these principal axes to yield principal components ( " scores " ). for the purposes of dimensionality reduction, one can keep only a subset of principal components and discard the rest. ( see here for a layman's introduction to pca. ) let $ \ mathbf x _ \ text { raw } $ be the $ n \ times p $ data matrix with $ n $ rows ( data points ) and $ p $ columns ( variables, or features ). after subtracting the mean vector $ \ boldsymbol \ mu $ from each row, we get the centered data matrix $ \ mathbf x $. let $ \ mathbf v $ be the $ p \ times k $ matrix of some $ k $ eigenvectors that we want to use ; these would most often be the $ k $ eigenvectors with the largest eigenvalues. then the $ n \ times k $ matrix of pca projections ( " scores " ) will be simply given by $ \ mathbf z = \ mathbf { xv } $. this is illustrated on the figure below : the first subplot shows some centered data ( the same data that i use in my animations in the linked thread ) and its projections on the first principal axis. the second subplot shows only the values of this projection ; the dimensionality has been reduced from two to one : in order to be able to reconstruct the original two variables from this one principal component, we can map it back to $ p $ dimensions with $ \ mathbf v ^ \ top $. indeed, the values of each pc should be placed on the same vector as was used for projection ; compare subplots 1 and 3. the result is then given by $ \ hat { \ mathbf x } = \ mathbf { zv } ^ \ top = \ mathbf { xvv } ^ \ top $. i am displaying it on the third subplot above. to get the final reconstruction $ \ hat { \ mathbf x } _ \ text { raw } $, we need to add the mean vector $ \ boldsymbol \ mu $ to that : $ $ \ boxed { \ text { pca reconstruction } = \ text { pc scores } \ cdot \ text {


eigenvectors } ^ \ top + \ text { mean } } $ $ note that one can go directly from the first subplot to the third one by multiplying $ \ mathbf x $ with the $ \ mathbf { vv } ^ \ top $ matrix ; it is called a projection matrix. if all $ p $ eigenvectors are used, then $ \ mathbf { vv } ^ \ top $ is the identity matrix ( no dimensionality reduction is performed, hence " reconstruction " is perfect ). if only a subset of eigenvectors is used, it is not identity. this works for an arbitrary point $ \ mathbf z $ in the pc space ; it can be mapped to the original space via $ \ hat { \ mathbf x } = \ mathbf { zv } ^ \ top $. discarding ( removing ) leading pcs sometimes one wants to discard ( to remove ) one or few of the leading pcs and to keep the rest, instead of keeping the leading pcs and discarding the rest ( as above ). in this case all the formulas stay exactly the same, but $ \ mathbf v $ should consist of all principal axes except for the ones one wants to discard. in other words, $ \ mathbf v $ should always include all pcs that one wants to keep. caveat about pca on correlation when pca is done on correlation matrix ( and not on covariance matrix ), the raw data $ \ mathbf x _ \ mathrm { raw } $ is not only centered by subtracting $ \ boldsymbol \ mu $ but also scaled by dividing each column by its standard deviation $ \ sigma _ i $. in this case, to reconstruct the original data, one needs to back - scale the columns of $ \ hat { \ mathbf x } $ with $ \ sigma _ i $ and only then to add back the mean vector $ \ boldsymbol \ mu $. image processing example this topic often comes up in the context of image processing. consider lenna - - one of the standard images in image processing literature ( follow the links to find where it comes from ). below on the left, i display the grayscale variant of this $ 512 \ times 512 $ image ( file available here ). we can treat this grayscale image as a $ 512 \ times 512 $ data matrix $ \ mathbf x _ \ text { raw } $. i perform pc


##a on it and compute $ \ hat { \ mathbf x } _ \ text { raw } $ using the first 50 principal components. the result is displayed on the right. reverting svd pca is very closely related to singular value decomposition ( svd ), see relationship between svd and pca. how to use svd to perform pca? for more details. if a $ n \ times p $ matrix $ \ mathbf x $ is svd - ed as $ \ mathbf x = \ mathbf { usv } ^ \ top $ and one selects a $ k $ - dimensional vector $ \ mathbf z $ that represents the point in the " reduced " $ u $ - space of $ k $ dimensions, then to map it back to $ p $ dimensions one needs to multiply it with $ \ mathbf s ^ \ phantom \ top _ { 1 : k, 1 : k } \ mathbf v ^ \ top _ { :, 1 : k } $. examples in r, matlab, python, and stata i will conduct pca on the fisher iris data and then reconstruct it using the first two principal components. i am doing pca on the covariance matrix, not on the correlation matrix, i. e. i am not scaling the variables here. but i still have to add the mean back. some packages, like stata, take care of that through the standard syntax. thanks to @ stask and @ kodiologist for their help with the code. we will check the reconstruction of the first datapoint, which is : 5. 1 3. 5 1. 4 0. 2 matlab load fisheriris x = meas ; mu = mean ( x ) ; [ eigenvectors, scores ] = pca ( x ) ; ncomp = 2 ; xhat = scores ( :, 1 : ncomp ) * eigenvectors ( :, 1 : ncomp )'; xhat = bsxfun ( @ plus, xhat, mu ) ; xhat ( 1, : ) output : 5. 083 3. 5174 1. 4032 0. 21353 r x = iris [, 1 : 4 ] mu = colmeans ( x ) xpca = prcomp ( x ) ncomp = 2 xhat = xpca $ x [, 1 : ncomp ] % * % t ( xpca $ rotation [, 1 : ncomp ] ) xhat


= scale ( xhat, center = - mu, scale = false ) xhat [ 1, ] output : sepal. length sepal. width petal. length petal. width 5. 0830390 3. 5174139 1. 4032137 0. 2135317 for worked out r example of pca reconstruction of images see also this answer. python import numpy as np import sklearn. datasets, sklearn. decomposition x = sklearn. datasets. load _ iris ( ). data mu = np. mean ( x, axis = 0 ) pca = sklearn. decomposition. pca ( ) pca. fit ( x ) ncomp = 2 xhat = np. dot ( pca. transform ( x ) [ :, : ncomp ], pca. components _ [ : ncomp, : ] ) xhat + = mu print ( xhat [ 0, ] ) output : [ 5. 08718247 3. 51315614 1. 4020428 0. 21105556 ] note that this differs slightly from the results in other languages. that is because python's version of the iris dataset contains mistakes. stata webuse iris, clear pca sep * pet *, components ( 2 ) covariance predict _ seplen _ sepwid _ petlen _ petwid, fit list in 1 iris seplen sepwid petlen petwid _ seplen _ sepwid _ petlen _ petwid setosa 5. 1 3. 5 1. 4 0. 2 5. 083039 3. 517414 1. 403214. 2135317


i prefer to treat software tools and computers in a similar fashion to laboratory equipment, and in some sense biology in general. biologists are used to unexpected things happening in their experiments, and it's not uncommon for a new discovery to change the way that people look at something. things break down, cells die off quicker on a wednesday afternoon, results are inconsistent, and that third reviewer keeps on about doing that thing that's been done a hundred times before without anything surprising happening ( just not this time ). it's a good idea to record as much as can be thought of that might influence an experiment, and for software that includes any input data or command line options, and especially software version numbers. in this sense, a discovered software bug can be treated as a new discovery of how the world works. if the discovery is made public, and other people consider that it's important enough, then some people might revisit old research to see if it changes things. of course, the nice thing about software is that bugs can be reported back to the creators of programs, and possibly fixed, resulting in an improved version of the software at a later date. if the bug itself doesn't spark interest and the program gets fixed anyway, people unknowingly use newer versions, and there might be a bit more confusion and discussion about why results don't match similar studies carried out before the software change. if you want a bit of an idea of the biological equivalent of a major software bug, have a look at the barcode index switching issue, or the cell line contamination issue.


you have some very strong color and geometry cues you can leverage. i would try the following : extract the green channel & apply watershed type algorithm on it, followed by connected components. subsequently compute component statistics ( area & bounding box ) for each component. retain only the components with area ~ = bounding box size. this will be true only for rectangular objects and will eliminate forests / wooded areas etc. isolate the white channel ( r = g = b ) and apply hough transform on the output. this will give you the lines. combine 1 & 2 to get your tennis courts.


the problem with equispaced points is that the interpolation error polynomial, i. e. $ $ f ( x ) - p _ n ( x ) = \ frac { f ^ { ( n + 1 ) } ( \ xi ) } { ( n + 1 )! } \ prod _ { i = 0 } ^ n ( x - x _ i ), \ quad \ xi \ in [ x _ 0, x _ n ] $ $ behaves differently for different sets of nodes $ x _ i $. in the case of equispaced points, this polynomial blows up at the edges. if you use gauss - legendre points, the error polynomial is significantly better behaved, i. e. it doesn't blow up at the edges. if you use chebyshev nodes, this polynomial equioscillates and the interpolation error is minimal.


no - one mentioned the inverse hyperbolic sine transformation. so for completeness i'm adding it here. this is an alternative to the box - cox transformations and is defined by \ begin { equation } f ( y, \ theta ) = \ text { sinh } ^ { - 1 } ( \ theta y ) / \ theta = \ log [ \ theta y + ( \ theta ^ 2y ^ 2 + 1 ) ^ { 1 / 2 } ] / \ theta, \ end { equation } where $ \ theta > 0 $. for any value of $ \ theta $, zero maps to zero. there is also a two parameter version allowing a shift, just as with the two - parameter bc transformation. burbidge, magee and robb ( 1988 ) discuss the ihs transformation including estimation of $ \ theta $. the ihs transformation works with data defined on the whole real line including negative values and zeros. for large values of $ y $ it behaves like a log transformation, regardless of the value of $ \ theta $ ( except 0 ). the limiting case as $ \ theta \ rightarrow0 $ gives $ f ( y, \ theta ) \ rightarrow y $. it looks to me like the ihs transformation should be a lot better known than it is.


without a transformer the live wire is live relative to ground. if you are at " ground " potential then touching the live wire makes you part of the return path. { this image taken from an excellent discussion here with a transformer the output voltage is not referenced to ground - see diagram ( a ) below. there is no " return path " so you could ( stupidly ) safely touch the " live " conductor and ground and not received a shock. from the electricians guide i say " stupidly " as, while this arrangement is safer it is not safe unconditionally. this is because, if there is leakage or hard connection from the other side of the transformer to ground then there may still be a return path - as shown in ( b ) above. in the diagram the return path is shown as either capacitive or direct. if the coupling is capacitive then you may feel a " tickle " or somewhat mild " bite " from the live conductor. if the other conductor is grounded then you are back to the original transformlerless situation. ( capacitive coupling may occur when an appliance body is connected to a conductor but there is no direct connection from body to ground. the body to ground proximity forms a capacitor. ) so a transformer makes things safer by providing isolation relative to ground. murphy / circumstance will work to defeat this isoation. this is why, ideally, an isolating transformer should be used to protect only one item of equipment at a time. with one item a fault in the equipment will propbably not produce a dangerous situation. the transformer has done its job. but with n items of equipment - if one has a fault from neutral to case or is wired wrongly this may defeat the transformer such that a second faulty device may then present a hazard to the user. in figure ( b ) above, the first faulty device provides the link at bottom and the second provides the link at top. similarly :


the probabilistic way : this is $ p [ n _ n \ leqslant n ] $ where $ n _ n $ is a random variable with poisson distribution of parameter $ n $. hence each $ n _ n $ is distributed like $ x _ 1 + \ cdots + x _ n $ where the random variables $ ( x _ k ) $ are independent and identically distributed with poisson distribution of parameter $ 1 $. by the central limit theorem, $ y _ n = \ frac1 { \ sqrt { n } } ( x _ 1 + \ cdots + x _ n - n ) $ converges in distribution to a standard normal random variable $ z $, in particular, $ p [ y _ n \ leqslant 0 ] \ to p [ z \ leqslant0 ] $. finally, $ p [ z \ leqslant0 ] = \ frac12 $ and $ [ n _ n \ leqslant n ] = [ y _ n \ leqslant 0 ] $ hence $ p [ n _ n \ leqslant n ] \ to \ frac12 $, qed. the analytical way, completing your try : hence, i know that what i need to do is to find $ \ lim \ limits _ { n \ to \ infty } i _ n $, where $ $ i _ n = \ frac { e ^ { - n } } { n! } \ int _ { 0 } ^ n ( n - t ) ^ ne ^ tdt. $ $ to begin with, let $ u ( t ) = ( 1 - t ) e ^ t $, then $ i _ n = \ dfrac { e ^ { - n } n ^ n } { n! } nj _ n $ with $ $ j _ n = \ int _ { 0 } ^ 1 u ( t ) ^ n \ mathrm dt. $ $ now, $ u ( t ) \ leqslant \ mathrm e ^ { - t ^ 2 / 2 } $ hence $ $ j _ n \ leqslant \ int _ 0 ^ 1 \ mathrm e ^ { - nt ^ 2 / 2 } \ mathrm dt \ leqslant \ int _ 0 ^ \ infty \ mathrm e ^ { - nt ^ 2 / 2 } \ mathrm dt = \ sqrt { \ frac { \ pi } {


2n } }. $ $ likewise, the function $ t \ mapsto u ( t ) \ mathrm e ^ { t ^ 2 / 2 } $ is decreasing on $ t \ geqslant0 $ hence $ u ( t ) \ geqslant c _ n \ mathrm e ^ { - t ^ 2 / 2 } $ on $ t \ leqslant1 / n ^ { 1 / 4 } $, with $ c _ n = u ( 1 / n ^ { 1 / 4 } ) \ mathrm e ^ { - 1 / ( 2 \ sqrt { n } ) } $, hence $ $ j _ n \ geqslant c _ n \ int _ 0 ^ { 1 / n ^ { 1 / 4 } } \ mathrm e ^ { - nt ^ 2 / 2 } \ mathrm dt = \ frac { c _ n } { \ sqrt { n } } \ int _ 0 ^ { n ^ { 1 / 4 } } \ mathrm e ^ { - t ^ 2 / 2 } \ mathrm dt = \ frac { c _ n } { \ sqrt { n } } \ sqrt { \ frac { \ pi } { 2 } } ( 1 + o ( 1 ) ). $ $ since $ c _ n \ to1 $, all this proves that $ \ sqrt { n } j _ n \ to \ sqrt { \ frac \ pi2 } $. stirling formula shows that the prefactor $ \ frac { e ^ { - n } n ^ n } { n! } $ is equivalent to $ \ frac1 { \ sqrt { 2 \ pi n } } $. regrouping everything, one sees that $ i _ n \ sim \ frac1 { \ sqrt { 2 \ pi n } } n \ sqrt { \ frac \ pi { 2n } } = \ frac12 $. moral : the probabilistic way is shorter, easier, more illuminating, and more fun. caveat : my advice in these matters is, clearly, horribly biased.


it's a rather rough algorithm, but i'd use the following procedure for a crude estimate : if, as you say, the purported $ f ( x ) $ that represents your $ ( x _ i, y _ i ) $ is already almost linear as $ x $ increases, what i'd do is to take differences $ \ dfrac { y _ { i + 1 } - y _ i } { x _ { i + 1 } - x _ i } $, and then use an extrapolation algorithm like the shanks transformation to estimate the limit of the differences. the result is hopefully a good estimate of this asymptotic slope. what follows is a mathematica demonstration. the wynn $ \ epsilon $ algorithm is a convenient implementation of the shanks transformation, and it is built in as the ( hidden ) function sequencelimit [ ]. we try out the procedure on the function $ $ \ frac4 { x ^ 2 + 3 } + 2 x + e ^ { - 4 x } + 3 $ $ xdata = randomreal [ { 20, 40 }, 25 ] ; ydata = table [ ( 3 + 13 * e ^ ( 4 * x ) + 6 * e ^ ( 4 * x ) * x + x ^ 2 + 3 * e ^ ( 4 * x ) * x ^ 2 + 2 * e ^ ( 4 * x ) * x ^ 3 ) / ( e ^ ( 4 * x ) * ( 3 + x ^ 2 ) ), { x, xdata } ] ; sequencelimit [ differences [ ydata ] / differences [ xdata ], method - > { " wynnepsilon ", degree - > 2 } ] 1. 999998 i might as well show off how simple the algorithm is : wynnepsilon [ seq _? vectorq ] : = module [ { n = length [ seq ], ep, res, v, w }, res = { } ; do [ ep [ k ] = seq [ [ k ] ] ; w = 0 ; do [ v = w ; w = ep [ j ] ; ep [ j ] = v + ( if [ abs [ ep [ j + 1 ] - w ] > 10 ^ - ( precision [ w ] ), ep [ j + 1 ] - w, 10 ^ - ( precision [ w ] ) ] ) ^ - 1 ;, { j, k - 1, 1, -


1 } ] ; res = { res, ep [ if [ oddq [ k ], 1, 2 ] ] } ;, { k, n } ] ; flatten [ res ] ] last [ wynnepsilon [ differences [ ydata ] / differences [ xdata ] ] ] 1. 99966 this implementation is adapted from weniger's paper.


all models are wrong, but some are useful. ( george e. p. box ) reference : box & draper ( 1987 ), empirical model - building and response surfaces, wiley, p. 424. also : g. e. p. box ( 1979 ), " robustness in the strategy of scientific model building " in robustness in statistics ( launer & wilkinson eds. ), p. 202.


the exact mechanism is unclear. here are some possible causes : rapid collapsing of cavities inside the joint [ 1 ] ; rapid ligament stretching [ 1 ] ; breaking of intra - articular adhesions [ 1 ] ; escaping gases from synovial fluid [ 2 ] ; movements of joints, tendons and ligaments [ 2 ] ; mechanic interaction between rough surfaces [ 2 ], mostly in pathological situations like arthritis ( and it is called crepitus [ 3 ] ). there are no known bad effects of joint cracking [ 1, 4 ]. there are no long term sequelae of these noises, and they do not lead to future problems. there is no basis for the admonition to not crack your knuckles because it can lead to arthritis. there are no supplements or exercises to prevent these noises [ 4 ]. and no good effects either : knuckle " cracking " has not been shown to be harmful or beneficial. more specifically, knuckle cracking does not cause arthritis [ 5 ]. references : wikipedia contributors, " cracking joints, " wikipedia, the free encyclopedia, ( accessed july 22, 2014 ). the library of congress. everyday mysteries. what causes the noise when you crack a joint? available from ( accessed 22. 07. 2014 ) wikipedia contributors, " crepitus, " wikipedia, the free encyclopedia, ( accessed july 22, 2014 ). johns hopkins sports medicine patient guide to joint cracking & popping. available from ( accessed 22. 07. 2014 ) webmd, llc. will joint cracking cause osteoarthritis? available from ( accessed 22. 07. 2014 )


( this is a fairly long answer, there is a summary at the end ) you are not wrong in your understanding of what nested and crossed random effects are in the scenario that you describe. however, your definition of crossed random effects is a little narrow. a more general definition of crossed random effects is simply : not nested. we will look at this at the end of this answer, but the bulk of the answer will focus on the scenario you presented, of classrooms within schools. first note that : nesting is a property of the data, or rather the experimental design, not the model. also, nested data can be encoded in at least 2 different ways, and this is at the heart of the issue you found. the dataset in your example is rather large, so i will use another schools example from the internet to explain the issues. but first, consider the following over - simplified example : here we have classes nested in schools, which is a familiar scenario. the important point here is that, between each school, the classes have the same identifier, even though they are distinct if they are nested. class1 appears in school1, school2 and school3. however if the data are nested then class1 in school1 is not the same unit of measurement as class1 in school2 and school3. if they were the same, then we would have this situation : which means that every class belongs to every school. the former is a nested design, and the latter is a crossed design ( some might also call it multiple membership. edit : for a discussion of the differences between multiple membership and crossed random effects, see here ), and we would formulate these in lme4 using : ( 1 | school / class ) or equivalently ( 1 | school ) + ( 1 | class : school ) and ( 1 | school ) + ( 1 | class ) respectively. due to the ambiguity of whether there is nesting or crossing of random effects, it is very important to specify the model correctly as these models will produce different results, as we shall show below. moreover, it is not possible to know, just by inspecting the data, whether we have nested or crossed random effects. this can only be determined with knowledge of the data and the experimental design. but first let us consider a case where the class variable is coded uniquely across schools : there is no longer any ambiguity concerning nesting or crossing. the nesting is explicit. let us now see this with an example in r, where we


have 6 schools ( labelled i - vi ) and 4 classes within each school ( labelled a to d ) : > dt < - read. table ( " header = true, sep = ", ", na. strings = " na ", dec = ". ", strip. white = true ) > # update 1 : data was previously publicly available from > # > # but the link is now broken. > # update 2 : the link is broken again. a new link is used. the previous link was : > xtabs ( ~ school + class, dt ) class school a b c d i 50 50 50 50 ii 50 50 50 50 iii 50 50 50 50 iv 50 50 50 50 v 50 50 50 50 vi 50 50 50 50 we can see from this cross tabulation that every class id appears in every school, which satisfies your definition of crossed random effects ( in this case we have fully, as opposed to partially, crossed random effects, because every class occurs in every school ). so this is the same situation that we had in the first figure above. however, if the data are really nested and not crossed, then we need to explicitly tell lme4 : > m0 < - lmer ( extro ~ open + agree + social + ( 1 | school / class ), data = dt ) > summary ( m0 ) random effects : groups name variance std. dev. class : school ( intercept ) 8. 2043 2. 8643 school ( intercept ) 93. 8421 9. 6872 residual 0. 9684 0. 9841 number of obs : 1200, groups : class : school, 24 ; school, 6 fixed effects : estimate std. error t value ( intercept ) 60. 2378227 4. 0117909 15. 015 open 0. 0061065 0. 0049636 1. 230 agree - 0. 0076659 0. 0056986 - 1. 345 social 0. 0005404 0. 0018524 0. 292 > m1 < - lmer ( extro ~ open + agree + social + ( 1 | school ) + ( 1 | class ), data = dt ) summary ( m1 ) random effects : groups name variance std. dev. school ( intercept ) 95. 887 9. 792 class ( intercept ) 5. 790 2. 406 residual 2. 787 1. 669 number of obs : 1200, groups : school


, 6 ; class, 4 fixed effects : estimate std. error t value ( intercept ) 60. 198841 4. 212974 14. 289 open 0. 010834 0. 008349 1. 298 agree - 0. 005420 0. 009605 - 0. 564 social - 0. 001762 0. 003107 - 0. 567 as expected, the results differ because m0 is a nested model while m1 is a crossed model. now, if we introduce a new variable for the class identifier : > dt $ classid < - paste ( dt $ school, dt $ class, sep = ". " ) > xtabs ( ~ school + classid, dt ) classid school i. a i. b i. c i. d ii. a ii. b ii. c ii. d iii. a iii. b iii. c iii. d iv. a iv. b i 50 50 50 50 0 0 0 0 0 0 0 0 0 0 ii 0 0 0 0 50 50 50 50 0 0 0 0 0 0 iii 0 0 0 0 0 0 0 0 50 50 50 50 0 0 iv 0 0 0 0 0 0 0 0 0 0 0 0 50 50 v 0 0 0 0 0 0 0 0 0 0 0 0 0 0 vi 0 0 0 0 0 0 0 0 0 0 0 0 0 0 classid school iv. c iv. d v. a v. b v. c v. d vi. a vi. b vi. c vi. d i 0 0 0 0 0 0 0 0 0 0 ii 0 0 0 0 0 0 0 0 0 0 iii 0 0 0 0 0 0 0 0 0 0 iv 50 50 0 0 0 0 0 0 0 0 v 0 0 50 50 50 50 0 0 0 0 vi 0 0 0 0 0 0 50 50 50 50 the cross tabulation shows that each level of class occurs only in one level of school, as per your definition of nesting. this is also the case with your data, however it is difficult to show that with your data because it is very sparse. both model formulations will now produce the same output ( that of the nested model m0 above ) : > m2 < - lmer ( extro ~ open + agree + social + ( 1 | school / classid ), data = dt ) > summary ( m2 ) random effects : groups name variance std. dev. classid : school


( intercept ) 8. 2043 2. 8643 school ( intercept ) 93. 8419 9. 6872 residual 0. 9684 0. 9841 number of obs : 1200, groups : classid : school, 24 ; school, 6 fixed effects : estimate std. error t value ( intercept ) 60. 2378227 4. 0117882 15. 015 open 0. 0061065 0. 0049636 1. 230 agree - 0. 0076659 0. 0056986 - 1. 345 social 0. 0005404 0. 0018524 0. 292 > m3 < - lmer ( extro ~ open + agree + social + ( 1 | school ) + ( 1 | classid ), data = dt ) > summary ( m3 ) random effects : groups name variance std. dev. classid ( intercept ) 8. 2043 2. 8643 school ( intercept ) 93. 8419 9. 6872 residual 0. 9684 0. 9841 number of obs : 1200, groups : classid, 24 ; school, 6 fixed effects : estimate std. error t value ( intercept ) 60. 2378227 4. 0117882 15. 015 open 0. 0061065 0. 0049636 1. 230 agree - 0. 0076659 0. 0056986 - 1. 345 social 0. 0005404 0. 0018524 0. 292 it is worth noting that crossed random effects do not have to occur within the same factor - in the above the crossing was completely within school. however, this does not have to be the case, and very often it is not. for example, sticking with a school scenario, if instead of classes within schools we have pupils within schools, and we were also interested in the doctors that the pupils were registered with, then we would also have nesting of pupils within doctors. there is no nesting of schools within doctors, or vice versa, so this is also an example of crossed random effects, and we say that schools and doctors are crossed. a similar scenario where crossed random effects occur is when individual observations are nested within two factors simultaneously, which commonly occurs with so - called repeated measures subject - item data. typically each subject is measured / tested multiple times with / on different items and these same items are measured / tested by different subjects. thus, observations are clustered within subjects and within items, but


items are not nested within subjects or vice - versa. again, we say that subjects and items are crossed. summary : tl ; dr the difference between crossed and nested random effects is that nested random effects occur when one factor ( grouping variable ) appears only within a particular level of another factor ( grouping variable ). this is specified in lme4 with : ( 1 | group1 / group2 ) where group2 is nested within group1. crossed random effects are simply : not nested. this can occur with three or more grouping variables ( factors ) where one factor is separately nested in both of the others, or with two or more factors where individual observations are nested separately within the two factors. these are specified in lme4 with : ( 1 | group1 ) + ( 1 | group2 )


there is considerable overlap among these, but some distinctions can be made. of necessity, i will have to over - simplify some things or give short - shrift to others, but i will do my best to give some sense of these areas. firstly, artificial intelligence is fairly distinct from the rest. ai is the study of how to create intelligent agents. in practice, it is how to program a computer to behave and perform a task as an intelligent agent ( say, a person ) would. this does not have to involve learning or induction at all, it can just be a way to'build a better mousetrap '. for example, ai applications have included programs to monitor and control ongoing processes ( e. g., increase aspect a if it seems too low ). notice that ai can include darn - near anything that a machine does, so long as it doesn't do it'stupidly '. in practice, however, most tasks that require intelligence require an ability to induce new knowledge from experiences. thus, a large area within ai is machine learning. a computer program is said to learn some task from experience if its performance at the task improves with experience, according to some performance measure. machine learning involves the study of algorithms that can extract information automatically ( i. e., without on - line human guidance ). it is certainly the case that some of these procedures include ideas derived directly from, or inspired by, classical statistics, but they don't have to be. similarly to ai, machine learning is very broad and can include almost everything, so long as there is some inductive component to it. an example of a machine learning algorithm might be a kalman filter. data mining is an area that has taken much of its inspiration and techniques from machine learning ( and some, also, from statistics ), but is put to different ends. data mining is carried out by a person, in a specific situation, on a particular data set, with a goal in mind. typically, this person wants to leverage the power of the various pattern recognition techniques that have been developed in machine learning. quite often, the data set is massive, complicated, and / or may have special problems ( such as there are more variables than observations ). usually, the goal is either to discover / generate some preliminary insights in an area where there really was little knowledge beforehand, or to be able to predict future observations accurately. moreover, data mining procedures could be either'unsupervised'( we don't know


the answer - - discovery ) or'supervised'( we know the answer - - prediction ). note that the goal is generally not to develop a more sophisticated understanding of the underlying data generating process. common data mining techniques would include cluster analyses, classification and regression trees, and neural networks. i suppose i needn't say much to explain what statistics is on this site, but perhaps i can say a few things. classical statistics ( here i mean both frequentist and bayesian ) is a sub - topic within mathematics. i think of it as largely the intersection of what we know about probability and what we know about optimization. although mathematical statistics can be studied as simply a platonic object of inquiry, it is mostly understood as more practical and applied in character than other, more rarefied areas of mathematics. as such ( and notably in contrast to data mining above ), it is mostly employed towards better understanding some particular data generating process. thus, it usually starts with a formally specified model, and from this are derived procedures to accurately extract that model from noisy instances ( i. e., estimation - - by optimizing some loss function ) and to be able to distinguish it from other possibilities ( i. e., inferences based on known properties of sampling distributions ). the prototypical statistical technique is regression.


" touch not the cat, bot a glove " dttah / acnr / ianal / ymmv * equipment : high impedance voltmeter / oscilloscope with hv probe. high voltage low capacitance capacitors ( 1 10 100 1000 pf ) x 2 of each. pretest - charge capacitors to some semi known high voltage and measure with voltmeter to determine measurement ability. for purrfect results there should be minimal paws between first and second iterations of 2. 3. 4. select cap - say 100 pf. discharge cap ( short ) connect one end of cap to ground - one end of cap to cat..... ( how " to cat " is achieved is left as an exercise for the reader. ).... ( cap and cat are now at same purrtential ) disconnect cap from cat measure vcap repeat 2. 3. 4. compare readings. repeat with higher and lower caps. aim is range where v1 / v2 is usefully high - say about 2 : 1. processing. when cap connects to cat cap is charged. cat and cap share charge in proportion to capacitances. overall voltage drops to reflect increase in system capacitance from addin cap to ccat. if vcat before and after transfer was known you could calculate ccat. but vcat'a bit hard'to determine. repeating process gives a second point and 2 simultaneous equations can be solved to give ccat. if ccap < < ccat the delta v is small and results are ill conditioned. if ccap > > ccat the delta v is large and results are ill conditioned. if ccap ~ ~ ~ = ccat the porridge is just right and the bed is just right. if ccap = ccat then voltage will halve on second reading. v = vcat _ original / 2 otherwise ratio change is related to inverse proportion to capacitances. v2 = v1 x ccat / ( ccat + ccap ) or say v1 / v2 = 0. 75 ccat = 3 x ccap. e & oe.... dttah...... don't try this at home acnr........ all care, no responsibility ianal....... i am not a lawyer ymmv....... your mileage will vary e


& oe........ errors & omissions excepted.


it's an interesting question and one that has been asked before. npr did a story in 2013 on this topic, but their question was a bit more focused than just " why are so many black people good runners? " the observation that led to their story wasn't just that black people in general were over - represented among long - distance running medalists, but that kenyans in particular were over - represented. digging deeper, the story's investigators found that the best runners in kenya also tended to come from the same tribal group : the kalenjin. i'm not going to repeat all the details in that story ( which i encourage you to read ), but the working answer that the investigators came up with is that there are both genetic traits and certain cultural practices that contribute to this tribe's success on the track. unfortunately, from the point of view of someone who wants a concise answer, it is very difficult to separate and quantify the exact contributions that each genetic and cultural modification makes to the runners'successes. pubmed also has a number of peer - reviewed papers detailing the kalenjin running phenomenon, but i could only find two with free full - access and neither had the promising title of " analysis of the kenyan distance - running phenomenon, " for which you have to pay. insert annoyed frowning face here. i did a quick search of some kenyan gold medalist runners in the 2016 olympics and sure enough, several ( though certainly not all ) are kalenjin. i'm less sure about the ethiopian runners, since most research that i found online seems to focus on the kenyans, but i'd feel safe hypothesizing that something similar can explain their dominance at the podium. so, the short answer to your question is that it's not just " black people " who dominate the world of competitive long - distance running, but that very specific subsets of people ( who, as it turns out, are black ) do display a competitive advantage and that both genetics and culture account for much of this advantage.


why do we age is a classical question in evolutionary biology. there are several things to consider when we think of how genes that cause disease, aging, and death to evolve. one explanation for the evolution of aging is the mutation accumulation ( ma ) hypothesis. this hypothesis by p. medawar states that mutations causing late life deleterious ( damaging ) effects can build up in the genome more than diseases that cause early life disease. this is because selection on late acting mutations is weaker. mutations that cause early life disease will more severely reduce the fitness of its carrier than late acting mutations. for example, if we said in an imaginary species that all individuals cease to reproduce at 40 years old and a mutation arises that causes a fatal disease at 50 years old then selection can not remove it from the population - carriers will have as many children as those who do not have the gene. under the mutation accumulation hypothesis it is then possible for mutations to drift through the population. another hypothesis which could contribute to aging is the antagonistic pleiotropy ( ap ) hypothesis of g. c. williams. pleiotropy is when genes have more than one effect, such genes tend to cause correlations between traits, height and arm length probably have many of the same genes affecting them, otherwise there would be no correlation between arm length and height ( though environment and linkage can also cause these patterns )... back to ap as an explanation for aging, if a gene improves fitness early in life, but causes late life disease it can spread through the population via selection. the favourable early effect spreads well because of selection and, just as with ma, selection can not " see " the late acting disease. under both ma and ap the key point is that selection is less efficient at removing late acting deleterious mutations, and they may spread more rapidly thanks to beneficial early life effects. also if there is extrinsic mortality ( predation etc. ) then the effect of selection is also weakened on alleles that affect late life. the same late - life reduction in the efficacy of selection also slows the rate at which alleles increasing lifespan spread. a third consideration is the disposable - soma model, a description by t. kirkwood of life - history trade - offs which might explain why aging and earlier death could be favoured. the idea is that individuals have a limited amount of resources available to them - perhaps because of environmental constraints or ability to acquire / allocate the resources. if we then assume that individuals have


to use their energy for two things, staying alive via repair and maintenance ( somatic - maintenance ) and making offspring ( reproductive - investment ), then any energy devoted to one will take away from the other. if an individual carries a gene that makes it devote all of its energy to somatic maintenance then its fitness will be very low ( probably 0! ) and that gene will not spread. if the level of maintenance required to live forever costs more energy than an individual can spare without suffering from low fitness ( very likely ) or can even acquire and efficiently convert in the first place ( also very likely ) then high - maintenance alleles will not spread ( and aging & death will continue to occur ). to go a little further, it is common for sexes to age differently ( this is what i work on ) and one possible explanation is that the sexes favour different balances of the trade off between somatic - maintenance and reproductive investment, this can lead to conflict over the evolution of genes affecting this balance and slow the rates of evolution to sex specific optima. this paper provides a good review of the area. to summarise, evolution has not managed to get rid of death via genetic disease etc. ( intrinsic mortality ) because the effect is only weakly selected against, and those alleles may provide some early life benefit, and resource limitation may also reduce the potential to increase lifespan due to trade - offs with reproductive effort. adaptive evolution is not about the survival of the fittest but the reproduction of the fittest - the fittest allele is the one which spreads the most effectively. edit : thanks to remi. b for also pointing out some other considerations. another thought is that of altruistic aging - aging for the good of the population ( the population is likely to contain related individuals, you are related to all other humans to some degree ). in this model aging is an adaptive process ( unlike in ma where it is just a consequence of weak selection ). by dying an individual makes space for it's offspring / relatives to survive ( because resources are then less likely to limit populations ). this will stop excessive population growth which could lead to crashes in the population and so, by dying earlier, an individual promotes the likelihood that its progeny will survive. arguments of altruistic sacrifice are often hard to promote but recent work suggests that this is a more plausible model than once thought. evolvabilty theories also suggest that aging is an adaptive process. these suggest that populations, composed of a mixture of young and old,


have biases in how well adapted the members of the population are - where younger individuals are better adapted ( because they were produced more recently it is likely that the environment is similar to the environment they are favoured in ). thus by removing the less well adapted individuals from a population via senescence and freeing up resources for younger better adapted individuals, a population evolves more rapidly towards it optimal state.


first, we need to understand what is a markov chain. consider the following weather example from wikipedia. suppose that weather on any given day can be classified into two states only : sunny and rainy. based on past experience, we know the following : $ p ( \ text { next day is sunny } \, \ vert \, \ text { given today is rainy ) } = 0. 50 $ since, the next day's weather is either sunny or rainy it follows that : $ p ( \ text { next day is rainy } \, \ vert \, \ text { given today is rainy ) } = 0. 50 $ similarly, let : $ p ( \ text { next day is rainy } \, \ vert \, \ text { given today is sunny ) } = 0. 10 $ therefore, it follows that : $ p ( \ text { next day is sunny } \, \ vert \, \ text { given today is sunny ) } = 0. 90 $ the above four numbers can be compactly represented as a transition matrix which represents the probabilities of the weather moving from one state to another state as follows : $ p = \ begin { bmatrix } & s & r \ \ s & 0. 9 & 0. 1 \ \ r & 0. 5 & 0. 5 \ end { bmatrix } $ we might ask several questions whose answers follow : q1 : if the weather is sunny today then what is the weather likely to be tomorrow? a1 : since, we do not know what is going to happen for sure, the best we can say is that there is a $ 90 \ % $ chance that it is likely to be sunny and $ 10 \ % $ that it will be rainy. q2 : what about two days from today? a2 : one day prediction : $ 90 \ % $ sunny, $ 10 \ % $ rainy. therefore, two days from now : first day it can be sunny and the next day also it can be sunny. chances of this happening are : $ 0. 9 \ times 0. 9 $. or first day it can be rainy and second day it can be sunny. chances of this happening are : $ 0. 1 \ times 0. 5 $. therefore, the probability that the weather will be sunny in two days is : $ p ( \ text { sunny 2 days from now } = 0. 9 \ times 0. 9 + 0. 1 \ times 0. 5 = 0. 81 +


0. 05 = 0. 86 $ similarly, the probability that it will be rainy is : $ p ( \ text { rainy 2 days from now } = 0. 1 \ times 0. 5 + 0. 9 \ times 0. 1 = 0. 05 + 0. 09 = 0. 14 $ in linear algebra ( transition matrices ) these calculations correspond to all the permutations in transitions from one step to the next ( sunny - to - sunny ( $ s _ 2s $ ), sunny - to - rainy ( $ s _ 2r $ ), rainy - to - sunny ( $ r _ 2s $ ) or rainy - to - rainy ( $ r _ 2r $ ) ) with their calculated probabilities : on the lower part of the image we see how to calculate the probability of a future state ( $ t + 1 $ or $ t + 2 $ ) given the probabilities ( probability mass function, $ pmf $ ) for every state ( sunny or rainy ) at time zero ( now or $ t _ 0 $ ) as simple matrix multiplication. if you keep forecasting weather like this you will notice that eventually the $ n $ - th day forecast, where $ n $ is very large ( say $ 30 $ ), settles to the following'equilibrium'probabilities : $ p ( \ text { sunny } ) = 0. 833 $ and $ p ( \ text { rainy } ) = 0. 167 $ in other words, your forecast for the $ n $ - th day and the $ n + 1 $ - th day remain the same. in addition, you can also check that the'equilibrium'probabilities do not depend on the weather today. you would get the same forecast for the weather if you start of by assuming that the weather today is sunny or rainy. the above example will only work if the state transition probabilities satisfy several conditions which i will not discuss here. but, notice the following features of this'nice'markov chain ( nice = transition probabilities satisfy conditions ) : irrespective of the initial starting state we will eventually reach an equilibrium probability distribution of states. markov chain monte carlo exploits the above feature as follows : we want to generate random draws from a target distribution. we then identify a way to construct a'nice'markov chain such that its equilibrium probability distribution is our target distribution. if we can construct such a chain then we arbitrarily start from some point and iterate the markov


chain many times ( like how we forecast the weather $ n $ times ). eventually, the draws we generate would appear as if they are coming from our target distribution. we then approximate the quantities of interest ( e. g. mean ) by taking the sample average of the draws after discarding a few initial draws which is the monte carlo component. there are several ways to construct'nice'markov chains ( e. g., gibbs sampler, metropolis - hastings algorithm ).


in a previous answer in the theoretical computer science site, i said that category theory is the " foundation " for type theory. here, i would like to say something stronger. category theory is type theory. conversely, type theory is category theory. let me expand on these points. category theory is type theory in any typed formal language, and even in normal mathematics using informal notation, we end up declaring functions with types $ f : a \ to b $. implicit in writing that is the idea that $ a $ and $ b $ are some things called " types " and $ f $ is a " function " from one type to another. category theory is the algebraic theory of such " types " and " functions ". ( officially, category theory calls them " objects " and " morphisms " so as to avoid treading on the set - theoretic toes of the traditionalists, but increasingly i see category theorists throwing such caution to the wind and using the more intuitive terms : " type " and " function ". but, be prepared for protests from the traditionalists when you do so. ) we have all been brought up on set theory from high school onwards. so, we are used to thinking of types such as $ a $ and $ b $ as sets, and functions such as $ f $ as set - theoretic mappings. if you never thought of them that way, you are in good shape. you have escaped set - theoretic brain - washing. category theory says that there are many kinds of types and many kinds of functions. so, the idea of types as sets is limiting. instead, category theory axiomatizes types and functions in an algebraic way. basically, that is what category theory is. a theory of types and functions. it does get quite sophisticated, involving high levels of abstraction. but, if you can learn it, you will acquire a deep understanding of types and functions. type theory is category theory by " type theory, " i mean any kind of typed formal language, based on rigid rules of term - formation which make sure that everything type checks. it turns out that, whenever we work in such a language, we are working in a category - theoretic structure. even if we use set - theoretic notations and think set - theoretically, still we end up writing stuff that makes sense categorically. that is an amazing fact. historically, dana scott may have been the first to realize this. he worked on producing semantic models of programming languages based on


typed ( and untyped ) lambda calculus. the traditional set - theoretic models were inadequate for this purpose, because programming languages involve unrestricted recursion which set theory lacks. scott invented a series of semantic models that captured programming phenomena, and came to the realization that typed lambda calculus exactly represented a class of categories called cartesian closed categories. there are plenty of cartesian closed categories that are not " set - theoretic ". but typed lambda calculus applies to all of them equally. scott wrote a nice essay called " relating theories of lambda calculus " explaining what is going on, parts of which seem to be available on the web. the original article was published in a volume called " to h. b. curry : essays on combinatory logic, lambda calculus and formalism ", academic press, 1980. berry and curien came to the same realization, probably independently. they defined a categorical abstract machine ( cam ) to use these ideas in implementing functional languages, and the language they implemented was called " caml " which is the underlying framework of microsoft's f #. standard type constructors like $ \ times $, $ \ to $, $ list $ etc. are functors. that means that they not only map types to types, but also functions between types to functions between types. polymorphic functions preserve all such functions resulting from functor actions. category theory was invented in 1950's by eilenberg and maclane precisely to formalize the concept of polymorphic functions. they called them " natural transformations ", " natural " because they are the only ones that you can write in a type - correct way using type variables. so, one might say that category theory was invented precisely to formalize polymorphic programming languages, even before programming languages came into being! a set - theoretic traditionalist has no knowledge of the functors and natural transformations that are going on under the surface when he uses set - theoretic notations. but, as long as he is using the type system faithfully, he is really doing categorical constructions without being aware of them. all said and done, category theory is the quintessential mathematical theory of types and functions. so, all programmers can benefit from learning a bit of category theory, especially functional programmers. unfortunately, there do not seem to be any text books on category theory targeted at programmers specifically. the " category theory for computer science " books are typically targeted at theoretical computer science students / researchers. the book by benjamin pierce, basic category theory


for computer scientists is perhaps the most readable of them. however, there are plenty of resources on the web, which are targeted at programmers. the haskellwiki page can be a good starting point. at the midlands graduate school, we have lectures on category theory ( among others ). graham hutton's course was pegged as a " beginner " course, and mine was pegged as an " advanced " course. but both of them cover essentially the same content, going to different depths. university of chalmers has a nice resource page on books and lecture notes from around the world. the enthusiastic blog site of " sigfpe " also provides a lot of good intuitions from a programmer's point of view. the basic topics you would want to learn are : definition of categories, and some examples of categories functors, and examples of them natural transformations, and examples of them definitions of products, coproducts and exponents ( function spaces ), initial and terminal objects. adjunctions monads, algebras and kleisli categories my own lecture notes in the midlands graduate school covers all these topics except for the last one ( monads ). there are plenty of other resources available for monads these days. so that is not a big loss. the more mathematics you know, the easier it would be to learn category theory. because category theory is a general theory of mathematical structures, it is helpful to know some examples to appreciate what the definitions mean. ( when i learnt category theory, i had to make up my own examples using my knowledge of programming language semantics, because the standard text books only had mathematical examples, which i didn't know anything about. ) then came the brilliant book by lambek and scott called " introduction to categorical logic " which related category theory to type systems ( what they call " logic " ). it is now possible to understand category theory just by relating it to type systems even without knowing a lot of examples. a lot of the resources i mentioned above use this approach to explain category theory.


here is a scatterplot of some multivariate data ( in two dimensions ) : what can we make of it when the axes are left out? introduce coordinates that are suggested by the data themselves. the origin will be at the centroid of the points ( the point of their averages ). the first coordinate axis ( blue in the next figure ) will extend along the " spine " of the points, which ( by definition ) is any direction in which the variance is the greatest. the second coordinate axis ( red in the figure ) will extend perpendicularly to the first one. ( in more than two dimensions, it will be chosen in that perpendicular direction in which the variance is as large as possible, and so on. ) we need a scale. the standard deviation along each axis will do nicely to establish the units along the axes. remember the 68 - 95 - 99. 7 rule : about two - thirds ( 68 % ) of the points should be within one unit of the origin ( along the axis ) ; about 95 % should be within two units. that makes it easy to eyeball the correct units. for reference, this figure includes the unit circle in these units : that doesn't really look like a circle, does it? that's because this picture is distorted ( as evidenced by the different spacings among the numbers on the two axes ). let's redraw it with the axes in their proper orientations - - left to right and bottom to top - - and with a unit aspect ratio so that one unit horizontally really does equal one unit vertically : you measure the mahalanobis distance in this picture rather than in the original. what happened here? we let the data tell us how to construct a coordinate system for making measurements in the scatterplot. that's all it is. although we had a few choices to make along the way ( we could always reverse either or both axes ; and in rare situations the directions along the " spines " - - the principal directions - - are not unique ), they do not change the distances in the final plot. technical comments ( not for grandma, who probably started to lose interest as soon as numbers reappeared on the plots, but to address the remaining questions that were posed. ) unit vectors along the new axes are the eigenvectors ( of either the covariance matrix or its inverse ). we noted that undistorting the ellipse to make a circle divides the distance along each eigenvector by the standard deviation :


the square root of the covariance. letting $ c $ stand for the covariance function, the new ( mahalanobis ) distance between two points $ x $ and $ y $ is the distance from $ x $ to $ y $ divided by the square root of $ c ( x - y, x - y ) $. the corresponding algebraic operations, thinking now of $ c $ in terms of its representation as a matrix and $ x $ and $ y $ in terms of their representations as vectors, are written $ \ sqrt { ( x - y )'c ^ { - 1 } ( x - y ) } $. this works regardless of what basis is used to represent vectors and matrices. in particular, this is the correct formula for the mahalanobis distance in the original coordinates. the amounts by which the axes are expanded in the last step are the ( square roots of the ) eigenvalues of the inverse covariance matrix. equivalently, the axes are shrunk by the ( roots of the ) eigenvalues of the covariance matrix. thus, the more the scatter, the more the shrinking needed to convert that ellipse into a circle. although this procedure always works with any dataset, it looks this nice ( the classical football - shaped cloud ) for data that are approximately multivariate normal. in other cases, the point of averages might not be a good representation of the center of the data or the " spines " ( general trends in the data ) will not be identified accurately using variance as a measure of spread. the shifting of the coordinate origin, rotation, and expansion of the axes collectively form an affine transformation. apart from that initial shift, this is a change of basis from the original one ( using unit vectors pointing in the positive coordinate directions ) to the new one ( using a choice of unit eigenvectors ). there is a strong connection with principal components analysis ( pca ). that alone goes a long way towards explaining the " where does it come from " and " why " questions - - if you weren't already convinced by the elegance and utility of letting the data determine the coordinates you use to describe them and measure their differences. for multivariate normal distributions ( where we can carry out the same construction using properties of the probability density instead of the analogous properties of the point cloud ), the mahalanobis distance ( to the new origin ) appears in place of the " $ x $ " in the expression $ \ ex


##p ( - \ frac { 1 } { 2 } x ^ 2 ) $ that characterizes the probability density of the standard normal distribution. thus, in the new coordinates, a multivariate normal distribution looks standard normal when projected onto any line through the origin. in particular, it is standard normal in each of the new coordinates. from this point of view, the only substantial sense in which multivariate normal distributions differ among one another is in terms of how many dimensions they use. ( note that this number of dimensions may be, and sometimes is, less than the nominal number of dimensions. )


you are right. notice that the term $ o ( n + m ) $ slightly abuses the classical big - o notation, which is defined for functions in one variable. however there is a natural extension for multiple variables. simply speaking, since $ $ \ frac { 1 } { 2 } ( m + n ) \ le \ max \ { m, n \ } \ le m + n \ le 2 \ max \ { m, n \ }, $ $ you can deduce that $ o ( n + m ) $ and $ o ( \ max \ { m, n \ } ) $ are equivalent asymptotic upper bounds. on the other hand $ o ( n + m ) $ is different from $ o ( \ min \ { n, m \ } ) $, since if you set $ n = 2 ^ m $, you get $ $ o ( 2 ^ m + m ) = o ( 2 ^ m ) \ supsetneq o ( m ) = o ( \ min \ { 2 ^ m, m \ } ). $ $


sometimes we can " augment knowledge " with an unusual or different approach. i would like this reply to be accessible to kindergartners and also have some fun, so everybody get out your crayons! given paired $ ( x, y ) $ data, draw their scatterplot. ( the younger students may need a teacher to produce this for them. : - ) each pair of points $ ( x _ i, y _ i ) $, $ ( x _ j, y _ j ) $ in that plot determines a rectangle : it's the smallest rectangle, whose sides are parallel to the axes, containing those points. thus the points are either at the upper right and lower left corners ( a " positive " relationship ) or they are at the upper left and lower right corners ( a " negative " relationship ). draw all possible such rectangles. color them transparently, making the positive rectangles red ( say ) and the negative rectangles " anti - red " ( blue ). in this fashion, wherever rectangles overlap, their colors are either enhanced when they are the same ( blue and blue or red and red ) or cancel out when they are different. ( in this illustration of a positive ( red ) and negative ( blue ) rectangle, the overlap ought to be white ; unfortunately, this software does not have a true " anti - red " color. the overlap is gray, so it will darken the plot, but on the whole the net amount of red is correct. ) now we're ready for the explanation of covariance. the covariance is the net amount of red in the plot ( treating blue as negative values ). here are some examples with 32 binormal points drawn from distributions with the given covariances, ordered from most negative ( bluest ) to most positive ( reddest ). they are drawn on common axes to make them comparable. the rectangles are lightly outlined to help you see them. this is an updated ( 2019 ) version of the original : it uses software that properly cancels the red and cyan colors in overlapping rectangles. let's deduce some properties of covariance. understanding of these properties will be accessible to anyone who has actually drawn a few of the rectangles. : - ) bilinearity. because the amount of red depends on the size of the plot, covariance is directly proportional to the scale on the x - axis and to


the scale on the y - axis. correlation. covariance increases as the points approximate an upward sloping line and decreases as the points approximate a downward sloping line. this is because in the former case most of the rectangles are positive and in the latter case, most are negative. relationship to linear associations. because non - linear associations can create mixtures of positive and negative rectangles, they lead to unpredictable ( and not very useful ) covariances. linear associations can be fully interpreted by means of the preceding two characterizations. sensitivity to outliers. a geometric outlier ( one point standing away from the mass ) will create many large rectangles in association with all the other points. it alone can create a net positive or negative amount of red in the overall picture. incidentally, this definition of covariance differs from the usual one only by a constant of proportionality. the mathematically inclined will have no trouble performing the algebraic demonstration that the formula given here is always twice the usual covariance. for a full explanation, see the follow - up thread at


first off, may i say that i applaud your decision to test this through an experiment. it's rare to see that than i would like. now, on to the matter at hand. it's fairly well known from industrial chemistry that non - polar solvents degrade latex quite heavily. i work with latex seals a lot, and the hexanes we use routinely break the seals down in under a day. of course, if you're lubricating your condoms with hexanes, you're a ) an idiot or b ) absolutely insane. a paper i managed to find suggests that there really isn't too much direct data on condoms, and it muses that the warnings might have arisen from industry, where nonpolar solvents decidedly do degrade latex. to find out, they did a burst experiment with condoms that had been treated with various oils. glycerol and vaseline - treated condoms showed a very, very minor decrease in strength, while mineral oil / baby oil - treated ones burst at less than 10 % of the volume of an untreated condom. they also found that 10 month - old condoms have half the burst volume of 1 - month old ones, so you could argue that using 1 - month - old condoms that have been slathered in vaseline is still much safer than using older ones. as for the actual chemistry of the weakening, i honestly don't know. if i were to hazard a guess, i would note that the latex looks like a bunch of ethylenes glued together, so my guess would be that the solvents get between the chains and force them apart, weakening them. for this to happen, the solvent must be nonpolar, but still small enough to slip between the chains of the polymer. that's probably why vaseline and canola oil don't have much of an effect - - - they're just too big to fit between the chains. again though, i don't know for sure, so don't quote me on this last paragraph.


i managed to crack the formula for optical isomers with odd chiral centers, so i'll share my attempt here. hopefully others may innovate on it and post solutions for other formulae. pseudo - chiral carbon atoms - an introduction the gold book defines pseudo - chiral / pseudo - asymmetric carbon atom as : a tetrahedrally coordinated carbon atom bonded to four different entities, two and only two of which have the same constitution but opposite chirality sense. this implies that, in your case : if chiral carbons 2 and 4 both have configuration r ( or both s ), then the central carbon 3 will be achiral / symmetric, because now " two and only two of its groups which have the same constitution " will have the same chirality sense instead. ( your approach by " plane of symmetry " is wrong. find more details on this question ) hence, there can be two stereoisomers ( r and s ) possible on the 3rd carbon due to its pesudochirality. but, there will be only one if both substituents on left and right have the same optical configurations. building up an intuition by manual counting for optical isomers with odd number of chiral centers and similar ends, you can guess that, if there are $ n $ chiral centers, then the middle ( $ \ frac { n + 1 } 2 $ - th ) carbon atom will be pseudo - chiral. to build up an intuition, we'll manually count optical isomers for $ n = 3 $ and $ n = 5 $ : case $ n = 3 $ take the example of pentane - 2, 3, 4 - triol itself. we find four ( = $ 2 ^ { n - 1 } $ ) isomers : $ $ \ begin { array } { | c | c | c | } \ hline \ text { c2 } & \ text { c3 } & \ text { c4 } \ \ \ hline r & - & r \ \ \ hline s & - & s \ \ \ hline r & s & r \ \ \ hline r & s & s \ \ \ hline \ end { array } $ $ as expected from the relevant formula, we find that the first two ( $ = 2 ^ \ frac { n - 1 } 2 $ ) are meso compounds, and the remaining two ( $ = 2 ^ { n - 1 } - 2 ^ \ frac


{ n - 1 } 2 $ ) are enantiomers. case $ n = 5 $ take the example of heptane - 2, 3, 4, 5, 6 - pentol : we expect $ 16 ~ ( = 2 ^ { n - 1 } ) $ isomers, with the c4 carbon being pseudo - chiral. to avoid a really large table, we observe that the number of meso isomers is easily countable ( < < number of enantiomers ). here is a table of those four ( = $ 2 ^ \ frac { n - 1 } 2 $ ) meso isomers : $ $ \ begin { array } { | c | c | c | c | c | c | } \ hline \ text { c2 } & \ text { c3 } & \ text { c4 } & \ text { c5 } & \ text { c6 } \ \ \ hline r & r & - & r & r \ \ \ hline r & s & - & s & r \ \ \ hline s & r & - & r & s \ \ \ hline s & s & - & s & s \ \ \ hline \ end { array } $ $ note that the total optical isomers are given by $ 2 ^ { n - 1 } $ isomers ( more on that below ). hence, the number of enantiomers is easily $ 12 ( = 2 ^ { n - 1 } - 2 ^ \ frac { n - 1 } 2 ) $. a formula for the number of meso isomers as you must have observed from the table, the sequence of optical configurations, when read from the fourth carbon atom, is exactly the same towards both left and right. in other words, if we fix an arbitrary permutation for the optical configurations of the carbon atoms on the left ( say rss ), then we will get only one unique permutation of the optical configurations on the right ( ssr ). we know that each carbon on the left has two choices ( r or s ), and there are $ \ frac { n - 1 } { 2 } $ carbon atoms on the left. hence, the total number of permutations will be $ 2 \ times2 \ times2 \ cdots \ frac { n - 1 } { 2 } \ text { times } = 2 ^ \ frac { n - 1 } { 2 } $. since, our


description ( " the sequence of optical configurations, when read from the fourth carbon atom, is exactly the same on both left and right " ) describes meso isomers, we have hence counted the number of meso isomers, which is $ 2 ^ \ frac { n - 1 } { 2 } $. a formula for the number of total isomers we note that there are $ n $ chiral carbons ( including that pseudo chiral carbon ). again, each chiral carbon has $ 2 $ choices. hence, the maximum possible number of optical isomers is $ 2 \ times2 \ times2 \ cdots n \ text { times } = 2 ^ n $. this is the maximum possible, not the actual total number of isomers, which is much lower. the reduction in number of isomers is because the string of optical configurations reads exactly the same from either terminal carbon. example : rssrs is the same as srssr. this happens because the compound has " similar ends " hence, each permutation has been over counted exactly twice. thus, the actual total number of isomers is half of the maximum possible, and is $ = \ frac { 2 ^ n } 2 = 2 ^ { n - 1 } $. conclusion hence, we have derived that, if'n'( number of chiral centers ) is odd for a compound with similar ends, then : $ \ text { number of meso isomers } = 2 ^ { ( n - 1 ) / 2 } $ $ \ text { total number of optical isomers } = 2 ^ { n - 1 } $ $ \ text { number of enantiomers } = 2 ^ { n - 1 } - 2 ^ { ( n - 1 ) / 2 } $


the ancient greeks had a theory that the sun, the moon, and the planets move around the earth in circles. this was soon shown to be wrong. the problem was that if you watch the planets carefully, sometimes they move backwards in the sky. so ptolemy came up with a new idea - the planets move around in one big circle, but then move around a little circle at the same time. think of holding out a long stick and spinning around, and at the same time on the end of the stick there's a wheel that's spinning. the planet moves like a point on the edge of the wheel. well, once they started watching really closely, they realized that even this didn't work, so they put circles on circles on circles... eventually, they had a map of the solar system that looked like this : this " epicycles " idea turns out to be a bad theory. one reason it's bad is that we know now that planets orbit in ellipses around the sun. ( the ellipses are not perfect because they're perturbed by the influence of other gravitating bodies, and by relativistic effects. ) but it's wrong for an even worse reason that that, as illustrated in this wonderful youtube video. in the video, by adding up enough circles, they made a planet trace out homer simpson's face. it turns out we can make any orbit at all by adding up enough circles, as long as we get to vary their size and speeds. so the epicycle theory of planetary orbits is a bad one not because it's wrong, but because it doesn't say anything at all about orbits. claiming " planets move around in epicycles " is mathematically equivalent to saying " planets move around in two dimensions ". well, that's not saying nothing, but it's not saying much, either! a simple mathematical way to represent " moving around in a circle " is to say that positions in a plane are represented by complex numbers, so a point moving in the plane is represented by a complex function of time. in that case, moving on a circle with radius $ r $ and angular frequency $ \ omega $ is represented by the position $ $ z ( t ) = re ^ { i \ omega t } $ $ if you move around on two circles, one at the end of the other, your position is $ $ z ( t ) = r _ 1e ^ { i \ omega _ 1