statistics topological insulators : book recommendations - topological insulators for dummies iron - based superconductors : reference needed for iron - based superconductors soft matter : soft condensed matter book for self - study intermolecular forces : resource for intermolecular forces in soft condensed matter materials science : best materials science introduction book? quantum chemistry : is there any quantum physics book that treats covalent bonding systematically? particle physics popular : good book about elementary particles for high school students? general : books for particle physics and the standard model experimental : enlightening experimental physics books / resources detectors : reference for solid state particle detector data analysis : textbook about the handiwork of a hep analysis? heavy ion collisions : reference on stages of heavy ion collisions in particle physics theories of everything : what is a good non - technical introduction to theories of everything? quantum field theory background : textbook on group theory to be able to start qft basics : a no - nonsense introduction to quantum field theory relativistic qm : any suggestion for a book that includes quantum mechanics principles and smoothly introduces you to qed ( quantum electrodynamics )? introductory : what is a complete book for introductory quantum field theory? lectures : online qft video lectures s - matrix theory : materials about $ s $ - matrix and $ s $ - matrix theory renormalization : are there books on regularization and renormalization in qft at an introductory level? renormalization ( in general ) : suggested reading for renormalization ( not only in qft ) for mathematicians : quantum field theory from a mathematical point of view rigorous / axiomatic : rigorous approaches to quantum field theory algebraic qft : which are some best sources to learn algebraic quantum field theory ( aqft )? topological field theory : reading list in topological qft nonperturbative : books on non - perturbative phenomena in quantum field theory curved spacetime : suggested reading for quantum field theory in curved spacetime curved spacetime ( advanced ) : modern treatment of effective qft in curved spacetime general relativity introductory : books for general relativity mathematical : mathematically - oriented treatment of general relativity exercises : recommendation on books with problems for general relativity? exact solutions : a book containing a large subset of known exact solutions to the efes high energy theory string theory ( introductory ) : introduction to string theory string theory ( advanced ) : advanced topics in string theory string theory ( matrix ) : good introductory text for matrix string theory supersymmetry ( with


exercises ) : problems book recommendation on supersymmetry, supergravity and superstring theory kahler manifolds : kahler and complex manifolds conformal field theory : reading list and book recommendation on conformal field theory conformal bootstrap : looking for intro to conformal bootstrap ads / cft : introduction to ads / cft integrability : what is a good introduction to integrable models in physics? entanglement entropy : quantum field theory text on entanglement entropy twistors : gentle introduction to twistors loop quantum gravity : lqg demystified book? quantum gravity in general : obligated bibliography for quantum gravity miscellaneous free : list of freely available physics books lecture notes : best sets of physics lecture notes and articles historical : physics history book with some math acoustics : books about musical acoustics chemistry : where should a physicist go to learn chemistry? biophysics : what are good references for learning about biophysics at graduate level? computational : textbook recommendation for computational physics experimental : what's a good book on experimental methods for physics? plasma physics : book suggestion for introductory plasma physics problems olympiad : best physics olympiad resources graduate exams : graduate physics problems books puzzles site : is there a physics puzzles site like project euler?


in no particular order : algebraic number theory notes by sharifi : dalawat's first course in local arithmetic : intro to top grps : representation theory resources : classical invariant theory : cring project : - the notes are huge & has many authors - including mse's zev, akhil ( no longer active ) & darij. check the toc. partitions bijections, a survey : hidden subgroup problem ( review, open stuff ) : spirit of moonshine : vertex operator algebras and modular forms : categorified algebra & quantum mechanics : exponential sums over finite fields : gauss sums : adeles over $ \ bbb q $ : followed by automo reps over gl ( 1, a ) invariant thry : species : flt : categorical concepts : groups, rings, fields ( lenstra ) : which is part of algebra notes : if we're going to mention hatcher ( famous to me for the algebraic topology notes ), we might as well also mention a few other books that are online, like algebra chapter 0, stanley's insane first volume of enumerative combinatorics ( which reminds me : generatingfunctionology ). also i don't see topology without tears mentioned. the sheer number of books and notes on differential geometry and lie theory is mind - boggling, so i'll have to update later with the juicier ones. let's not forget the ams notes online back through 1995 - they're very nice reading as well.


there area few different influenza virus database resources : the influenza research database ( ird ) ( a. k. a fludb - based upon url ) a niaid bioinformatics resource center or brc which highly curates the data brought in and integrates it with numerous other relevant data types the ncbi influenza virus resource a sub - project of the ncbi with data curated over and above the genbank data that is part of the ncbi the gisaid epiflu database a database of sequences from the global initiative on sharing all influenza data. has unique data from many countries but requires user agree to a data sharing policy. the openfludb former gisaid database that contains some sequence data that genbank does not have. for those who also may be interested in other virus databases, there are : virus pathogen resource ( vipr ) a companion portal to the ird, which hosts curated and integrated data for most other niaid a - c virus pathogens including ( but not limited to ) ebola, zika, dengue, enterovirus, and hepatitis c lanl hiv database los alamos national laboratory hiv database with hiv data and many useful tools for all virus bioinformatics pave : papilloma virus genome database ( from quintik comment ) niaid developed and maintained papilloma virus bioinformatics portal disclaimer : i used to work for the ird / vipr and currently work for niaid.


the linear model is written as $ $ \ left | \ begin { array } { l } \ mathbf { y } = \ mathbf { x } \ mathbf { \ beta } + \ mathbf { \ epsilon } \ \ \ mathbf { \ epsilon } \ sim n ( 0, \ sigma ^ 2 \ mathbf { i } ), \ end { array } \ right. $ $ where $ \ mathbf { y } $ denotes the vector of responses, $ \ mathbf { \ beta } $ is the vector of fixed effects parameters, $ \ mathbf { x } $ is the corresponding design matrix whose columns are the values of the explanatory variables, and $ \ mathbf { \ epsilon } $ is the vector of random errors. it is well known that an estimate of $ \ mathbf { \ beta } $ is given by ( refer, e. g., to the wikipedia article ) $ $ \ hat { \ mathbf { \ beta } } = ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } \ mathbf { x } ^ { \ prime } \ mathbf { y }. $ $ hence $ $ \ textrm { var } ( \ hat { \ mathbf { \ beta } } ) = ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } \ mathbf { x } ^ { \ prime } \ ; \ sigma ^ 2 \ mathbf { i } \ ; \ mathbf { x } ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } = \ sigma ^ 2 ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } = \ sigma ^ 2 ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 }, $ $ [ reminder : $ \ textrm { var } ( ax ) = a \ times \ textrm { var } ( x ) \ times a ′ $, for some random vector $ x $ and some non - random matrix $ a $ ] so that $ $ \ widehat


{ \ textrm { var } } ( \ hat { \ mathbf { \ beta } } ) = \ hat { \ sigma } ^ 2 ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 }, $ $ where $ \ hat { \ sigma } ^ 2 $ can be obtained by the mean square error ( mse ) in the anova table. example with a simple linear regression in r # - - - - - - generate one data set with epsilon ~ n ( 0, 0. 25 ) - - - - - - seed < - 1152 # seed n < - 100 # nb of observations a < - 5 # intercept b < - 2. 7 # slope set. seed ( seed ) epsilon < - rnorm ( n, mean = 0, sd = sqrt ( 0. 25 ) ) x < - sample ( x = c ( 0, 1 ), size = n, replace = true ) y < - a + b * x + epsilon # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # - - - - - - using lm - - - - - - mod < - lm ( y ~ x ) # - - - - - - - - - - - - - - - - - - - - # - - - - - - using the explicit formulas - - - - - - x < - cbind ( 1, x ) betahat < - solve ( t ( x ) % * % x ) % * % t ( x ) % * % y var _ betahat < - anova ( mod ) [ [ 3 ] ] [ 2 ] * solve ( t ( x ) % * % x ) # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # - - - - - - comparison - - - - - - # estimate > mod $ coef ( intercept ) x 5. 020261 2. 755577 > c ( betahat [ 1 ], betahat [ 2 ] ) [ 1 ] 5. 020261 2. 755577 #


standard error > summary ( mod ) $ coefficients [, 2 ] ( intercept ) x 0. 06596021 0. 09725302 > sqrt ( diag ( var _ betahat ) ) x 0. 06596021 0. 09725302 # - - - - - - - - - - - - - - - - - - - - - - when there is a single explanatory variable, the model reduces to $ $ y _ i = a + bx _ i + \ epsilon _ i, \ qquad i = 1, \ dotsc, n $ $ and $ $ \ mathbf { x } = \ left ( \ begin { array } { cc } 1 & x _ 1 \ \ 1 & x _ 2 \ \ \ vdots & \ vdots \ \ 1 & x _ n \ end { array } \ right ), \ qquad \ mathbf { \ beta } = \ left ( \ begin { array } { c } a \ \ b \ end { array } \ right ) $ $ so that $ $ ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } = \ frac { 1 } { n \ sum x _ i ^ 2 - ( \ sum x _ i ) ^ 2 } \ left ( \ begin { array } { cc } \ sum x _ i ^ 2 & - \ sum x _ i \ \ - \ sum x _ i & n \ end { array } \ right ) $ $ and formulas become more transparant. for example, the standard error of the estimated slope is $ $ \ sqrt { \ widehat { \ textrm { var } } ( \ hat { b } ) } = \ sqrt { [ \ hat { \ sigma } ^ 2 ( \ mathbf { x } ^ { \ prime } \ mathbf { x } ) ^ { - 1 } ] _ { 22 } } = \ sqrt { \ frac { n \ hat { \ sigma } ^ 2 } { n \ sum x _ i ^ 2 - ( \ sum x _ i ) ^ 2 } }. $ $ > num < - n * anova ( mod ) [ [ 3 ] ] [ 2 ] > denom < - n * sum ( x ^ 2 ) - sum ( x ) ^ 2 > sqrt ( num / denom ) [ 1 ] 0. 09


it ’ s a matter of preference i guess but i recommend the ensembl builds. decide whether you want the toplevel or primary assembly, and whether you want soft - masked, repeat - masked or unmasked files. the naming schema is very straightforward ; the combinations are described in the readme file, and all files reside in one directory. for example, if you want the unmasked primary assembly, the file to download would be homo _ sapiens. grch37. 75. dna. primary _ assembly. fa. gz. as for goldenpath / ucsc, there ’ s no need to download and concatenate separate chromosomes ( contrary to what the other answer said ) ; you can download the whole ( toplevel ) reference from the bigzips directory ; from the readme : this directory contains the feb. 2009 assembly of the human genome ( hg19, grch37 genome reference consortium human reference 37 ( gca _ 000001405. 1 ) ), as well as repeat annotations and genbank sequences. there are essentially three options here : chromfa. tar. gz, which contains the whole genome in one chromosome per file ; chromfamasked. tar. gz, the same with repeats masked by n ; hg19. 2bit, which is the whole genome in one file, but needs to be extracted using the utility program twobittofa, which needs to be downloaded separately. in any case, i always download the reference and build my own index for mapping, since this allows me more control ; not everybody might need this much control, but then building the index once is fairly fast anyway.


let's denote the sigmoid function as $ \ sigma ( x ) = \ dfrac { 1 } { 1 + e ^ { - x } } $. the derivative of the sigmoid is $ \ dfrac { d } { dx } \ sigma ( x ) = \ sigma ( x ) ( 1 - \ sigma ( x ) ) $. here's a detailed derivation : $ $ \ begin { align } \ dfrac { d } { dx } \ sigma ( x ) & = \ dfrac { d } { dx } \ left [ \ dfrac { 1 } { 1 + e ^ { - x } } \ right ] \ \ & = \ dfrac { d } { dx } \ left ( 1 + \ mathrm { e } ^ { - x } \ right ) ^ { - 1 } \ \ & = - ( 1 + e ^ { - x } ) ^ { - 2 } ( - e ^ { - x } ) \ \ & = \ dfrac { e ^ { - x } } { \ left ( 1 + e ^ { - x } \ right ) ^ 2 } \ \ & = \ dfrac { 1 } { 1 + e ^ { - x } \ } \ cdot \ dfrac { e ^ { - x } } { 1 + e ^ { - x } } \ \ & = \ dfrac { 1 } { 1 + e ^ { - x } \ } \ cdot \ dfrac { ( 1 + e ^ { - x } ) - 1 } { 1 + e ^ { - x } } \ \ & = \ dfrac { 1 } { 1 + e ^ { - x } \ } \ cdot \ left ( \ dfrac { 1 + e ^ { - x } } { 1 + e ^ { - x } } - \ dfrac { 1 } { 1 + e ^ { - x } } \ right ) \ \ & = \ dfrac { 1 } { 1 + e ^ { - x } \ } \ cdot \ left ( 1 - \ dfrac { 1 } { 1 + e ^ { - x } } \ right ) \ \ & = \ sigma ( x ) \ cdot ( 1 - \ sigma ( x ) ) \ end { align } $ $


tldr : harmony can work on 106 ~ cell samples but has less versatility then methods like batman. batman is useful if you need your data for differential gene expression and single - cell eqtl and can work on ~ 200 cell data. well their certainly is a minimum which comes down to several factors. as many of these rely on algorithms like knn and other clustering algorithms for integration the threshold is dependent on the quality of the data and the algorithms abilities to predict reasonable clusters from that data. as well as the downstream uses of the data... batman highest performance on simulated set 200 cells and 1000 cells with 200 cells from one batch and and 1000 in the other only batman succeeded in clustering the high dimensionality simulated data well. if you need your data for differential gene expression and single - cell eqtl, batman seems to be the correct choice, for small sets. " the downside of these methods is that they operate in latent space, which limits their interpretability and use in downstream analyses such as differential gene expression and single - cell eqtl analyses. " " a shows that the two original datasets become better mixed with each other. batman has the highest performance not only when considering the top two principal components but also when considering more top principal components. it is the only method that manages to efficiently maintain a high ilisi score in a larger number of dimensions. " harmony is best if... differential gene expression and single - cell eqtl is not necessary " moreover, we show that harmony requires dramatically fewer computational resources. it is the only available algorithm that makes the integration of ~ 106 cells feasible on a personal computer. " computing time benchmarks - section it seems as the runtime of these algorithms is less than 2 minutes, using a set with less than ~ 2000 cells will not benefit your computational time much. " when processing a small data set of ~ 2000 cells, all four methods took less than 2 min. " " to obtain such datasets, we downsampled the mca and tm datasets to obtain a total of 9 sets of data containing between ~ 2000 and ~ 140, 000 cells, while the number of highly variable genes ( hvgs ) was controlled in a range from ~ 2000 to ~ 3000 ( table s1 ). "


short answer : the exterior derivative acts on differential forms ; the lie derivative acts on any tensors and some other geometric objects ( they have to be natural, e. g. a connection, see the paper of p. petersen below ) ; both the exterior and the lie derivatives don't require any additional geometric structure : they rely on the differential structure of the manifold ; the covariant derivative needs a choice of connection which sometimes ( e. g. in a presence of a semi - riemannian metric ) can be made canonically ; there are relationships between these derivatives. for a longer answer i would suggest the following selection of papers t. j. willmore, the definition of lie derivative r. palais, a definition of the exterior derivative in terms of lie derivatives p. petersen, the ricci and bianchi identities of course, there is a lot more to say. edit. i decided to extend my answer as i believe that there are some essential points which have not been discussed yet. an encyclopedic reference that treats all these derivatives concurrently at a modern level of generality is i. kolar, p. w. michor, j. slovak, natural operations in differential geometry ( springer 1993 ), freely available online here. i would not even dare to summarize this resource since it has an abysmal deepness and all - round completeness, and indeed covers all the parts of the original question. moreover, i believe that the bibliography list of this book contains almost any further relevant reference. as it has been already mentioned by many in this discussion, these operations are intimately related. it cannot be overemphasized that the most important feature that they all share is naturality ( they commute with pullback, and this, in particular, makes them coordinate - free ). see kms cited above and its bibliography, and specifically the following references may be useful : r. palais, natural operations on differential forms, e. g. here or here. c. l. terng, natural vector bundles and natural differential operators, e. g. here it turns out that their naturality forces them to be unique if we impose on them some basic properties, such as $ d \ circ d = 0 $ for the exterior derivative. one way to prove that and further references could be found in : d. krupka, v. mikolasova, on the uniqueness of some differential invariants : $ d $, $ [, ] $, $ \ na


##bla $, see here. also it is interesting that the bianchi identities for the connection follow from the naturality and the property $ d \ circ d = 0 $ for the exterior derivative, see ph. delanoe, on bianchi identities, e. g. here. the reference list that i produce here is too far from being complete in any sense. i only would add one classical treatment that i personally used to comprehend some of the fundamental notions related to lie derivatives ( in particular, the lie derivative of a connection! ) : k. yano, the theory of lie derivatives and its applications, freely available here indeed, my comments are speculative and sparse. i wish if this question were answered by someone like p. michor, to be honest : - )


mkl ( from intel ) is optimized for intel processors, and probably has the " upper hand " there in many cases. but it is also " famous " for choosing the " worst " code - paths for amd processors, as described here.


i think one reason is that often one does not want to remember what the variable names really represent. as an example, when we choose to talk about the matrix $ ( a _ { ij } ) $ instead of the matrix $ ( \ mathrm { transitionprobability } _ { ij } ) $, this expresses the important fact that once we have formulated our problem in terms of matrices, it is perfectly safe to forget where the problem came from originally - - in fact, remembering what the matrix " really " describes might only be unnecessary psychological baggage that prevents us from applying all linear - algebraic tools at our disposal. ( as an aside, have you ever seen code written by a mathematician? it very often looks exactly like your first example. )


your informal descriptions of the algorithms is wonderful. i think in both cases the author was trying to come up with the simplest solution they could think of that guaranteed both mutual exclusion and deadlock freedom. neither algorithm is starvation free or fair. [ ed : as pointed out in the comments, both algorithms are starvation free, and peterson's algorithm is also fair ]. dekker's solution was the first mutual exclusion algorithm using just load and store instructions. it was introduced in dijkstra, edsger w. ; " cooperating sequential processes ", in f. genuys, ed., programming languages : nato advanced study institute, pp. 43 - 112, academic press, 1968. if you read through the paper you see dijkstra work through a number of attempts, recognizing the problem with each, and then adding a little bit more for the next version. part of the inefficiency of his algorithm comes from the fact that he starts with a turn - taking algorithm and then tries to modify it to allow the processes to progress in any order. ( not just 0, 1, 0, 1,... ) peterson's algorithm was published in 1981, after more than a decade of experience and hindsight about dekker's algorithm. peterson wanted a much simpler algorithm than dekker so that the proof of correctness is much easier. you can see that he was feeling some frustration with the community from the title of his paper. peterson, g. l. ; " myths about the mutual exclusion problem, " inf. proc. lett., 12 ( 3 ) : 115 - 116, 1981. very quick read and very well written. ( and the snide remarks about formal methods are priceless. ) peterson's paper also discusses the process by which he built his solution from simpler attempts. ( since his solution is simpler, it required fewer intermediate steps. ) note that the main difference ( what you call " dominance " rather than " submissiveness " ) is that because peterson started out fresh ( not from the turn - taking algorithm dijkstra started with ) his wait loop is simpler and more efficient. he realizes that he can just get away with simple looped testing while dijkstra had to backoff and retry each time. i feel i must also mention lamport's classic bakery algorithm paper : lamport, leslie ; " a new solution of dijkstra's concurrent programming problem ", comm acm 17 ( 8 ) : 453 - 45


##5, 1974. the bakery algorithm is arguably simpler than dekker's algorithm ( and certainly simpler in the case of more than 2 processors ), and is specifically designed to be fault tolerant. i specifically mention it for two reasons. first, because it gives a little bit of history about the definition of the mutual exclusion problem and attempts to solve it up to 1974. second because the bakery algorithm demonstrates that no hardware atomicity is required to solve the mutual exclusion problem. reads that overlap writes to the same location can return any value and the algorithm still works. finally, a particular favorite of mine is lamport, leslie ; " a fast mutual exclusion algorithm, " acm trans. comp. sys., 5 ( 1 ) : 1 - 11, 1987. in this paper lamport was trying to optimize a solution to the mutual exclusion problem in the ( common ) case that there is little contention for the critical section. again, it guarantees mutual exclusion and deadlock freedom, but not fairness. it is ( i believe ) the first mutual exclusion algorithm using only normal reads and writes that can synchronize n processors in o ( 1 ) time when there is no contention. ( when there is contention, it falls back on an o ( n ) test. ) he gives an informal demonstration that the best you can do in the contention free case is seven memory accesses. ( dekker and peterson both do it with 4, but they can only handle 2 processors, when you extend their algorithms to n they have to add an extra o ( n ) accesses. ) in all : i'd say dekker's algorithm itself is interesting mainly from a historical perspective. dijkstra's paper explained the importance of the mutual exclusion problem, and demonstrated that it could be solved. but with many years of hindsight simpler ( and more efficient ) solutions than dekker's have been found.


wow, this one has been over - answered already, i know... but it is such a fun question! so, here's an answer that hasn't been, um, " touched " on yet... : ) you, sir, whatever your age may be ( anyone with kids will know what i mean ), have asked for an answer to one of the deepest questions of quantum mechanics. in the quantum physics dialect of high nerdese, your question boils down to this : why do half - integer spin particles exhibit pauli exclusion - that is, why do they refuse to the be in the same state, including the same location in space, at the same time? you are quite correct that matter as a whole is mostly space. however, the specific example of bound atoms is arguably not so much an example of touching as it is of bonding. it would be the equivalent of a 10 - year - old son not just poking his 12 - year - old sister, but of poking her with superglue on his hand, which is a considerably more drastic offense that i don't think anyone would be much amused by. touching, in contrast, means that you have to push - that is, exert some real energy - into making the two objects contact each other. and characteristically, after that push, the two object remain separate ( in most cases ) and even bound back a bit after the contact is made. so, i think one can argue that the real question behind " what is touching? " is " why do solid objects not want to be compressed when you try to push them together? " if that were not the case, the whole concept of touching sort of falls apart. we would all become at best ghostly entities who cannot make contact with each other, a bit like chihiro as she tries to push haku away during their second meeting in spirited away. now with that as the sharpened version of the query, why do objects such a people not just zip right through each other when they meet, especially since they are ( as noted ) almost entirely made of empty space? now the reflex answer - and it's not a bad one - is likely to be electrical charge. that's because we all know that atoms are positive nuclei surrounded by negatively charged electrons, and that negative charges repel. so, stated that way, it's perhaps not too surprising that, when the outer " edges " of these rather fuzzy atoms get too close, their respective sets of electrons would get


close enough to repel each other. so by this answer, " touching " would simply be a matter of atoms getting so close to each other that their negatively charged clouds of electrons start bumping into each other. this repulsion requires force to overcome, so the the two objects " touch " - reversibly compress each other without merging - through the electric fields that surround the electrons of their atoms. this sounds awfully right, and it even is right... to a limited degree. here's one way to think of the issue : if charge was the only issue involved, then why do some atoms have exactly the opposite reaction when their electron clouds are pushed close to each other? for example, if you push sodium atoms close to chlorine atoms, what you get is the two atoms leaping to embrace each other more closely, with a resulting release of energy that at larger scales is often described by words such as " boom! " so clearly something more than just charge repulsion is going on here, since at least some combinations of electrons around atoms like to nuzzle up much closer to each other instead of farther away. what, then, guarantees that two molecules will come up to each other and instead say " howdy, nice day... but, er, could you please back off a bit, it's getting stuffy? " that general resistance to getting too close turns out to result not so much from electrical charge ( which does still play a role ), but rather from the pauli exclusion effect i mentioned earlier. pauli exclusion is often skipped over in starting texts on chemistry, which may be why issues such as what touching means are also often left dangling a bit. without pauli exclusion, touching - the ability of two large objects to make contact without merging or joining - will always remain a bit mysterious. so what is pauli exclusion? it's just this : very small, very simple particles that spin ( rotate ) in a very peculiar way always, always insist on being different in some way, sort of like kids in large families where everyone wants their unique role or ability or distinction. but particles, unlike people, are very simple things, so they only have a very limited set of options to choose from. when they run out of those simple options, they have only one option left : they need their own bit of space, apart from any other particle. they will then defend that bit of space very fiercely indeed. it is that defense of their own space that leads large collections of electrons


to insist on taking up more and more overall space, as each tiny electron carves out its own unique and fiercely defended bit of turf. particles that have this peculiar type of spin are called fermions, and ordinary matter is made of three main types of fermions : protons, neutrons, and electrons. for the electrons, there is only one identifying feature that distinguishes them from each other, and that is how they spin : counterclockwise ( called " up " ) or clockwise ( called " down " ). you'd think they'd have other options, but that, too, is a deep mystery of physics : very small objects are so limited in the information they carry that they can't even have more than two directions from which to choose when spinning around. however, that one option is very important for understanding that issue of bonding that must be dealt with before atoms can engage in touching. two electrons with opposite spins, or with spins that can be made opposite of each other by turning atoms around the right way, do not repel each other : they attract. in fact, they attract so much that they are an important part of that " boom! " i mentioned earlier for sodium and chlorine, both of which have lonely electrons without spin partners, waiting. there are other factors on how energetic the boom is, but the point is that, until electrons have formed such nice, neat pairs, they don't have as much need to occupy space. once the bonding has happened, however - once the atoms are in arrangements that don't leave unhappy electrons sitting around wanting to engage in close bonds - then the territorial aspect of electrons comes to the forefront : they begin defending their turf fiercely. this defense of turf first shows itself in the ways electrons orbit around atoms, since even there the electrons insist on carving out their own unique and physically separate orbits, after that first pairing of two electrons is resolved. as you can imagine, trying to orbit around an atom while at the same time trying very hard to stay away from other electron pairs can lead to some pretty complicated geometries. and that, too, is a very good thing, because those complicated geometries lead to something called chemistry, where different numbers of electrons can exhibit very different properties due to new electrons being squeezed out into all sorts of curious and often highly exposed outside orbits. in metals, it gets so bad that the outermost electrons essentially become community children that zip around the entire metal crystal instead of sticking to single atoms. that '


s why metals carry heat and electricity so well. in fact, when you look at a shiny metallic mirror, you are looking directly at the fastest - moving of these community - wide electrons. it's also why, in outer space, you have to be very careful about touching two pieces of clean metal to each other, because with all those electrons zipping around, the two pieces may very well decide to bond into a single new piece of metal instead of just touching. this effect is called vacuum welding, and it's an example of why you need to be careful about assuming that solids that make contact will always remain separate. but many materials, such a you and your skin, don't have many of these community electrons, and are instead full of pairs of electrons that are very happy with the situations they already have, thank you. and when these kinds of materials and these kinds of electrons approach, the pauli exclusion effect takes hold, and the electrons become very defensive of their turf. the result at out large - scale level is what we call touching : the ability to make contact without easily pushing through or merging, a large - scale sum of all of those individual highly content electrons defending their small bits of turf. so to end, why do electrons and other fermions want so desperately to have their own bits of unique state and space all to themselves? and why, in every experiment ever done, is this resistance to merger always associated with that peculiar kind of spin i mentioned, a form of spin that is so minimal and so odd that it can't quite be described within ordinary three - dimensional space? we have fantastically effective mathematical models of this effect. it has to do with antisymmetric wave functions. these amazing models are instrumental to things such as the semiconductor industry behind all of our modern electronic devices, as well as chemistry in general, and of course research into fundamental physics. but if you ask the " why " question, that becomes a lot harder. the most honest answer is, i think, " because that is what we see : half - spin particles have antisymmetric wave functions, and that means they defend their spaces. " but linking the two together tightly - something called the spin - statistics problem - has never really been answered in a way that richard feynman would have called satisfactory. in fact, he flatly declared more than once that this ( and several other items in quantum physics ) were still basically mysteries for which we lacked really deep insights into why the universe we know works that way


. and that, sir, is why your question of " what is touching? " touches more deeply on profound mysteries of physics than you may have realized. it's a good question. 2012 - 07 - 01 addendum here is a related answer i did for s. e. chemistry. it touches on many of the same issues, but with more emphasis on why " spin pairing " of electrons allows atoms to share and steal electrons from each other - - that is, it lets them form bonds. it is not a classic textbook explanation of bonding, and i use a lot of informal english words that are not mathematically accurate. but the physics concepts are accurate. my hope is that it can provide a better intuitive feel for the rather remarkable mystery of how an uncharged atom ( e. g. chlorine ) can overcome the tremendous electrostatic attraction of a neutral atom ( e. g. sodium ) to steal one or more of its electrons.


you mention biopython, which contains tests : some of the tests consist in reading files present in the folders listed in the above link. these files could be a starting point for a database of test files. whenever one comes across a test case not covered with these files, one could construct a new test file and contribute it to biopython, along with a test, or at least file an issue : that would be a way to contribute to biopython while constituting a database of test files.


first of all, if you want to understand mapping quality ( mapq ), ignore rna - seq mappers. they often produce misleading mapq because mapq is not important to rna - seq anyway. strictly speaking, you have two questions, one in the title : the meaning of mapq ; and the other in a comment : how mapq is computed. on the meaning, mapq is nearly the same as baseq – the phred scaled probability of the alignment / base being wrong. it often amuses me that we question mapq but take baseq for granted. baseq is also scaled and discretized differently ; even fewer people know how illumina / pacbio / nanopore / historical sequencers estimates baseq. on the second question, section 2 of the maq supplementary explains the theoretical aspects of mapq, which is still correct today. briefly, mapping quality consists of three components : 1 ) the probability of contamination ; 2 ) the effect of mapping heuristics and 3 ) the error due to the repetitiveness of the reference. only 3 ) can be modeled theoretically. in case of bwa - mem, if we assume the matching score is 1, type - 3 error is estimated with : $ $ 10 / \ log10 \ cdot [ \ log4 \ cdot ( s _ 1 - s _ 2 ) - \ log n _ { \ rm sub } ] $ $ where $ s _ 1 $ is the best alignment score, $ s _ 2 $ is the second best and $ n _ { \ rm sub } $ is the number of suboptimal alignments. factor $ \ log 4 $ comes from the scoring matrix. factor $ 10 / \ log10 $ is the phred scale. this equation assumes gap - free alignment and is very close to section 2. 5. 2 in the maq supplementary. it is ok - ish for short reads, but often overestimates for long reads. i am not aware of a practical approach in general cases. in addition to this method, you can estimate mapq by read simulation : just try to find a function such that it fits empirical mapq. some have tried machine learning, too.


the question and the accepted answer are not about k - mer data structure at all, which i will explain in detail below. i will first answer the actual question op intends to ask. the simplest way to keep k - mers is to use an ordinary hash table. the performance is mostly determined by the hash table library. std : : unordered _ map in gcc / clang is one of the worst choices because for integer keys, it is very slow. google dense, ska : : bytell _ hash _ map and ska : : flat _ hash _ map, tsl : : robin _ map and absl : : flat _ hash _ map are much faster. there are a few libraries that focus on smaller footprint, such as google sparse and sparsepp, but those can be a few times slower. in addition to the choice of hash table, how to construct the key is critical. for k < = 32, the right choice is to encode a k - mer with a 64 - bit integer, which will be vastly better than std : : string. memory alignment is also important. in c / c + +, as long as there is one 8 - byte member in a struct, the struct will be 8 - byte aligned on x86 _ 64 by default. most c + + hash table libraries pack key and value in std : : pair. if you use 64 - bit keys and 32 - bit values, std : : pair will be 8 - byte aligned and use 16 bytes, even though only 12 bytes are actually used – 25 % of memory is wasted. in c, we can explicitly define a packed struct with _ _ attribute _ _ ( ( _ _ packed _ _ ) ). in c + +, probably you need to define special key types. a better way to get around memory alignment is to go down to the bit level. for read mapping, for example, we only use 15 – 23bp seeds. then we have 18 ( = 64 - 23 * 2 ) bits left unused. we can use these 18 bits to count k - mers. such bit - level management is quite common. the above is just basic techniques. there are a few other tricks. for example, 1 ) instead of using one hash table, we can use 4096 ( = 2 ^ 12 ) hash tables. then we can store 12 bits of k - mer information into the 4096 part. this gives us invaluable 12 bits


in each bucket to store extra information. this strategy also simplifies parallel k - mer insertions as with a good hash function, it is rare to insert into two tables at the same time. 2 ) when most k - mers are unique, the faster way to count k - mers is to put k - mers in an array and then sort it. sorting is more cache friendly and is faster than hash table lookups. the downside is that sort counting can be memory demanding when most k - mers are highly repetitive. the other answer is spending considerable ( probably the majority of ) time on k - mer iteration, not on hash table operations. the program loops through each position on the sequence and then each k - mer position. for an $ l $ - long sequence, this is an $ o ( kl ) $ algorithm. it has worse theoretical time complexity than hash table operations, which is $ o ( l ) $. although hash table operations are slow due to cache misses, a factor of k = 29 is quite significant. another issue is that all programs in the question and in the other answer are compiled without - o3. adding this option brings the bytell _ hash _ map lookup time from 314s to 34s on my machine. the c program at the end of my post shows the proper way to iterate k - mers. it is an $ o ( l ) $ algorithm with a tiny constant. the program keeps track of both forward and reverse k - mers at the same time and update them with a few bit operations at each sequence position. this echoes my previous comment " you should not reverse complement the whole k - mer ". on the same machine, the program looks up k - mers in 5. 5s using 792mb ram at the peak. this 6 - fold ( = 34 / 5. 5 ) speedup mostly comes from k - mer iteration, given that the hash table library in use is known to have comparable performance to bytell _ hash _ map. # include < stdio. h > # include < stdint. h > # include " khash. h " static inline uint64 _ t hash _ 64 ( uint64 _ t key ) { / / more sophisticated hash function to reduce collisions key = ( ~ key + ( key < < 21 ) ) ; / / key = ( key < < 21 ) - key - 1 ; key = key ^ key > > 24 ; key = ( ( key +


( key < < 3 ) ) + ( key < < 8 ) ) ; / / key * 265 key = key ^ key > > 14 ; key = ( ( key + ( key < < 2 ) ) + ( key < < 4 ) ) ; / / key * 21 key = key ^ key > > 28 ; key = ( key + ( key < < 31 ) ) ; return key ; } khash _ init ( 64, khint64 _ t, int, 1, hash _ 64, kh _ int64 _ hash _ equal ) unsigned char seq _ nt4 _ table [ 128 ] = { / / table to change " acgtn " to 01234 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 4, 1, 4, 4, 4, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 4, 1, 4, 4, 4, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4 } ; static uint64 _ t process _ seq ( khash _ t ( 64 ) * h, int k, int len, char * seq, int is _ ins ) { int i, l ; uint64 _ t x [ 2 ], mask = ( 1ull < < k * 2 ) - 1, shift = ( k - 1 ) * 2, tot = 0 ; for ( i = l = 0, x [ 0 ] = x [ 1 ] = 0 ; i < len ; + + i


) { int c = ( uint8 _ t ) seq [ i ] < 128? seq _ nt4 _ table [ ( uint8 _ t ) seq [ i ] ] : 4 ; if ( c < 4 ) { / / not an " n " base x [ 0 ] = ( x [ 0 ] < < 2 | c ) & mask ; / / forward strand x [ 1 ] = x [ 1 ] > > 2 | ( uint64 _ t ) ( 3 - c ) < < shift ; / / reverse strand if ( + + l > = k ) { / / we find a k - mer uint64 _ t y = x [ 0 ] < x [ 1 ]? x [ 0 ] : x [ 1 ] ; khint _ t itr ; if ( is _ ins ) { / / insert int absent ; itr = kh _ put ( 64, h, y, & absent ) ; if ( absent ) kh _ val ( h, itr ) = 0 ; tot + = + + kh _ val ( h, itr ) ; } else { / / look up itr = kh _ get ( 64, h, y ) ; tot + = itr = = kh _ end ( h )? 0 : kh _ val ( h, k ) ; } } } else l = 0, x [ 0 ] = x [ 1 ] = 0 ; / / if there is an " n ", restart } return tot ; } # include < zlib. h > # include < time. h > # include < unistd. h > # include " kseq. h " kseq _ init ( gzfile, gzread ) int main ( int argc, char * argv [ ] ) { khash _ t ( 64 ) * h ; int i, k = 29 ; while ( ( i = getopt ( argc, argv, " k : " ) ) > = 0 ) if ( i = ='k') k = atoi ( optarg ) ; h = kh _ init ( 64 ) ; for ( i = 1 ; i > = 0 ; - - i ) { uint64 _ t tot = 0 ; kseq _ t * ks ; gzfile fp ; clock _ t t ; fp = gzopen


( argv [ optind ], " r " ) ; ks = kseq _ init ( fp ) ; t = clock ( ) ; while ( kseq _ read ( ks ) > = 0 ) tot + = process _ seq ( h, k, ks - > seq. l, ks - > seq. s, i ) ; fprintf ( stderr, " [ % d ] %. 3f \ n ", i, ( double ) ( clock ( ) - t ) / clocks _ per _ sec ) ; kseq _ destroy ( ks ) ; gzclose ( fp ) ; } kh _ destroy ( 64, h ) ; return 0 ; }


so she is doing \ begin { align * } 61 - 17 = ( 60 + 1 ) - ( 10 + 7 ) & = ( 60 - 10 ) - ( 7 - 1 ) \ \ & = 50 - 6 \ \ & = 44 \ end { align * } she manage to have positive results on each power of ten group up to a multiplication by $ \ pm 1 $ and sums at the end the pieces ; this is kind of smart : ) conclusion : if she is comfortable with this system, let her do...


this is mostly the case for sulfuric acid. commercially available sulfuric acid is dense ( ~ 1. 8 g / ml ) and when water is added, it may not mix. in this case a layer of hot weak acid solution is formed, which boils and sprays around. when acid is poured into water, it flows down the flask and mixes much better, so no boiling occurs. the reason this occurs is due to the large amount of energy released in the hydration reaction of sulfuric acid ions. do not believe that heat comes from dissociation, as the dissociation of acids, bases, and salts always consumes energy. the energy is released from subsequent hydration, and the release may be high, especially if $ \ ce { h + } $ or $ \ ce { oh - } $ ions are hydrated.


the problem lasers do all sorts of cool things in research and in applications, and there are many good reasons for it, including their coherence, frequency stability, and controllability, but for some applications, the thing that really matters is raw power. as a simple example, it had long been understood that if the intensity of light gets high enough, then the assumption of linearity that underpins much of classical optics would break down, and nonlinear optical phenomena like second - harmonic generation would become available, getting light to do all sorts of interesting things. using incoherent light sources, the required intensities are prohibitively high, but once the laser was invented, it took only one year until the first demonstration of second - harmonic generation, and a few short years after that until third - harmonic generation, a third - order nonlinear process that requires even higher intensities. put another way, power matters, and the more intensity you have available, the wider a range of nonlinear optical phenomena will be open for exploration. because of this, a large fraction of laser science has been focused on increasing the available intensities, generally using pulsed lasers to achieve this and with notable milestones being q - switching and mode - locking. however, if you try to push onward with a bigger laser amplifier and more and more power, you are basically destined sooner or later to hit a brick wall, rather brusquely, in the form of catastrophic self - focusing. this is a consequence of yet another nonlinear effect, the kerr effect, happening inside the laser medium itself. at face value, the kerr effect looks harmless enough : basically, it says that if the intensity is high enough, the refractive index of the material will rise slightly, in proportion to the intensity : $ $ n ( i ) = n _ 0 + n _ 2 \ : i. $ $ so, what's the big deal? in short, if you have a laser beam propagating through such a medium, then the intensity of the light will be higher in the center, which means that the refractive index will be higher in the center. in other words, the material's optical properties will look like those of a convex lens, and it will tend to focus the beam. this will tend to make the beam sharper, which will increase the intensity at the center, which will raise the refractive index at the center even higher...... which will then focus the beam even more tightly, leading to higher and higher int


##ensities. this makes up a positive feedback loop, and if the initial intensity is high enough, the medium is long enough, and there isn't enough initial diffraction to counteract it, then it will spiral out of control and cause catastrophic laser - induced damage in the very medium that you're trying to use to amplify that laser beam. ( moreover, it is quite common, particularly in air, that the laser will diffract on the damaged spot and then re - self - focus a bit further down the line, a phenomenon known as laser filamentation. if you get things just right wrong, this can propagate a failure in the gain medium up to the destruction of an entire beamline. ) image source this sounds like a funky mechanism, but it was a huge roadblock for a very long time. if you plot the highest laser intensity available at different times since the invention of the laser, it climbs quickly up during the sixties, and then it hits a wall and stays put for some ten to fifteen years : image source this represents the barrier of kerr - lens self - focusing, and at the time the only way to overcome it was to build a laser which was physically bigger, to dilute the intensity over more gain medium to try to prevent the problem. until, that is, chirped pulse amplification came around to solve the problem. the solution at its core, chirped pulse amplification ( cpa ) works by diluting the light, so that it can be amplified to a larger total power without reaching a dangerous intensity, but it does this stretching in time, i. e. longitudinally along the laser pulse. the basic sequence consists of four steps : first of all, you start with a short laser pulse that you want to amplify you then stretch it in time, by introducing chirp into the signal : that is, you use some sort of dispersive element, like a prism or a diffraction grating, which decomposes the pulse into all of its constituent colors and sends the longer wavelengths first and the shorter wavelengths last. this will naturally reduce the intensity of the pulse. ( why " chirp "? because the upward ( or downward ) sweep of frequencies over the pulse is precisely what gives bird chirps their characteristic sound. ) you then pass this lower - intensity pulse through your laser amplifier, which is safe because the instantaneous intensity is below the self - focusing damage threshold of your gain medium. finally,


you pass your pulse through a reversed set of gratings which will undo the relative delay between the longer - and shorter - wavelengths of your pulse, putting them all together into a single pulse of the same shape and length as your original pulse...... but at the much higher amplified power, and at intensities which would be impossible to achieve safely using direct amplification of the pulse. the core feature that makes the method tick is the fact that, when done correctly, the stretching of the pulse will completely conserve the coherence between the different frequency components, which means that it is fully reversible and when you add a cancelling chirp the pulse will go back to its initial shape. furthermore, the method relies on the fact that stimulated emission will completely duplicate, in a coherent way, the photons that it is amplifying, which means that the photons that are introduced by the amplification will have the same frequency and phase characteristics as the initial pulse, which means that when you remove the chirp from the amplified pulse the added - in photons will also compress into a tight envelope. applications like i said at the beginning, cpa is particularly useful in places where raw laser power, and particularly concentrated laser power, is of paramount importance. here are some examples : in the same way that lasers gave us nonlinear optics, cpa has been integral in the development of high - order harmonic generation which has pushed past the second - or third - order harmonics to happily produce tens or hundreds of harmonics. ( the current record goes all the way to harmonic 5, 000. ) this isn't only'more ', it's qualitatively different : it pushes nonlinear optics to regimes where the usual perturbative expansion completely breaks down, and where it needs to be replaced with a completely new set of tools, which revolve around the so - called three - step model, and which involve a nice and quite particular new interface between classical and quantum mechanics, where trajectories do ( sort of ) exist but over complex - valued time and space, due to the presence of quantum tunnelling. it has also helped push the study of light - matter interaction past that same perturbative limit, giving us the tools to extract electrons from molecules and control them in very precise ways, thereby allowing for the creation of tools like e. g. laser - driven electron diffraction, which can be used to image the shapes of molecules as they undergo bending


and other vibrations. cpa also underpins several breakthrough measurements which have been touched on previously on this site, including the observation of the time - dependent waveform of a light pulse, itself done using high - order harmonic radiation ; the observation of charge oscillations when atoms are placed in excited states, again using hhg ; or performing electron holography from an atomic target using electrons pulled from that same atom. of course, all the cool laser - driven qed stuff at the top of that second diagram : if your laser is strong enough that, if you release an electron into the focus, the kinetic energy of its oscillations will exceed $ m _ e c ^ 2 $, then you can start to have things like laser - driven pair creation, and all sorts of fun stuff. some of it is already on the table, some of it is in achievable plans for the future, and all of it is made possible by cpa. cpa is also extremely useful in delivering sharply controlled bursts of power to materials. this is extremely useful in laser micromachining, for example, where it is routinely used in e. g. using short laser pulses to etch waveguides into dielectrics, which are then extremely useful for chip - based quantum computation and quantum information processing. similarly, the ability to deliver sharply controlled bursts of power is extremely useful in laser microsurgery, and there are several types of eye surgery that exclusively use cpa pulses to provide sharp'kicks'of power which perform cleaner incisions. on a much larger scale, when you really turn up the power to the maximum, cpa is a vital component of laser wakefield acceleration, which uses the ionized pocket left behind by an intense laser pulse as it travels through a gas to accelerate electrons to energies that would otherwise require an extremely large particle accelerator, but which are now available using a much more modest table - top laser system. further reading some additional resources for further reading : the nobel prize's scientific background and popular information documents make excellent reading, and people don't go looking at those sources anywhere near enough. go check them out! the original paper : compression of amplified chirped optical pulses. d. strickland and g. mourou. optics comms. 55, 447 ( 1985 ). what power limitations was chirped radar designed to overcome?, a previous question of mine on similar radar technologies that predated cpa. this is a nice tutorial


on u michigan.


thanks to manu tamminen for this solution : echo accttgaaa | tr acgtacgt tgcatgca | rev


i don't know if it's the fastest, but the following provides an approximately 10x speed up over your functions : import string tab = string. maketrans ( " actg ", " tgac " ) def reverse _ complement _ table ( seq ) : return seq. translate ( tab ) [ : : - 1 ] the thing with hashing is that it adds a good bit of overhead for a replacement set this small. for what it's worth, i added that to your code as " with a translation table " and here is what i got on my workstation : global dict implementation 1. 37599s total, 363374. 8 strings per second 3. 3 % increase over baseline naive implementation 1. 44126s total, 346919. 4 strings per second - 1. 3 % increase over baseline with a translation table 0. 16780s total, 2979755. 6 strings per second 88. 2 % increase over baseline if you need python 3 rather than python 2, then substitute tab = str. maketrans ( " actg ", " tgac " ) for tab = string. maketrans ( " actg ", " tgac " ), since maketrans is now a static method on the str type. for those wondering, using biopython is slower for this ( ~ 50 % slower than the naive implementation ), presumably due to the overhead of converting the strings to seq objects. if one were already reading sequences in using biopython, though, i wouldn't be surprised if the performance was much different.


what you really want to do is essentially called as voice activity detection or speech detection. basically any pure speech signal ( which contains no music ) has three parts. the voiced sound - which is basically caused by vowels the unvoiced sound - which contains consonants. the characteristic of human sound is such that while a lot of energy is used in voiced sound the real information is contained in consonants. also, voiced sound is usually lower frequency where as unvoiced sounds are higher frequencies. [ to be precise all voiced sound are resonated more or less a constant frequency for a given person which is his / her pitch ]. now, as any system there is noise. the voiced sound is usually quite powerful enough that it can be distinguished visible. when you apply a lower frequency filtering it is possible to collect good magnitude of voiced sounds however, the unvoiced sound ( with all the rich information ) will get lost. coming to the question how to solve it : the trick lies in the fact that unvoiced sound still come from a resonating source ; and inherently restricted over a certain frequency. where as, the noise is rather uniform. so a simple measure that distinguish all three is " local power " or alternatively but equivalent is to take the windowed auto - correlation. if you take at a time say 100 samples - and auto correlate itself, if it contains only noise the results will be pretty much zero ( this is the property of white noise ) where as for speech signal, this magnitude will be observable because the signal still has better structure. this has worked for me in the past. vad has been an active research areas - because almost all mobile phone communications wants to detect non speech part and remove them from encoding. but if they would remove non - voiced speech this would make telephony useless. the g. 729 standard computes vad based on features like : line spectral frequencies, full - band energy, low - band energy ( < 1 khz ), and zero - crossing rate. the gsm standard works as follows : option 1 computes the snr in nine bands and applies a threshold to these values. option 2 calculates different parameters : channel power, voice metrics, and noise power. it then thresholds the voice metrics using a threshold that varies according to the estimated snr. ( from wikipedia ) for more advanced techniques i am listing some references on this subject. most sited reference : jongseo sohn ; nam soo kim ; wonyong sung


; " a statistical model - based voice activity detection " signal processing letters, ieee, jan 1999, volume : 6 issue : 1 pp : 1 - 3 most relevant for you : mark marzinzik and birger kollmeier " speech pause detection for noise spectrum estimation by tracking power envelope dynamics " ieee transactions on speech and audio processing, vol. 10, no. 2, february 2002 pp. 109 ramirez, j. ; j. m. gorriz, j. c. segura ( 2007 ). " voice activity detection. fundamentals and speech recognition system robustness ". in m. grimm and k. kroschel. robust speech recognition and understanding. pp. 1 – 22. isbn 978 - 3 - 902613 - 08 - 0. introductory : jonathan kola, carol espy - wilson and tarun pruthi " voice activity detection "


switched mode power supplies use what is known as a " flyback converter " to provide voltage conversion and galvanic isolation. a core component of this converter is a high frequency transformer. practical transformers have some stray capacitance between primary and secondary windings. this capacitance interacts with the switching operation of the converter. if there is no other connection between input and output this will result in a high frequency voltage between the output and input. this is really bad from an emc perspective. the cables from the power brick are now essentially acting as an antenna transmitting the high frequency generated by the switching process. to suppress the high frequency common mode is is necessary to put capacitors between the input and output side of the power supply with a capacitance substantially higher than the capacitance in the flyback transformer. this effectively shorts out the high frequency and prevents it escaping from the device. when desinging a class 2 ( unearthed ) psu we have no choice but to connect these capacitors to the input " live " and / or " neutral ". since most of the world doesn't enforce polarity on unearthed sockets we have to assume that either or both of the " live " and " neutral " terminals may be at a sinificant voltage relative to earth and we usually end up with a symmetrical design as a " least bad option ". that is why if you measure the output of a class 2 psu relative to mains earth with a high impedance meter you will usually see around half the mains voltage. that means on a class 2 psu we have a difficult tradeoff between safety and emc. making the capacitors bigger improves emc but also results in higher " touch current " ( the current that will flow through someone or something who touches the output of the psu and mains earth ). this tradeoff becomes more problematic as the psu gets bigger ( and hence the stray capacitance in the transformer gets bigger ). on a class 1 ( earthed ) psu we can use the mains earth as a barrier between input and output either by connecting the output to mains earth ( as is common in desktop pc psus ) or by using two capacitors, one from the output to mains earth and one from mains earth to the input ( this is what most laptop power bricks do ). this avoids the touch current problem while still providing a high frequency path to control emc. short circuit failure


of these capacitors would be very bad. in a class 1 psu failure of the capacitor between the mains supply and mains earth would mean a short to earth, ( equivalent to a failure of " basic " insulation ). this is bad but if the earthing system is functional it shouldn't be a major direct hazard to users. in a class 2 psu a failure of the capacitor is much worse, it would mean a direct and serious safety hazard to the user ( equivilent to a failure or " double " or " reinforced " insulation ). to prevent hazards to the user the capacitors must be designed so that short circuit failure is very unlikely. so special capacitors are used for this purpose. these capacitors are known as " y capacitors " ( x capacitors on the other hand are used between mains live and mains neutral ). there are two main subtypes of " y capacitor ", " y1 " and " y2 " ( with y1 being the higher rated type ). in general y1 capacitors are used in class 2 equipment while y2 capacitors are used in class 1 equipment. so does that capacitor between the primary and secondary sides of the smps mean that the output is not isolated? i've seen lab supplies that can be connected in series to make double the voltage. how do they do that if it isn't isolated? some power supplies have their outputs hard - connected to earth. obviously you can't take a pair of power supplies that have the same output terminal hard - connected to earth and put them in series. other power supplies only have capactive coupling from the output to either the input or to mains earth. these can be connected in series since capacitors block dc ( though don't go crazy, the capacitors will have a limited working voltage ).


combustion of small materials, such as a match or birthday candle, actually involves the release of volatile vapours, which themselves burn. it is not the solid material that burns. there needs to be a minimum amount of volatile material present in this combustion zone ( just above the burning match ) for the ignition to occur. as the combustion process continues, heat is given off, and more volatile materials are released, which in turn continues the combustion cycle. now, if you shake a match or blow on a candle, you rapidly disperse these volatile fuels from the combustion zone, and there is no longer sufficient fuel to ignite. it is effectively removing the fuel component of the fire triangle, for just a brief moment. large fires can reignite because there is sufficient heat left in the fuel to further release volatile fuels, which can either self - ignite or ignite through the presence of embers. blowing gently on a small wood fire increases the oxygen content in the combustion zone, without dispersing the fuel. similarly, if you experiment with a small enough wood fire, you will see that blowing on different parts of the fire will have a different outcome : blowing on the base of the fuels will increase oxygen content, and not affect volatile fuels. blowing on the top of the fire ( where the flame starts to burn ) will probably put the fire out. will this put out a small paper fire? that will depend on the heat retained by the burning fuel. a single piece of a4 paper if shaken hard enough will extinguish. a ream of a4 paper that has burned halfway down the page will be put out this way, but could easily reignite if the paper pages are pulled apart to allow oxygen into the released volatile fuels. generally, forest fires are accelerated by strong winds. winds affect forest fires in a number of ways and increase the rate of spread significantly. this is a topic for a whole other question.


there are numerous containments known. let $ \ subseteq $ denote containment and $ \ subset $ proper containment. let $ \ times $ denote incomparability. let $ ll = \ bigcup _ k ll ( k ) $, $ lr = \ bigcup _ k lr ( k ) $. grammar level for ll $ ll ( 0 ) \ subset ll ( 1 ) \ subset ll ( 2 ) \ subset ll ( 2 ) \ subset \ cdots \ subset ll ( k ) \ subset \ cdots \ subset ll \ subset ll ( * ) $ $ sll ( 1 ) = ll ( 1 ), sll ( k ) \ subset ll ( k ), sll ( k + 1 ) \ times ll ( k ) $ most of these are proven in properties of deterministic top down grammars by rosenkrantz and stearns. $ sll ( k + 1 ) \ times ll ( k ) $ is a rather trivial exercise. this presentation by terence parr places $ ll ( * ) $ on slide 13. the paper ll - regular grammars by jarzabek and krawczyk show $ ll \ subset llr $, and their proof trivially extends to $ ll \ subset ll ( * ) $ for lr $ lr ( 0 ) \ subset slr ( 1 ) \ subset lalr ( 1 ) \ subset lr ( 1 ) $ $ slr ( k ) \ subset lalr ( k ) \ subset lr ( k ) $ $ slr ( 1 ) \ subset slr ( 2 ) \ subset \ cdots \ subset slr ( k ) $ $ lalr ( 1 ) \ subset lalr ( 2 ) \ subset \ cdots \ subset lalr ( k ) $ $ lr ( 0 ) \ subset lr ( 1 ) \ subset lr ( 2 ) \ subset \ cdots \ subset lr ( k ) \ subset \ cdots \ subset lr $ these are all simple exercises. ll versus lr $ ll ( k ) \ subset lr ( k ) $ ( properties of deterministic top down grammars plus any left recursive grammar ) $ ll ( k ) \ times slr ( k ), lalr ( k ), lr ( k - 1 ) $ ( simple exercise ) $ ll \ subset lr $ ( any left recursive grammar ) $ ll ( * ) \ times lr $ ( left recursion versus arbitrary look


##ahead ) language level for ll $ ll ( 0 ) \ subset ll ( 1 ) \ subset ll ( 2 ) \ subset \ cdots \ subset ll ( k ) \ subset \ cdots \ subset ll \ subset ll ( * ) $ $ sll ( k ) = ll ( k ) $ most of these are proven in properties of deterministic top down grammars. the equivalence problem for ll - and lr - regular grammars by nijholt makes references to papers showing $ ll ( k ) \ subset ll ( * ) $. the paper ll - regular grammars by jarzabek and krawczyk show $ ll \ subset llr $, and their proof trivially extends to $ ll \ subset ll ( * ) $ for lr $ lr ( 0 ) \ subset slr ( 1 ) = lalr ( 1 ) = lr ( 1 ) = slr ( k ) = lalr ( k ) = lr ( k ) = lr $ some of these were proven by knuth in his paper on the translation of languages from left to right in which he introduced lr ( k ), the rest are proven in transforming lr ( k ) grammars to lr ( 1 ), slr ( 1 ), and ( 1, 1 ) bounded right - context grammars by mickunas et al. ll versus lr $ ll \ subset lr ( 1 ) $ ( containment follows from the above, $ \ { a ^ i b ^ j | i \ geq j \ } $ is the canonical example for strict containment ) $ ll ( * ) \ times lr $ ( the language $ \ { a ^ i b ^ j | i \ geq j \ } $ shows half the claim, and the introduction of the equivalence problem for ll - and lr - regular grammars by nijholt makes references to papers showing the other half ) $ lr ( 1 ) = dcfl $ ( see e. g. reference here ).


according to h. c. urey and g. failla, science 15 mar 1935, vol. 81, issue 2098, pp. 273, there's no difference in the taste of ordinary and heavy water.


analog filters are stable if the poles are in the left half of the s - plane ( figure on the left ) and digital filters are stable if the poles are inside the unit circle ( figure on the right ). so mathematically all that is needed to convert from analog to digital is a mapping ( conformal? ) from the half - space to the unit disk and the $ \ jmath \ omega $ axis to the unit circle $ \ vert z \ vert = 1 $. any transformation that does this is a possible candidate for being an alternative to the bilateral transform. two of the well known methods are the impulse invariance method and the matched z - transform method. conceptually, both of these are similar to sampling a continuous waveform that we're familiar with. denoting the inverse laplace transform by $ \ mathcal { l } ^ { - 1 } $ and the z transform as $ \ mathcal { z } $, both these methods involve calculating the impulse response of the analog filter as $ $ a ( t ) = \ mathcal { l } ^ { - 1 } \ { a ( s ) \ } $ $ and sampling $ a ( t ) $ at a sampling interval $ t $ that is high enough so as to avoid aliasing. the transfer function of the digital filter is then obtained from the sampled sequence $ a [ n ] $ as $ $ d _ a ( z ) = \ mathcal { z } \ { a [ n ] \ } $ $ however, there are key differences between the two. impulse invariance method : in this method, you expand the analog transfer function as partial fractions ( not in the matched z transform as mentioned by peter ) as $ $ a ( s ) = \ sum _ m \ frac { c _ m } { s - \ alpha _ m } $ $ where $ c _ m $ is some constant and $ \ alpha _ m $ are the poles. mathematically, any transfer function with a numerator of lesser degree than the denominator can be expressed as a sum of partial fractions. only low - pass filters satisfy this criterion ( high - pass and bandpass / bandstop have at least the same degree ), and hence impulse invariant method cannot be used to design other filters. the reason why it fails is also quite clear. if you had a polynomial in the numerator of the same degree as in the denominator, you will have a free standing constant term, which upon inverse


transforming, will give a delta function that cannot be sampled. if you carry out the inverse laplace and forward z transforms, you'll see that the poles are transformed as $ \ alpha _ m \ to e ^ { \ alpha _ m t } $ which means that if your analog filter is stable, the digital filter will also be stable. hence it preserves the stability of the filter. matched z - transform in this method, instead of splitting the impulse response as partial fractions, you do a simple transform of both the poles and the zeros in a similar manner ( matched ) as $ \ beta _ m \ to e ^ { \ beta _ m t } $ and $ \ alpha _ m \ to e ^ { \ alpha _ m t } $ ( also stability preserving ), giving $ $ a ( s ) = \ frac { \ prod _ m ( s - \ beta _ m ) } { \ prod _ n ( s - \ alpha _ n ) } \ longrightarrow \ frac { \ prod _ m \ left ( 1 - z ^ { - 1 } e ^ { \ beta _ m t } \ right ) } { \ prod _ n \ left ( 1 - z ^ { - 1 } e ^ { \ alpha _ n t } \ right ) } $ $ you can easily see the limitation of both these methods. impulse invariant is applicable only if your filter is low pass and matched z - transform method is applicable to bandstop and bandpass filters ( and high pass up to the nyquist frequency ). they are also limited in practice by the sampling rate ( after all, you can only go up to a certain point ) and suffer from the effects of aliasing. the bilinear transform is by far the most commonly used method in practice and the above two are rather more for academic interests. as for conversion back to analog, i'm sorry but i do not know and can't be of much help there as i hardly ever use analog filters.


i looked into asic's a while ago and here's what i found : everybody has different definitions for the word " asic ". there are ( very roughly ) three categories : fpga conversions, " normal " asic, and " full custom ". as expected, these are in order of increasing price and increasing performance. before describing what these are, let me tell you how a chip is made... a chip has anywhere from 4 to 12 + " layers ". the bottom 3 or 4 layers contains the transistors and some basic interconnectivity. the upper layers are almost entirely used to connect things together. " masks " are kind - of like the transparencies used in the photo - etching of a pcb, but there is one mask per ic layer. when it comes to making an asic, the cost of the masks is huge. it is not uncommon at all for a set of masks ( 8 layers, 35 to 50 nm ) to run us $ 1 million! so it is no great surprise to know that most of the " cheaper " asic suppliers try very hard to keep the costs of the masks down. fpga conversions : there are companies that specialize in fpga to asic conversions. what they do is have a somewhat standard or fixed " base " which is then customized. essentially the first 4 or 5 layers of their chip is the same for all of their customers. it contains some logic that is similar to common fpga's. your " customized " version will have some additional layers on top of it for routing. essentially you're using their logic, but connecting it up in a way that works for you. performance of these chips is maybe 30 % faster than the fpga you started with. back in " the day ", this would also be called a " sea of gates " or " gate array " chip. pros : low nre ( us $ 35k is about the lowest ). low minimum quantities ( 10k units / year ). cons : high per - chip costs - - maybe 50 % the cost of an fpga. low performance, relative to the other solutions. " normal " asic : in this solution, you are designing things down to the gate level. you take your vhdl / verilog and compile it. the design for the individual gates are taken from a library of gates & devices that has been approved by the chip manufacturer ( so they know


it works with their process ). you pay for all the masks, etc. pros : this is what most of the chips in the world are. performance can be very good. per - chip costs is low. cons : nre for this starts at us $ 0. 5 million and quickly goes up from there. design verification is super important, since a simple screw - up will cost a lot of money. nre + minimum order qty is usually around us $ 1 million. full custom : this is similar to a normal asic, except that you have the flexibility to design down to the transistor level ( or below ). if you need to do analog design, super low power, super high performance, or anything that can't be done in a normal asic, then this is the thing for you. pros : this requires a very specialized set of talents to do properly. performance is great. cons : same con's as normal asic, only more so. odds of screwing something up is much higher. how you go about this really depends on how much of the work you want to take on. it could be as " simple " as giving the design files to a company like tsmc or umc and they give you back the bare wafers. then you have to test them, cut them apart, package them, probably re - test, and finally label them. of course there are other companies that will do most of that work for you, so all you get back are the tested chips ready to be put on a pcb. if you have gotten to this point and it still seems like an asic is what you want to do then the next step would be to start googling for companies and talking with them. all of those companies are slightly different, so it makes sense to talk with as many of them as you can put up with. they should also be able to tell you what the next step is beyond talking with them.


i have no familiarity with the multitaper method. that said, you've asked quite a question. in pursuit of my msee degree, i took an entire course that covered psd estimation. the course covered all of what you listed ( with exception to the multitaper method ), and also subspace methods. even this only covers some of the main ideas, and there are many methods stemming from these concepts. for starters, there are two main methods of power spectral density estimation : non - parametric and parametric. non - parametric methods are used when little is known about the signal ahead of time. they typically have less computational complexity than parametric models. methods in this group are further divided into two categories : periodograms and correlograms. periodograms are also sometimes referred to as direct methods, as they result in a direct transformation of the data. these include the sample spectrum, bartlett's method, welch's method, and the daniell periodogram. correlograms are sometimes referred to as indirect methods, as they exploit the wiener - khinchin theorem. therefore these methods are based on taking the fourier transform of some sort of estimate of the autocorrelation sequence. because of the high amount of variance associated with higher order lags ( due to a small amount of data samples used in the correlations ), windowing is used. the blackman - tukey method generalizes the correlogram methods. parametric methods typically assume some sort of signal model prior to calculation of the psd estimate. therefore, it is assumed that some knowledge of the signal is known ahead of time. there are two main parametric method categories : autoregressive methods and subspace methods. autoregressive methods assume that the signal can be modeled as the output of an autoregressive filter ( such as an iir filter ) driven by a white noise sequence. therefore all of these methods attempt to solve for the iir coefficients, whereby the resulting power spectral density is easily calculated. the model order ( or number of taps ), however, must be determined. if the model order is too small, the spectrum will be highly smoothed, and lack resolution. if the model order is too high, false peaks from an abundant amount of poles begin to appear. if the signal may be modeled by an ar process of model'p ', then the output of the filter of order > = p driven by the signal will produce white noise. there


are hundreds of metrics for model order selection. note that these methods are excellent for high - to - moderate snr, narrowband signals. the former is because the model breaks down in significant noise, and is better modeled as an arma process. the latter is due to the impulsive nature of the resulting spectrum from the poles in the fourier transform of the resulting model. ar methods are based on linear prediction, which is what's used to extrapolate the signal outside of its known values. as a result, they do not suffer from sidelobes and require no windowing. subspace methods decompose the signal into a signal subspace and noise subspace. exploiting orthogonality between the two subspaces allows a pseudospectrum to be formed where large peaks at narrowband components can appear. these methods work very well in low snr environments, but are computationally very expensive. they can be grouped into two categories : noise subspace methods and signal subspace methods. both categories can be utilized in one of two ways : eigenvalue decomposition of the autocorrelation matrix or singular value decomposition of the data matrix. noise subspace methods attempt to solve for 1 or more of the noise subspace eigenvectors. then, the orthogonality between the noise subspace and the signal subspace produces zeros in the denominator of the resulting spectrum estimates, resulting in large values or spikes at true signal components. the number of discrete sinusoids, or the rank of the signal subspace, must be determined / estimated, or known ahead of time. signal subspace methods attempt to discard the noise subspace prior to spectral estimation, improving the snr. a reduced rank autocorrelation matrix is formed with only the eigenvectors determined to belong to the signal subspace ( again, a model order problem ), and the reduced rank matrix is used in any one of the other methods. now, i'll try to quickly cover your list : psd using burg method : the burg method leverages the levinson recursion slightly differently than the yule - walker method, in that it estimates the reflection coefficients by minimizing the average of the forward and backward linear prediction error. this results in a harmonic mean of the partial correlation coefficients of the forward and backward linear prediction error. it produces very high resolution estimates, like all autoregressive methods, because it uses linear prediction to extrapolate the signal outside of its known data record


. this effectively removes all sidelobe phenomena. it is superior to the yw method for short data records, and also removes the tradeoff between utilizing the biased and unbiased autocorrelation estimates, as the weighting factors divide out. one disadvantage is that it can exhibit spectral line splitting. in addition, it suffers from the same problems all ar methods have. that is, low to moderate snr severely degrades the performance, as it is no longer properly modeled by an ar process, but rather an arma process. arma methods are rarely used as they generally result in a nonlinear set of equations with respect to the moving average parameters. psd using covariance method : the covariance method is a special case of the least - squares method, whereby the windowed portion of the linear prediction errors is discarded. this has superior performance to the burg method, but unlike the yw method, the matrix inverse to be solved for is not hermitian toeplitz in general, but rather the product of two toeplitz matrices. therefore, the levinson recursion cannot be used to solve for the coefficients. in addition, the filter generated by this method is not guaranteed to be stable. however, for spectral estimation this is a good thing, resulting in very large peaks for sinusoidal content. psd using periodogram : this is one of the worst estimators, and is a special case of welch's method with a single segment, rectangular or triangular windowing ( depending on which autocorrelation estimate is used, biased or unbiased ), and no overlap. however, it's one of the " cheapest " computationally speaking. the resulting variance can be quite high. psd using modified covariance method : this improves on both the covariance method and the burg method. it can be compared to the burg method, whereby the burg method only minimizes the average forward / backward linear prediction error with respect to the reflection coefficient, the mc method minimizes it with respect to all of the ar coefficients. in addition, it does not suffer from spectral line splitting, and provides much less distortion than the previously listed methods. in addition, while it does not guarantee a stable iir filter, it's lattice filter realization is stable. it is more computationally demanding than the other two methods as well. psd using welch's method : welch's method improves upon the periodogram by addressing the lack of the ensemble


averaging which is present in the true psd formula. it generalizes barlett's method by using overlap and windowing to provide more psd " samples " for the pseudo - ensemble average. it can be a cheap, effective method depending on the application. however, if you have a situation with closely spaced sinusoids, ar methods may be better suited. however, it does not require estimating the model order like ar methods, so if little is known about your spectrum a priori, it can be an excellent starting point. psd using yule - walker ar method : this is a special case of the least squares method where the complete error residuals are utilized. this results in diminished performance compared to the covariance methods, but may be efficiently solved using the levinson recursion. it's also known as the autocorrelation method. spectrogram using short - time fourier transform : now you're crossing into a different domain. this is used for time - varying spectra. that is, one whose spectrum changes with time. this opens up a whole other can of worms, and there are just as many methods as you have listed for time - frequency analysis. this is certainly the cheapest, which is why its so frequently used. spectral estimation : this is not a method, but a blanket term for the rest of your post. sometimes the periodogram is referred to as the " sample spectrum " or the " schuster periodogram ", the former of which may be what you're referring to. if you are interested, you may also look into subspace methods such as music and pisarenko harmonic decomposition. these decompose the signal into signal and noise subspace, and exploits the orthogonality between the noise subspace and the signal subspace eigenvectors to produce a pseudospectrum. much like the ar methods, you may not get a " true " psd estimate, in that power most likely is not conserved, and the amplitudes between spectral components is relative. however, it all depends on your application.


it may be necessary to distinguish between methods that use unique molecular identifiers ( umis ), such as 10x's chromium, drop - seq, etc, and non - umi methods, such as smrt - seq. at least for umi - based methods, the alternative perspective, that there is no significant zero - inflation in scrna - seq, is also advocated in the single - cell research community. the argument is straight - forward : the empirical mean expression vs. dropout rate curve matches the theoretically predicted one, given the current levels of capture efficiency. examples svensson blog a couple of blog posts from valentine svensson argue this point rather pedagogically, and include citations from across the literature : droplet scrna - seq is not zero - inflated count - depth variation makes poisson scrna - seq data negative binomial baynorm there is a more extensive preprint by tang, shahrezaei, et al. ( biorxiv, 2018 ) that claims to show a binomial model is sufficient to account for the observed dropout noise. here is a snippet of a relevant conclusion : importantly, as baynorm recovered dropout rates successfully in both umi - based and non - umi protocols without the need for specific assumptions, we conclude that invoking zero - inflation models is not required to describe scrna - seq data. consistent with this, the differences in mean expression levels of lowly expressed genes observed between bulk and scrna - seq data, which were suggested to be indicative of zero - inflation, were recovered by our simulated data using the binomial model only. multinomial modeling there is also a very clearly written preprint by townes, irizarry, et al. ( biorxiv, 2019 ) where the authors consider scrna - seq as a proper compositional sampling ( i. e., multinomial process ) and they come to a similar conclusion, though specifically for umi - based methods. from the paper : the multinomial model makes two predictions which we verified using negative control data. first, the fraction of zeros in a sample ( cell or droplet ) is inversely related to the total number of umis in that sample. second, the probability of an endogenous gene or ercc spike - in having zero counts is a decreasing function of its mean expression ( equations provided in methods ). both of these


predictions were validated by the negative control data ( figure 1 ). in particular, the empirical probability of a gene being zero across droplets was well calibrated to the theoretical prediction based on the multinomial model. this also demonstrates that umi counts are not zero inflated. furthermore, by comparing raw read counts ( prior to umi - based deduplication ) and umi counts, they conclude that pcr is indeed the cause of zero - inflation : the results suggest that while read counts appear zero - inflated and multimodal, umi counts follow a discrete distribution with no zero inflation ( figure s1 ). the apparent zero inflation in read counts is a result of pcr duplicates. i highly recommend giving this a read, especially because it nicely situates other common generative models ( e. g., binomial, poisson ) as valid simplifying assumptions of the multinomial model. it should be noted that this same group previously published a work ( hicks, irizarry, et al. 2018 ), mostly focused on non - umi - based datasets ( smrt - seq ), where they showed evidence that, relative to bulk rna - seq, there was significant zero - inflation.


" if the map and the terrain disagree, trust the terrain. " it's not really understood why deep learning works as well as it does, but certainly old concepts from learning theory such as vc dimensions appear not to be very helpful. the matter is hotly debated, see e. g. : h. w. lin, m. tegmark, d. rolnick, why does deep and cheap learning work so well? c. zhang, s. bengio, m. hardt, b. recht, o. vinyals, understanding deep learning requires rethinking generalization. d. krueger, b. ballas, s. jastrzebski, d. arpit, m. s. kanwal, t. maharaj, e. bengio, a. fischer, a. courville, deep nets dont learn via memorization. regarding the issue of adversarial examples, the problem was discovered in : c. szegedy, w. liu, y. jia, p. sermanet, s. reed, d. anguelov, d. erhan, v. vanhoucke, a. rabinovich, going deeper with convolutions. it is further developed in : i. goodfellow, j. shlens, c. szegedy, explaining and harnessing adversarial examples. there is a lot of follow - on work. update march 2020. a new hypothesis that appears to explain some of the mismatch between clear over - parameterisation of modern ( feed - forward ) nns and good recognition performance is frankle and carbin's lottery ticket hypothesis from 2018 : j. frankle, m. carbin, the lottery ticket hypothesis : finding sparse, trainable neural networks. the claim is that a " randomly - initialised, dense [ feed - forward ] neural network contains a subnetwork that is initialised such that when trained in isolation it can match the test accuracy of the original network after training for at most the same number of iterations. " regarding the original question, the lottery ticket hypothesis might be understood as saying that : training by stochastic gradient descent searches for small subnetworks that work well and deemphasises the rest of the overparameterised network's learning capacity. the bigger the original network, the more likely it is to contain a small subnetwork with good performance on the task at hand. this


has found empirical support, e. g. in h. zhou, j. lan, r. liu, j. yosinski, deconstructing lottery tickets : zeros, signs, and the supermask. and theoretical support in : e. malach, g. yehudai, s. shalev - shwartz, o. shamir, proving the lottery ticket hypothesis : pruning is all you need. as far as i'm aware, it has not yet been possible to generalise the lottery ticket hypothesis to recurrent nns. update march 2021. the original 2016 paper understanding deep learning requires rethinking generalization has been updated. here is the new version : c. zhang, s. bengio, m. hardt, b. recht, o. vinyals, understanding deep learning ( still ) requires rethinking generalization.


the answer to your question is yes it is certainly possible. at one time it was thought that there was something special about " organic " chemicals which meant that they could not be artificially synthesised out of fundamental elements. in 1828 frederick wohler synthesised urea ( co ( nh2 ) 2 ) which is often taken as the first demonstration that the organic v inorganic distinction was not a sound one ( for more on this see the wikipedia article on wohler synthesis. as far as we know all essential human nutrients can be synthesised from inorganic ingredients, even complex molecules such as vitamin b12. other contributors have pointed out that organic pathways for synthesising our food have evolved over long periods to be very efficient - at least in the conditions prevailing on earth. you haven't ruled out copying biochemical pathways using chemicals that are entirely of inorganic origin. anyone trying to do this seriously could create glucose ( for example ) by artificially creating enzymes ( perhaps via artificial dna ) to do the job. the thing is that we already have self - replicating and repairing machines to do that already ( plants ). there might be circumstances when we needed to use artificial synthesis. i can think of two science - fiction stories that deal with this question, the first of which goes into some detail : the moon is hell by john w. campbell, in which astronauts are stranded on the moon and forced to make food from what they find there. technical error by arthur c. clarke, in which a man is accidentally rotated through the fourth dimension. his employers contemplate the difficulty caused by the " handedness " of many biological molecules meaning they would have to artificially synthesise many of his foods. it may be that a future expedition to mars ( say ) might have to think about these things. a little searching fails to come up with standard inorganic syntheses of glucose and similar substances. the reason for this is almost certainly because it is so easy to use organic inputs. glucose is easily made by the hydrolysis of starch. starch is very common and cheap. even l - glucose is usually made out of organically derived precursors ( or sometimes even using d - glucose ). update : sources etc one problematic question is : where do you get your input for making nutrients? as others have pointed out, exactly where to draw the line is difficult. this problem starts in defining what is alive in the first place. do you count viruses ( which can go down to a few thousand base pairs of rna ) or satellite viruses ( stobrv has


only 359 base pairs ) or prions? in a sense these are " just " very large molecules. but then really simple bacteria are not many orders of magnitude more complex. as an aside most systems of ethics that do not permit eating meat do not make an alive / non - alive distinction, choosing some other aspect such as sentience, though jainism comes close to doing so. the second problem is, if we reject living things as sources of food, how far removed from those living things are we allowed to get? you say no cells in any state including " dead ". that would exclude ( say ) fruit even though most fruits are expressly created by plants in order to be eaten ( and in some cases must be eaten ) - something that vegans, jains, fruitarians and others would be happy with eating. if we could use dead material things would be much easier. but would you also include hydrocarbons ( coal, oil, gas ) which were once living organisms? if you do, then you are in difficulty because terrestrial carbon is recycled through the biosphere. all co2 was ( to a close approximation ) once a part of a living thing. if you take that position then of course you are going to have to go off - planet to find your source chemicals and your problem becomes very much harder. i was assuming that you were restricting yourself to consuming cells that retain some of their cell structure but had not completely degraded. if that is where you draw the line then there are ample sources of raw materials on earth. genetic modification is much more science fiction though not entirely impossible. some nutrients could be made by humans without much difficulty. our inability to manufacture vitamin c is down to one missing enzyme ( l - gulono - gamma - lactone oxidase ) which is present in most vertebrates ( i think of mammals only guinea pigs, humans and some bats are unable to synthesise it ). you could certainly imagine some very careful genetic modification changing humans so they no longer need to consume vitamin c. but photosynthesis would be much harder. chloroplasts ( which do the job in most plants ) are really a very primitive form of life living in plant cells which may independently reproduce ( and for that reason might be excluded by you - they aren't " cells " but they have membranes ). they could easily end up in conflict with our mitochondria ( since intracellular conflict between organelles is possible ) and you would need to do enormous amounts of work to make human


cells co - operate with them properly. more in keeping with your theme would be adding photosynthetic systems directly to human cells along with a suite of enzymes to manufacture all the things we cannot. that is of course in principle scientifically possible ( since plants do it ) but much harder than it looks. living systems are very complicated and small changes can have unexpected consequences. even very minor genetic modifications are problematic. the human autotroph is likely to be some way off.


the $ \ theta ( \ log n ) $ complexity is usually connected with subdivision. when using lists as an example, imagine a list whose elements are sorted. you can search in this list in $ \ mathcal { o } ( \ log n ) $ time - you do not actually need to look at each element because of the sorted nature of the list. if you look at the element in the middle of the list and compare it to the element you search for, you can immediately say whether it lies in the left or right half of the array. then you can just take this one half and repeat the procedure until you find it or reach a list with 1 item which you trivially compare. you can see that the list effectively halves each step. that means if you get a list of length $ 32 $, the maximum steps you need to reach one - item list is $ 5 $. if you have a list of $ 128 = 2 ^ 7 $ items, you need only $ 7 $ steps, for a list of $ 1024 = 2 ^ { 10 } $ you need only $ 10 $ steps etc. as you can see, the exponent $ n $ in $ 2 ^ n $ always shows the number of steps necessary. logarithm is used to " extract " exactly this exponent number, for example $ \ log _ 2 2 ^ { 10 } = 10 $. it also generalizes to list lengths that are not powers of two long.


cellular respiration in plants is slightly different than in other eukaryotes because the electron transport chain contains an additional enzyme called alternative oxidase ( aox ). aox takes some electrons out of the pathway prematurely - basically the energy is used to generate heat instead of atp. the exact purpose of aox in plants is still unclear. plants will make more aox in response to cold, wounding, and oxidative stress. we know of at least one plant ( skunk cabbage ) that exploits this pathway to generate enough heat to melt snow. this link gives a pretty good overview. ( aox is dear to my heart, since my first 3 years working in a laboratory were spent studying this gene < 3 )