fast.ai: Intro to Machine Learning 1 (v1) (2018)
Jeremy starts with a selection of studentsâ€™ posts.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
All right welcome back to machine learning. I am really excited to be able to share some amazing stuff that University of San Francisco students have built during the week or written about during the week, and quite a few things are going to show. You have already spread around the internet quite a bit: lots of tweets and posts and all kinds of stuff happening, one of the the first to be widely shared. Was this one by Tyler? Who did something really interesting? He he started out by saying like what. If I like, create the synthetic data set where the independent variables is like the X and the y and the dependent variable is like color right and interestingly, he showed me an earlier version of this, where he wasn't using color. He was just like putting the actual numbers in here and this thing kind of wasn't really working at all and as soon as he started using Kawa, it started working really well, and so I wanted to mention that one of the things that unfortunately we we don't Teach you at USF is a theory of human perception. Perhaps we should, because actually, when it comes to visualization, its kind of the most important thing to know is what is the human eye or what is what what it was the human brain good at perceiving? There's a whole area of academic study on this, and one of the things that we're best at perceiving is differences in color right.
So that's why, as soon as we look at this picture of this synthetic data, he created, you can immediately say. Oh there's kind of four areas of you know lighter red color, so what he did was he said: okay, what if we like tried to create a machine learning model of this synthetic data set, and so specifically, he created a tree, and the cool thing is that You can actually draw the tree right so after he created the tree. He did this all in matplotlib. Matplotlib is very flexible right. He actually drew the tree boundaries. So that's already. A pretty neat trick is to be actually able to draw the tree, but then he did something even cleverer, which is he said, okay. So what predictions does the tree make well as the average of each of these areas, and so to do that we can actually draw the average color. Alright, there's actually kind of pretty here is the predictions that the tree makes now. Here's where it gets really interesting is like you can, as you know, randomly generate trees through resampling, and so here are four trees generated through resampling they're, all like pretty similar, but a little bit different, and so now we can actually visualize bagging and to visualize bagging. We literally take the average of the four pictures, all right, that's what bagging is and there it is alright, and so here is like the the fuzzy decision boundaries of a random forest, and I think this is kind of amazing right, because it's it's like a.
I wish I had this actually when I started teaching you all random forest, because I could have skipped a couple of classes. It's just like. Okay, that's what we do. You know we create the decision, boundaries, we average each area and then we we do it. A few times in average, all of them, okay. So that's what a random forest does, and I think like this is just such a great example of making the complex easy through through pictures so congrats to Tyler, for that it actually turns out that he has actually reinvented something that somebody else has already done. A guy called Christian any who went on to be one of the world's foremost machine learning. Researchers actually included almost exactly this technique. In a book he wrote about decision forests, so it's actually kind of cool that Tyler ended up reinventing something that one of the world's foremost authorities on v decision forests. Actually it has created. So I
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Thought that was me, that's nice, because when we pup when we posted this on Twitter, you know got a lot of attention and finally, somebody with it was able to say like: oh, you know what this this actually already exists. So Tyler has gone away and you know started reading that book. Something else which is super cool is Jason Carpenter created a whole new library called Parfitt and Parfitt is a parallelized fitting of multiple models for the purpose of selecting hyper parameters and there's a lot. I really like about this he's shown a clear example of how to use it right and, like the API, looks very similar to other grid search based approaches, but it uses the validation techniques that Rachel wrote about and that we learnt about a couple of weeks ago. Of using a good validation set - and you know what he's done here is in his blog post that introduces it, you know, he's he's gone right back and said like what are hyper parameters. Why do we have to train them and he's kind of explained every step and then the the module itself is like it's? It's very polished. You know he's added documentation to it. He's added a nice readme to it and it's kind of interesting when you actually look at the code, you realize you know it's very simple. You know which is it's definitely not a bad thing.
That's a good thing as to is to make things simple, but by kind of writing this little bit of code and then packaging it up so nicely he's made it really easy for other people to use this technique, which is great, and so one of the things I've been really thrilled to see is then Vinay went along and combined two things from our class. One was to take profit and then the other was to take the kind of accelerated SGD approach to classification. We turn learned about in the last lesson and combine the two to say like okay. Well, let's now use half it to help us find the parameters of a SGD logistic regression. So I think that's really a really great idea. Something else which I thought was terrific is print sexually basically went through and summarized pretty much all the stuff we learnt in the random and random forest interpretation plus, and he went even further than that, as he described each of the different approaches to random forest interpretation. He described how it's done so here, for example, is feature importance, a variable permutation a little picture of each one and then super cool here is the code to implement it from scratch. So I think this is like really nice post. You know describing something that not many people understand and showing you know exactly how it works both with pictures and with code that implements it from scratch. So I think that's really really great.
One of the things I really like here is that for like the tree interpreter, but he actually showed how you can take the tree interpreter output and feed it into the new waterfall chart package that Chris USF student built to show how you can actually visualize the Contributions of the tree interpreter in a waterfall chart so again kind of a nice combination of multiple pieces of technology. We both learned about and and built as a group. I also really thought this kernel. There's been a few interesting kernels share it and I'll share some more next week and diverse wrote this really nice kernel showing this is quite challenging: careful competition on detecting icebergs versus chips and it's a kind of a weird two channel satellite data, which is very hard To visualize - and he actually went through and basically described kind of the formulas for how these, like radar scattering things, actually work and then actually managed to come up with a code that allowed him to recreate. You know the actual 3d icebergs or ships, and I have not seen that done before or like I, you know it's it's quite challenging to know how to visualize his data and then he went on to show how to build a neural net. To try to interpret this so that was pretty fantastic as well. So yeah! Congratulations for all of you! I know for a lot of you.
You know you're posting stuff out there to the rest of the world for the first time you know, and it's kind of intimidating you're used to writing stuff, that you got a hand into a teacher and there any ones who see it - and you know it's kind Of scary, the first time you do it, but then the first time somebody you
Back to the course.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Know that votes your cable kernel or ATS, a clap to your medium post. He suddenly realized. Oh I'm, actually I've written something that people like. That's that's pretty great. So if you haven't tried yourself yet I again invite you to try writing something and if you're not sure you could write a summary of a lesson. You could write a summary of like if there's something you found hard like. Maybe you found it hard to fire up a gpu-based AWS instance. You eventually figured it out. You know you could write down just describe how you solve that problem or if one of your classmates didn't understand something, and you explained it to them, then you could like write down something saying like oh there's, this concept that some people have trouble understanding. Here's a good way, I think, of explaining it there's all kinds of stuff you could. You could do
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Okay, so let's go back to SGD and so we're going back through this logbook, which Rachel put together basically taking us through kind of SGD from scratch for the purpose of digit recognition and actually quite a lot of the stuff we look at today is going to Be closely following part of the computation or linear algebra course, which you can both find the MOOCs on faster, I or at USF it'll, be an elective next year. Alright. So if you find some of this this stuff interesting - and I hope you do - then please consider signing up for the elective or checking out the video online, so we're building neural networks and we're starting with an assumption that we've downloaded the eminence data. We've normalized it by subtracting the main and divided by the standard, deviation. Okay. So the data is it's slightly unusual in that, although they represent images they where they were downloaded as each image was a 784 long rank one tensor, so it's been flattened out, okay and so for the purpose of drawing pictures of it. We had to resize it to 28 by 28, but the
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Actual data we've got is not 28 by 28. It says it's, it's 784 long, flattened out. Okay, four basic steps we're gon na take here is to start out with training the world's simplest neural network, basically a logistic regression right, so no hidden layers and we're going to Train it using a library, fastai and we're going to build the network using a Library, plate watch right and then we're going to gradually get rid of all the libraries right. So, first of all well get rid of the N n neural net library and pytorch and write that ourselves then we'll get rid of the fast a I fit function and write that ourselves and then we'll get rid of the pytorch optimizer and write that Ourselves, and so by the end of this notebook, we'll have written all the pieces ourselves. The only thing that will end up relying on is the two key things that pytorch gives us, which is a the ability to write Python code and have it run on the GPU and be the ability to write Python code and have it automatically differentiated. For us, okay, so there are two things: we're not going to attempt to write ourselves because it's boring and pointless, but everything else we'll try and write ourselves on top of those two things. Ok, so our starting point is like not doing anything ourselves. It's basically having it all done for us, and so pytorch has an N n library, which is where the neural net stuff lives. You can create a multi-layer neural network by using the sequential function and then passing in a list of the layers that you want and we asked for a linear layer, followed by a softmax layer and that defines our logistic regression.
Okay, the input to our linear layer is 28 by 28. As we just discussed, the output is 10 because we want a probability for each of the numbers, not through 9, for each of our images. Okay, CUDA sticks it on the GPU and then fit fits a model. Ok, so we start out with a random set of weights and then fit users gradient descent to make it better. We had to tell the fit function. What criterion to use, in other words, what counts is better and we told it to use negative log likelihood we'll learn about that in the next lesson. What that is exactly we had to tell it what optimizer to use - and we said please use - opt M, not Adam. The details of that we won't cover in this course we're going to use something, build something simpler, called SGD.  We decided to print out accuracy, ok, so that was that, and so, if we do that, ok, so after we fit it, we get an accuracy of generally somewhere around 91 92 percent. So what we going to do from here is we're going to gradually we're going to repeat this exact same thing, so we're going to rebuild this model, you know four or five times fitting it, building it and fitting it with less and less libraries. Ok, so the second thing that we did last time was to try to start to define the the module ourselves right. So instead of saying the network is a sequential bunch of these layers. Let's not use that like at all and try and define it ourself from scratch. Okay! So to do that, we have to use our because that's how we build everything in play torch and we have to create a class which inherits from an end module.
So, n, n dot module is a pytorch class that takes our class and turns it into a neural network module which basically means we'll anything that you inherit from an end module like this. You can pretty much insert into a neural network as a layer or you can treat it as a neural network. It's going to get all the stuff that it needs automatically to to work as a part of or a full neural network. Now we'll talk about exactly what that means today in the next lesson right, so we need to construct the object. So that means we need to define the constructor Thunder in it and then importantly, this is a Python thing is, if you inherit from some other object, then you have to create the thing you inherit from first so when you say super dot, dunder init that says Construct the enn module piece of that first right. If you don't do that, then the NN dot module stuff never gets a chance to actually get constructed right. So this is just like a standard Python oo, subclass, constructor, okay, and if any, if that's on unclear to you, then you know this is where you definitely want to just grab a Python intro 200, because this is the standard approach all right so inside our constructor. We want to do the equivalent of an end linea all right, so what n n dot linea is doing? Is it's taking our it's taking our 28 by 28 vector so 768 long vector and we're going to be that's going to be the input to a matrix multiplication.
So we now need to create a something with 768 rows and that's 768 and 10 columns. Ok! So because the input to this is going to be a mini batch of size. Actually, let's move this into a new window 768 by 10, and the input to this is going to be a mini batch of size 64 by 768 right. So we're going to do this matrix product, ok! So when we say in pie chart and in linea it's going to construct this matrix for us right. So since we are not using that we're doing things from scratch, we need to make it ourselves so to make it ourselves, we can say, generate normal random numbers with this dimensionality, which we passed in here, 768 by 10, okay, so that gives us our our randomly Initialized matrix okay, then we want to add on to this. You know we don't just want y equals ax. We want y equals ax plus B, all right, so we need to add on what we call in neural Nets of bias vector. So we create here a bias vector of length: 10, okay, again randomly initialized, and so now here are our two randomly initialized weight tensors. So that's our constructor okay! Now we need to find for word. Why do we need to define for word? This is a pytorch. Specific thing what's going to happen is this is when you create a module in pytorch, the objects that you get back behaves as if it's a function, you can call it with parentheses, which will do it that a moment, and so you need to somehow define What happens when you call it as if it's a function and the answer is tight, which calls a method called forward? Okay, that's just that! That's the pie that the pytorch kind of approach that they picked right.
So when it calls forward, we need to do our actual calculation of the output of this module or letter. Okay. So here is the thing that actually gets calculated in a logistic regression. So basically we take our input X, which gets passed to forward. That's basically how forward works. It gets past the mini-batch and we matrix multiply it by the layer, one weights which we defined up here and then we add on the layer one bias which we defined up here: okay and actually nowadays, we can define this a little bit more elegantly using the Python three matrix multiplication operator, which is the at sign and when you, when you use that, I think you kind of end up with something that looks closer to what the mathematical notation looked like, and so I find that nicer. Okay, all right! So that's that's our linear layer in our logistic regression. You know a zero hidden layer, neural net. So then the next thing we do to that is soft. Next, okay, so we get the output of this matrix model play. Okay, who wants to tell me what the dimensionality of my output of this matrix model play is sorry 64 by 10. Thank you Karen. I should mention for those of you that weren't at deep learning class yesterday, we actually looked at a really cool post from Karam who described how to do structured data analysis with neural nets, which has been like super popular and a whole bunch of people who kind Of said that they've read it and found it interesting, so that was really exciting.
So we get this matrix of outputs and we put this through a softmax, and why do we put it through a softmax? We put it through a softmax because in the end we want - probably you know, for every image. We want a probability that is a 0 or a 1 or a 2 or 3 or 4 all right.  So a softmax does exactly that for us. So, for example, if we weren't picking out you know, numbers from nought to 10, but instead of picking it out, cat dog play an official building. The output of that matrix multiplied for one particular image might look like that. These are just some random numbers and to turn that into a softmax. I first go a to the power of each of those numbers. I sum up those eight of the power offs and then I take each of those eight of the power of z' and divide it by the sum and that softmax that's the definition of softmax so because it was in the power of it means it's always positive, Because it was divided by the sum, it means that it's always between zero and one, and it also means because it's divided by the sum that they always add up to one. So by applying this softmax activation function. So anytime we have a layer of outputs which we call activations, and then we apply some function. Some nonlinear function to that that map's 1:1 scale at a one scalar like softmax.
Does we call that an activation function? Okay, so the softmax activation function takes our outputs and turns it into something which behaves like a probability right. We don't, strictly speaking, need it. We could still try and train something which, where the output directly is the probabilities right, but by creating using this function, that automatically makes them always behave like probabilities. It means there's less for the network to learn, so it's going to learn better alright. So, generally speaking, whenever we design an architecture, we try to design it in a way where it's as easy as possible for it to create something of the form that we want. So that's why we use softmax right. So that's the basic steps right. We have our input, which is a bunch of images right, which is here. It gets multiplied by a weight metrics. We actually also add on a bias right to get a output of the linear function. We put it through a nonlinear activation function in this case softmax, and that gives us our probabilities, so there there that all is pytorch also tends to use the log of softmax for reasons that don't particularly need fatherís. Now, it's basically a numerical stability convenience. Okay, so to make this the same as our version up here that you saw log softmax, I'm going to use log here as well. Okay, so we can now instantiate this class. That is create an object of this class. So I have a question back for the probabilities where we were before hmm.
So if we were to have a photo with a cat and a dog together, would that change the way that that works or does it work in the same basic? Yes, that's a great question, so if you had a photo with a cat and a dog together and you wanted it to spit out both cat and dog, this would be a very poor choice. So softmax is specifically the activation function. We use for categorical predictions where we only ever want to predict one of those things right and so part of the reason. Why is that, as you can see, because we're using e to the right e to the slightly bigger numbers creates much bigger numbers? As a result of which we generally have just one or two large and everything else is pretty small right. So if I like recalculate these rounded numbers a few times, you'll see like it tends to be a bunch of zeroes and one or two high numbers right. So it's really designed to try to kind of make it easy to predict like this. One thing is the thing I want if you're doing, multi-label prediction, so I want to just find all the things in this image rather than using softmax. We would instead use sigmoid. That's a sigmoid recall it would cause each of these between to be between 0 & amp 1, but they would no longer add to 1. It's a good question and like a lot of these details about like best practices, are things that we cover in the deep learning course, and we won't cover heaps of them here and the machine learning course we're more interested in the mechanics.
I guess, but we're trying to do them. We've they're, quick, all right so now that we've got that we can instantiate an object of that class and of course we want to copy it over to the GPU, so we can do computations over there again. We need an optimizer we're we talking about what this is shortly, but you'll see here. We've called a function on our class called parameters, but we never defined a method called parameters and the reason that is going to work is because it actually was defined Forest inside an end up module and so an end up module actually automatically go through the attributes. We've created and finds anything that basically we we said this is a parameter. So the way you say something is a parameter. Is you wrap it in an end off parameter? So this is just the way that you tell pytorch. This is something that I want to optimize. Ok, so when we created the weight matrix, we just wrapped it with an end up parameter, it's exactly the same as a regular 5 torch variable which we'll learn about shortly. It's just a little flag to say: hey, you should you should optimize this, and so, when you call net to parameters on our net to object, we created it goes through everything that we created in the constructor checks to see if any of them are of type Parameter and if so, it sets all of those being things that we to train with the optimizer and we'll be implementing the optimizer from scratch later. Okay, so having done that, we can fit and we should get basically the same answers before 91 ish.
So that looks good all right. So what if we actually built here well, what we've actually built, as I said, is something that can behave like a regular function all right. So I want to show you how we can actually call this as a function so to be able to call it as a function. We need to be able to pass data to it to be able to pass data to it. I'm going to need to grab a mini batch of analyst images. Okay, so we used for convenience the image classifier data from arrays method from fastai, and what that does is it creates a pytorch data loader for us a pytorch data. Loader is something that grabs a few images and sticks them into a mini batch that makes them available, and you can basically say give me another mini batch pick me. Another mini batch, give me another mini batch and so in Python we call these things. Generators generators are things where you can basically say. I want another. I want another. I want another right there's. This kind of very close connection between iterators and generators are not going to worry about the difference between them right now, but you'll see basically to turn to actually get hold of something which we can say. Please give me another of in order to grab something that we can. We can use to generate mini batches. We have to take our data loader, and so you can ask for the training data loader from our model. Data object.
You'll see there's a bunch of different data loader, as you can ask, for you can ask for the test data loader, the Train date, loader, the validation, loader or wintered images, data, loader and so forth. So we're going to grab the training data loader that was created for us. This is a pice and plate, or data loader well slightly optimized by us, but same idea, and you can then say this is a standard Python thing we can say turn that into an iterator turn that into something where we can grab another one at a time From and so once you've done that we've now got something that we can iterate through. You can use the standard Python next function to grab one more thing from that generator: okay, so that's returning and the X's from a mini-batch and the Y's found our mini batch. The other way that you can use generators and iterators in python is with a for loop. I could also said like, for you know, X, mini batch, comma Y mini batch in data loader and then like do something right. So when you do that, it's actually behind the scenes - it's basically syntactic sugar for calling next lots of times. Okay. So this is all standard Python stuff, so that returns a tensor of size 64 by 784, as we would expect right, the the FASTA I library we used defaults to a mini batch size of 64. That's why it's that long! These are all of the background. 0 pixels, but they're not actually zero.
In this case, why aren't they zero yeah they're normalized exactly right, so we subtracted the mean divided by the standard, deviation right, so there there it is so now what we want to do is we want to pass that into our our logistic regression? So what we might do is we'll go variable X, M B equals variable. Okay, I can take my X mini-batch. I can move it onto the GPU because remember my net to object is on the GPU, so our data, for it also has to be on the GPU and then the second thing I do is: I have to wrap it in variable. So what is variable? Do this is how we get for free automatic differentiation. pytorch can automatically differentiate. You know pretty much anything right, any tensor, but to do so takes memory and time. So it's not going to always keep track like to do what have any differentiation. It has to keep track of exactly how something was calculated. We added these things together, we multiplied it by that. We then took the sign blah blah blah right. You have to know all of the steps, because then to do the automatic differentiation it has to take the derivative of each step using the chain rule, multiply them all together right. So that's slow and memory intensive. So we have to opt in to saying like okay. This particular thing we're going to be taking the derivative of later so please keep track of all of those operations for us, and so the way we opt-in is by wrapping a tensor in a variable right.
So that's how we do it and you'll see that it looks almost exactly like a tensor, but it now says variable containing this tensor right. So in pytorch, a variable has exactly identical api to a tensor or actually more specifically a superset with the api of a tensor anything we can do to a tensor. We can do to a variable, but it's going to keep track of exactly what we did. So we can later on, take the derivative okay, so we can now pass that into our net to object. Remember I said you can treat this as if it's a function right so notice we're not calling dot forward, we're just treating it as a function and then remember. We took the log so to undo that I'm taking the X and that will give me my probabilities. Okay, so there's my probabilities and it's got return something of size 64 by 10. So for each image in the mini batch, we've got ten probabilities and you'll see. Most probabilities are pretty close to zero right, and a few of them are quite a bit bigger, which is exactly what we do. We're hope right is that it's like okay, it's not a zero, it's not a one. It's not a two! It is a three. It's not a four, it's not a five and so forth. So maybe this would be a bit easier to read if we just grabbed like the first three of them: okay, just like 10 to the negative, the neg two five, five, four okay and then suddenly here's one which is ten to Nick one right.
So you can kind of see what it's trying to I was trying to do here I mean we could call like net two dot forward and it will do exactly the same thing right, but that's not how all of the pytorch mechanics actually work. It's actually. They actually call it as if it's a function right, and so this is actually a really important idea like because it means that when we define our own architectures or whatever anywhere that you would put in a function you could put in a layer anyway, you put In a layer you can put in a neural net anyway, put it on your neck. You can put in a function because, as far as pytorch is concerned, they're all just things that it's going to call just like as if they're functions so they're all like interchangeable - and this is really important, because that's how we create really good neural nets is By mixing and matching lots of pieces and putting them all together right, let me give an example. Here is my logistic regression which got 91 and a bit percent accuracy. I'm now going to turn it into a neural network with one hidden layer, all right and the way I'm going to do. That is I'm going to create my more layer, I'm going to change this, so it's spits out a hundred rather than ten, which means this one input is going to be a hundred rather than ten.
Now this as it is, can't possibly make things any better at all, yet why is this definitely not going to be better than what I had before yeah? Can somebody pass the yeah bishop, your combination of two linear layers, which is just the same as exactly right? So we've got two linear layers, which is just a linear layer right so to make things interesting, I'm going to replace all of the negatives from the first layer with zeros, because that's a nonlinear transformation and so that nonlinear transformation is called a rectified linear unit. Okay, so n n dot sequential simply is going to call each of these layers in turn for each mini batch right. So dual linear layer replace all of the negatives with zero, do another linear layer and do it softbank's. This is now a neural network with one hidden layer, and so let's try trading that instead, yeah accuracies now been up to 96 %, okay, so the this is. The idea is that the basic techniques we're learning in this lesson like become powerful at the point where you start stacking them together. Okay, can somebody pass the green box there and then yes, Daniel, no reason it was like easier to type an extra zero. Like this question of like how many activations should I have a neural network, layer is kind of part of the the scale of a deep learning practitioner we covered in the deep learning course and not in this class.
When adding that additional, I guess transformation additional layer. Additional this one here is called a nonlinear layer or an activation activation function. Direct activation function. Does it matter that like if you would have done for like to soft Max's, or is that something you cannot do like yo? You can absolutely use the softmax there, but it it's probably not gon na give you what you want and the reason why is that a soft max tends to push most of its activations to zero and an activation just be clear, like I've had a lot of Questions in deep learning course about like what's an activation. An activation is the value that is calculated in a layer right. So this is an activation right. It's not a weight. A weight is not an activation. It's the value that you calculate from a layer, so soft max will tend to make most of its activations pretty close to zero, and that's the opposite of what you want. You generally want your activations to be kind of as rich and diverse and and used as possible, so nothing to stop you doing it, but it probably won't work very well. Basically, pretty much all of your layers will be followed by non by nonlinear activation functions. That will nearly always be value, except for the last layer. It's going to or three layers deep. Do you want to switch up these activation layers? That's a great question. So, if I wanted to go deeper, I would just do that.
Okay, that's an outer hidden layer Network, so I think I'd heard you said that there are a couple of different activation functions like that rectified linear units. What are some examples, and why would you use each yeah great question so basically like, as you add, like more linear layers, you kind of got your input comes in and you put it through a linear layer and then a nonlinear layer, linear layer, nonlinear layer learning A linear layer and then the final nonlinear layer, the final nonlinear layer, as we've discussed, you know if it's a multi category classification, but you only ever pick one of them. You would use softmax if it's a binary classification or a multi-label classification, where you're predicting multiple things. You would use sigmoid. If it's a regression, you would often have nothing at all right, although we learnt in last night's DL course where sometimes you can use sigmoid there as well so they're, basically the options main options for the final layer for the hidden layers. You pretty much always use value right, but there is a another another one. You can pick, which is kind of interesting, which is called leaky value, and it looks like this and basically, if it's above zero, it's y equals x and if it's below zero. It's like y equals 0.1 X, okay, so that's very similar to value, but it's you know, rather than being equal to 0 under X. It's it's like something close to that so they're, the main to rally and lake here Lu. There are various others, but they're kind of like things that just look very close to that. So, for example, there's something called Lu, which is quite popular but, like you know the details, don't matter too much honestly like that they're like aou, is something that looks like this, but it's slightly more curvy in the middle and it's kind of like it's not generally, Something that you so much pick based on the data set, it's more like over time.
We just find better activation functions so two or three years ago, everybody is value. You know a year ago, pretty much everybody used Lake Erie Lu today I guess probably most people are starting to move towards ALU, but honestly that the choice of activation function doesn't matter terribly much actually, and you know, people have actually showed that you can use like A pretty arbitrary nonlinear activation functions like even a sine wave, and it still works okay. So, although what we're going to do today is showing how to create this network with no hidden layers to turn it into that Network, which is 96 % ish accurate, is it will be trivial right and in fact it's something. You should probably try and do during the week right is to create that version. Okay, so now that we've got something where we can take our network parsing our variable and get back some predictions, that's basically all that happened when we called fish. So we're going to see how how that that approach can be used to create this to cast a gradient descent. One thing to note is that the to turn the predicted probabilities into a predicted like which digit is it we would need to use AG max. Unfortunately, pytorch doesn't call it ad max instead, pytorch just calls it max and max returns two things: it returns the actual max across this axis. So this is across the columns right and the second thing it returns is the index of that maximum right.
So so the equivalent of arc max is to call max and then get the first indexed thing. Okay, so there's our predictions right. If this was in numpy, we would instead use MP. Dot admits. Okay, all right. So here are the predictions from our hand, created logistic regression and, in this case, looks like we've got all, but one correct. So the next thing we're going to try and get rid of in terms of using libraries is for try to avoid using the matrix multiplication operator and instead we're going to try and write that by hand.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
So this next part we're going to learn about something which kind of seems it kind of it's. Gon na seem like a a minor little kind of programming idea, but actually it's going to turn out that, at least in my opinion, it's the most important programming concept that will teach in this course and it's possibly the most important programming kind of concept. In all of all the things you need to build machine learning, algorithms and it's the idea of broadcasting and the idea I will show by example, if we create an array of 10 6 NIC 4 and an array of 2 8 7 and then add the two Together, it adds each of the components of those two arrays in turn. We call that element wise. So in other words, we didn't have to write a loop right back in the old days. We would have to have looped through each one and added them and then concatenate them together. We don't have to do that today. It happens for us automatically. So in lump a we automatically get element wise operations. We can do the same thing with pytorch, so in first day I would just add a little capital T to turn something into a pipe watch. Tensor all right and if we add those together exactly the same thing, all right so element wise operations are pretty standard in these kinds of libraries. It's interesting not just because we don't have to write the for loop right, but it's actually much more interesting because of the performance things that are happening here.
The first is, if we were doing a for loop right, if we were doing a four loop that would happen in Python right. Even when you use play torch, it still does the for loop in Python. It has no way of like optimizing a for loop, and so a for loop in Python is something like 10,000 times slower than in C. So that's your first problem, like I remember, is like 1,000 or 10,000. The second problem, then, is that you don't just want it to be optimized in C, but you want C to take advantage of the thing that your all of your CPUs do to something called Cindy single instruction, multiple data, which is yours, your CPU, is capable of Taking eight things at a time right in a vector and adding them up to another vector with eight things in in a single CPU instruction right. So if you can take advantage of Sim D you're immediately eight times faster, it depends on how big the data type is. It might be, four might be eight. The other thing that you've got in your computer is you've, got multiple processes, multiple cores, so you've probably got like. If this is inside happening on one side, one core you've probably got about four of those okay. So if you're using Cindy your eight times faster, if you can use multiple cores than your 32 times faster and then, if you're doing that in C, you might be something like 32 times per thousand times faster right. And so the nice thing is that when we do that, it's taking advantage of all of these things - okay, better still, if you do it in pytorch and your data was created with CUDA to stick it on the GPU, then your GPU can do about 10,000. Things at a time all right so that'll be another hundred times faster than C all right, so this is critical to getting good performance. Is you have to learn how to write Lewis code by taking advantage of these element-wise operations and like it's not it's a lot more than just plus.
I can also use less then right and that's going to return 0, 1 1 or if we go back to numpy false true true, and so you can kind of use this to do all kinds of things without looping. So, for example, I could now multiply that by a and here are all of the values of a as long as they're less than B or we could take the mean this is the percentage of values in AE that are less than B, all right. So, like there's a lot of stuff, you can do with this simple idea, but to take it further right to take it further than just this element: wise operation, we're going to
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Have to go the next step to something called broadcasting, so, let's take a five-minute break, come back at 2:17 and we'll talk about broadcasting so broadcasting. This is the definition from the numpy documentation of broadcasting and I'm going to come back to it in a moment rather than reading it now. But let's start by looking an example of broadcasting. So a is a array with one dimension, also known as a Rank: 1 tensor, also known as a vector we can say a greater than 0. So here we have a Rank 1 tensor right and a rank 0 tensor right. A rank. 0 tensor is also called a scalar. The rank 1 tensor is also called a vector, and we've got an operation between the two all right. Now, you've probably done it a thousand times without even noticing that's kind of weird right that you've got these things of different ranks and different sizes. So what is it actually doing right? But what it's actually doing is it's, taking that scaler and copying it here-here-here right and then it's actually going element-wise tan is greater than zero. Six is greater than zero. Minus pore is greater than zero. You have been giving us back the three answers right and that's called broadcasting broadcasting means copying one or more axes of my tensor to allow it to be the same shape as the other tensor.
It doesn't really copy it, though, what it actually does is it stores this kind of internal indicator that says pretend that this is a vector of three zeros, but it actually just like, rather than kind of going to the next row, we're going to the next scaler. It goes back to where it came from if you're interested in learning about this. Specifically it's they set the stride on that axis to be zero. That's a minor advanced concept for those who procures, so we could do a plus one right. It's going to broadcast the scalar 1 to be 1 1, 1 and then do element wise addition. If we could do the same with a matrix right, here's our matrix 2 times the matrix is going to broadcast to to be to to to to to to to to to and then do element wise multiplication right. So that's our kind of most simple version of broadcasting, so here's a slightly more complex version of broadcasting. Here's an array called C right. So this is a rank. One tensor and here's our matrix M from before rank two tensor. We can add M plus C alright. So, what's going on here, one two, three: four: five: six: seven, eight nine, that's M all right and then C: 10. 20. 30. You can see that what it's done is to add that to each row right 11, 22, 33, 14, 25, 36, and so we can kind of figure. It seems to have done. The same kind of idea is broadcasting a scaler. It's like made copies of it and then it treats those as if it's a rank two matrix, and now we can do element wise addition that make sense right now.
That's yes can. Can you pass that Devin over there? Thank you so it's like by looking at this example. It like copies it done, making new rows. So how would we want to do it if we wanted to get new columns, I'm so glad you asked so. Instead we would do this 10. 20. 30. All right and then copy that 10, 20, 30, 10, 20, 30 and now treat that as our matrix, so to get numpy to do that, we need to not pass in a vector but to pass in a matrix with one column, a rank, two tensor right. So basically, it turns out that numpy is going to think of a rank 1 tensor for these purposes, as if it was a rank, two tensor, which represents a row right, so in other words, that it is 1 by 3. All right, so we want to create a tensor which is 3 by 1 there's a couple of ways to do that: one is to use NP expand, imps and if you then pass in this argument, it says please insert a length 1 axis here, please so in Our case, we want to turn it into a 3 by 1, so if we said expanding c comma 1, okay, so if we say expanding C comma one, it changes the shape to three comma one. So if we look at what that looks like that looks like a column okay, so if we now go that plus M, you can see it's doing exactly what we hoped it would do, alright, which is to add ten, twenty thirty to the column. Ten, twenty thirty to the column, ten, twenty thirty to the column.
Okay. Now because the location of a unit axis turns out to be so important, it's really helpful to kind of experiment with creating these extra unit axes and know how to do it easily and MP. Dot, expand ins isn't, in my opinion, the easiest way to do this. The easiest way the easiest way is to index into the tensor with a special index. None and what none does is. It creates a new axis in that location of length, one right. So this is going to add a new axis at the start of length, one. This is going to add a new axis at the end at length one or why not they're both right. So if you think about it like a tensor which has like three things in it could be of any rank, you like right, you can just add, you know, taxis all over the place, and so that way we can kind of decide how we want our broadcasting To work so there's a pretty convenient thing in numpy called broadcast, and what that does is it takes our vector and broadcasts it to that shape and shows us what that would look like right. So if you ever like unsure of what's going on in some broadcasting operation, you can save broadcast too, and so, for example, here we could say like rather than three comma three, we could say MJ right and see exactly what's happening, gon na happen, and so that's. What's gon na happen before we add it to em right so if we said turn it into a column, that's what that looks like makes sense, so that's kind of like the intuitive definition of broadcasting, and so now, hopefully we can go back to that numpy documentation And understand what it means right broadcasting describes how numpy is going to treat arrays of different shapes when we do some operation right.
The smaller array is broadcast across the larger array. By smaller array, they mean lower rank tensor, basically, our broadcast across the light. The higher rank tensor, so they have compatible shapes it vector, eise's array operation, so vectorizing generally means like using sim D and stuff like that, so that multiple things happen. At the same time, all the looping occurs in C, but it doesn't actually make needless copies of theta. It kind of just acts as if it had okay. So there's our definition. Now, in deep learning, you very often deal with tensors of rank four or more, and you very often combine them with tensors of rank one or two and trying to just rely on intuition to do that correctly as nearly impossible. So you really need to know the rules. So here are the rules. Okay, here's my shop, here's C dot shape. So the rule are that we're going to compare the shapes of our two tensors element-wise, we're going to look at one at a time and we're going to start at the end. All right so look at the trailing dimensions and then go towards the front. Okay and so two dimensions are going to be compatible when one of these two things is true right. So let's check right, we've got our our M & amp C. Compatible m is 3. 3 c is 3 right, so we're going to start at the end, trailing dimensions first and check. Are they compatible they're compatible if the dimensions are equal? Okay, so these ones are equal, so they are compatible right. All right.
Let's go to the next one. Uh-Oh we're missing all right C is missing something. So what happens if something is missing, as we insert a 1 okay, that's the rule, all right, and so let's now check are these compatible. One of them is one yes, they're compatible. Okay, so now you can see why it is that numpy treats the one dimensional array as if it is a rank, two tensor which is representing a row. It's because we're basically inserting a one at the front. Okay, so that's the rule, so, for example, this is something that you very commonly have to do, which is you start with like an image there, like 256 pixels by 256 pixels by 3 channels, and you want to subtract the mean of each channel right. So you've got 256 by 256 by 3 and you want to subtract something of length 3 right. So yeah. You can do that absolutely because 3 & amp, 3 are compatible because they're the same right, 256 and empty is compatible. It's going to insert a 1 256 and empty is compatible. It's going to insert of 1 okay, so you're going to end up with this is going to be broadcast over all of this access and then that whole thing will be broadcast over this access and so we'll end up with a 256 by 256 by 3, effective Tensor here right so, interestingly, like very few people in the data science or machine learning, communities, understand broadcasting and the vast majority of the time.
For example, when I see people doing pre-processing for computer vision like subtracting the mean they always write loose over the channels right - and I kind of think like it's it - it's like so handy to not have to do that, and it's often so much faster to not Have to do that so if you get good at broadcasting, you'll have this like super useful skill that very very few people have and and
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Like it's, it's it's an ancient skill. You know it goes. It goes all the way back to the days of APL, so APL was from the late. 50S stands for a programming language and Kenneth Iverson wrote this paper called notation as a tool for thought in which he proposed a new math notation, and he proposed that if we use this new math notation, it gives us new tools for thought and allows us to Think things we couldn't before and one of his ideas was broadcasting not as a computer programming tool but as a piece of math notation, and so he ended up implementing this notation as a tool for thought. As a programming, language called APL and his son has gone on to further develop that into a piece of software called J, which is basically what you get when you put sixty years of very smart people working on this idea, and with this programming language, you can Express very complex mathematical ideas, often just where the line of code or two, and so I mean it's great, that we have J, but it's even greater that these ideas have found their ways into the languages we all use like in Python. The numpy and pytorch libraries all right - these are not just little kind of niche ideas is like fundamental ways to think about math and to do programming. Let me give an example of like this kind of notation as a tool for thought. Let's, let's look here: we've got C right here: we've got C, none right notice. This is now up to square brackets right, so this is kind of like a one row vector tensor.
Here it is a little column. So what is round ones? Okay? What's that gon na do ever think about it, anybody want to have a go, can't even talk through your thinking. Okay, can we pass the check over there? Thank you. Yes, absolutely so. Take us through your thinking, how's, that going to work so the diagonally lumens can be directly visualized from the squares then cross 1020 clothes, 20 and 30 plus 30. And if you multiply the first row for this column, you get the first row of the matrix mm-hmm. So, finally, we will get our 3 cross 3 matrix, yeah and so to think of this in terms of like those broadcasting rules, we're basically taking this column right, which is a rank 3 comma 1 right and this kind of roll sorry have dimension 3, comma 1 And this row, which is of dimension 1, comma, 3 right and so to make these compatible with our broadcasting rules right. This one here has to be duplicated 3 times because it needs to match this. Okay and now this one's going to have to be duplicated three times to match this okay, and so now I've got two matrices to do an element-wise product of, and so, as you say there is that outer product right now. The interesting thing here is that suddenly now that this is not a special mathematical case, but just a specific version of the general idea of broadcasting we can do like and out a plus or we can do an order greater than right or whatever right.
So it's suddenly we've kind of got this this this concept, that we can use to build new ideas, and then we can start to experiment with those new ideas, and so you know, interestingly, numpy actually uses this. Sometimes, for example, if you want to create a grid, this is how numpy does it all right? Actually, this is kind of the sorry. Let me show you this way. If you want to create a grid, this is how numpy does it. It actually returns zero. One two three four and zero one, two three four one is a column. One is a row, so we could say like okay, that's X, grid, comma Y grid, and now you could do something like row. I mean we could obviously go like that right and so suddenly we've expanded that out into a grid right and so yeah. It's kind of interesting how like some of these, like simple little concepts, kind of get built on and built on and built on. So if you lose something like APL or J, it's this whole environment of layers and layers and layers of this. We don't have such a deep environment in numpy, but you know you can certainly see these ideas of like broadcasting coming through in in simple things like how do we create a grid in non-pay, so yeah? So that's that's.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Broadcasting - and so what we can do with this now is use this to implement matrix multiplication ourselves. Ok, now, why would we want to do that? Well, obviously, we don't write matrix. Multiplication has already been handled perfectly nicely for us by our libraries, but very often you'll, find in all kinds of areas in in machine learning and particularly in deep learning that there'll be particular types of linear function that you want to do that aren't quite done for You right so, for example, there's like whole areas called like tensor regression and tensor decomposition, which are really being developed a lot at the moment and they're kind of talking about, like how do we take like higher rank, tensors and kind of turn them into combinations of Rows columns and faces - and it turns out that when you can kind of do this, you can basically like deal with really high dimensional data structures with not much memory and not with not much computation time. For example, there's a really terrific library called tensile e, which does a whole lot of this kind of stuff for you. So it's a really really important area. It covers like all of deep learning lots of modern machine learning in general, and so even though you're not going to like define matrix multiplication, you're very likely to wanted to find some other slightly different tensor product. You know. So it's really useful to kind of understand how to do that.
So let's go back and look at our matrix and our our 2d array in 1d array rank two tensor rank 1 tensor and remember. We can do a matrix multiplication using the @ sign or the old way and PMML okay, and so what that's actually doing when we do that is we're basically saying ok, 1 times 10 plus 2 times 20 plus 3 times 30 is 140 right, and so we Do that for each row and we can go through and do the same thing for the next one and for the next one to get our result right. You could do that in torch as well. We could make this a little shorter, okay, same thing, okay, but that is not matrix multiplication. What's that, okay element, wise specifically, we've got a matrix and a vector so broadcasting, okay, good, so we've got this is element wise with broadcasting but notice the numbers it's created. 10. 40. 90. Are the exact three numbers that I needed to calculate when I did that first piece of my matrix multiplication? So in other words, if we sum this over the columns, which is axis equals 1, we get our matrix vector product okay. So we can kind of do this stuff without special help from our library. So now, let's expand this out to a matrix matrix product. So a matrix, matrix product looks like this. This is this great site called matrix multiplication XYZ and it shows us this is what happens when we multiply two matrices.
Okay, that's what matrix multiplication is operationally speaking, so, in other words, what we just did there was we first of all took the first column with the first row to get this one, and then we took the second column with the first row to get that one Right so we're basically doing the thing we just did the matrix vector product we're just doing it twice right once with this column and once with this column, and then we concatenate the two together okay, so we can now go ahead and do that, like so M Times the first column dot some M times the second column, that's some, and so there are the two columns of our matrix multiplication. Okay, so I didn't want to like make our code too messy, so I'm not going to actually like use that, but like we have it there now, if we want to, we don't need to use torch or numpy matrix multiplication anymore. We've got, we've got our own! That we can use using nothing but element wise operations, Broadcasting and some okay. So this is our logistic regression from scratch. Class again I just copied it here: here's where we instantiate the object copy to the GPU. We create an optimizer which we'll learn about in a moment and we call fit okay. So the goal is to now repeat this without needing to call fit so to do that. We're going to need a loop which grabs a mini batch of data at a time and with each mini batch of data.
We need to pass it to the optimizer and say: please try to come up with a slightly better set of predictions for this mini batch right. So, as we learnt in order to grab a mini batch of the training set at a time, we have to ask the model data object for the training data loader. We have to wrap it an iterator to create an iterator or a generator, and so that gives us our our data. Loader. Okay, so pytorch calls this a data loader. We actually wrote our own class today as a loader, but it's it's all. It's basically the same idea, and so the next thing we do is we grab the X and the y tensor the next one from our data. Loader, okay wrap it in a variable to say: I need to be able to take the derivative of the calculations using this, because if I can't take the derivative, then I can't get the gradients and I can't update the weights all right and I need to put It on the GPU, because my module is on the GPU, and so we can now take that variable and pass it to the objects that we instantiated our logistic regression remember now module we can use it as if it's a function, because that's how high-touch works and That gives us a set of predictions, as we saw on scene before okay. So now we can check the loss and the loss, we're defined as being a negative log likelihood, loss object, okay and we're going to learn about how that's calculated in the next lesson and for now think of it.
Just like a root mean squared error, but for classification problems, so we can call that also just like a function, so you can kind of see this. It's very general idea in pytorch that you know kind of treat everything ideally like it's a function. So in this case we have a loss, negative log likelihood loss object. We could treat it like a function, we pass in our predictions and we pass in our actuals all right again. The actuals need to be turned into a variable and put on the GPU, because the loss is specifically the thing that we actually want to take the derivative of right, so that gives us our loss and there it is that's our loss to point four three. Okay, so it's a variable and because it's a variable, it knows how it was calculated all right. It knows it was calculated with this loss function. It knows that the predictions were calculated with this network. It knows that this network consisted of these operations, and so we can get the gradient automatically and so to get the gradient recall, I'll drop backward. Remember, L is the thing that contains our loss right, so L drop backward is, is something which is added to anything. That's a variable, you even called drop backward, and that says please calculate the gradients, okay and so that calculates the gradients and stores them inside that that the basically for each of the weights that was used. It used each of the parameters that was used to calculate that it's now stored a dot grad.
Well, we'll see it later. It's basically stored the gradient right, so we can then call optimizer dot step and we're going to do this bit step manually shortly and that's the bit that says: please make the weights a little bit better right, and so what optimizer dot step is doing. Is it saying like okay, if you had like a really simple function like this right, then what the optimizer does is it says: okay, let's pick a random starting point right and let's calculate the value of the loss right. So here's our parameter. Here's our loss right! Let's take the derivative right, the derivative tells us which way is down, so it tells us. We need to go that direction. Okay and we take a small step, and then we take the derivative again and we take a small step derivative again. Take a small step drove it again, take a small step until eventually we're taking such small steps that we stop. Okay. So that's what gradient descent does?  Well, we basically take the derivative here. So let's say derivative, there is like eight right and we multiply it by a small number like say 0.01, and that tells us what step size to take. This small number here is called the learning rate and it's the most important hyper parameter to set right. If you pick two smaller learning rate, then your steps down are going to be like tiny and it's going to take you forever.
Okay, too big a learning rate and your jump too far right and then you'll jump too far and your diverge rather than converge. Okay, we're not going to talk about how to pick a learning rate in this class, but in the deep learning class we actually show you a specific technique that very reliably picks a very good learning rate. So that's basically what's happening right, so we calculate the derivatives and we call the optimizer that does a step, in other words, update the weights based on the gradients and the learning rate. We should hopefully find that, after doing that, we have a better loss than we did before, so I just really ran this and got a loss here of 4.16 and after one step it's now 4.0. 3. Okay, so it worked the way we hoped it word based on this mini batch. It updated all of the weights in our network to be a little better than they were, as a result of which our loss went down. Okay, so let's turn that into a training loop, all right, we're going to go through a hundred steps grab one more mini batch of data from the data loader calculate our predictions from our network calculate our loss from the predictions and the actuals. Every 10 goes we'll print out the accuracy just take the mean of the whether they're, equal or not, 1 pi taut, specific thing you have to 0 the gradients. Basically, you can have networks where, like you've, got lots of different loss functions that you might want to add all of the gradients together right.
So you have to tell play torch like when to set the gradients back to zero right, so this just says: set all the gradients to zero calculate the gradients, let's quote backward and then take one step of the optimizer, so update the weights using the gradients and The learning rate - and so once we run it, you can see, the loss goes down and the accuracy goes up. Okay, so that's the basic approach, and so next lesson we'll see what that does alright. Well, we're looking in detail we're not going to look inside here, as I say, we're going to basically take the calculation of the derivative says, that's a given right, but basically what's happening there in any kind of deep network. You have kind of like a function.  That might be like a rally and you pass the output of that into another function that might be another linear net Lenny Olea. You pass that into another function. That might be another value and so forth right. So at these, these deep networks are just functions of functions of functions, so you could write them. Mathematically like that right and so all backprop does is. It says, let's just simplify this down to the two version, as we can say: okay, u equals f of X right and so therefore the derivative of G of f of X is we can calculate with the chain rule as being G. You f dash X, right, and so you can see we can do the same thing for the functions of the functions of the functions, and so when you apply a function to a function of a function, you can take the derivative just by taking the product of The derivatives of each of those layers, okay and in neural networks.
We call this back propagation okay. So when you hear back propagation, it just means use the chain rule to calculate the derivatives, and so when you see a neural network to find like here right like if it's defined sequentially literally all this means is apply. This function to the input apply this function to that apply. This function to that apply this function to that right, so this is just defining a composition of a function, to a function, to a function, to a function, okay and so yeah. So, although we're not going to bother with calculating the gradients ourselves, you can now see why it can do it right as long as it has internally. You know a it knows like: what's the what's the derivative of ^, what's the derivative of sine, what's the derivative of + and so forth, then our Python code in here is just combining those things together, so it just needs to know how to compose them together With the chain rule and where it goes, okay, okay, so I think we can leave it there for now and yeah and in the next class, we'll go and we'll see how to write our own optimizer and then we'll have solved em mist from scratch. Ourselves see you then


Installation¶
Note
In order to make use of the GPU, make sure that you have the correct prerequisites installed.
zyglrox
relies on Tensorflow 1.15.0, which has the same requirements as 1.14.0:
Python 3.3-3.7
CUDNN 7.4
CUDA 10.0
See the tensorflow documentation for more information.
Setup a Conda environment using the environment.yml
file
Clone the git
git clone https://gitlab.com/rooler/zyglrox.git
Create a virtual environment
conda env create -f environment.ymlIt is good to know that you can easily update the env
conda env update -f environment.ymlTo remove the environment use
conda remove --name qve --all
Activate the environment as follows:
conda activate zyglrox
Finally, navigate to the zyglrox
directory and run
python setup.py install
If you care able to run the tutorials, you’re good to go!


Let us make the Echo program just a little bit more interesting: We add an initial welcome phrase and a prompt with a line number. Here is the program:
module Echo2 where import POSIX echo2 env = class count := 1 prompt = do env.stdout.write (show count++"> ") count := count+1 echo str = action env.stdout.write str prompt result action env.stdin.installR echo env.stdout.write "Welcome to Echo2!\n" prompt root = newRoot echo2
The class definition now has two more components:
In addition, the interface returned upon instantiation is a dedicated method that takes care of all program-specific initialization. Such a class allows for a particularly simple root definition, as shown in the last line of the example. The wrapper function newRoot has the following definition in module POSIX:
newRoot :: (Env -> Class Action) -> World -> Cmd () () newRoot cl world = do env = new posix world init = new cl env init
Some additional details:
show is a function that converts the integer value count to a string; ++ denotes string concatenation. Both these are defined in the Prelude, a library module which is implicitly imported by all modules.
The user input string given to echo will include the newline; hence so will the string echoed to stdout. But to write the welcome phrase on a separate line, it will have to end with an explicit newline.


| Trees | Indices | Help |
|---|
|
|
*************************************************************************** Persistence of Vision Ray Tracer Scene Description File File: 10036-1_car.pov Vers: 3.6 Desc: Lego Set "Pizza To Go" Subpart: car Link: www.peeron.com/inv/sets/10036-1 Date: 04/2013 Auth: Chocokiko ***************************************************************************
|
|||
|
| Trees | Indices | Help |
|---|
| Generated by Epydoc 3.0.1 on Sat Feb 6 12:15:27 2016 | http://epydoc.sourceforge.net |


Vel aliquam quis, nulla pede mi commodo tristique nam hac Luctun torquent velit felis commodo pellentesque nulla cras vel aliqua quisan nulla pede mi commoda
Vel aliquam quis, nulla pede mi commodo tristique nam hac. Luctus torquent velit felis commodo pellentesque nulla cras. Tincidunt hacvel alivquam quis nulla pede mi commodo tristique nam hac luctus torquent
Molestie lorem est faucibus faucibus erat phasellus placerat proin aptent
Molestie lorem est faucibus faucibus erat phasellus placerat proin aptent
Molestie lorem est faucibus faucibus erat phasellus placerat proin aptent
Molestie lorem est faucibus Faucibus erat phasellus placerat proinint aptent pulvinar fusce nostra porta sem platea nec, donec fusce erat Molestie lorem est faucibus faucibus erat phasellus placerat proin aptent pulvinar fusce nostra port
Vel aliquam quis, nulla pede mi commodo tristique nam hac. Luctus torquent velit felis commodo pellentesque nulla cras. Tincidunt hacvel alivquam quis nulla pede mi commodo tristique nam hac luctus torquent
Posuere porttitor justo ornare dictum ultricies enim imperdiet integer habitant.
Posuere porttitor justo ornare dictum ultricies enim imperdiet integer habitant.
Posuere porttitor justo ornare dictum ultricies enim imperdiet integer habitant.
Posuere porttitor justo ornare dictum ultricies enim imperdiet integer habitant.
Molestie lorem est faucibus Faucibus erat phasellus placerat proinint aptent pulvinar fusce nostra porta sem platea nec, donec fusce erat Molestie lorem est faucibus faucibus erat phasellus placerat proin aptent pulvinar fusce nostra port
Vel aliquam quis, nulla pede mi commodo tristique nam hac. Luctus torquent velit felis commodo pellentesque nulla cras. Tincidunt hacvel alivquam quis nulla pede mi commodo tristique nam hac luctus torquent
Attend only first day
Attend only first day
Attend only first day
Vel aliquam quis, nulla pede mi commodo tristique nam hac. Luctus torquent velit felis commodo pellentesque nulla cras. Tincidunt hacvel alivquam
Executive, ACI Group
Also made from. Give may saying meat there from heaven it lights face had is gathered god earth light for life may itself shall whales made they're blessed whales also made from give may saying meat. There from heaven it lights face had also made from. Give may saying meat there from heaven
Executive, ACI Group
Also made from. Give may saying meat there from heaven it lights face had is gathered god earth light for life may itself shall whales made they're blessed whales also made from give may saying meat. There from heaven it lights face had also made from. Give may saying meat there from heaven
Executive, ACI Group
Also made from. Give may saying meat there from heaven it lights face had is gathered god earth light for life may itself shall whales made they're blessed whales also made from give may saying meat. There from heaven it lights face had also made from. Give may saying meat there from heaven


20-year-old forgotten left-back Gallacher trains with Klopp’s first-team
by Wash N'GogNot many will remember Liverpool signing Tony Gallacher from Falkirk back in January 2018…
The then-teenager arrived from Scotland to compete for our U23s – but we’ve barely heard about him since.
The likes of Adam Lewis and Yasser Larouci have been given the occasional chance in friendlies at left-back, but Gallacher, now 20, has never been in contention.
During the international break though, the Scot has been training with the first-teamers not away on duty with their countries which according to This Is Anfield is ‘rare’…
Liverpool have another left-back coming through the ranks in Colombian Anderson Arroyo, but he’s currently on loan at Mladá Boleslav in the Czech Republic!
Out of the current crop of young left-backs, we have highest hopes for Larouci based on what we saw in the pre-season friendlies – but it’ll be a very long time before one of them is competing with Andy Robertson for minutes.
James Milner is his first reserve, and we may see the vice-captain v Crystal Palace on Saturday due to Robbo’s injury.


| Public Fields | |
|---|---|
| deviceCount | |
| driver | |
| inputDevices | A stack of available MIDI inputs. |
| openInputs | A map of open MIDI inputs |
| openOutputs | A map of open MIDI outputs |
| outputDevices | A stack of available MIDI outputs. |
| sysexBuffer | |
| version |
| Public Constructors | |
|---|---|
| New | scan midi bus |
| Public Methods | |
|---|---|
| CloseAll | close all open MIDI channels |
| CloseInput | Close specified MIDI input channel |
| CloseOutput | Close specified MIDI output channel |
| EventContent | Return contents of Sysex message |
| EventData | Return packed 4 bytes of current MIDI event |
| EventDataBytes | Return separate 4 bytes of current MIDI event |
| EventTime | Return timestamp of event in seconds |
| HasEvent | Returns number of MIDI events in queue |
| InputName | Name of input channel |
| OpenInput | Open specified MIDI input channel |
| OpenOutput | Open specified MIDI output channel |
| OutputName | Name of output channel |
| SendMessage | Send MIDI data to specified MIDI output channel |
| SendMessages | Send raw MIDI Sysex data (MSB ignored) |
| Sleep | High resolution PortMidi sleep command. |


Which Logo do you prefer?
Select your favorite logo, insert your email and submit. Max 3 votes.
Get Results


Highlights From The 2019 Pandas Hack
Source: pandas | NumFOCUS | Author: nf-admin | Published: Sep 13, 2019
The post Highlights From The 2019 Pandas Hack appeared first on NumFOCUS.
Dataframe summit @ EuroSciPy write up
Source: datapythonista blog - pandas | Author: Marc Garcia | Published: Sep 11, 2019
Last week took place in Bilbao, Spain, EuroSciPy 2019. This year we introduced the maintainers track a room dedicated to discussions among maintainers. The idea is similar to the birds of a feather or unconference sessions of other conferences. But focussed on open source maintainers and contributors. And we scheduled …
GeoPandas now uses the pandas ExtensionArray interface
Source: Joris Van den Bossche | Author: Joris Van den Bossche | Published: Aug 13, 2019
Short summary: the upcoming 0.6.0 release of GeoPandas will feature a refactor based on the pandas ExtensionArray interface. Although this change should keep the user interface mostly stable, it enables more robust integration with pandas and allows for more upcoming changes in the future. And given the invasive code changes under the hood, testing is very welcome!
pandas: The two cultures
Source: datapythonista blog - pandas | Author: Marc | Published: Jul 22, 2019
Leo Breiman was a distinguished statistician at UC Berkeley, known among other things for his major contributions to CART (decision trees), and ensemble techniques, mainly bootstrap aggregation. Combining both, he was able to define one of the most popular machine learning models even today (18 years after the publication of …
pandas + binder
Source: datas-frame | Author: Tom Augspurger | Published: Jul 21, 2019
This post describes the start of a journey to get pandas' documentation running on Binder. The end result is this nice button:
For a while now I've been jealous of Dask's examples repository. That's a repository containing a collection of Jupyter notebooks demonstrating Dask in action. It stitches together some …
Inaugural NumFOCUS Awards and New Contributor Recognition
Source: pandas | NumFOCUS | Author: Gina Helfrich, Ph.D. (Communications Director, NumFOCUS) | Published: Sep 27, 2018
The post Inaugural NumFOCUS Awards and New Contributor Recognition appeared first on NumFOCUS.
Tabular Data in Scikit-Learn and Dask-ML
Source: datas-frame | Author: Tom Augspurger | Published: Sep 17, 2018
Scikit-Learn 0.20.0 will contain some nice new features for working with tabular data. This blogpost will introduce those improvements with a small demo. We'll then see how Dask-ML was able to piggyback on the work done by scikit-learn to offer a version that works well with Dask Arrays …
Distributed Auto-ML with TPOT with Dask
Source: datas-frame | Author: Tom Augspurger | Published: Aug 30, 2018
This work is supported by Anaconda Inc.
This post describes a recent improvement made to TPOT. TPOT is an automated machine learning library for Python. It does some feature engineering and hyper-parameter optimization for you. TPOT uses genetic algorithms to evaluate which models are performing well and how to choose …


{% extends "_post.html" %} {%hyde title: "Hula Hula Hula" created: 2005-02-15 19:39:01 %} {% block article %} {%article%}
I didn’t want to be left out.


ATTRACT
Copyright (c) 2005 Jaroslaw Cichocki
This game is played on an empty NxN square board (say 8x8):
- TURNS - At each turn, each player must do the following actions:
- First, drop a friendly stone on an empty cell;
- Second, all pieces in the same horizontal and vertical rows move one cell nearer to the placed piece, if possible.
- GOAL - If at the end of the move, there are any pieces that have an enemy piece at each diagonally adjacent cell, 4 in all, then the player with the most such pieces wins. If these numbers are equal, the game continues.
An example White's turn.
He plays at [1] attracting four different stones (check next diagram), and by doing so, he wins by making one winning pattern (the marked stones) while Black does not have any.
Check Jaroslaw's website for more detailed information.


5 Types of branches
- Master branch-the master branch is the production branch. The code living in the master branch
is what the running product has.
- Release branch-the release branch is pulled from the master branch. The release branch is the
branch where the code is pushed to when ready for release. This is where the code lives before it makes it
to production. Additional unit testing is run on code on this branch.
- Hotfix branch-the hotfix branch is pulled from master branch. The purpose of the hotfix branch
is to fix issues in production in a quick manner without a sprint. This branch is only used it's an occasion
of a urgent issue.
- Develop branch-the develop branch is pulled from the master branch. The develop branch is the
branch where any code that has been developed in the current sprint lives. This is code meant for the next
release but is not ready for release yet.
- Feature branch-the feature branch is pulled from the development branch. The feature branch
is where the development of the code happens. A feature branch is used per feature.
A ticket process
- First you will need your ticket for the feature that is being worked on.
- Checkout master branch.
By default you should be on master branch unless if switched to a different branch.
- Do a git fetch and git pull in order to pull exact code from master into local.
- Checkout develop branch.
- Do a git fetch and git pull in order to ensure code on develop branch is up to date with code on master.
- Checkout a new branch. The newly created branch is considered a feature branch and will branch out
out from develop.
- New branch will have the code from develop up to this point
- Coding can not begin on feature branch.
- Once feature has been completed and ready to be tested, merge feature branch into development branch.
- If no merge conflicts occured, code should have successfully joined the code in develop./li>
- Testing will occur on develop branch.
- Once feature is verified to be ready for production, develop branch is merged into release branch.
- Once code is on release branch, additional testing is done to verify code is absolutely ready to be in production.
- Now release branch will be merged into master branch.
- Master branch will now have the new code.
- If any P1 issues occur that NEED to be fixed asap, a new branch is pulled from master branch in order to
crate a hotfix branch and fix the code.
- When code on hotfix branch has been tested and accepted, hotfix branch will be merged into master branch.


| Home | Trees | Indices | Help |
|
|---|
|
|
The classes defined in this module should correspond 1-to-1 to the specifications in the PamFax API docs. See the link below for more info:
https://sandbox-apifrontend.pamfax.biz/summary/processors/
Users should not need to instantiate this class in applications using PamFax. Instead, just call the action directly on the PamFax object, and it will be delegated to the correct processor automatically.
|
|||
|
Common Class encapsulating common actions for the PamFax API |
|||
|
FaxHistory Class encapsulating actions related to fax history for this account |
|||
|
FaxJob Class encapsulating a specific fax job |
|||
|
NumberInfo Class encapsulating information for a given fax number |
|||
|
OnlineStorage Class encapsulating actions related to online storage |
|||
|
Session Class encapsulating a PamFax session |
|||
|
Shopping Class encapsulating shopping options available through PamFax |
|||
|
UserInfo Class encapsulating user info |
|
|||
|
|||
|
|||
|
|||
|
|||
|
|
|||
IP_ADDR =
|
|||
USER_AGENT =
|
|||
ORIGIN =
|
|||
CONTENT_TYPE =
|
|||
CONTENT_TYPE_JSON =
|
|||
__package__ =
|
|
Construct the URL that corresponds to a given action. All kwargs whose value is None are filtered out. Arguments: base_url -- The base of the URL corresponding to the PamFax processor name action -- The PamFax method to append to the base URL api_credentials -- The API credentials including user token extracted from /Session/VerifyUser Keyword arguments: **kwargs -- optional HTTP parameters to send to the PamFax URL |
Wait for the HTTP response and throw an exception if the return status is not OK. Return either a dict based on the HTTP response in JSON, or if the response is not in JSON format, return a tuple containing the data in the body and the content type. |
Encode multipart form data per mime spec and return (content_type, body) Arguments: fields -- A sequence of (name, value) elements for regular form fields. files -- A sequence of (name, filename, value) elements for data to be uploaded as files |
| Home | Trees | Indices | Help |
|
|---|
| Generated by Epydoc 3.0.1 on Wed Sep 21 21:30:31 2011 | http://epydoc.sourceforge.net |


Sudheeran backs Comtrust strike
by Staff ReporterSenior Congress leader V.M. Sudheeran on Wednesday offered his support to the ongoing indefinite fast being staged by the employees of the now-defunct Comtrust Weaving Factory, outside the Kerala State Industrial Development Corporation office in Thiruvananthapuram.
The agitation entered the 30th day on November 30. Mr. Sudheeran said that it was the responsibility of the government to implement the provisions of the bill passed by the Legislative Assembly and there need not be a strike for the purpose. Both the Left Democratic Front and the United Democratic Front were in support of the bill. “Even then, if there are difficulties to implement it, the Chief Minister should intervene,” Mr. Sudheeran said.


Change the user's session ID during authentication.
Change session IDs during authentication to make sure that the session state corresponds to the authenticated user.
To change session IDs during login:
Identify authentication functionality. Find the code that is responsible for authentication, including repeated authentication that is required for sensitive functionality.
Change session IDs during authentication. Add code that invalidates the current session ID and creates a new one. For example:
HttpSession session = request.getSession();
session.invalidate();
session = request.getSession(true);


{% extends "base.html" %} {% block title %}Signup is disabled{% endblock %} {% block h1 %}
Sorry, due to an API change on Google's end (v2 reaching end of life), signups are disabled until I'm able to get EventBot updated. Sorry for the inconvenience.
{% endblock %}


WITNESS LIST State Affairs Committee May 16, 2005 - 8:30A HB 57 AGAINST: Casey, Danieal T. (Fast Growth School Coalition), Austin, TX Vasquez, Sylvester (TASB - TASA), San Antonio, TX ON: McGeehan, Ann (Texas Secretary of State), Austin, TX Registering, but not testifying: For: Adams, Cathie (Texas Eagle Forum), Dallas, TX Finch, Mary A. (The League of Women Voters of Texas), Austin, TX Gerstenschlager, MerryLynn (Self), Weatherford, TX Against: Baray, Ray (City of San Antonio), San Antonio, TX HB 418 Registering, but not testifying: On: Nail, William S. (ERS), Austin, TX HB 617 Registering, but not testifying: On: Nail, William S. (ERS), Austin, TX HB 1071 FOR: Valkavich, Helen (Office of the City Attorney - City of San Antonio), San Antonio, TX Registering, but not testifying: For: Finch, Mary (League fo Women Voters of Texas), Austin, TX HB 1114 Registering, but not testifying: On: Nail, William S. (ERS), Austin, TX HB 1138 FOR: Fenoglio, Stepen (Bingo Interest Group and Over 950 Charitable and Business Organizations), Austin, TX ON: Atkins, Billy Director (Charitable Bingo Division, TX Lottery Commission), Austin, TX FOR COMMITTEE SUBSTITUTE: Fenoglio, Stephan (Bingo Interest Group and Over 950 Charitable and Business Organizations), Austin, TX Registering, but not testifying: For: Bresnen, Steve (Bingo Interest Group), Austin, TX Pewitt, Bill (Texas Charitable Bingo Association), Austin, TX Woodard, Shayne (Gametech International, Inc.), Austin, TX On: Paynter, Suzii (Christian Life Commission, Texas Babtist General Convention), Austin, TX HB 1271 FOR: Hammerlein, George (Harris County Tax Officer), Houston, TX Registering, but not testifying: For: Adams, Cathie (Texas Eagle Forum), Dallas, TX Gerstenschlager, MerryLynn (Self), Weatherford, TX Hammerlein, George (Harris County Tax Office), Houston, TX On: McGeehan, Ann Director of Elections (Texas Secretary of State), Austin, TX HB 1353 FOR: Beachley, Pam (Texas Cotton - Ginners Trust), Austin, TX Registering, but not testifying: For: Engelke, Tommy (Texas Agricultural Cooperation Council), Austin, TX Freeman, Harold (Texas Construction Association), Austin, TX Fuelberg, Curtis (Associated Plumbing, Heating Cooling Contractors), Austin, TX Loonam, Timothy W. (Texas Cotton Ginners' Trust), Austin, TX McAlister, Todd (Texas Air Conditioning Contractors Association), Austin, TX Moore, Mickey (First Cardinal Corp.), Austin, TX Williams, Tony (Texas Cotton Ginners Association), Austin, TX HB 1382 Registering, but not testifying: For: Finch, Mary (League of Women Voters of Texas), Austin, TX HB 1426 Registering, but not testifying: For: Copeland, Chris (Texas Veterinary Medical Association), Austin, TX On: Sidwa, Tom DVM (Texas Department of State Health Services), Austin, TX HB 1509 Registering, but not testifying: On: McGeehan, Ann (Texas Secretary of State), Austin, TX HB 1614 Registering, but not testifying: On: McGeehan, Ann (Texas Secretary of State), Austin, TX HB 1690 FOR: Carlton, Kathy (Apartment Association of Greater Dallas), Dallas, TX Davenport, Freddy (Jim's Carwash), Naples, TX Edmondson, Barbara (One Great Bentley, Inc.), Richardson, TX Jamal, Sam (Property Owner "3609 S. Central Expway, Dallas 75215"), Arlington, TX LeMasters, William A. (Self), Dallas, TX McKinnie, Donnelle (Centerpoint Apartments LTD), Frisco, TX Morrison, Elsie L. (Century Mgt. - 3220 Grand, Dallas, TX), Garland, TX Space, Chuck (Southwest Car Wash Association), Austin, TX Stefani, Steven A. (Budget Suites of America), Las Vegas, TX AGAINST: Patterson, Jr., T.J. (City of Ft. Worth - City Attorney's Office), Ft. Worth, TX ON: Bernal, Julian (City of Dallas), Dallas, TX Perkins, Tom (City of Dallas), Dallas, TX Registering, but not testifying: For: Mintz, David (Texas Apartment Association), Austin, TX On: Richie, Jennifer (City of Dallas City Attorney's Office), Dallas, TX Providing written testimony: For: Clouse, Richard (Bucket Suites), Irving, TX Davenport, Dale (Jim's Car Wash), Royse City, TX Phillips, Russell (Citizens National Bank in Waxahachie), Waxahachie, TX HB 2039 FOR: Bass, Robert (Texas Buildiong Branch - Association General Contractors), Austin, TX Enoch, Craig (Texas Building Branch - A.G.C.), Austin, TX AGAINST: Schroeter, Thomas (Port of Houston Authority), Houston, TX Registering, but not testifying: For: Bruns, Richard (DFW - Drywall & Accoustical Contractors Association), Dallas, TX Castillo, Yvonne General Counsel (Texas Society of Architects), Austin, TX Ford, David President, CEO (Associated Builders and Contractors - Central Texas Chapter), Austin, TX Freeman, Harold (Texas Construction Association), Austin, TX Marwitz, David (Texas Surety Federation), Austin, TX Nelson, Larry C. (LJ Co., Inc dba Larry Nelson Co.), Austin, TX Against: Johnson, Steven (Texas Association of Community Colleges), Austin, TX HB 2068 Registering, but not testifying: On: McGeehan, Ann (Texas Secretary of State), Austin, TX HB 2069 Registering, but not testifying: On: McGeehan, Ann (Texas Secretary of State), Austin, TX HB 2280 Registering, but not testifying: On: McGeehan, Ann (Texas Secretary of State), Austin, TX HB 2322 Registering, but not testifying: On: McGeehan, Ann (Texas Secretary of State), Austin, TX HB 2465 AGAINST: Santana, Sonia (ACLU - TX), Austin, TX ON: McGeehan, Ann (Texas Secretary of State), Austin, TX Registering, but not testifying: Against: Smith, Tom "Smitty" (Public Citizen), Austin, TX HB 2678 Registering, but not testifying: For: Young, Jenny (Texas Dental Association), Austin, TX HB 3200 Registering, but not testifying: On: McCarley, James (City of Plano), Dallas, TX HJR 6 Registering, but not testifying: For: Nelson, Larry C. (Self), Austin, TX Against: Albion, Estella (Self), Austin, TX Baker, David (Self), Austin, TX Behr, Allison LMSW (Self), Austin, TX Bodoin, Michael (Self), Austin, TX Braddy, David (Self), Austin, TX Clarendon, James (Self), Austin, TX Cooper, Gary L. (Self), Austin, TX Cunningham, Joy (Self), Austin, TX DeBerry, Debra P. (Self), Austin, TX Dumond, Marie (Self), Austin, TX Friss-Hansen, Dana (Self), Austin, TX Gerland, Bill (Self), Little River, TX Gibbs, Rodney (Self), Austin, TX Goertz, Keith (Self), Austin, TX Grevlich, Rivkah (Self), Austin, TX Grffin, Scott (Self), Austin, TX Hebert, Bryan (Self), Austin, TX Hewitt, Joseph R. (Self), Austin, TX Hobbs, Ellen (Myself: My Family), Austin, TX Holbrook, Jake Director (Members if StandOut), Austin, TX Holbrook, Jake (Rodney Gonzales), San Marcos, TX Holland, Stephanie (LGRL), Harlingen, TX Holzbach, Mark (Self), Austin, TX Jones, R. Seabrook (Self), Austin, TX Kleinman, Erica (LGRL), Austin, TX Levi, Victor (Self), Austin, TX O'Connor, Caroline (Self), Austin, TX Parrish, Jerry A. M.D. (My Family - Citizens of the United States), Austin, TX Paulis, Gretchen (Self), Austin, TX Pierre, Sachielle (Self), Austin, TX Preve, Francis (Self), Austin, TX Puchek, Alexis (Self), Austin, TX Putnam, Daryl (Self), Austin, TX Ray, Charlie Despain (Self), Austin, TX Richardson, Philip C. (Self), Austin, TX Savicki, Kristen (Austin), Austin, TX Sralla, Jeffery (Self), Pflugerville, TX Stewart, Blake (Self), Austin, TX Swainston, Dave (Self), Austin, TX Thomas, Kathleen (Self), Austin, TX Trowe, Paul (Palse Interactive - a small business in Austin), Austin, TX Vance, Lane (Self), Austin, TX Waddill, Stephen (Self), Austin, TX Wagers, Kelley (Self), Buffalo, NY Walker, David (Myself and My Partner Charles Mitchell), Austin, TX Wallace, Alex (Self), Austin, TX Westbrook, Elizabeth M. (Self), Austin, TX Whiteford, Steve (Self), Austin, TX Zimmer, Robert (Self), Austin, TX Providing written testimony: Against: Andrews, Robert F. (Self as an attorney involved in Constitution litigation), Austin, TX


"This company is pretty darn great." - A satisifed client
123 Fake St., San Francisco, CA
© kstar designs 2011-2015. // Contact Kyle at kyleastark@gmail.com.


PSALMS 57
A psalm written by David for the choir director, when David went into a cave to escape from Saul; to be sung using the tune "Do not destroy."
1 God, act mercifully toward me!
Act mercifully toward me because I am coming to you to protect me.

But my enemies are human, and they have spears and arrows, not teeth;
they say false things about me.
5 God, show in the heavens that you are very great!
Show your glory to people all over the earth!
6 It is as if my enemies have spread a net to seize me,
and I became very distressed.
It is as if they have dug a deep pit along the path where I walk,
but they themselves fell into it!
7 God, I am very confident in you.
I will sing to you,
and I will praise you while I sing.
8 It is an honor to wake up and praise you.
I will arise before the sun rises
and praise you while I play my big harp or my lyre.
9 Lord, I will thank you among all the people;
and I will sing to praise you among many people groups,
10 because your faithful love for us is as great as the distance from the earth to the sky,
and because your faithfulness.
11 God, show in the heavens that you are very great!
Show your glory to people all over the earth!


A Web Programmer
I am an Android and Web Developer.CEO of AFRO-Devs Ins. I buy my own beers😁
It makes me well informed on all kinda facts especially academically, financially and even in my romance life😉
Uhm, now this is something i tried running away from around a couple of years back. But you know "A man can't run from his destiny".
If you have in anyway fell in love with this website.Don't hesistate to contact me.
Tel:09034025220
Whatsapp:09034025220


--- layout: homepage title: homepage.title description: homepage.description navName: Home mainNavLinkName: main.navigation.Home permalink: / ---
{% t homepage.intro.details %}
{% t homepage.welcome_teaser.headline %} {% t homepage.welcome_teaser.action_button %}


thunk
`Prev `__
T
`Next `__
--------------
**thunk**: /thuhnk/, n.
1. [obs.]“A piece of coding which provides an address:”, according to P.
Z. Ingerman, who invented thunks in 1961 as a way of binding actual
parameters to their formal definitions in Algol-60 procedure calls. If a
procedure is called with an expression in the place of a formal
parameter, the compiler generates a thunk which computes the expression
and leaves the address of the result in some standard location.
2. Later generalized into: an expression, frozen together with its
environment, for later evaluation if and when needed (similar to what in
techspeak is called a closure). The process of unfreezing these thunks
is called forcing.
3. A `*stubroutine* <../S/stubroutine.html>`__, in an overlay
programming environment, that loads and jumps to the correct overlay.
Compare `*trampoline* `__.
4. Microsoft and IBM have both defined, in their Intel-based systems, a
“16-bit environment” (with bletcherous segment registers and 64K address
limits) and a “32-bit environment” (with flat addressing and semi-real
memory management). The two environments can both be running on the same
computer and OS (thanks to what is called, in the Microsoft world, WOW
which stands for Windows On Windows). MS and IBM have both decided that
the process of getting from 16- to 32-bit and vice versa is called a
“thunk”; for Windows 95, there is even a tool THUNK.EXE called a “thunk
compiler”.
5. A person or activity scheduled in a thunklike manner. “It occurred to
me the other day that I am rather accurately modeled by a thunk — I
frequently need to be forced to completion.:” — paraphrased from a
`*plan file* <../P/plan-file.html>`__.
Historical note: There are a couple of onomatopoeic myths circulating
about the origin of this term. The most common is that it is the sound
made by data hitting the stack; another holds that the sound is that of
the data hitting an accumulator. Yet another suggests that it is the
sound of the expression being unfrozen at argument-evaluation time. In
fact, according to the inventors, it was coined after they realized (in
the wee hours after hours of discussion) that the type of an argument in
Algol-60 could be figured out in advance with a little compile-time
thought, simplifying the evaluation machinery. In other words, it had
‘already been thought of’; thus it was christened a thunk, which is “the
past tense of ‘think’ at two in the morning”.
--------------
+--------------------------------------------+----------------------------+-------------------------+
| `Prev `__ | `Up <../T.html>`__ | `Next `__ |
+--------------------------------------------+----------------------------+-------------------------+
| thundering herd problem | `Home <../index.html>`__ | tick |
+--------------------------------------------+----------------------------+-------------------------+