This series will cover the K2 Graph, a novel indirect encoding for neuroevolution. The reader should have a strong understanding of the NEAT algorithm by Kenneth Stanley and HyperNEAT before continuing. The research papers introducing these algorithms are quite accessible and don't require advanced mathematical knowledge. I highly recommend reading through them a couple of times.
The K2 is an indirect encoding taking cues from HyperNEAT. Like HyperNEAT and its variants, it aims to generate phenotypes utilizing biologically inspired principles such as repetition, regularity, symmetry, hierarchy, and modularity. In addition, it is designed to use an evolvable substrate, meaning that the number of neurons can grow indefinitely and that the neural architecture is not fixed or chosen by a human programmer. Thus it is more an alternative to Evolvable Substrate HyperNEAT (ES-HyperNEAT) than to vanilla HyperNEAT.
The K2 Graph was designed with the intent of simplifying indirect encoding schemes conceptually and for implementation. It's matrix-based design lends itself to be implemented in modern tensor-based deep learning frameworks. Future posts will contain implementation details using Numpy, a popular Python library for scientific computing.
Let's imagine a set of neurons connected by directed weights. A neural network can be described in mathematical terms as a directed, weighted graph where the neurons are the nodes of the graph and the weights are the edges.
Illustration of a Neural Network by CBurnett licensed under CC BY-SA 3.0A graph can be written in matrix form as an adjacency matrix. An adjacency matrix is a representation of a graph that is a type of square matrix where each element represents an edge. The corresponding nodes connected by a weight are indicated by the row and column of the edge in the matrix.
A weighted directed graph and its corresponding adjacency matrix.The K2 Graph heavily utilizes adjacency matrices and can be viewed as a nesting of adjacency matrices.
The K2 Tree is tree structure originally designed for efficiently compressing web graphs [1]. It represents an adjacency matrix as a tree structure where if a node contains a 1, it's subtree also contains a one and if a node contains a 0, the node is a leaf and has no subtree. All internal nodes have k2 children.
An example of a K2 tree where k = 2.To go from the K2 Tree to the K2 Graph some generalizations will need to be made.
The first necessary change to make K2 Tree into K2 Graph is the addition of real numbers. For the K2 Graph, real numbers can exist in the nodes and the links. Let's start with a simple example. Note that there is some restructuring here to help make the difference between nodes and links more explicit by packaging the nodes into matrices of k by k.
An simple example of a K2 graph, again where k = 2.Giving values to the links allows for more expressive power so that different sub-matrices can have different weightings, which will be important for evolution.
Graph structures differ from tree structures in that they allow reuse of nodes. The K2 graph will thus be a Directed Acyclic Graph (DAG). In this example, the lower node is referred to twice but with a different weighting (0.5 and 0.6) each time.
K2 graph with node reuse.Take a look at the picture below. Can you imagine a way to use a K2 Graph to roughly compress the adjacency matrix? Are there any symmetries or patterns in the adjacency matrix that can be exploited?
Average Regional Connection Matrix of the Human Brain by Patric Hagmann, Leila Cammoun, Xavier Gigandet, Reto Meuli, Christopher J. Honey, Van J. Wedeen, Olaf Sporns licensed under CC Generic 2.5In the next post, we'll see how K2 Graphs can be expressed as genes.
Part 2 of this series1. Brisaboa, N.R., Ladra, S., Navarro, G.: k2-trees for compact web graph representation. In: Karlgren, J., Tarhio, J., Hyyro, H. (eds.) SPIRE 2009. LNCS, vol. 5721, pp. 18-30. Springer, Heidelberg (2009)


harmonise_data.Rd
In order to perform MR the effect of a SNP on an outcome and exposure must be harmonised to be relative to the same allele.
harmonise_data(exposure_dat, outcome_dat, action = 2)
| exposure_dat | Output from |
|---|---|
| outcome_dat | Output from |
| action | Level of strictness in dealing with SNPs.
|
Data frame with harmonised effects and alleles
Expects data in the format generated by read_exposure_data
and extract_outcome_data
.
This means the inputs must be dataframes with the following columns:
outcome_dat
:
SNP
beta.outcome
se.outcome
effect_allele.outcome
other_allele.outcome
eaf.outcome
outcome
exposure_dat
:
SNP
beta.exposure
se.exposure
effect_allele.exposure
other_allele.exposure
eaf.exposure
The function tries to harmonise INDELs. If they are coded as sequence strings things work more smoothly. If they are coded as D/I in one dataset it will try to convert them to sequences if the other dataset has adequate information. If coded as D/I in one dataset and as a variant with equal length INDEL alleles in the other, the variant is dropped. If one or both the datasets only has one allele (i.e. the effect allele) then harmonisation is naturally going to be more ambiguous and more variants will be dropped.


Walkthrough Part I: Generate a Simple Documentation Website
By completing this walkthrough, you'll become familiar with the workflow of docfx
and the general principle of organizing documents inside docfx
. You will finish this walkthrough with a static website that can be published to any host service. Download the files used in this walkthrough here.
Step1. Setup DocFX
Download docfx from http://dotnet.github.io/docfx/. Getting Started with docfx describes three ways to install docfx. This walkthrough uses the first one: Use docfx.exe directly.
- Download docfx.zip and unzip it to
D:\docfx\
- Add
D:\docfx\
toPATH
so that the commanddocfx
can be directly called from everywhere for convenience. (For example, for Windows,set PATH=%PATH%;D:\docfx\
.)
Step2. Init a DocFX project
- Create a new folder
D:\docfx_walkthrough
- Start Command Line under
D:\docfx_walkthrough
- Call
docfx init -q
. This command generates adocfx_project
folder with the defaultdocfx.json
file under it.docfx.json
is the configuration filedocfx
uses to generate documentation.-q
option means generating the project quietly using default values, you can also trydocfx init
and follow the instructions to provide your own settings.
Step3. Build our website
Run command docfx docfx_project/docfx.json
. Note that a new subfolder _site
is generated under that folder. This is where the static website is generated.
Step4. Preview our website
The generated static website can be published to GitHub pages, Azure websites, or your own hosting services without any further changes. You can also run command docfx serve docfx_project/_site
to preview the website locally.
If port 8080
is not in use, docfx
will host _site
under http://localhost:8080
. If 8080
is in use, you can use docfx serve _site -p <port>
to change the port to be used by docfx
.
Congrats! You can now see a simple website similar to:
Step5. Add a set of articles to the website
Place more
.md
files intoarticles
, e.g.,details1.md
,details2.md
,details3.md
. If the files reference any resources, put those resources into theimages
folder.In order to organize these articles, we add these files into
toc.yml
underarticles
subfolder. The content oftoc.yml
is as below:- name: Introduction href: intro.md - name: Details 1 href: details1.md - name: Details 2 href: details2.md - name: Details 3 href: details3.md
So now our folder layout is:
|- index.md |- toc.yml |- articles | |- intro.md | |- details1.md | |- details2.md | |- details3.md | |- toc.yml |- images |- details1_image.png
Run Step3 and Step4 again, and the website is now: .
Notice that more items are added to the sidebar for Articles nav page. The title inside the sidebar is exactly what we set in the toc.yml
inside articles
subfolder.
Conclusion
In this walkthrough, we built a website from a set of .md
files. We call these .md
files Conceptual Documentation. In walkthrough part 2, we will learn to add API Documentation to our website. The API Documentation is extracted directly from .NET source code. In a series of advanced walkthroughs, we will learn advanced concepts in docfx
, such as cross reference between articles, external reference to other documentation sets, etc. We will also learn to customize our websites, from theme to layout to metadata extraction.


| DOSE: |
repeated |
oral |
GBL |
(liquid) |
I had been using GBL for four months. I'd read all the horror stories on the web about addiction, withdrawals, etc. Of course I believed I had self-control. To me, these stories I had read of other peoples misfortunes were as I said merely stories. It would never happen to me. I began using GBL knowing damn well it was addictive. I had a dosing schedule I followed religiously at first. Soon I abandoned my schedule and dosed whenever I felt I needed the boost. Well I needed the boost all the time. I never relied on G for sleep, which was the only thing I did right; however, everything else I did wrong. I became emotionally neglectful of my family, my girlfriend, and my friends. I was only concerned with feeling good. The terrible thing is I didn't notice everything unfolding in front of me.
Under stress from school, and partially my addiction, an unfortunate landslide of events began to occur. My girlfriend and I had an argument about something simple and easily resolvable. In my intoxication, a simple argument transformed into a nervous breakdown. I had a hopeless feeling of doubt and sadness in which I put my girlfriend in an extremely uncomfortable position that she definitely did not deserve.
Understandably she left me the next day. She had enough of her own problems and I wasn't helping either of us. She has always disapproved of my drug use. She has been with me through a drug related hospitalization, many drug-fueled arguments about something easily resolvable. I have been given many chances and I blew it. I knew I had to change the day of our argument. I gave up G, but the damage had been done and it was too late. She is a wonderful person and I lost her because of my stupidity. Now I would do anything to get her back, absolutely anything, but I fear it's too late.
 Let me tell you. Drugs don't ruin lives.  Please take it from me and the many unfortunate souls before me to quit this nonsense while you still have something worth holding on to. Happiness doesn't come on a plant, in a pill, or a liquid. Happiness comes from the relationships we have with one another and ourselves. If this report can prevent even a single person from making my same mistakes hopefully my trouble was not all in vain.
Any benefits like relief from anxiety and depression were be short-lived and I ended up worse than when I started. Itís not worth my life or happiness. I got out while I still could!
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


Home
Chatbot
Translator
GPT3 Framework Navigation Panel
Check the Top Bar for navigating to the applications.


public class BinomialHeap extends java.lang.Object implements Heap
| Constructor and Description |
|---|
BinomialHeap()
Constructor for the binomial heap.
|
| Modifier and Type | Method and Description |
|---|---|
int |
deleteMax()
Finds the maximum value of the heap, deletes it and then returns it.
|
BinomialNode |
getHead()
Returns the head node of the current binomial heap.
|
void |
increaseKey(BinomialNode nodeToIncrease,
int newKey)
Increases the value of the key in the given node.
|
void |
insert(int newValue)
Inserts a new value into the heap.
|
void |
link(BinomialNode firstNode,
BinomialNode secondNode)
Helper method for the union operation.
|
BinomialHeap |
makeBinomialHeap()
Creates and returns a new binomial heap with head set to null.
|
BinomialNode |
merge(BinomialHeap firstHeap,
BinomialHeap secondHeap)
Helper method for the union operation.
|
int |
returnMax()
Returns the maximum value of the heap.
|
BinomialHeap |
union(BinomialHeap secondHeap)
Unites two binomial heaps into one while making sure that the resulting
heap abides by the binomial heap properties.
|
public BinomialHeap()
public BinomialHeap makeBinomialHeap()
public BinomialNode getHead()
public int returnMax()
public int deleteMax()
public void insert(int newValue)
public void increaseKey(BinomialNode nodeToIncrease, int newKey)
nodeToIncrease
- the node containing the key to be increased.newKey
- the new value of the key.public BinomialHeap union(BinomialHeap secondHeap)
secondHeap
- the heap to be united with the current heap.public void link(BinomialNode firstNode, BinomialNode secondNode)
firstNode
- the first node to be linked.secondNode
- the second node to be linked.public BinomialNode merge(BinomialHeap firstHeap, BinomialHeap secondHeap)
firstHeap
- the first heap to be merged.secondHeap
- the second heap to be merged.


{% extends "base.html" %} {% import "bootstrap/wtf.html" as wtf %} {% import "_macros.html" as macros %} {% block title %}Products staging area{% endblock %} {% block page_content %}


organizing idea-share
Where would you like to start?
What seems most pressing or intriguing?
*Note: In organizing, a "campaign" refers to the entity/movement of people coming together to try to change something.


# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 114 def primary_key(name, type = :primary_key, options = {}) column(name, type, options.merge(:primary_key => true)) end
class ActiveRecord::ConnectionAdapters::TableDefinition
Represents the schema of an SQL table in an abstract way. This class provides methods for manipulating the schema representation.
Inside migration files, the t
object in
create_table
is actually of this type:
class SomeMigration < ActiveRecord::Migration def up create_table :foo do |t| puts t.class # => "ActiveRecord::ConnectionAdapters::TableDefinition" end end def down ... end end
The table definitions The Columns are stored as a ColumnDefinition in the
columns
attribute.
Attributes
An array of ColumnDefinition objects, representing the column changes that have been defined.
Public Class Methods
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 99 def initialize(types, name, temporary, options, as = nil) @columns_hash = {} @indexes = {} @foreign_keys = {} @native = types @temporary = temporary @options = options @as = as @name = name end
Public Instance Methods
Returns a ColumnDefinition for the
column with name name
.
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 119 def [](name) @columns_hash[name.to_s] end
Instantiates a new column for the table. The type
parameter is
normally one of the migrations native types, which is one of the following:
:primary_key
, :string
, :text
,
:integer
, :float
, :decimal
,
:datetime
, :time
, :date
,
:binary
, :boolean
.
You may use a type not in this list as long as it is supported by your database (for example, “polygon” in MySQL), but this will not be database agnostic and should usually be avoided.
Available options are (none of these exists by default):
-
:limit
- Requests a maximum column length. This is number of characters for:string
and:text
columns and number of bytes for:binary
and:integer
columns. -
:default
- The column's default value. Use nil for NULL. -
:null
- Allows or disallowsNULL
values in the column. This option could have been named:null_allowed
. -
:precision
- Specifies the precision for a:decimal
column. -
:scale
- Specifies the scale for a:decimal
column. -
:index
- Create an index for the column. Can be eithertrue
or an options hash.
Note: The precision is the total number of significant digits and the scale is the number of digits that can be stored following the decimal point. For example, the number 123.45 has a precision of 5 and a scale of 2. A decimal with a precision of 5 and a scale of 2 can range from -999.99 to 999.99.
Please be aware of different RDBMS implementations behavior with
:decimal
columns:
-
The SQL standard says the default scale should be 0,
:scale
<=:precision
, and makes no comments about the requirements of:precision
. -
MySQL:
:precision
[1..63],:scale
[0..30]. Default is (10,0). -
PostgreSQL:
:precision
[1..infinity],:scale
[0..infinity]. No default. -
SQLite2: Any
:precision
and:scale
may be used. Internal storage as strings. No default. -
SQLite3: No restrictions on
:precision
and:scale
, but the maximum supported:precision
is 16. No default. -
Oracle:
:precision
[1..38],:scale
[-84..127]. Default is (38,0). -
DB2:
:precision
[1..63],:scale
[0..62]. Default unknown. -
SqlServer?:
:precision
[1..38],:scale
[0..38]. Default (38,0).
This method returns self
.
Examples¶ ↑
# Assuming +td+ is an instance of TableDefinition td.column(:granted, :boolean) # granted BOOLEAN td.column(:picture, :binary, limit: 2.megabytes) # => picture BLOB(2097152) td.column(:sales_stage, :string, limit: 20, default: 'new', null: false) # => sales_stage VARCHAR(20) DEFAULT 'new' NOT NULL td.column(:bill_gates_money, :decimal, precision: 15, scale: 2) # => bill_gates_money DECIMAL(15,2) td.column(:sensor_reading, :decimal, precision: 30, scale: 20) # => sensor_reading DECIMAL(30,20) # While <tt>:scale</tt> defaults to zero on most databases, it # probably wouldn't hurt to include it. td.column(:huge_integer, :decimal, precision: 30) # => huge_integer DECIMAL(30) # Defines a column with a database-specific type. td.column(:foo, 'polygon') # => foo polygon
Short-hand examples¶ ↑
Instead of calling column
directly, you can also work with the
short-hand definitions for the default types. They use the type as the
method name instead of as a parameter and allow for multiple columns to be
defined in a single statement.
What can be written like this with the regular calls to column:
create_table :products do |t| t.column :shop_id, :integer t.column :creator_id, :integer t.column :item_number, :string t.column :name, :string, default: "Untitled" t.column :value, :string, default: "Untitled" t.column :created_at, :datetime t.column :updated_at, :datetime end add_index :products, :item_number
can also be written as follows using the short-hand:
create_table :products do |t| t.integer :shop_id, :creator_id t.string :item_number, index: true t.string :name, :value, default: "Untitled" t.timestamps null: false end
There's a short-hand method for each of the type values declared at the
top. And then there's #timestamps that'll
add created_at
and updated_at
as datetimes.
#references will add
an appropriately-named _id column, plus a corresponding _type column if the
:polymorphic
option is supplied. If :polymorphic
is a hash of options, these will be used when creating the
_type
column. The :index
option will also create
an index, similar to calling add_index
. So what can be written
like this:
create_table :taggings do |t| t.integer :tag_id, :tagger_id, :taggable_id t.string :tagger_type t.string :taggable_type, default: 'Photo' end add_index :taggings, :tag_id, name: 'index_taggings_on_tag_id' add_index :taggings, [:tagger_id, :tagger_type]
Can also be written as follows using references:
create_table :taggings do |t| t.references :tag, index: { name: 'index_taggings_on_tag_id' } t.references :tagger, polymorphic: true, index: true t.references :taggable, polymorphic: { default: 'Photo' } end
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 256 def column(name, type, options = {}) name = name.to_s type = type.to_sym options = options.dup if @columns_hash[name] && @columns_hash[name].primary_key? raise ArgumentError, "you can't redefine the primary key column '#{name}'. To define a custom primary key, pass { id: false } to create_table." end index_options = options.delete(:index) index(name, index_options.is_a?(Hash) ? index_options : {}) if index_options @columns_hash[name] = new_column_definition(name, type, options) self end
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 110 def columns; @columns_hash.values; end
Adds index options to the indexes hash, keyed by column name This is primarily used to track indexes that need to be created after the table
index(:account_id, name: 'index_projects_on_account_id')
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 287 def index(column_name, options = {}) indexes[column_name] = options end
Appends a primary key definition to the table definition. Can be called multiple times, but this is probably not a good idea.
Adds a reference.
t.references(:user) t.belongs_to(:supplier, foreign_key: true)
See ActiveRecord::ConnectionAdapters::SchemaStatements#add_reference for details of the options you can use.
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 312 def references(*args) options = args.extract_options! polymorphic = options.delete(:polymorphic) index_options = options.delete(:index) foreign_key_options = options.delete(:foreign_key) type = options.delete(:type) || :integer if polymorphic && foreign_key_options raise ArgumentError, "Cannot add a foreign key on a polymorphic relation" end args.each do |col| column("#{col}_id", type, options) column("#{col}_type", :string, polymorphic.is_a?(Hash) ? polymorphic : options) if polymorphic index(polymorphic ? %w(type id).map { |t| "#{col}_#{t}" } : "#{col}_id", index_options.is_a?(Hash) ? index_options : {}) if index_options if foreign_key_options to_table = Base.pluralize_table_names ? col.to_s.pluralize : col.to_s foreign_key(to_table, foreign_key_options.is_a?(Hash) ? foreign_key_options : {}) end end end
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 271 def remove_column(name) @columns_hash.delete name.to_s end
Appends :datetime
columns :created_at
and
:updated_at
to the table. See ActiveRecord::ConnectionAdapters::SchemaStatements#add_timestamps
t.timestamps null: false
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 299 def timestamps(*args) options = args.extract_options! emit_warning_if_null_unspecified(:timestamps, options) column(:created_at, :datetime, options) column(:updated_at, :datetime, options) end
Private Instance Methods
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 362 def aliased_types(name, fallback) 'timestamp' == name ? :datetime : fallback end
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 354 def create_column_definition(name, type) ColumnDefinition.new name, type end
# File lib/active_record/connection_adapters/abstract/schema_definitions.rb, line 358 def native @native end


MONTAG SHORT FICTION #002
I'm sorry that my toaster won't stop calling you.
I used to know the phone numbers of everyone I cared to know, before everyone had a hundred-person worldwide network of semi-acquaintances. Then like everyone else I outsourced all my personal information to the smartphone, and later my RFID chip, so that when I walked into the house, they would tell all the other devices, hey! Human alert! Warm up my pipe and slippers!
I enjoyed the predictive smart home tech reading the electromagnetic waves coming out of my body so that even if I didn’t have the phone or the chip, the lights would turn on for me. If I got hungry enough to even think about ordering a pizza, my kitchen would simply ask me to confirm my credit info. It always ordered just what I craved. And then as the technology got better at reading biological data, information was stored in my body again. A neural network connected to the chip helps locate the phone numbers of my friends in my brain and sends them to the chip that transmits to all my devices.
I’ll admit that the biohacking I did when you left was irresponsible, I tried to burn you out of my neural network with drugs and alcohol, and messed with some subroutines that I had no business trying to reprogram sober, let alone heartbroken. I was sick of all my smart picture frames knowing what date I was thinking about and showing me pictures of us.
With the toaster, I know exactly the memory I got caught up in. I was thinking about making you brunch, which is also why I had to blacklist bacon and eggs from my shopping algorithm - they just kept ordering themselves every time I triggered this memory and I couldn't stop triggering it when I looked at the toaster. They wouldn't keep, and there's no way I could eat that much bacon and eggs alone.
A lot of people get stuck in pleasure loops of salt, fat, and sugar, and never leave their houses so the shopping algorithm gradually adds cholesterol medication or insulin to their shopping lists. Some people don't mess with their home algorithms and think they want what they want. My home with my chip and my phone know me better than I know myself.
But I’m trying to take more control of my life now. After manually restricting meat and dairy from my list, I couldn't bring myself to give up bread. So the toaster stayed connected in this terrible feedback loop where I think about eating some toast, then try not to think about brunch with you, which inevitably backfires, and the poor little toaster tries to be helpful, so it logs on the network and tries to call you because it thinks we're still having breakfast together.
 It doesn’t know how to stop itself from reaching out because it knows what I want, and no matter how many times I try to delete the number from my brain somehow it’s still in there.
I can usually stop it before the dial tone so you wouldn't know, but sometimes I forget and it squeaks a ring out, so I’m sorry about that.


Problem 3. Underage people
Write a function that prints all underaged persons of an array of person
Use Array#filter and Array#forEach
You could use Chrome console (F12) or Firebug to see the results.


UniWare
General
Device
Sensors
Settings
WiFi
MQTT
Sensors
Save
Error!
Delete sensor failed
Error!
Load sensors failed.
Error!
Load device settings failed.
Reboot
Trying to reboot
Success!
Sensors senttings saved.
Reboot?
Error!
Save sensors senttings failed


funny how a beautiful song could tell such a sad story.
Bollywood Videos
BollyWood Audio
Hollywood Videos
Hollywood Audio


Epicodus offers so many different courses, we often get students unsure of what languages they should learn. However, deciding which track you should take doesn't have to hard! Take this simple survey to determine what track best fits your goals:
Explore the Epicodus Programming Tracks
Copyright (c) 2016 Copyright Holder All Rights Reserved.


|
Allegro CL version 10.0 |
This chapter contains the following sections:
3.1 A simple example with two communicating controls
3.2 Building an application
3.3 The Doodler Tutorial
3.3.1 What the doodler application does
3.3.2 Developing an application
3.3.3 The steps in developing the doodler application
3.3.4 The doodler application windows
3.3.5 Popping up a common dialog
This is chapter 3 of the User Guide for the Allegro CL 10.0 Integrated Development Environment (IDE).
The chapters of the IDE User Guide are:
Chapter 1: Introduction to the IDE
Chapter 2: The Allegro CL Development Environment (IDE)
Chapter 3: An example (this chapter)
Chapter 4: Projects
Chapter 5: Components
Chapter 6: Designing a user interface using forms
Chapter 7: Menus
Chapter 8: Events
The problem with examples is that to be interesting, they have to do something interesting (to you, the reader) and anything that does something interesting needs some programming. The Allegro CL IDE is a tool for designing and implementing user interfaces to applications, along with a collection of tools which make the task of programming easier, but it does not write the underlying program (the part that actually does the work, often called the engine).
We will discuss two examples in this chapter. First is a simple example where we make a form with two controls. We modify the event handler of one control so clicking on it prompts the user for a string and then displays the string in the other control. We show all steps and illustrate various windows and dialogs as we go.
Second, we discuss the application created by the Interface Builder Tutorial. We do not discuss the steps of the Tutorial in detail. Rather we discuss aspects of the finished application. If you see a feature of interest to your needs, you know that information on implementing it can be found in the Tutorial.
In this example, we create a dialog with a button and single-item-list. We will arrange it so that clicking on the button displays a dialog asking the user to enter some text that is then added to the items in the single-item-list.
Here is approximately what the system looks like when we have a project with one form (form1):
We got this configuration by opening a project and then choosing New Form from the File menu, specifying Dialog for the type of form. We were asked where we wanted the project files to be saved, and accepted the default c:\allegro-projects\project1\. (If you have used projects before, you may be using project2 or project3 rather than project1.) You should have a directory associated with the project. If we save files, we will save them here, as that makes cleanup easier. It is actually not necessary to save files for the first example, but if you do, it is best to place them is the project directory.
Second, we need to define a function that will be used in the application. This function will display the dialog asking for a string. Go to the editor workbook window (click View | Editor if the editor workbook is not visible -- that command brings it to the front). This window uses a tab control to display editor buffers, which may or may not be associated with files. It starts with a single unsaved buffer named Untitled (there may be a form1 buffer as well):
Add the following to Untitled:
(in-package :cg-user) (defun ask-for-string (prompt &optional (string "")) (multiple-value-bind (string1 string2 result) (ask-user-for-string prompt string "~OK" "~Cancel") (declare (ignore string2)) (when (string-equal result "~OK") string1)))
While typing this in, try some editor tricks:
Bring up the Apropos dialog by clicking Search | Apropos. Enter ask-user-for in the String box and click Search. All symbols whose names start with ask-user-for are displayed; ask-user-for-string is at the bottom.
Select it by clicking on it (it is selected in the illustration). Edit | Copy copies it to the clipboard. Back in the Untitled buffer, Edit | Paste pastes it into the buffer. Pressing F1 causes the help page for ask-user-for-string to be displayed.
Search | Complete Symbol displays a menu of choices (a dialog if there are a lot of choices). When you get to multiple-value-bind, only type multiple-v and click Search | Complete Symbol. This pop-up menu is displayed:
Now press `a'. The `a' choice, multiple-value-bind, is printed in the editor.
Notice too that the Lisp code indents as you go to new lines. When you are done, the buffer should look like this:
Place the cursor just after the final parenthesis (or anywhere within the definition) and click Tools | Incremental Evaluation. You should see ask-for-string printed in the Debug window. That indicates that the function definition has been evaluated in the Lisp environment, as we want.
Note that there are shortcut keys for most of the menu choices we have mentioned (typically Control-C for Edit | Copy, Control-V for Paste, Control-. for Symbol Completion, and Control-E for Incremental Evaluation). However, your editor mode may use those key combinations for editing operations making them unavailable as menu shortcuts. The shortcuts are shown in the menu. Use them in preference to the menu if desired and if they are not used by the editor.
Now we can design our application window. We will work with the current project and form. Display the Project Manager window by clicking View | Project Manager.
Now select the blank form. If it is not visible, click on (and so select) form1 (not saved) in the Project Manager and click on the View Selected Form button (shown in the illustration).
The form should appear. The Inspector should be inspecting Form1. If it is not (or is not visible), double-click on the interior of the blank form. The Inspector, inspecting form1, will appear.
Also now display the code associated with form1 in an editor buffer. Do this by selecting form1 in the Project Manager dialog and clicking on the View Selected Code button.
This will display an unsaved form1 buffer in the Editor Workbook.
We will be adding code for event handlers for controls on the form in this buffer presently. (Note: if we do not display this buffer before we start editing event handling code, the initial event handling function code is stored in a hidden buffer. Then when this form1 buffer is used, redefinition warnings will be displayed when the code is compiled. Such warnings, if they appear, may be safely ignored.)
Now back to the inspection of Form1. (Double-click on the form if the inspector is hidden). The inspector looks like this:
Change the title of the form to Simple Example by selecting the original value (Form1) as follows. Select the text (Form1) which is the value of the title property and type Simple Example. Alternatively, click on the extended editor button (with the three dots, on the right, when the title property is selected, and replace Form1 with Simple Example in the dialog that appears and click OK. The title of form1 will change to Simple Example. (This change is already made in the illustration.)
The form will now look like this:
We want to change the name and the range of the single-item-list. Select the
single-item-list by clicking on it. Sizing handles (the solid squares) appear around the
selected component, as in the illustration. The Inspector window should now be inspecting
the single-item-list component. Click in the name
property, and type :list-box
. Then click in the range field and type nil
.
Here is the Inspector after both changes have been made. Notice the single-item-list on
the form is now blank (not illustrated). That is because we have set the range (the things
displayed) to nil
. In the application, when a user clicks the button, a
dialog asking for a string will appear and whatever the user enters will become an item in
the single-item-list.
Now we want to change the name and the title of the button. Select the button, and go to the Inspector, change the name field to :add-button and the title field to Add . Here is the Inspector when those changes are made. Also notice the title of the button on the form is changed to Add .
We want to specify the behavior when a user clicks on the button. When the user clicks, the on-change event handler for the button (if there is one) is called. We need a handler function that asks the user for a string and makes the result an item on the single-item-list.
To modify the on-change event handler for the button, select the button and look
at the Inspector. Click on the Events button and you will see the event handlers for the
button. Actually, there aren't any all the values are nil
meaning all
events are ignored (the on-change-test,
the only item whose value is not nil
, is an exception we do not discuss
here).
The cursor should be in the blank line in the middle of the function definition. Enter the following line. The modified editor should look like the illustration.
(ask-for-string "Enter string to add to list" "Greetings")
After you have saved the files or not, an active dialog appears. When you click on the Add button, a dialog asking for a string, with default Greetings, should appear. We show part of the form and the two dialogs.
(declare
, like this:Add the following code, replacing all the current code below the declare line. The illustration shows the appearance after these lines are typed in. Note that the text is indented a couple of spaces and is indented after pasting in the editor buffer. It is important that no left parenthesis except the very first in a Lisp form (the one before defun in our window) be in the first column, or the editor will be confused about where the Lisp form actually begins.
(let ((list-box (find-sibling :list-box widget)) (string (ask-for-string "Enter string to add to list" "Greetings"))) (when string (setf (range list-box) (adjoin string (range list-box) :test #'string-equal)) )) t)
Note that list-box was found using the function find-sibling and the name, :list-box
.
There are a number of find- functions. find-sibling is best in this case because
only the other controls on the form have to be examined. This function now changes the
range of list-box to include the entered string (adjoin adds an item to a list if
it is not already there if the new string is equal to a string already in the
range, it is not added).
We now have a dialog with the controls communicating with each other. And we were able to modify the behavior while the form was running.
We can convert this simple example into a standalone application. Early in the example, we mentioned the Project manager window and used it (if needed) to display the blank form. Since then we haven't said much about projects.
A project is a collection of modules associated with an application. With the modules and Allegro CL itself, you can build your application. Modules are parts of the application and have one or more associated files. Form1 is a module and has associated files form1.cl and form1.bil (we discuss these types of files in more detail in chapter 4).
We have to be sure that the project contains all the modules necessary, form1 and the untitled (or excode.cl if you saved it) files. Lots of files are included in an application so we strongly recommend that you start with an empty folder (like the C:\allegro-projects\project1 subfolder suggested by the system early in this discussion).
If you have saved excode.cl already, you were asked whether you wanted the file added to the current project. If you answered `yes', the file will be included in the project. If you answered `no', the file would be saved without adding it to the project. Suppose that is what you did. You can add that file (or any other) to the project at any time as follows:
A Creating Image window is created (it is not displayed by default but it appears on the run bar). Information about what is happening while the application is being built is printed in that window. Note that you cannot do anything while the image is being created other than waiting. When the build completes (assuming it is successful), a dialog like this appears:
If you click Run the EXE File, the application will run. If you click OK, you return to the running Lisp and the IDE.
Look in the C:\allegro-projects\project1 folder.
You can start project1.exe by double-clicking on it to run the application. End the application by clicking on the Close button of the dialog.
The Doodler tutorial is another application whose complete source is included with the IDE. The tutorial illustrates features of the Allegro CL IDE. We recommend that you run the tutorial separately. Here we discuss features of the final product rather than taking you through it step by step.
The code, examples, and so on can be found in the gui-builder-tutorial folder and its subfolders. We strongly recommend that you use the tutorial to familiarize yourself with the product. It exercises many of the features of the system, often showing more than one way to achieve a particular end. It provides many programming examples (already written). Our hope is that parts of the tutorial will give you examples that will be useful to you in your own program design. And the result is a program that is at least visually interesting even if it is not directly relevant to whatever application you want to write.
The tutorial application is called the doodler (the name of the project is the :interface-builder-tutorial project). It has four windows and dialogs: one for choosing a background color,
one for defining a cycloidal curve (by specifying the three relevant coefficients),
one for listing and managing the curves defined,
and the main window upon which the specified curve or curves are drawn.
The files in the final subfolder of the tutorial folder show all the files associated with the interface-builder-tutorial application after it is completed. Except for the files util.cl, cycloid.cl, colorx.cl, all of which implement the application drawing the curve and choosing a color -- and the several bmp files, providing the illustrations for the buttons, all the files needed for the application are generated using the Allegro CL IDE. (Some files, such as the Help file doodler.hlp, provide assistance in using the tutorial but are not needed by the doodler application.)
Here are the elements of the project listed by the doodler project manager.
We shall return to the doodler application as this chapter progresses. Now we discuss developing applications in general.
Broadly, the steps to developing an application are these:
Just as there are many ways to say something, there are many ways to collect input from a user. The doodler mostly uses buttons, but some numeric input uses an editable text control with an associated up-down control:
It also uses a color choice control that is a standard Windows dialog for choosing colors. (Note: that dialog is not associated with one of the forms of the interface-builder-tutorial project. It is displayed by ask-user-for-color.) Many are designed for and can be used for collecting user input. User input from the mouse is still input.
The background color of the drawing pane.
How many curves to draw.
The A, B, and C coefficients of each curve.
The line color of each curve.
Additionally, the user can erase the drawing pane, center the drawing pane, and scroll the drawing pane (all information about user desires that must be input somehow).
It is reasonable to have a window that lists all curves and allows the user to add and delete curves from that list. Another window can be used for specifying the details of each curve, and another for specifying the background color. Finally, the user must somehow give the command to draw the specified curves. This arrangement (a description of the actual interface-builder-tutorial windows) is obviously one choice among many. The draw command could be placed on the doodler window rather than on the curves window, or the erase button could be placed below the Draw All button on the curves window. The background color could be implemented by a Background Color button on the doodler window (that displayed a color choice dialog immediately) rather than having a background color window with a few choices and an option to get more choices. There are no end of possible arrangements, some being obviously unsatisfactory (dialogs for each coefficient rather than one dialog for all three) but many being equally satisfactory.
Notice that this step is longer to describe that the others, because there are so many possibilities and so few reasons to strongly prefer one choice over another.
There are four windows associated with the doodler application. Each window is designed by a form during the design process, so there are four forms in the interface-builder-tutorial project.
These forms are listed on the Options tab of the interface-builder-tutorial Project Manager. The project started (as all new projects do) with one form. Three others were added after. Each form was added by clicking on New Form in the File menu. When you click on File | New Form, this dialog appears asking for the Window class of the form:
There are more choices shown in the illustration then will be present in a fresh Allegro CL. The additional choices, doodler, curve-dialog, coefficient-dialog and background-palette, were all added as the project was worked on. You are choosing the type of window corresponding to the form. The default choices (everything but the four listed as new) provide many capabilities, but creating new classes of devices is common because you get to add features to the new device class without affecting any existing example of that class. Consider the doodler choice (the device of the doodler window). The doodler class is defined as follows (in final\doodler.cl):
(defclass doodler (bitmap-window) ((doodler-curve-dialog :accessor doodler-curve-dialog :initform nil) (doodler-background-palette :accessor doodler-background-palette :initform nil)))
That says doodler is a subclass of bitmap-window (a bitmap window is one with an interior bitmap-pane upon which you can draw). It has two additional slots: doodler-curve-dialog and doodler-background-palette. They are not initialized. Later, they are given as values the two dialogs (corresponding to the curve-dialog and background-palette forms). A show-curve-dialog method is defined as follows:
(defmethod show-curve-dialog ((window doodler)) (let* ((dialog (doodler-curve-dialog window)) (curve-list nil)) (when (or (not dialog) (not (windowp dialog))) (setf dialog (make-curve-dialog :parent window)) (setf (doodler-curve-dialog window) dialog) (setf curve-list (find-widget :curve-list dialog)) (setf (range curve-list) (list (make-instance 'cycloidal-curve))) (move-window dialog (window-to-screen-units window (make-position (- (+ (exterior-width dialog) 10)) 40)))) (select-window dialog)))
This method is specialized to instances of the doodler class (and thus cannot affect any other window in the system). Without analyzing the code too closely, if the value of the doodler-curve-dialog slot is non-nil, it is assumed to be a curve-dialog and displayed; if it is nil, a curve-dialog is created and displayed. The argument to this method is the current window (a doodler window).
Why are things done this way? The design is to have a button on the doodler window which, when clicked, displays the associated curve-dialog. This is implemented by having the system call show-curve-dialog with the parent (doodler) window as its argument when the button is clicked. That action displays (creating if necessary) a curve-dialog associated with the doodler window. The dialog knows which doodler window it is associated with (because that doodler window is its parent) and the doodler window knows which curve-dialog it has (because it is the value of the doodler-curve-dialog slot of the doodler window).
Windows has provided many standard (called common) dialogs for common actions. One is the file choice dialog (a dialog that lets you pick a file on the system) that everyone is probably familiar with. Another is the color choice dialog. The background-color dialog allows a user to choose a background color for the doodler window. Several choices are provided along with a way to get many more choices and even make a custom color using a common color-choice dialog (clicking the Other Color button).
Let us look at how this is implemented. The code is in background-palette.cl (the code associated with the background-palette form) and colorx.cl, an auxiliary source file not specifically associated with a form. In background-palette.cl, we find the function called when the Add Colors button is clicked:
(defun background-palette-color-button-on-change (widget new-value old-value) (declare (ignore-if-unused widget new-value old-value)) (when new-value (add-other-color (parent widget))) (not new-value))
add-other-color, defined in colorx.cl, is defined as follows:
(defmethod add-other-color ((dialog color-mixin)) (let* ((new-color (ask-user-for-color :initial-color (current-color dialog))) (color-list (find-widget :color-list dialog)) (color-name nil)) ;; do nothing if user canceled (when new-color ;; do not add color if it already is on the list.
#| So far, a color-choose common dialog has been displayed and the user has either specified a new color or canceled. If a new color is chosen, it is added to the list of defined background colors and the multi-picture button is expanded to include the new color. |# (when (not (setf color-name (find-color-name dialog new-color))) (setf color-name (new-color-name dialog)) (let* ((colors (range color-list))) (setf (range color-list) (append colors (list (make-instance 'button-info :name color-name :image new-color :string nil :height nil :tooltip nil :help-string nil)))))) ;; change which color is pressed (setf (value color-list) (list color-name)))))
The call to ask-user-for-color pops up a color choice dialog. The add-other-color function does what is necessary when a new color is chosen. You can mimic the code here when you want to pop up a common dialog and use the result of user action in your application.
Go to chapter 4. Go to beginning of this chapter.
Copyright (c) 1998-2015, Franz Inc. Berkeley, CA., USA. All rights reserved.
Documentation for Allegro CL version 10.0. This is the initial (unrevised) page.
Created 2015.1.25.
|
Allegro CL version 10.0 |


Welcome to the
imagemin-webp-webpack-plugin
If those images display correctly then the plugin successfully converted them!
From CSS:


Our Top 10 list for the roundest animals in the world.
Our Top 5 list for the cutest animals in the world. They all have big eyes...
How sharp are your eyes? Try to find these animals.
Do you have unanswered questions? Ask wise owl anything.
That's just sad :(
Bread Dog needs your help.


BROWSING HISTORY
» Here you can find history of websites you visited.
Today
Yesterday
Last 7 days
Last 30 days
Complete
History


To create even more advanced filters or to execute some actions at specific moments during the renaming, you can use External Processor filters. This filter execute an external software and can read the output of that software to rename the file.
The most powerfull use is for more for developpers. Create or use a console application that takes the file name as argument and outputs some text that can later be used to rename the file in Rename-It!. The output of your application could be for example the meta-title of the file, the size of an image, whatever you like. You may further use renaming filters to clean the output.
Another use is to execute some actions on the files at specific times. For example to run a VIM script, flip an image...
Note that those filters are not executed for the preview by default: They are only executed when the renaming really happen. If you check the "Execute for the preview" box, the filter will be executed for any filtering preview.


Getting Started¶
These tutorials do not attempt to make up for a graduate or undergraduate course in machine learning, but we do make a rapid overview of some important concepts (and notation) to make sure that we’re on the same page. You’ll also need to download the datasets mentioned in this chapter in order to run the example code of the up-coming tutorials.
Download¶
On each learning algorithm page, you will be able to download the corresponding files. If you want to download all of them at the same time, you can clone the git repository of the tutorial:
git clone https://github.com/lisa-lab/DeepLearningTutorials.git
On Linux or Mac systems, after cloning, all datasets can be downloaded at once with:
cd DeepLearningTutorials/data ./download.sh
Datasets¶
MNIST Dataset¶
The MNIST dataset consists of handwritten digit images and it is divided in 60,000 examples for the training set and 10,000 examples for testing. In many papers as well as in this tutorial, the official training set of 60,000 is divided into an actual training set of 50,000 examples and 10,000 validation examples (for selecting hyper-parameters like learning rate and size of the model). All digit images have been size-normalized and centered in a fixed size image of 28 x 28 pixels. In the original dataset each pixel of the image is represented by a value between 0 and 255, where 0 is black, 255 is white and anything in between is a different shade of grey.
Here are some examples of MNIST digits:
For convenience we pickled the dataset to make it easier to use in python. It is available for download here. The pickled file represents a tuple of 3 lists : the training set, the validation set and the testing set. Each of the three lists is a pair formed from a list of images and a list of class labels for each of the images. An image is represented as numpy 1-dimensional array of 784 (28 x 28) float values between 0 and 1 (0 stands for black, 1 for white). The labels are numbers between 0 and 9 indicating which digit the image represents. The code block below shows how to load the dataset.
import cPickle, gzip, numpy # Load the dataset f = gzip.open('mnist.pkl.gz', 'rb') train_set, valid_set, test_set = cPickle.load(f) f.close()When using the dataset, we usually divide it in minibatches (see Stochastic Gradient Descent). We encourage you to store the dataset into shared variables and access it based on the minibatch index, given a fixed and known batch size. The reason behind shared variables is related to using the GPU. There is a large overhead when copying data into the GPU memory. If you would copy data on request (each minibatch individually when needed) as the code will do if you do not use shared variables, due to this overhead, the GPU code will not be much faster then the CPU code (maybe even slower). If you have your data in Theano shared variables though, you give Theano the possibility to copy the entire data on the GPU in a single call when the shared variables are constructed. Afterwards the GPU can access any minibatch by taking a slice from this shared variables, without needing to copy any information from the CPU memory and therefore bypassing the overhead. Because the datapoints and their labels are usually of different nature (labels are usually integers while datapoints are real numbers) we suggest to use different variables for label and data. Also we recommend using different variables for the training set, validation set and testing set to make the code more readable (resulting in 6 different shared variables).
Since now the data is in one variable, and a minibatch is defined as a slice of that variable, it comes more natural to define a minibatch by indicating its index and its size. In our setup the batch size stays constant throughout the execution of the code, therefore a function will actually require only the index to identify on which datapoints to work. The code below shows how to store your data and how to access a minibatch:
def shared_dataset(data_xy): """ Function that loads the dataset into shared variables The reason we store our dataset in shared variables is to allow Theano to copy it into the GPU memory (when code is run on GPU). Since copying data into the GPU is slow, copying a minibatch everytime is needed (the default behaviour if the data is not in a shared variable) would lead to a large decrease in performance. """ data_x, data_y = data_xy shared_x = theano.shared(numpy.asarray(data_x, dtype=theano.config.floatX)) shared_y = theano.shared(numpy.asarray(data_y, dtype=theano.config.floatX)) # When storing data on the GPU it has to be stored as floats # therefore we will store the labels as ``floatX`` as well # (``shared_y`` does exactly that). But during our computations # we need them as ints (we use labels as index, and if they are # floats it doesn't make sense) therefore instead of returning # ``shared_y`` we will have to cast it to int. This little hack # lets us get around this issue return shared_x, T.cast(shared_y, 'int32') test_set_x, test_set_y = shared_dataset(test_set) valid_set_x, valid_set_y = shared_dataset(valid_set) train_set_x, train_set_y = shared_dataset(train_set) batch_size = 500 # size of the minibatch # accessing the third minibatch of the training set data = train_set_x[2 * batch_size: 3 * batch_size] label = train_set_y[2 * batch_size: 3 * batch_size]
The data has to be stored as floats on the GPU ( the right
dtype
for storing on the GPU is given by theano.config.floatX
).
To get around this shortcoming for the labels, we store them as float,
and then cast it to int.
Note
If you are running your code on the GPU and the dataset you are using is too large to fit in memory the code will crash. In such a case you should store the data in a shared variable. You can however store a sufficiently small chunk of your data (several minibatches) in a shared variable and use that during training. Once you got through the chunk, update the values it stores. This way you minimize the number of data transfers between CPU memory and GPU memory.
Notation¶
Dataset notation¶
We label data sets as \mathcal{D}. When the distinction is important, we indicate train, validation, and test sets as: \mathcal{D}_{train}, \mathcal{D}_{valid} and \mathcal{D}_{test}. The validation set is used to perform model selection and hyper-parameter selection, whereas the test set is used to evaluate the final generalization error and compare different algorithms in an unbiased way.
The tutorials mostly deal with classification problems, where each data set \mathcal{D} is an indexed set of pairs (x^{(i)},y^{(i)}). We use superscripts to distinguish training set examples: x^{(i)} \in \mathcal{R}^D is thus the i-th training example of dimensionality D. Similarly, y^{(i)} \in \{0, ..., L\} is the i-th label assigned to input x^{(i)}. It is straightforward to extend these examples to ones where y^{(i)} has other types (e.g. Gaussian for regression, or groups of multinomials for predicting multiple symbols).
Math Conventions¶
- W: upper-case symbols refer to a matrix unless specified otherwise
- W_{ij}: element at i-th row and j-th column of matrix W
- W_{i \cdot}, W_i: vector, i-th row of matrix W
- W_{\cdot j}: vector, j-th column of matrix W
- b: lower-case symbols refer to a vector unless specified otherwise
- b_i: i-th element of vector b
List of Symbols and acronyms¶
- D: number of input dimensions.
- D_h^{(i)}: number of hidden units in the i-th layer.
- f_{\theta}(x), f(x): classification function associated with a model P(Y|x,\theta), defined as {\rm argmax}_k P(Y=k|x,\theta). Note that we will often drop the \theta subscript.
- L: number of labels.
- \mathcal{L}(\theta, \cal{D}): log-likelihood \cal{D} of the model defined by parameters \theta.
- \ell(\theta, \cal{D}) empirical loss of the prediction function f parameterized by \theta on data set \cal{D}.
- NLL: negative log-likelihood
- \theta: set of all parameters for a given model
Python Namespaces¶
Tutorial code often uses the following namespaces:
import theano
import theano.tensor as T
import numpy
A Primer on Supervised Optimization for Deep Learning¶
What’s exciting about Deep Learning is largely the use of unsupervised learning of deep networks. But supervised learning also plays an important role. The utility of unsupervised pre-training is often evaluated on the basis of what performance can be achieved after supervised fine-tuning. This chapter reviews the basics of supervised learning for classification models, and covers the minibatch stochastic gradient descent algorithm that is used to fine-tune many of the models in the Deep Learning Tutorials. Have a look at these introductory course notes on gradient-based learning for more basics on the notion of optimizing a training criterion using the gradient.
Learning a Classifier¶
Zero-One Loss¶
The models presented in these deep learning tutorials are mostly used for classification. The objective in training a classifier is to minimize the number of errors (zero-one loss) on unseen examples. If f: R^D \rightarrow \{0,...,L\} is the prediction function, then this loss can be written as:
\ell_{0,1} = \sum_{i=0}^{|\mathcal{D}|} I_{f(x^{(i)}) \neq y^{(i)}}
where either \mathcal{D} is the training set (during training) or \mathcal{D} \cap \mathcal{D}_{train} = \emptyset (to avoid biasing the evaluation of validation or test error). I is the indicator function defined as:
I_x = \left\{\begin{array}{ccc} 1&\mbox{ if $x$ is True} \\ 0&\mbox{ otherwise}\end{array}\right.
In this tutorial, f is defined as:
f(x) = {\rm argmax}_k P(Y=k | x, \theta)
In python, using Theano this can be written as:
# zero_one_loss is a Theano variable representing a symbolic
# expression of the zero one loss ; to get the actual value this
# symbolic expression has to be compiled into a Theano function (see
# the Theano tutorial for more details)
zero_one_loss = T.sum(T.neq(T.argmax(p_y_given_x), y))
Negative Log-Likelihood Loss¶
Since the zero-one loss is not differentiable, optimizing it for large models (thousands or millions of parameters) is prohibitively expensive (computationally). We thus maximize the log-likelihood of our classifier given all the labels in a training set.
\mathcal{L}(\theta, \mathcal{D}) = \sum_{i=0}^{|\mathcal{D}|} \log P(Y=y^{(i)} | x^{(i)}, \theta)
The likelihood of the correct class is not the same as the number of right predictions, but from the point of view of a randomly initialized classifier they are pretty similar. Remember that likelihood and zero-one loss are different objectives; you should see that they are correlated on the validation set but sometimes one will rise while the other falls, or vice-versa.
Since we usually speak in terms of minimizing a loss function, learning will thus attempt to minimize the negative log-likelihood (NLL), defined as:
NLL(\theta, \mathcal{D}) = - \sum_{i=0}^{|\mathcal{D}|} \log P(Y=y^{(i)} | x^{(i)}, \theta)
The NLL of our classifier is a differentiable surrogate for the zero-one loss, and we use the gradient of this function over our training data as a supervised learning signal for deep learning of a classifier.
This can be computed using the following line of code:
# NLL is a symbolic variable ; to get the actual value of NLL, this symbolic
# expression has to be compiled into a Theano function (see the Theano
# tutorial for more details)
NLL = -T.sum(T.log(p_y_given_x)[T.arange(y.shape[0]), y])
# note on syntax: T.arange(y.shape[0]) is a vector of integers [0,1,2,...,len(y)].
# Indexing a matrix M by the two vectors [0,1,...,K], [a,b,...,k] returns the
# elements M[0,a], M[1,b], ..., M[K,k] as a vector. Here, we use this
# syntax to retrieve the log-probability of the correct labels, y.
 For the purpose of ordinary gradient descent we consider that the training data is rolled into the loss function. Then the pseudocode of this algorithm can be described as:
# GRADIENT DESCENT
while True:
loss = f(params)
d_loss_wrt_params = ... # compute gradient
params -= learning_rate * d_loss_wrt_params
if <stopping condition is met>:
return params
Stochastic gradient descent (SGD) works according to the same principles as ordinary gradient descent, but proceeds more quickly by estimating the gradient from just a few examples at a time instead of the entire training set. In its purest form, we estimate the gradient from just a single example at a time.
# STOCHASTIC GRADIENT DESCENT
for (x_i,y_i) in training_set:
# imagine an infinite generator
# that may repeat examples (if there is only a finite training set)
loss = f(params, x_i, y_i)
d_loss_wrt_params = ... # compute gradient
params -= learning_rate * d_loss_wrt_params
if <stopping condition is met>:
return params
The variant that we recommend for deep learning is a further twist on stochastic gradient descent using so-called “minibatches”. Minibatch SGD (MSGD) works identically to SGD, except that we use more than one training example to make each estimate of the gradient. This technique reduces variance in the estimate of the gradient, and often makes better use of the hierarchical memory organization in modern computers.
for (x_batch,y_batch) in train_batches:
# imagine an infinite generator
# that may repeat examples
loss = f(params, x_batch, y_batch)
d_loss_wrt_params = ... # compute gradient using theano
params -= learning_rate * d_loss_wrt_params
if <stopping condition is met>:
return params
There is a tradeoff in the choice of the minibatch size B. The reduction of variance and use of SIMD instructions helps most when increasing B from 1 to 2, but the marginal improvement fades rapidly to nothing. With large B, time is wasted in reducing the variance of the gradient estimator, that time would be better spent on additional gradient steps. An optimal B is model-, dataset-, and hardware-dependent, and can be anywhere from 1 to maybe several hundreds. In the tutorial we set it to 20, but this choice is almost arbitrary (though harmless).
Note
If you are training for a fixed number of epochs, the minibatch size becomes important because it controls the number of updates done to your parameters. Training the same model for 10 epochs using a batch size of 1 yields completely different results compared to training for the same 10 epochs but with a batchsize of 20. Keep this in mind when switching between batch sizes and be prepared to tweak all the other parameters according to the batch size used.
All code-blocks above show pseudocode of how the algorithm looks like. Implementing such algorithm in Theano can be done as follows:
# Minibatch Stochastic Gradient Descent
# assume loss is a symbolic description of the loss function given
# the symbolic variables params (shared variable), x_batch, y_batch;
# compute gradient of loss with respect to params
d_loss_wrt_params = T.grad(loss, params)
# compile the MSGD step into a theano function
updates = [(params, params - learning_rate * d_loss_wrt_params)]
MSGD = theano.function([x_batch,y_batch], loss, updates=updates)
for (x_batch, y_batch) in train_batches:
# here x_batch and y_batch are elements of train_batches and
# therefore numpy arrays; function MSGD also updates the params
print('Current loss is ', MSGD(x_batch, y_batch))
if stopping_condition_is_met:
return params
Regularization¶
There is more to machine learning than optimization. When we train our model from data we are trying to prepare it to do well on new examples, not the ones it has already seen. The training loop above for MSGD does not take this into account, and may overfit the training examples. A way to combat overfitting is through regularization. There are several techniques for regularization; the ones we will explain here are L1/L2 regularization and early-stopping.
L1 and L2 regularization¶
L1 and L2 regularization involve adding an extra term to the loss function, which penalizes certain parameter configurations. Formally, if our loss function is:
NLL(\theta, \mathcal{D}) = - \sum_{i=0}^{|\mathcal{D}|} \log P(Y=y^{(i)} | x^{(i)}, \theta)
then the regularized loss will be:
E(\theta, \mathcal{D}) = NLL(\theta, \mathcal{D}) + \lambda R(\theta)\\
or, in our case
E(\theta, \mathcal{D}) = NLL(\theta, \mathcal{D}) + \lambda||\theta||_p^p
where
||\theta||_p = \left(\sum_{j=0}^{|\theta|}{|\theta_j|^p}\right)^{\frac{1}{p}}
which is the L_p norm of \theta. \lambda is a hyper-parameter which controls the relative importance of the regularization parameter. Commonly used values for p are 1 and 2, hence the L1/L2 nomenclature. If p=2, then the regularizer is also called “weight decay”.
In principle, adding a regularization term to the loss will encourage smooth network mappings in a neural network (by penalizing large values of the parameters, which decreases the amount of nonlinearity that the network models). More intuitively, the two terms (NLL and R(\theta)) correspond to modelling the data well (NLL) and having “simple” or “smooth” solutions (R(\theta)). Thus, minimizing the sum of both will, in theory, correspond to finding the right trade-off between the fit to the training data and the “generality” of the solution that is found. To follow Occam’s razor principle, this minimization should find us the simplest solution (as measured by our simplicity criterion) that fits the training data.
Note that the fact that a solution is “simple” does not mean that it will generalize well. Empirically, it was found that performing such regularization in the context of neural networks helps with generalization, especially on small datasets. The code block below shows how to compute the loss in python when it contains both a L1 regularization term weighted by \lambda_1 and L2 regularization term weighted by \lambda_2
# symbolic Theano variable that represents the L1 regularization term
L1 = T.sum(abs(param))
# symbolic Theano variable that represents the squared L2 term
L2 = T.sum(param ** 2)
# the loss
loss = NLL + lambda_1 * L1 + lambda_2 * L2
Early-Stopping¶
Early-stopping combats overfitting by monitoring the model’s performance on a validation set. A validation set is a set of examples that we never use for gradient descent, but which is also not a part of the test set. The validation examples are considered to be representative of future test examples. We can use them during training because they are not part of the test set. If the model’s performance ceases to improve sufficiently on the validation set, or even degrades with further optimization, then the heuristic implemented here gives up on much further optimization.
The choice of when to stop is a judgement call and a few heuristics exist, but these tutorials will make use of a strategy based on a geometrically increasing amount of patience.
# early-stopping parameters
patience = 5000 # look as this many examples regardless
patience_increase = 2 # wait this much longer when a new best is
# found
improvement_threshold = 0.995 # a relative improvement of this much is
# considered significant
validation_frequency = min(n_train_batches, patience/2)
# go through this many
# minibatches before checking the network
# on the validation set; in this case we
# check every epoch
best_params = None
best_validation_loss = numpy.inf
test_score = 0.
start_time = time.clock()
done_looping = False
epoch = 0
while (epoch < n_epochs) and (not done_looping):
# Report "1" for first epoch, "n_epochs" for last epoch
epoch = epoch + 1
for minibatch_index in range(n_train_batches):
d_loss_wrt_params = ... # compute gradient
params -= learning_rate * d_loss_wrt_params # gradient descent
# iteration number. We want it to start at 0.
iter = (epoch - 1) * n_train_batches + minibatch_index
# note that if we do `iter % validation_frequency` it will be
# true for iter = 0 which we do not want. We want it true for
# iter = validation_frequency - 1.
if (iter + 1) % validation_frequency == 0:
this_validation_loss = ... # compute zero-one loss on validation set
if this_validation_loss < best_validation_loss:
# improve patience if loss improvement is good enough
if this_validation_loss < best_validation_loss * improvement_threshold:
patience = max(patience, iter * patience_increase)
best_params = copy.deepcopy(params)
best_validation_loss = this_validation_loss
if patience <= iter:
done_looping = True
break
# POSTCONDITION:
# best_params refers to the best out-of-sample parameters observed during the optimization
If we run out of batches of training data before running out of patience, then we just go back to the beginning of the training set and repeat.
Note
The validation_frequency
should always be smaller than the
patience
. The code should check at least two times how it
performs before running out of patience. This is the reason we used
the formulation validation_frequency = min( value, patience/2.)
Note
This algorithm could possibly be improved by using a test of statistical significance rather than the simple comparison, when deciding whether to increase the patience.
Testing¶
After the loop exits, the best_params variable refers to the best-performing model on the validation set. If we repeat this procedure for another model class, or even another random initialization, we should use the same train/valid/test split of the data, and get other best-performing models. If we have to choose what the best model class or the best initialization was, we compare the best_validation_loss for each model. When we have finally chosen the model we think is the best (on validation data), we report that model’s test set performance. That is the performance we expect on unseen examples.
Recap¶
That’s it for the optimization section. The technique of early-stopping requires us to partition the set of examples into three sets (training \mathcal{D}_{train}, validation \mathcal{D}_{valid}, test \mathcal{D}_{test}). The training set is used for minibatch stochastic gradient descent on the differentiable approximation of the objective function. As we perform this gradient descent, we periodically consult the validation set to see how our model is doing on the real objective function (or at least our empirical estimate of it). When we see a good model on the validation set, we save it. When it has been a long time since seeing a good model, we abandon our search and return the best parameters found, for evaluation on the test set.
Theano/Python Tips¶
Loading and Saving Models¶
When you’re doing experiments, it can take hours (sometimes days!) for gradient-descent to find the best parameters. You will want to save those weights once you find them. You may also want to save your current-best estimates as the search progresses.
Pickle the numpy ndarrays from your shared variables
The best way to save/archive your model’s parameters is to use pickle or
deepcopy the ndarray objects. So for example, if your parameters are in
shared variables w, v, u
, then your save command should look something
like:
>>> import cPickle
>>> save_file = open('path', 'wb') # this will overwrite current contents
>>> cPickle.dump(w.get_value(borrow=True), save_file, -1) # the -1 is for HIGHEST_PROTOCOL
>>> cPickle.dump(v.get_value(borrow=True), save_file, -1) # .. and it triggers much more efficient
>>> cPickle.dump(u.get_value(borrow=True), save_file, -1) # .. storage than numpy's default
>>> save_file.close()
Then later, you can load your data back like this:
>>> save_file = open('path')
>>> w.set_value(cPickle.load(save_file), borrow=True)
>>> v.set_value(cPickle.load(save_file), borrow=True)
>>> u.set_value(cPickle.load(save_file), borrow=True)
This technique is a bit verbose, but it is tried and true. You will be able to load your data and render it in matplotlib without trouble, years after saving it.
Do not pickle your training or test functions for long-term storage
Theano functions are compatible with Python’s deepcopy and pickle mechanisms, but you should not necessarily pickle a Theano function. If you update your Theano folder and one of the internal changes, then you may not be able to un-pickle your model. Theano is still in active development, and the internal APIs are subject to change. So to be on the safe side – do not pickle your entire training or testing functions for long-term storage. The pickle mechanism is aimed at for short-term storage, such as a temp file, or a copy to another machine in a distributed job.
Read more about serialization in Theano, or Python’s pickling.
Plotting Intermediate Results¶
Visualizations can be very powerful tools for understanding what your model or
training algorithm is doing. You might be tempted to insert matplotlib
plotting commands, or PIL
image-rendering commands into your model-training
script. However, later you will observe something interesting in one of those
pre-rendered images and want to investigate something that isn’t clear from
the pictures. You’ll wished you had saved the original model.
If you have enough disk space, your training script should save intermediate models and a visualization script should process those saved models.
You already have a model-saving function right? Just use it again to save these intermediate models.
Libraries you’ll want to know about: Python Image Library (PIL), matplotlib.


Check the boxes below to ignore/unignore words, then click save at the bottom. Ignored words will never appear in any learning session.
All None


GURPS Micro-Mecha v0.1
There’s no reason not to admit that I don’t know what I’m doing, since it’s on a private blog! Even still, I don’t know what I’m doing. That’s important to establish off the bat. But let’s get on to the content.
GURPS Mecha Is A Mess
I’m not being very fair, am I? After all, David Pulver worked hard on GURPS Mecha! Or did he work hard on GURPS Spaceships? Or did he work hard on GURPS Spaceships 4: Fighters, Carriers and Mecha? Or did Sean Punch work hard on the Fourth Edition Basic Set, and I should build a Mech as a character on a 1000 point budget, giving his main cannons as Innate Attacks and scaling up handheld weapon damage somehow?
Well, the answer is “Yes,” and “All of the above.” Of course Pulver worked hard. Of course the Basic Set is good. And there’s a good argument in favor of doing it both ways. But there are five separate systems for designing Mecha in GURPS, between 3rd and 4th editions. And all of them are complicated. Even GURPS Spaceships is complicated, to a degree. It takes 28 pages to explain the systems on offer. GURPS Spaceships 4 has an explanation of Mecha which is drastically undercut, with only SM+4 designs and legs added, but the removal of Medium/Minor weapons for those designs, and no explanation of how to scale handheld weapons.
So, in this mixture of 6 separate systems (Mecha as Allies, 3e GURPS Mecha, 3e GURPS Vehicles 2nd Edition, 4e GURPS Spaceships 4, Pyramid #3/40’s Mecha Operations and Pyramid #3/51’s Modular Mecha), and with the hopes of running an Ultra-Lite Mecha game via Play-by-Post, I came up with a way to make things worse: I made another one, and I made my best effort to fit it onto a single page.
Micro Mecha
Mecha At War
The default setting assumed here uses futuristic, laser-firing ‘Mechs of roughly 7 yards in height. For ‘Mechs at different levels of technology, increase Loaded Weight by 1 rank (with the matching decrease in Move) and decrease all other traits by 1 (do not double-tax Move) for every major technological leap backwards, or the opposite (decrease LWt and increase everything else) for the product of improved technology. For ‘Mechs of lesser size, reduce all ranks including LWt by 1 per major size jump; of greater, increase all ranks including LWt by 1.
To build a mech, you spend 20 Hard Points. Each of these represents a piece of equipment or mechanical feature of your mech. These fall into roughly five categories: Armor, Movement, Power, Weapons and Support.
Armor
Each Hard Point assigned to armor grants 6 dDR, and increases the Tonnage by 1 rank on the table (the default is 20 tons), along with a matching decrease in Move by 1 rank. When a Mech takes damage, each die of dDamage dealt reduces the overall armor by 1 dDR; Sundering weapons deal 2 dDR worth of damage per die.
Damage
When taking damage that penetrates DR, roll a 20-sided die and compare to the Hard Point list. The Mech makes an HT roll to avoid the system being disabled. If the system is already disabled, then it is destroyed on a failed HT roll. If the system is already destroyed, then roll HT to avoid the entire Mech being blown up. If more than 5 dDamage penetrates DR, continue rolling HT after a failure to see how badly the attack penetrates.
Movement
Each Hard Point assigned to additional legs ascends Move by 1 rank, with a default (for two legs) of 10. Regardless of the number of legs, movement costs 1 Power Point. This requirement may be increased, with each Power Point further increasing Move by 1 rank per point. This must be decided when creating the Mech.
Power
Each Hardpoint assigned to power generation grants 2 Power Points. These Power Points must be assigned during combat situations to allow movement, fire powerful weapons, and power support equipment. Assigning and re-assigning Power Points occurs during the combat round.
Weapons
Each Hardpoint assigned to weapons can be grouped together into a single weapon or separated into multiple weapons, depending on preference, though this choice must be made at the time of generation (not in real-time). Adding a Hardpoint ascends the damage scale by 1 rung, beginning with 1D. Up to 2 modifications can be added per Hard Point, which each add a 1-Power-Point requirement to fire (the default is 0). The modifications are chosen from the following:
- Heavy Damage ascends 1 rank.
- Sundering Deals extra damage when destroying armor; overall damage is reduced by 1 rank.1
- Armor Piercing Divides dDR by two.1 Multi-Hardpoint weapons systems may take Heavy up to twice per Hard Point, each instance costing 1 Power Point as normal.
Handheld Weapons
One-Handed Ranged deals Weapon Damage of 1 rank lower, and can take up to 1 Power Point upgrade if limited to 10 shots. Two-Handed Ranged deals Weapon Damage, and can take up to 1 Power Point upgrade if limited to 10 shots. Handheld Melee weapons deal Weapon Damage, and automatically choose between being Armor Piercing or Sundering. They then choose 1 further Power Point upgrade, which may be additional Sundering or Armor Piercing, or extra damage. Handheld Melee still cannot have both AP and Sundering.
Support
Additional equipment may be assigned to Hard Points, each of which is listed below, along with any Power Point requirements.
- Weapons Jammer (1 Power) Attacks made against the Mech are made with a –2 penalty.
- Targeting System (1 Power) Attacks made by the Mech are made at +1.
- Arm (1 Power for all Arms) Either holds handheld weapons, or can be used for striking in melee (Weapon damage +1 Rank).
- Radar Allows an IQ-based Electronics Operation (Sensors)/TL check to locate other nearby Mecha.
- Stealth Countermeasures Attempts to use Radar to detect the Mech are made with a –2 penalty.
Mecha Combat
Each turn, choose one action from those below, and move up to your Move in yards:
- Attack Gunnery (Mecha)/TL check to deal damage to one target per weapons battery. Each target past the first assesses a –1 penalty to all attacks that round.
- Evade Driving (Mecha)/TL check to give enemy Mechs –2 penalty to attack you for 1 turn.
- Lock-On Electronics Operation (Sensors)/TL to grant a +2 to attacking a specified target on the next turn.
- Power Allocation Decide whether to act quickly or carefully. Careful allocation takes the full turn. If done quickly, test Repair (Mecha)/TL. On failure, your Power pool shrinks by 1 until you reallocate. On a critical success, your Power pool expands by 1, until combat ends. On a critical failure, your Power pool shrinks by 1 until combat ends.
| Rank | LWt | Move | Damage |
|---|---|---|---|
| -14 | 150 lbs | 2/3” | |
| -13 | 200 lbs | 1” | |
| -12 | 300 lbs | 1.5” | |
| -11 | 500 lbs | 2” | |
| -10 | 700 lbs | 3” | 1 |
| -9 | 1,000 lbs | 5” | 1D-2 |
| -8 | 1 ton | 8” | 1D-1 |
| -7 | 1.5 ton | 1 ft | 1D |
| -6 | 2 ton | 1.5 ft | 1D+2 |
| -5 | 3 ton | 2 ft | 2D |
| -4 | 5 ton | 1 | 3D |
| -3 | 7 ton | 1.5 | 5D |
| -2 | 10 ton | 3 | 7D |
| -1 | 15 ton | 5 | 1D dDMG |
| +0 | 20 ton | 7 | 1D+2 dDMG |
| +1 | 30 ton | 10 | 2D dDMG |
| +2 | 50 ton | 15 | 3D dDMG |
| +3 | 70 ton | 20 | 5D dDMG |
| +4 | 100 ton | 30 | 7D dDMG |
| +5 | 150 ton | 50 | 5D dDMG |
| +6 | 200 ton | 70 | 7D dDMG |
| +7 | 300 ton | 100 | 1D cDMG |
| +8 | 500 ton | 150 | 1D+2 cDMG |


I have recently completed my PhD, working in Uncertainty Quantification in the Numerical Analysis group of the School of Mathematics at the University of Manchester (UK). My area of research is 'Efficient Inverse Uncertainty Quantification for PDEs with Uncertain Data'. I am the recipient of a Smith Institute CASE studentship award part funded by the National Physical Laboratory (Teddington, UK). My academic supervisors are Professor Catherine Powell and Dr Simon Cotter, my industrial supervisor is Louise Wright. My research interests include Bayesian Inverse Problems, Markov chain Monte Carlo methods, Finite Element Methods and Stochastic Galerkin Finite Element Methods.
Image credits: Nick Higham (Alan Turing Building - left) & Nuclear Engineering International Magazine (NPL Reception - right)
Many real world phenomena may be modelled through the use of partial differential equations (PDEs). It is often the case however that inputs to the PDE such as coefficients, boundary conditions and even domains are unknown or, 'uncertain'. In such scenarios, one may choose to model such unknowns as random variables, allowing for forward propogation of uncertainty through the PDE model. An example of such a PDE could be the transient heat flow equation $$ \rho c_{\scriptsize \mbox{p}} \frac{\partial u}{\partial t}(\mathbf{x},t) = \nabla \cdot \left( \lambda(\mathbf{x},t) \nabla u(\mathbf{x},t)\right) + Q(\mathbf{x},t), \quad \quad \quad \mathbf{x} \in D, \quad t \in [0,T], $$ which models the flow of heat through a homogeneous isotropic material subject to a heat source \(Q\). We may wish to propogate uncertain inputs through such a model to quantify uncertainty in the solution, this is known as forward uncertainty quantification. Alternatively, given (often indirect and/or noisy) observations of the solution, we may wish to quantify the uncertainty on the inputs, this is known as inverse uncertainty quantification). I am particularly interested in the latter approach, tackling these problems using the Bayesian approach outlined below.
Consider the inverse uncertainty quantification problem of recovering an unknown input given noisy, indirect measurements of the output. For example, we may wish to know the value of a PDE input such as the thermal conductivity of a material (\(\lambda\), in the equation above) but only have access to noisy observations of the temperature of the material at points in time at various spatial locations. Assuming the observations are subject to some form of noise, modelling this probabilistically allows us to define a posterior probability distribution for the unknown quantity given the data. This may be formulated in terms of our model and prior knowledge using Bayes' rule as $$ \mathbb{P}(\lambda|D) \propto \mathbb{P}(D|\lambda)\mathbb{P}(\lambda) \quad \quad \mbox{or} \quad \quad \pi(\lambda|D) \propto L(D|\lambda) \pi_{0}(\lambda). $$
Here our likelihood function, \(L(D|\lambda)\), is formulated using our forward model (for this example the PDE above) and our assumptions on the distribution of the noise. Our prior \(\pi_{0}(\lambda)\) encodes all our existing information about the unknown, usually from previous experiments or expert information. Using Markov chain Monte Carlo (MCMC) methods one is then able to sample from the posterior distribution allowing estimation of the posterior density itself, mean values or any other quantity of interest.My research revolves around the use of MCMC to efficiently perform inverse uncertainy quantification for PDEs with uncertain data. I aim to find efficient ways of solving the forward problem and use these along with intelligent MCMC algorithms to reduce the cost of sampling from the posterior.
My research involves concepts studied in a variety of the University of Manchester's research groups. I am a member of the following research groups:
I am a student member of SIAM, a student member of the IMA, an e-student member of the RSS and a member of the Manchester SIAM-IMA Student Chapter.


Software Developer
Graphic Designer
Chief Editor
Support Manager
Marketing Manager
Founder / CEO
Software Developer
Graphic Designer
Chief Editor
Support Manager
Marketing Manager
Founder / CEO
Software Developer
Graphic Designer
Chief Editor
Support Manager
Marketing Manager
Founder / CEO
Software Developer
Graphic Designer
Chief Editor
Support Manager
Marketing Manager
Founder / CEO


We know how important meditation and relaxation are to have positive outlook on life. We believe that self-enrichment is key to success. Here, in Aura, we use scientific methods to help you to truly take time out of your hectic schedule so that you can dedicate time for yourself. Simply scroll down on the "Home Page" and click on the image that reflects the mood that you are feeling. This will take you to a gallery of images with music that will enhance your mood.Spending ten minutes a day dedicating time your wellbeing is life-changing and helps to offer a new perspective to life.