[ all classes ]
[ edu.umn.csci5801.model ]
Coverage Summary for Class: MasterB (edu.umn.csci5801.model)
| Class | Class, % | Method, % | Line, % |
|---|---|---|---|
| MasterB | 100% (1/ 1) | 100% (3/ 3) | 100% (90/ 90) |
1 /**
2 * Copyright 2014 CSCI5801 Fall2014 Group1
3 *
4 * Licensed under the Apache License, Version 2.0 (the "License");
5 * you may not use this file except in compliance with the License.
6 * You may obtain a copy of the License at
7 *
8 * http://www.apache.org/licenses/LICENSE-2.0
9 *
10 * Unless required by applicable law or agreed to in writing, software
11 * distributed under the License is distributed on an "AS IS" BASIS,
12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 * See the License for the specific language governing permissions and
14 * limitations under the License.
15 */
16
17 package edu.umn.csci5801.model;
18
19 import java.util.ArrayList;
20 import java.util.List;
21
22 /**
23 * MasterB.java - Class containing all graduation requirements for students
24 * pursuing a Master's degree on Plan B. Also included is a mechanism to access
25 * all requirements externally.
26 *
27 * @author CSCI5801 Fall2014 Group1
28 * @version 1.0
29 */
30 public class MasterB {
31
32 private List<Requirement> reqList;
33
34 /**
35 * getReqList() - Method for getting the requirements list for Master Plan B
36 *
37 * @return reqList - Return the list of all requirements for graduation for Master Plan B
38 */
39 public List<Requirement> getReqList() {
40 if(reqList == null) {
41 setReqList();
42 }
43 return reqList;
44 }
45
46 /**
47 * setReqList() - Method for setting the requirements list for Master Plan B
48 */
49 private void setReqList() {
50
51 List<Requirement> tempReqList = new ArrayList<Requirement>();
52
53 List<String> ls = new ArrayList<String>();
54 ls.add("");
55
56 // REQUIREMENT #1 BREADTH_REQUIREMENT_MS
57 List<CourseTaken> ctl1 = new ArrayList<CourseTaken>();
58 ctl1.add(new CourseTaken(new Course("Analysis of Numerical Algorithms","csci5302","3",CourseArea.THEORY_ALGORITHMS), new Term(), Grade.C));
59 ctl1.add(new CourseTaken(new Course("Computational Aspects of Matrix Theory","csci5304","3",CourseArea.THEORY_ALGORITHMS), new Term(), Grade.C));
60 ctl1.add(new CourseTaken(new Course("Computational Complexity","csci5403","3",CourseArea.THEORY_ALGORITHMS), new Term(), Grade.C));
61 ctl1.add(new CourseTaken(new Course("Advanced Algorithms and Data Structures","csci5421","3",CourseArea.THEORY_ALGORITHMS), new Term(), Grade.C));
62 ctl1.add(new CourseTaken(new Course("Machine Learning","csci5525","3",CourseArea.THEORY_ALGORITHMS), new Term(), Grade.C));
63
64 ctl1.add(new CourseTaken(new Course("Operating Systems","csci5103","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
65 ctl1.add(new CourseTaken(new Course("System Modeling and Performance Evaluation","csci5104","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
66 ctl1.add(new CourseTaken(new Course("Introduction to Distributed Systems","csci5105","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
67 ctl1.add(new CourseTaken(new Course("Programming Languages","csci5106","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
68 ctl1.add(new CourseTaken(new Course("Introduction to Compilers","csci5161","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
69 ctl1.add(new CourseTaken(new Course("Advanced Computer Architecture","csci5204","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
70 ctl1.add(new CourseTaken(new Course("Data Communications and Computer Networks","csci5211","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
71 ctl1.add(new CourseTaken(new Course("Foundations of Advanced Networking","csci5221","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
72 ctl1.add(new CourseTaken(new Course("Wireless and Sensor Networks","csci5231","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
73 ctl1.add(new CourseTaken(new Course("Introduction to Parallel Computing: Architectures, Algorithms, and Programming","csci5451","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
74 ctl1.add(new CourseTaken(new Course("Functional Genomics, Systems Biology, and Bioinformatics","csci5461","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
75 ctl1.add(new CourseTaken(new Course("Architecture and Implementation of Database Management Systems","csci5708","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
76 ctl1.add(new CourseTaken(new Course("Software Engineering I","csci5801","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
77 ctl1.add(new CourseTaken(new Course("Software Engineering II","csci5802","3",CourseArea.ARCHITECTURE_SYSTEMS_SOFTWARE), new Term(), Grade.C));
78
79 ctl1.add(new CourseTaken(new Course("User Interface Design, Implementation, and Evaluation","csci5115","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
80 ctl1.add(new CourseTaken(new Course("Collaborative and Social Computing","csci5125","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
81 ctl1.add(new CourseTaken(new Course("Introduction to Computer Security","csci5271","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
82 ctl1.add(new CourseTaken(new Course("Modern Cryptography","csci5471","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
83 ctl1.add(new CourseTaken(new Course("Computational Techniques for Genomics","csci5481","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
84 ctl1.add(new CourseTaken(new Course("Artificial Intelligence I","csci5511","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
85 ctl1.add(new CourseTaken(new Course("Artificial Intelligence II","csci5512","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
86 ctl1.add(new CourseTaken(new Course("Introduction to Machine Learning","csci5521","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
87 ctl1.add(new CourseTaken(new Course("Introduction to Data Mining","csci5523","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
88 ctl1.add(new CourseTaken(new Course("Introduction to Intelligent Robotic Systems","csci5551","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
89 ctl1.add(new CourseTaken(new Course("Computer Vision","csci5561","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
90 ctl1.add(new CourseTaken(new Course("Fundamentals of Computer Graphics I","csci5607","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
91 ctl1.add(new CourseTaken(new Course("Fundamentals of Computer Graphics 2","csci5608","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
92 ctl1.add(new CourseTaken(new Course("Visualization","csci5609","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
93 ctl1.add(new CourseTaken(new Course("Animation and Planning in Games","csci5611","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
94 ctl1.add(new CourseTaken(new Course("Virtual Reality and 3D Interaction","csci5619","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
95 ctl1.add(new CourseTaken(new Course("Principles of Database Systems","csci5707","3",CourseArea.APPLICATIONS), new Term(), Grade.C));
96
97 Requirement req1 = new Requirement(Reqs.BREADTH_REQUIREMENT_MS, 3.25, ctl1, 9, ls);
98 tempReqList.add(req1);
99
100 //REQUIREMENT #2 PLAN_B_PROJECT
101 List<CourseTaken> ctl2 = new ArrayList<CourseTaken>();
102 ctl2.add(new CourseTaken(new Course("PlanB_project","csci8760","3"),new Term(), Grade.S));
103
104 Requirement req2 = new Requirement(Reqs.PLAN_B_PROJECT,ctl2,3,ls);
105 tempReqList.add(req2);
106
107 //REQUIREMENT #3 COLLOQUIUM
108 List<CourseTaken> ctl3 = new ArrayList<CourseTaken>();
109 ctl3.add(new CourseTaken(new Course("Colloquium","csci8970","1"),new Term(), Grade.S));
110
111 Requirement req3 = new Requirement(Reqs.COLLOQUIUM,ctl3,1,ls);
112 tempReqList.add(req3);
113
114 //REQUIREMENT #4 PHD_LEVEL_COURSES
115 List<CourseTaken> ctl4 = new ArrayList<CourseTaken>();
116 ctl4.add(new CourseTaken(new Course("Advanced Operating Systems","csci8101","3"),new Term(), Grade.C));
117 ctl4.add(new CourseTaken(new Course("Foundations of Distributed Computing","csci8102","3"),new Term(), Grade.C));
118 ctl4.add(new CourseTaken(new Course("Understanding the Social Web","csci8117","3"),new Term(), Grade.C));
119 ctl4.add(new CourseTaken(new Course("Advanced Compiler Techniques","csci8161.25","3"),new Term(), Grade.C));
120 ctl4.add(new CourseTaken(new Course("Parallel Computer Organization","csci8205","3"),new Term(), Grade.C));
121 ctl4.add(new CourseTaken(new Course("Advanced Computer Networks and Their Applications","csci8211","3"),new Term(), Grade.C));
122 ctl4.add(new CourseTaken(new Course("Security and Privacy in Computing","csci8271","3"),new Term(), Grade.C));
123 ctl4.add(new CourseTaken(new Course("Sparse Matrix Computations","csci8314","3"),new Term(), Grade.C));
124 ctl4.add(new CourseTaken(new Course("Numerical Linear Algebra in Data Exploration","csci8363","3"),new Term(), Grade.C));
125 ctl4.add(new CourseTaken(new Course("Computational Geometry and Applications","csci8442","3"),new Term(), Grade.C));
126 ctl4.add(new CourseTaken(new Course("Intelligent Agents","csci8551","3"),new Term(), Grade.C));
127 ctl4.add(new CourseTaken(new Course("Overview of Database Research","csci8701","3"),new Term(), Grade.C));
128 ctl4.add(new CourseTaken(new Course("Spatial Databases and Applications","csci8715","3"),new Term(), Grade.C));
129 ctl4.add(new CourseTaken(new Course("Databases for Bioinformatics","csci8725","3"),new Term(), Grade.C));
130 ctl4.add(new CourseTaken(new Course("Advanced Database Systems","csci8735","3"),new Term(), Grade.C));
131 ctl4.add(new CourseTaken(new Course("Advanced Software Engineering","csci8801","3"),new Term(), Grade.C));
132 ctl4.add(new CourseTaken(new Course("Special Advanced Topics in Computer Science","csci8980","3"),new Term(), Grade.C));
133
134 Requirement req4 = new Requirement(Reqs.PHD_LEVEL_COURSES,ctl4,3,ls);
135 tempReqList.add(req4);
136
137 //REQUIREMENT #5 TOTAL_CREDITS_MSB
138 Requirement req5 = new Requirement(Reqs.TOTAL_CREDITS_MSB);
139 tempReqList.add(req5);
140
141 //REQUIREMENT #6 OVERALL_GPA_MS
142 Requirement req6 = new Requirement(Reqs.OVERALL_GPA_MS, 3.25);
143 tempReqList.add(req6);
144
145 //REQUIREMENT #7 IN_PROGRAM_GPA_MS
146 Requirement req7 = new Requirement(Reqs.IN_PROGRAM_GPA_MS, 3.25);
147 tempReqList.add(req7);
148
149 //Requirement #8 MILESTONES_MS_B
150 List<CompletedMilestone> lm8 = new ArrayList<CompletedMilestone>();
151 lm8.add(new CompletedMilestone(Milestone.DPF_SUBMITTED));
152 lm8.add(new CompletedMilestone(Milestone.DPF_APPROVED));
153 lm8.add(new CompletedMilestone(Milestone.PROJECT_COMMITTEE_APPOINTED));
154 lm8.add(new CompletedMilestone(Milestone.GRADUATION_PACKET_REQUESTED));
155 lm8.add(new CompletedMilestone(Milestone.DEFENSE_PASSED));
156
157 Requirement req8 = new Requirement(Reqs.MILESTONES_MS_B,lm8,ls);
158 tempReqList.add(req8);
159
160 reqList = tempReqList;
161 }
162 }


{% extends 'base.html' %} {% block content %}
Django App is a simple django web application to demonstrate a web application deployment on docker in production environment through apache web server. visit github repo of Django App Here


--- layout: page title: About header-img: "img/about-bg.png" ---
Trained (former) musician, cat dad, gamer, stage manager, future developer, and part-time Wookie


Welcome to
The Solidify Interactive Zoo
Automating solutions using gamification.
Zoo Pages
expand_more
Home
Table of Content
Animal Manager
Build Method SVG
Zoo Introduction
Zoo SVG Diagram
Zoo Prototype
Zoo 3D Prototype
Zoo Slides
Other Stuff Luke Built
expand_more
Portfolio
Reference Logger
jcsc-Desktop
Data - Metrics - Reporting
Dashboards
BPHT5.com
Duke'z eBikes
message


Image Data Dictionary¶
The image data dictionary (IDD) is a way of formatting neuroimaging data from mixed sources (volumetric, surface, parcellated, etc.) conveniently for neuroimaging analysis through hdf5 files. The IDD also allows for sparse representation of data sources, and conversion from hdf5 files back to standard neuroimaging file formats (i.e. nifti, gifti).
Basic Schematic (with/without Parcellation)¶
The schematic of an IDD with and without a parcellation scheme can be seen in the following figure:
On the left, the example without the parcellation has a dataset named data, which can be a scalar (such as structural data) or vector (such as functional data). In this example the “data” dataset contains values from left hemisphere surfaces, right hemisphere surfaces, and volumetric sources, but can be similarly formatted from one or two of these three surfaces. The the datasets nifti_data_inds, lh_data_inds, and rh_data_inds can be used to index source specific elements of the “data” dataset, and the datasets nifti_ids, lh_ids, and rh_ids found under “ids” contain the native image indices that can be used to reconstruct nifti and gifti files from the hdf5 file without requiring external input.
The *_ids datasets also contain source specific metadata, and in the case that input parcellations are used when generating the IDD, the *_ids fields will be groups instead of datasets that contain the names of the different parcels for the source of interest and within each parcel’s dataset, the indices for the parcels constituent elements (voxel, or vertex) in native image space. In this case where a parcellation is used, each data source (*_ids) will have a metadata attribute with the list of all parcels represented in the data source.
Additional Options¶
For the purpose of denoising, it is useful to take the output of fMRIPREP and format it in an IDD. This can be any mix of sources (MNI, native space, LH, RH, etc.) and is built so that all the metadata needed for denoising is contained within the IDD.
This can be seen in the example below:
Beyond the basic schematic explored earlier, the main element is a group called “fmriprep_metadata”. Within this group, there is a dataset for each of the confounds used for fMRIPREP, and other relevant metrics are stored as metadata through the group’s attributes (such as mean_fd, TR, session, subject, mean_dvars, etc.)


{% include head.html %}
To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video


public final class ServiceConnector
extends java.lang.Object
To use a ServiceConnector,
@ServiceInfo(serviceIntent="com.myintent.MY_SERVICE")
private IMYService myService;
@ServiceConnectionCallback
public void onServiceConnectionChanged(String serviceIntent, boolean connected) {
}
Call bind(Object, Context)
ServiceConnector
| Modifier and Type | Method and Description |
|---|---|
static void |
bind(java.lang.Object target,
Context context)
Call this to process the given target object to look for any
IInterface fields annotated with ServiceInfo ,
and if so connects to those services. |
static boolean |
isAllConnected()
Returns true if all the annotated services are connected.
|
static boolean |
isConnected(java.lang.String serviceIntent)
Returns true if connected with a service of the given intent.
|
static void |
setEnableDebug(boolean enableDebug)
Call to enable or disable debug logs
|
static void |
unbind(java.lang.Object target)
Disconnects from all the services bounded to this target.
|
static void |
waitForAllConnected(long timeout)
Blocks for up to the given timeout for the connection with all the services
|
static void |
waitForConnected(long timeout,
java.lang.String serviceIntent)
Blocks for up to the given timeout for the connection to the given service.
|
public static void bind(java.lang.Object target, Context context)
IInterface
fields annotated with ServiceInfo
,
and if so connects to those services.
Those fields will get initialized with the remote
IInterface
service objects when the service is
successfully connected.target
- The object to analyzecontext
- Context used to connect to servicepublic static void unbind(java.lang.Object target)
target
- The target used to bind.public static boolean isAllConnected()
public static boolean isConnected(java.lang.String serviceIntent)
serviceIntent
- The intent of service to check forpublic static void waitForConnected(long timeout, java.lang.String serviceIntent) throws java.lang.InterruptedException
timeout
- Maximum timeout in ms to wait for. 0 Waits until service is connectedserviceIntent
- The service intentjava.lang.InterruptedException
public static void waitForAllConnected(long timeout) throws java.lang.InterruptedException
timeout
- Maximum timeout in ms to wait for. 0 Waits until service is connectedjava.lang.InterruptedException
public static void setEnableDebug(boolean enableDebug)
enableDebug
- Enable or disable


Predicting whether or not a loan will be paid off on time.
Lending Club
Identify loans that have a better chance of being paid off on time.
1. About Lending Club
We will be working with financial lending data from Lending Club. Lending Club is America’s largest marketplace connecting borrowers and investors, where consumers and small business owners lower the cost of their credit and enjoy a better experience than traditional bank lending, and investors earn attractive risk-adjusted returns.
How does an online credit marketplace work?
- Customers interested in a loan complete a simple application at LendingClub.com
- We leverage online data and technology to quickly assess risk, determine a credit rating and assign appropriate interest rates. Qualified applicants receive offers in just minutes and can evaluate loan options with no impact to their credit score
- Investors ranging from individuals to institutions select loans in which to invest and can earn monthly returns
Here is a diagram of the process:
2. The Data
Lending Club releases data for all of the approved and declined loan applications periodically on their website. We will be focusing on the approved loans data from 2007 to 2011, since a good number of the loans have already finished. In the datasets for later years, many of the loans are current and still being paid off.
The Data Dictionary contains information about what each column represents in the dataset and can be found here.
There are three sections to the Data Dictionary:
- LoanStats describes the approved loans dataset
- RejectStats describes the rejected loans dataset
- BrowseNotes describes additional information on loans in the dataset
Since rejected applications don’t appear on the Lending Club marketplace and aren’t available for investment, we will be focusing on data for approved loans only.
The approved loans datasets contain information on current loans, completed loans, and defaulted loans.
Before we can start doing machine learning, we need to define what features we want to use and which column repesents the target column we want to predict. Let’s start by reading in the dataset and exploring it.
3. Acquire The Data
import numpy as np
import pandas as pd
loans_2007 = pd.read_csv("LoanStats3a.csv")
loans_2007.head()
We need to format the data as followed:
- Remove the first row:
- The first row contains notes by Prospectus instead of the column names and prevents the dataset from being parsed correctly
- Remove the
desc
column:- Contains a long text explanation for each loan
- Remove the
url
column:- Contains a link to each loan on Lending Club which can only be accessed with an investor account
- Remove all columns containing more than 50% missing values:
- Allows us to move more efficiently since we can spend less time trying to fill in these values
loans_2007 = pd.read_csv('LoanStats3a.csv', skiprows=1)
half_count = len(loans_2007) / 2
loans_2007 = loans_2007.dropna(thresh=half_count, axis=1)
loans_2007 = loans_2007.drop(['desc', 'url'],axis=1)
loans_2007.to_csv('loans_2007.csv', index=False)
We will maintain the original dataset LoanStats3a.csv
by filtering out our newly parsed dataset into loans_2007.csv
3.1 Creating a Pandas DataFrame
import numpy as np
import pandas as pd
pandas.set_option('display.max_columns', None)
pandas.set_option('display.max_rows', None)
loans_2007 = pd.read_csv("loans_2007.csv", low_memory=False)
loans_2007.drop_duplicates()
print(loans_2007.shape)
loans_2007.head(5)
(42538, 52)
| id | member_id | loan_amnt | funded_amnt | funded_amnt_inv | term | int_rate | installment | grade | sub_grade | ... | last_pymnt_amnt | last_credit_pull_d | collections_12_mths_ex_med | policy_code | application_type | acc_now_delinq | chargeoff_within_12_mths | delinq_amnt | pub_rec_bankruptcies | tax_liens | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1077501 | 1296599.0 | 5000.0 | 5000.0 | 4975.0 | 36 months | 10.65% | 162.87 | B | B2 | ... | 171.62 | Jun-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 1 | 1077430 | 1314167.0 | 2500.0 | 2500.0 | 2500.0 | 60 months | 15.27% | 59.83 | C | C4 | ... | 119.66 | Sep-2013 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | 1077175 | 1313524.0 | 2400.0 | 2400.0 | 2400.0 | 36 months | 15.96% | 84.33 | C | C5 | ... | 649.91 | Jun-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 3 | 1076863 | 1277178.0 | 10000.0 | 10000.0 | 10000.0 | 36 months | 13.49% | 339.31 | C | C1 | ... | 357.48 | Apr-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 4 | 1075358 | 1311748.0 | 3000.0 | 3000.0 | 3000.0 | 60 months | 12.69% | 67.79 | B | B5 | ... | 67.79 | Jun-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 52 columns
4. Parse The Data
By exploring the data set and utilizing our understanding of the Data Dictionary we are going to:
-
Seperate the dataset into 3 groups
- Watch out for data leakage
- This can cause our model to overfit, and occures when the model uses data about the target column that wouldn’t be available when using the model on future loans.
- Identify features that don’t affect a borrower’s ability to pay back a loan
- (e.g. a randomly generated ID value by Lending Club)
-
Clean poorly formatted features
-
Explore features that require more data or a lot of processing to be useful
-
Determine if any of the features contain redundant information
- Select the target column for when we move on to the machine learning phase
4.1 Group 1
After analyzing each column, we can conclude that the following features need to be removed:
id
: randomly generated field by Lending Club for unique identification purposes onlymember_id
: also a randomly generated field by Lending Club for unique identification purposes onlyfunded_amnt
: leaks data from the future (after the loan is already started to be funded)funded_amnt_inv
: also leaks data from the future (after the loan is already started to be funded)grade
: contains redundant information as the interest rate column (int_rate)sub_grade
: also contains redundant information as the interest rate column (int_rate)emp_title
: requires other data and a lot of processing to potentially be usefulissue_d
: leaks data from the future (after the loan is already completed funded)
loans_2007 = loans_2007.drop(["id", "member_id", "funded_amnt", "funded_amnt_inv", "grade", "sub_grade", "emp_title", "issue_d"], axis=1)
Lending Club assigns a grade and a sub-grade based on the borrower’s interest rate. While the grade
and sub_grade
values are categorical, the int_rate
column contains continuous values, which are better suited for machine learning.
4.2 Group 2
Within this group of columns, we need to drop the following columns:
zip_code
: redundant with the addr_state column since only the first 3 digits of the 5 digit zip code are visible (which only can be used to identify the state the borrower lives in)out_prncp
: leaks data from the future, (after the loan already started to be paid off)out_prncp_inv
: also leaks data from the future, (after the loan already started to be paid off)total_pymnt
: also leaks data from the future, (after the loan already started to be paid off)total_pymnt_inv
: also leaks data from the future, (after the loan already started to be paid off)total_rec_prncp
: also leaks data from the future, (after the loan already started to be paid off)
The out_prncp
and out_prncp_inv
both describe the outstanding principal amount for a loan, which is the remaining amount the borrower still owes. These 2 columns as well as the total_pymnt
column describe properties of the loan after it’s fully funded and started to be paid off. This information isn’t available to an investor before the loan is fully funded and we don’t want to include it in our model.
loans_2007 = loans_2007.drop(["zip_code", "out_prncp", "out_prncp_inv", "total_pymnt", "total_pymnt_inv", "total_rec_prncp"], axis=1)
4.3 Group 3
In the last group of columns, we need to drop the following columns:
total_rec_int
: leaks data from the future, (after the loan already started to be paid off)total_rec_late_fee
: also leaks data from the future, (after the loan already started to be paid off)recoveries
: also leaks data from the future, (after the loan already started to be paid off)collection_recovery_fee
: also leaks data from the future, (after the loan already started to be paid off)last_pymnt_d
: also leaks data from the future, (after the loan already started to be paid off)last_pymnt_amnt
: also leaks data from the future, (after the loan already started to be paid off)
All of these columns leak data from the future, meaning that they’re describing aspects of the loan after it’s already been fully funded and started to be paid off by the borrower.
loans_2007 = loans_2007.drop(["total_rec_int", "total_rec_late_fee", "recoveries", "collection_recovery_fee", "last_pymnt_d", "last_pymnt_amnt"], axis=1)
print(loans_2007.shape)
loans_2007.head()
(42538, 32)
| loan_amnt | term | int_rate | installment | emp_length | home_ownership | annual_inc | verification_status | loan_status | pymnt_plan | ... | initial_list_status | last_credit_pull_d | collections_12_mths_ex_med | policy_code | application_type | acc_now_delinq | chargeoff_within_12_mths | delinq_amnt | pub_rec_bankruptcies | tax_liens | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 5000.0 | 36 months | 10.65% | 162.87 | 10+ years | RENT | 24000.0 | Verified | Fully Paid | n | ... | f | Jun-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 1 | 2500.0 | 60 months | 15.27% | 59.83 | < 1 year | RENT | 30000.0 | Source Verified | Charged Off | n | ... | f | Sep-2013 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | 2400.0 | 36 months | 15.96% | 84.33 | 10+ years | RENT | 12252.0 | Not Verified | Fully Paid | n | ... | f | Jun-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 3 | 10000.0 | 36 months | 13.49% | 339.31 | 10+ years | RENT | 49200.0 | Source Verified | Fully Paid | n | ... | f | Apr-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 4 | 3000.0 | 60 months | 12.69% | 67.79 | 1 year | RENT | 80000.0 | Source Verified | Current | n | ... | f | Jun-2016 | 0.0 | 1.0 | INDIVIDUAL | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 32 columns
We were able to reduce the number of columns from 52 to 32 columns. Now we need to decide on a target column that we want to use for modeling.
4.4 Target Column
The loan_status column is the only column that directly describes if a loan was paid off on time, had delayed payments, or was defaulted on the borrower. This column contains text values and needs to be converted to a numerical column for training our model.
We will use the value_counts()
method to display the frequency of each unique value. This will allow us to explore the different values in this column and come up with a strategy for converting them.
loans_2007['loan_status'].value_counts()
Fully Paid 33136
Charged Off 5634
Does not meet the credit policy. Status:Fully Paid 1988
Current 961
Does not meet the credit policy. Status:Charged Off 761
Late (31-120 days) 24
In Grace Period 20
Late (16-30 days) 8
Default 3
Name: loan_status, dtype: int64
4.5 The Quality of The Data
Binary Classification
There are 8 different possible values for the loan_status
column. The following table provides insight on each column as well as the counts in the Dataframe:
From an investor’s point of view, we are interested in trying to predict if a loan will be paid off on time or not. Only the Fully Paid
and Charged Off
values describe the final outcome of the loan. The other values describe loans that are still on going. Loans with a Charged Off
status essentially have no chance of being repaid, and loans marked with a Default
status have only a small chance of being repaid.
Since we are only interested in predicting which of these 2 values (Fully Paid
and Charged Off
) a loan will fall under, we can establish that this is a binary classification problem.
We can start by removing all the loans that don’t contain either a Fully Paid
or Charged Off
loan status and then assign the Fully Paid
values to 1
for the positive case and the Charged Off
values to 0
for the negative case. While there are a few different ways to transform all of the values in a column, we’ll use the Dataframe method replace.
Class Imbalance
One thing to keep in mind is the class imbalance between the positive and negative cases. While there are 33,136 loans that have been fully paid off, there are only 5,634 that were charged off. This class imbalance may result in the model having a stronger bias towards predicting the class with more observations. The stronger the imbalance, the more biased the model becomes.
loans_2007 = loans_2007[(loans_2007['loan_status'] == "Fully Paid") | (loans_2007['loan_status'] == "Charged Off")]
status_replace = {
"loan_status" : {
"Fully Paid": 1,
"Charged Off": 0,
}
}
loans_2007 = loans_2007.replace(status_replace)
Reducing Data Size
We can also further reduce the size of our data set by identifying columns that contain only one unique value. These columns won’t be useful for the model since they don’t add any information to each loan application.
orig_columns = loans_2007.columns
drop_columns = []
for col in orig_columns:
col_series = loans_2007[col].dropna().unique()
if len(col_series) == 1:
drop_columns.append(col)
loans_2007 = loans_2007.drop(drop_columns, axis=1)
print(drop_columns)
['pymnt_plan', 'initial_list_status', 'collections_12_mths_ex_med', 'policy_code', 'application_type', 'acc_now_delinq', 'chargeoff_within_12_mths', 'delinq_amnt', 'tax_liens']
We were able to remove 9 more columns since they only contained 1 unique value.
Let’s export our DataFrame and save it to a new filtered_loans_2007.csv
file
loans_2007.to_csv('filtered_loans_2007.csv', index=False)
5. Mine The Data
We need to make sure there are no missing values in the DataFrame, otherwise Scikit-learn will throw an error if you try to train a model using data that contain any missing or non-numerical values.
loans = pd.read_csv('filtered_loans_2007.csv')
null_counts = loans.isnull().sum()
null_counts
loan_amnt 0
term 0
int_rate 0
installment 0
emp_length 0
home_ownership 0
annual_inc 0
verification_status 0
loan_status 0
purpose 0
title 10
addr_state 0
dti 0
delinq_2yrs 0
earliest_cr_line 0
inq_last_6mths 0
open_acc 0
pub_rec 0
revol_bal 0
revol_util 50
total_acc 0
last_credit_pull_d 2
pub_rec_bankruptcies 697
dtype: int64
5.1 Handling Missing Values
We can keep the following columns and just remove columns where more than 1% of the rows contain missing values:
pub_rec_bankruptcies
- We should remove this column first because it contains the most rows with missing values
-
title
-
revol_util
last_credit_pull_d
loans = loans.drop("pub_rec_bankruptcies", axis=1)
loans = loans.dropna(axis=0)
5.2 Handling Non-numeric Values
We need to identify each columns data type and return the count
loans.dtypes.value_counts()
object 11
float64 10
int64 1
dtype: int64
The object columns that contain text need to be converted to numerical data types.
object_columns_df = loans.select_dtypes(include=["object"])
object_columns_df.head()
| term | int_rate | emp_length | home_ownership | verification_status | purpose | title | addr_state | earliest_cr_line | revol_util | last_credit_pull_d | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 36 months | 10.65% | 10+ years | RENT | Verified | credit_card | Computer | AZ | Jan-1985 | 83.7% | Jun-2016 |
| 1 | 60 months | 15.27% | < 1 year | RENT | Source Verified | car | bike | GA | Apr-1999 | 9.4% | Sep-2013 |
| 2 | 36 months | 15.96% | 10+ years | RENT | Not Verified | small_business | real estate business | IL | Nov-2001 | 98.5% | Jun-2016 |
| 3 | 36 months | 13.49% | 10+ years | RENT | Source Verified | other | personel | CA | Feb-1996 | 21% | Apr-2016 |
| 4 | 36 months | 7.90% | 3 years | RENT | Source Verified | wedding | My wedding loan I promise to pay back | AZ | Nov-2004 | 28.3% | Jan-2016 |
6. Refine The Data
Columns that represent categorical values:
home_ownership
: Can only be 1 of 4 categorical values based on the Data Dictionaryverification_status
: Indicates if income was verified by Lending Clubemp_length
: Number of years the borrower was employed at the time of applyingterm
: Number of payments on the loan, either 36 or 60addr_state
: Borrower’s state of residencepurpose
: A category provided by the borrower for the loan requesttitle
: Loan title provided the borrower
Columns that represent numeric values:
int_rate
: Interest rate of the loan in %revol_util
: Revolving Line Utilization Rate or the amount of credit the borrower is using relative to all available credit
Columns containing date values that require feature engineering will be removed:
earliest_cr_line
: The month the borrower’s earliest reported credit line was openedlast_credit_pull_d
: The most recent month Lending Club pulled credit for this loan
6.1 Categorical Values
Here is the unique value counts of the columnns that could potentially contain categorical values.
cols = ['home_ownership', 'verification_status', 'emp_length', 'term', 'addr_state']
for c in cols:
print(loans[c].value_counts())
RENT 18513
MORTGAGE 17112
OWN 2984
OTHER 96
NONE 3
Name: home_ownership, dtype: int64
Not Verified 16696
Verified 12290
Source Verified 9722
Name: verification_status, dtype: int64
10+ years 8545
< 1 year 4513
2 years 4303
3 years 4022
4 years 3353
5 years 3202
1 year 3176
6 years 2177
7 years 1714
8 years 1442
9 years 1229
n/a 1032
Name: emp_length, dtype: int64
36 months 29041
60 months 9667
Name: term, dtype: int64
CA 6958
NY 3713
FL 2791
TX 2667
NJ 1798
IL 1483
PA 1473
VA 1376
GA 1364
MA 1301
OH 1179
MD 1026
AZ 850
WA 822
CO 770
NC 753
CT 730
MI 712
MO 671
MN 603
NV 481
SC 462
WI 441
AL 437
OR 436
LA 430
KY 315
OK 290
KS 260
UT 254
AR 237
DC 209
RI 196
NM 184
WV 172
HI 166
NH 166
DE 113
MT 83
WY 80
AK 78
SD 61
VT 53
MS 19
TN 17
IN 9
ID 6
NE 5
IA 5
ME 3
Name: addr_state, dtype: int64
Below we will display the unique values of the following columns:
purpose
title
print(loans["purpose"].value_counts())
print(loans["title"].value_counts())
debt_consolidation 18130
credit_card 5039
other 3864
home_improvement 2897
major_purchase 2155
small_business 1762
car 1510
wedding 929
medical 680
moving 576
vacation 375
house 369
educational 320
renewable_energy 102
Name: purpose, dtype: int64
Debt Consolidation 2104
Debt Consolidation Loan 1632
Personal Loan 642
Consolidation 494
debt consolidation 485
Credit Card Consolidation 353
Home Improvement 346
Debt consolidation 324
Small Business Loan 310
Credit Card Loan 305
Personal 302
Consolidation Loan 251
Home Improvement Loan 234
personal loan 227
personal 211
Loan 208
Wedding Loan 201
Car Loan 195
consolidation 193
Other Loan 181
Credit Card Payoff 150
Wedding 149
Credit Card Refinance 143
Major Purchase Loan 139
Consolidate 125
Medical 118
Credit Card 115
home improvement 107
My Loan 92
Credit Cards 91
...
Moving expense loan 1
get rid of my credit cards loans 1
My Cessna Airplane Needs Engine Overhaul 1
Remodel office space so it can be leased 1
Happy Day 1
Consolidation Nov 2011 1
credit card and medical bill consolidati 1
Goodbye to Debt 1
Operation Pay Off Parents 1
Velma Loan 1
Med Bills 1
Young Engineer Looking For Home Help 1
Major Buy 1
megical 1
Debt Consildate CC 1
BOA 1
debt free in 36! 1
Chase-Citi-Dell-Apple-Exxon 1
Recently had a baby... 1
Debt Free Me ! 1
Gabi's Car Loan 1
Maria Elena 1
Credit consolidation and refinance 1
better opportunities for money saved on interest 1
Consolidation for Wedding Budget 1
Time to paint the exterior! 1
CC consolidator 1
Education/Furniture 1
Riddle family debt consolidation. 1
Duplex Investment 1
Name: title, Length: 19332, dtype: int64
Observation:
- The
purpose
andtitle
columns contain overlapping information. - The
purpose
column since it contains a few discrete values. - The
title
column has data quality issues. Many of the values are repeated with slight modifications (for example:Debt Consolidation
andDebt Consolidation Loan
anddebt consolidation
).
The emp_length
column needs to be cleaned and converted to numeric values since they have ordering.
We can use the following mapping to clean the emp_length
column:
- “10+ years”: 10
- “9 years”: 9
- “8 years”: 8
- “7 years”: 7
- “6 years”: 6
- “5 years”: 5
- “4 years”: 4
- “3 years”: 3
- “2 years”: 2
- “1 year”: 1
- ”< 1 year”: 0
- “n/a”: 0
We are going to map all individuals who may have been working more than 10 year to the value of 10.
For the individuals who have worked less than a year or have missing data will be assigned the value of 0.
mapping_dict = {
"emp_length": {
"10+ years": 10,
"9 years": 9,
"8 years": 8,
"7 years": 7,
"6 years": 6,
"5 years": 5,
"4 years": 4,
"3 years": 3,
"2 years": 2,
"1 year": 1,
"< 1 year": 0,
"n/a": 0
}
}
loans = loans.drop(["last_credit_pull_d", "earliest_cr_line", "addr_state", "title"], axis=1)
loans["int_rate"] = loans["int_rate"].str.rstrip("%").astype("float")
loans["revol_util"] = loans["revol_util"].str.rstrip("%").astype("float")
loans = loans.replace(mapping_dict)
6.2 Dummy Variables
The home_ownership
, verification_status
, emp_length
, and term
columns each contain a few discrete categorical values. We should encode these columns as dummy variables:
- Use the Series method
astype
to convert each column to the category data type - Use the
get_dummies
function to return a DataFrame containing the dummy columns - Use the
concat
method to add these dummy columns back toloans
- Remove the original, non-dummy columns (
home_ownership
,verification_status
,purpose
, andterm
) fromloans
cat_columns = ["home_ownership", "verification_status", "purpose", "term"]
dummy_df = pd.get_dummies(loans[cat_columns])
loans = pd.concat([loans, dummy_df], axis=1)
loans = loans.drop(cat_columns, axis=1)
Save our newly formatted Data:
loans.to_csv('cleaned_loans_2007.csv', index=False)
Import newly formatted Data:
import numpy as np
import pandas as pd
loans = pd.read_csv("cleaned_loans_2007.csv")
print(loans.info())
loans.head()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 38708 entries, 0 to 38707
Data columns (total 38 columns):
loan_amnt 38708 non-null float64
int_rate 38708 non-null float64
installment 38708 non-null float64
emp_length 38708 non-null int64
annual_inc 38708 non-null float64
loan_status 38708 non-null int64
dti 38708 non-null float64
delinq_2yrs 38708 non-null float64
inq_last_6mths 38708 non-null float64
open_acc 38708 non-null float64
pub_rec 38708 non-null float64
revol_bal 38708 non-null float64
revol_util 38708 non-null float64
total_acc 38708 non-null float64
home_ownership_MORTGAGE 38708 non-null int64
home_ownership_NONE 38708 non-null int64
home_ownership_OTHER 38708 non-null int64
home_ownership_OWN 38708 non-null int64
home_ownership_RENT 38708 non-null int64
verification_status_Not Verified 38708 non-null int64
verification_status_Source Verified 38708 non-null int64
verification_status_Verified 38708 non-null int64
purpose_car 38708 non-null int64
purpose_credit_card 38708 non-null int64
purpose_debt_consolidation 38708 non-null int64
purpose_educational 38708 non-null int64
purpose_home_improvement 38708 non-null int64
purpose_house 38708 non-null int64
purpose_major_purchase 38708 non-null int64
purpose_medical 38708 non-null int64
purpose_moving 38708 non-null int64
purpose_other 38708 non-null int64
purpose_renewable_energy 38708 non-null int64
purpose_small_business 38708 non-null int64
purpose_vacation 38708 non-null int64
purpose_wedding 38708 non-null int64
term_ 36 months 38708 non-null int64
term_ 60 months 38708 non-null int64
dtypes: float64(12), int64(26)
memory usage: 11.2 MB
None
| loan_amnt | int_rate | installment | emp_length | annual_inc | loan_status | dti | delinq_2yrs | inq_last_6mths | open_acc | ... | purpose_major_purchase | purpose_medical | purpose_moving | purpose_other | purpose_renewable_energy | purpose_small_business | purpose_vacation | purpose_wedding | term_ 36 months | term_ 60 months | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 5000.0 | 10.65 | 162.87 | 10 | 24000.0 | 1 | 27.65 | 0.0 | 1.0 | 3.0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
| 1 | 2500.0 | 15.27 | 59.83 | 0 | 30000.0 | 0 | 1.00 | 0.0 | 5.0 | 3.0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| 2 | 2400.0 | 15.96 | 84.33 | 10 | 12252.0 | 1 | 8.72 | 0.0 | 2.0 | 2.0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 |
| 3 | 10000.0 | 13.49 | 339.31 | 10 | 49200.0 | 1 | 20.00 | 0.0 | 1.0 | 10.0 | ... | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4 | 5000.0 | 7.90 | 156.46 | 3 | 36000.0 | 1 | 11.20 | 0.0 | 3.0 | 9.0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
5 rows × 38 columns
7. Build A Data Model
7.1 Calculating Error Metric
Our error metric will help us figure out whether or not our model is performing well, and will ultimately determine if our algorithm will make us money or lose us money.
In this case, we are primarily concerned with the following misclassifications: false positives and false negatives. Both of these are different types of misclassifications.
1. False Positive
- We predict that a loan will be paid off on time, but it actually isn’t.
2. False Negative
- We predict that a loan won’t be paid off on time, but it actually is paid off on time.
We need to treat false positives differently than false negatives. We want to minimize risk, and avoid false positives as much as possible because we would prefer to miss out on opportunities (false negatives) instead of funding a risky loan (false positives).
In the loan_status
and prediction
columns, a 0
means that the loan wont be paid off on time, and a 1
means that it will be paid off on time.
- Find the number of true negatives
- Find the number of items where
predictions
is0
, and the corresponding entry inloans["loan_status"]
is also0
- Assign the result to
tn
- Find the number of items where
- Find the number of true positives
- Find the number of items where
predictions
is1
, and the corresponding entry inloans["loan_status"]
is also1
- Assign the result to
tp
- Find the number of items where
- Find the number of false negatives
- Find the number of items where
predictions
is0
, and the corresponding entry inloans["loan_status"]
is1
- Assign the result to
fn
- Find the number of items where
- Find the number of false positives
- Find the number of items where
predictions
is1
, and the corresponding entry inloans["loan_status"]
is0
- Assign the result to
fp
- Find the number of items where
There is class imbalance in the loan_status
column. There are 6
times as many loans that were paid off on time (1
), than loans that weren’t paid off on time (0
). This causes a major issue when we use accuracy as a metric. This is because due to the class imbalance, a classifier can predict 1
for every row, and still have high accuracy.
it’s important to always be aware of imbalanced classes in machine learning models, and to adjust your error metric accordingly. In this case, we don’t want to use accuracy, and should instead use metrics that tell us the number of false positives and false negatives.
This means that we should optimize for:
- high recall (true positive rate)
- low fall-out (false positive rate)
We can calculate false positive rate and true positive rate, using the numbers of true positives, true negatives, false negatives, and false positives.
- False Positive Rate: “what percentage of my 1 predictions are incorrect?”
- In this case, “what percentage of the loans that I fund would not be repaid?”
- True Positive Rate: “what percentage of all the possible 1 predictions am I making?”
- In this case, “what percentage of loans that could be funded would I fund?”
# Predict that all loans will be paid off on time
predictions = pd.Series(np.ones(loans.shape[0]))
# False positives
fp_filter = (predictions == 1) & (loans["loan_status"] == 0)
fp = len(predictions[fp_filter])
# True positives
tp_filter = (predictions == 1) & (loans["loan_status"] == 1)
tp = len(predictions[tp_filter])
# False negatives
fn_filter = (predictions == 0) & (loans["loan_status"] == 1)
fn = len(predictions[fn_filter])
# True negatives
tn_filter = (predictions == 0) & (loans["loan_status"] == 0)
tn = len(predictions[tn_filter])
# Rates
tpr = tp / (tp + fn)
fpr = fp / (fp + tn)
print(tpr)
print(fpr)
1.0
1.0
both fpr
and tpr
were 1
. This is because we predicted 1
for each row. This means that we correctly identified all of the good loans (true positive rate), but we also incorrectly identified all of the bad loans (false positive rate). Now that we’ve setup error metrics, we can move on to making predictions using a machine learning algorithm.
7.2 Logistic Regression
A good first algorithm to apply to binary classification problems is logistic regression, for the following reasons:
- it’s quick to train and we can iterate more quickly,
- it’s less prone to overfitting than more complex models like decision trees,
- it’s easy to interpret.
import warnings
warnings.filterwarnings(action="ignore", module="scipy", message="^internal gelsd")
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression()
cols = loans.columns
train_cols = cols.drop("loan_status")
features = loans[train_cols]
target = loans["loan_status"]
lr.fit(features, target)
predictions = lr.predict(features)
In order to get a realistic depiction of the accuracy of the algorithm, we’ll need to use cross validation
to generate predictions. Cross validation splits the dataset into groups, then makes predictions on each group using the other groups as training data. This ensures that we don’t overfit by generating predictions on the same data that we train our algorithm with.
We can perform cross validation using the cross_val_predict
method of scikit-learn:
-
cross_val_predict allows us to pass in a classifier, the features, and the target.
-
We will create an instance of
KFold
, which will perform3
fold cross validation across our dataset. We setrandom_state
to1
to ensure that the folds are always consistent, and we can compare scores between runs. If we don’t, each fold will be randomized every time, making it hard to tell if we’re improving our model or not. -
We can pass the instance of
KFold
intocross_val_predict
, and it will then perform3
fold cross validation to generate unbiased predictions.
Once we have cross validated predictions, we can compute true positive rate and false positive rate.
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_predict, KFold
lr = LogisticRegression()
kf = KFold(features.shape[0], random_state=1)
predictions = cross_val_predict(lr, features, target, cv=kf)
predictions = pd.Series(predictions)
# False positives.
fp_filter = (predictions == 1) & (loans["loan_status"] == 0)
fp = len(predictions[fp_filter])
# True positives.
tp_filter = (predictions == 1) & (loans["loan_status"] == 1)
tp = len(predictions[tp_filter])
# False negatives.
fn_filter = (predictions == 0) & (loans["loan_status"] == 1)
fn = len(predictions[fn_filter])
# True negatives
tn_filter = (predictions == 0) & (loans["loan_status"] == 0)
tn = len(predictions[tn_filter])
# Rates
tpr = tp / (tp + fn)
fpr = fp / (fp + tn)
print(tpr)
print(fpr)
0.9988517209077449
0.9969723953695458
Penalizing The Classifier
Our classifier is not accounting for the imbalance in the classes and we need to tell the classifier to penalize misclassifications of the less prevalent class more than the other class.
We can do this by setting the class_weight
parameter to balanced
when creating the LogisticRegression instance. This tells scikit-learn to penalize the misclassification of the minority class during the training process. The penalty means that the logistic regression classifier pays more attention to correctly classifying rows where loan_status
is 0
. This lowers accuracy when loan_status
is 1
, but raises accuracy when loan_status
is 0
.
By setting the class_weight parameter to balanced, the penalty is set to be inversely proportional to the class frequencies. This would mean that for the classifier, correctly classifying a row where loan_status
is 0
is 6 times more important than correctly classifying a row where loan_status
is 1
.
from sklearn.linear_model import LogisticRegression
from sklearn.cross_validation import cross_val_predict
lr = LogisticRegression(class_weight="balanced")
kf = KFold(features.shape[0], random_state=1)
predictions = cross_val_predict(lr, features, target, cv=kf)
predictions = pd.Series(predictions)
# False positives.
fp_filter = (predictions == 1) & (loans["loan_status"] == 0)
fp = len(predictions[fp_filter])
# True positives.
tp_filter = (predictions == 1) & (loans["loan_status"] == 1)
tp = len(predictions[tp_filter])
# False negatives.
fn_filter = (predictions == 0) & (loans["loan_status"] == 1)
fn = len(predictions[fn_filter])
# True negatives
tn_filter = (predictions == 0) & (loans["loan_status"] == 0)
tn = len(predictions[tn_filter])
# Rates
tpr = tp / (tp + fn)
fpr = fp / (fp + tn)
print(tpr)
print(fpr)
0.6744024416039646
0.39269813000890474
Manual Penalties
We significantly improved false positive rate by balancing the classes, which reduced true positive rate. Our true positive rate is now around 67%
, and our false positive rate is around 40%
. From a conservative investor’s standpoint, it’s reassuring that the false positive rate is lower because it means that we’ll be able to do a better job at avoiding bad loans than if we funded everything. However, we’d only ever decide to fund 67%
of the total loans (true positive rate), so we’d immediately reject a good amount of loans.
We can try to lower the false positive rate further by assigning a harsher penalty for misclassifying the negative class. While setting class_weight to balanced will automatically set a penalty based on the number of 1s
and 0s
in the column, we can also set a manual penalty. In the last screen, the penalty scikit-learn imposed for misclassifying a 0
would have been around 5.89
(since there are 5.89
times as many 1s
as 0s
).
We can also specify a penalty manually if we want to adjust the rates more. To do this, we need to pass in a dictionary of penalty values to the class_weight
parameter:
from sklearn.linear_model import LogisticRegression
from sklearn.cross_validation import cross_val_predict
penalty = {
0: 10,
1: 1
}
lr = LogisticRegression(class_weight=penalty)
kf = KFold(features.shape[0], random_state=1)
predictions = cross_val_predict(lr, features, target, cv=kf)
predictions = pd.Series(predictions)
# False positives.
fp_filter = (predictions == 1) & (loans["loan_status"] == 0)
fp = len(predictions[fp_filter])
# True positives.
tp_filter = (predictions == 1) & (loans["loan_status"] == 1)
tp = len(predictions[tp_filter])
# False negatives.
fn_filter = (predictions == 0) & (loans["loan_status"] == 1)
fn = len(predictions[fn_filter])
# True negatives
tn_filter = (predictions == 0) & (loans["loan_status"] == 0)
tn = len(predictions[tn_filter])
# Rates
tpr = tp / (tp + fn)
fpr = fp / (fp + tn)
print(tpr)
print(fpr)
0.23340283443628562
0.08637577916295637
It looks like assigning manual penalties lowered the false positive rate to around 8%
, and thus lowered our risk. Note that this comes at the expense of true positive rate. While we have fewer false positives, we’re also missing opportunities to fund more loans and potentially make more money. Given that we’re approaching this as a conservative investor, this strategy makes sense, but it’s worth keeping in mind the tradeoffs.
8. Random Forest
Random Forests are able to work with nonlinear data, and learn complex conditionals. Logistic Regressions are only able to work with linear data. Training a Random Forest algorithm may enable us to get more accuracy due to columns that correlate nonlinearly with loan_status
.
We can use the RandomForestClassifer
class from scikit-learn to do this:
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import cross_val_predict
rf = RandomForestClassifier(class_weight="balanced", random_state=1)
kf = KFold(features.shape[0], random_state=1)
predictions = cross_val_predict(rf, features, target, cv=kf)
predictions = pd.Series(predictions)
# False positives.
fp_filter = (predictions == 1) & (loans["loan_status"] == 0)
fp = len(predictions[fp_filter])
# True positives.
tp_filter = (predictions == 1) & (loans["loan_status"] == 1)
tp = len(predictions[tp_filter])
# False negatives.
fn_filter = (predictions == 0) & (loans["loan_status"] == 1)
fn = len(predictions[fn_filter])
# True negatives
tn_filter = (predictions == 0) & (loans["loan_status"] == 0)
tn = len(predictions[tn_filter])
# Rates
tpr = tp / (tp + fn)
fpr = fp / (fp + tn)
print(tpr)
print(fpr)
0.971806726498051
0.9330365093499555
9. Results
Unfortunately, using a RandomForestClassifier didn’t improve our false positive rate. The model is likely weighting too heavily on the 1
class, and still mostly predicting 1s
. We could fix this by applying a harsher penalty for misclassifications of 0s
.
Ultimately, our best model had a false positive rate of 8%
, and a true positive rate of 23%
. For a conservative investor, this means that they make money as long as the interest rate is high enough to offset the losses from 8%
of borrowers defaulting, and that the pool of 23%
of borrowers is large enough to make enough interest money to offset the losses.
If we had randomly picked loans to fund, borrowers would have defaulted on 14.5%
of them, and our model is better than that, although we’re excluding more loans than a random strategy would.
9. Recommendations
- We can tweak the penalties further.
- We can try models other than a random forest and logistic regression.
- We can use some of the columns we discarded to generate better features.
- We can ensemble multiple models to get more accurate predictions.
- We can tune the parameters of the algorithm to achieve higher performance.


{% extends "base.html" %} {% block pageheading %} {% endblock %} {% block content %}
{{ location.name }}
{{ location.full_address }}
{{ location.authority }}
{{ location.owner}}
{{ location.site_size|floatformat:"0" }} m²
{{ location.estimated_floor_space|floatformat:"0" }} m²
{{location.nearest_broadband_fast}}%
{{ location.coverage_by_greenbelt }} %
{% for t in location.titles %}
{{ location.nearest_secondary_school_distance|floatformat:"0" }} metres
{% else %}None within 1km
{% endif %}{{ location.nearest_primary_school_distance|floatformat:"0" }} metres
{% else %}None within 1km
{% endif %}{{ location.nearest_metrotube_distance|floatformat:"0" }} metres
{% else %}None within 1km
{% endif %}{{ location.nearest_trainstop_distance|floatformat:"0" }} metres
{% else %}None within 1km
{% endif %}{{ location.nearest_motorway_distance|floatformat:"0" }} metres
{% else %}None within 6km
{% endif %}{{ location.nearest_busstop_distance|floatformat:"0" }} metres
{% else %}None within 1km
{% endif %}{{ location.nearest_ohl_distance|floatformat:"0" }} metres
{% else %}None within 3km
{% endif %}{{ location.nearest_broadband_distance|floatformat:"0" }} metres
{% else %}None within 500m
{% endif %}


User's Wish List
You can contribute money towards others WishList. Check their wishlist by clicking on thieir name.
Name
Email


{!-- Explanations and learning materials can be found in news/index and the other news template groups. In-line comments here are only for features not introduced in news/index. --} {html_head}
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat .
We'd love to hear from you. You can {encode='{webmaster_email}' title="email us"} or contact us at the address on your right.
12343 Valencia Street,
Mission District,
San Francisco,
California,
ZIP 123


Benchmark results are always dependent on a very individual setup. Normally it is not useful to generalize such results for every use case, but it can give you a hint. However, if you're really in the need of maximum performance, you should probably create an own benchmark with your objects or even use a profiler to detect the real hot spots in your application.
| Parser | Single Value Converters | Standard Converters | Reflection Converter | Serializable Converter |
|---|---|---|---|---|
| W3C DOM | 1823.0 | 1495.0 | 1437.0 | 1758.0 |
| JDOM | 2620.0 | 1796.0 | 1776.0 | 2030.0 |
| DOM4J | 1824.0 | 1811.0 | 2636.0 | 2177.0 |
| XOM | 571.0 | 716.0 | 995.0 | 958.0 |
| StAX (BEA) | 430.0 | 359.0 | 531.0 | 737.0 |
| StAX (Woodstox) | 357.0 | 344.0 | 535.0 | 725.0 |
| StAX (SJSXP) | 332.0 | 445.0 | 491.0 | 667.0 |
| XPP (Xpp3) | 351.0 | 395.0 | 544.0 | 743.0 |
| XPP (kXML2) | 299.0 | 353.0 | 517.0 | 761.0 |
| XppDom (Xpp3) | 351.0 | 395.0 | 544.0 | 743.0 |
| XppDom (kXML2) | 299.0 | 353.0 | 517.0 | 761.0 |
The values have been generated running the ParserBenchmark harness of the XStream benchmark module's test code.
The values above's unit is ms measured after 1000 unmarshalling operations with the object graphs described in the setup using XStream 1.4. The benchmark was run on an AMD Athlon with 2.1GHz running a JVM of Sun JDK 1.6.0_13 (32-bit) in Gentoo Linux. Note again, that these values are no replacement for real profiler results and they may vary from run to run for ~100ms due to this machine's background processes on a single CPU. However, it can give you some idea of what you can expect using different parser technologies.


McLaren Elva revealed
McLaren's latest special edition harks back to one of Bruce McLaren's racers from the 1960s.
by Scott Collie- shares
McLaren has revealed its latest special edition, the limited-run Elva roadster harking back to Bruce McLaren's open-cockpit Elva from the 1960s.
Just 399 examples will be built, priced from US$1.69 million ($2.47 million) each before McLaren Special Operations (MSO) gets its hands on the car.
It's the first modern McLaren to be built with an open cockpit – not including the Formula 1 car, of course – and has no roof, windows, or windscreen. The lack of bodywork makes this the lightest road car produced by McLaren Automotive.
To make sure the driver and passenger aren't buffeted into submission by the wind, the Elva uses an active aerodynamics system to channel air from the nose, through the bodywork and over the cockpit.
A windscreen will be optional, too. Given the nature of Australian Design Rules (ADR), we'd suggest any Elva that's to come Down Under will need the screen.
Power comes from a 4.0-litre twin-turbocharged V8 engine derived from the Senna's motor, outputting 600kW and 800Nm. That's good for a 100km/h sprint of under three seconds, and a 200km/h sprint of 6.7 seconds.
As with the Sport Series line-up, the Elva channels air from its front end through the doors to feed the mid-mounted engine. McLaren also says the doors are the lightest it's ever developed.

Inside (so to speak) is a more stylised, sculptural interpretation of the design language we've seen in McLaren cars since the MP4-12C. The driver is faced with a digital instrument binnacle, and there's a vertically-oriented touchscreen infotainment system protruding from the dash.
The seats are sculptural, carbon-backed units developed specifically for the Elva. They have slightly shorter bases than those in the Senna, because the driver and passenger might need to stand up when entering and exiting the car – and they can be specced with six-point harnesses for track work.
Owners will be able to choose from a range of materials, starting with high-end aniline leather and rolling through a moisture-resistant trim called Ultrafabric.
Interior storage comes behind the rear tonneau, which is a single carbon panel. There's space for two helmets back there.
MSO will be on hand to heavily customise the 399 Elvas. The car will be offered white gold or platinum badges, and a 24 carat gold heat shield for the engine bay.
Naked carbon-fibre is an option for the exterior, while MSO will also blend colours across the car's length to create a 'Velocity' paint finish.
Is the Elva coming to Australia?
We've contacted McLaren to see if the Elva is bound for our shores, and will update this story when details are available.
1
9
nextprevious
McLaren Elva revealed


Main Dialog Box
Installation
So what is being installed? Usually a bunch of files, some of which have shortcuts in the start menu and/or on the desktop. So that means we must have a way of specifying the various files. This is done in a dialog box with lists which are kept in combo boxes. The user may specify a new Feature and it will be added to the Feature combo box so that it may be modified later. The user may specify a new component and it will be added to the Component/Shortcut Name combo box so that its attributes may be specified and modified later.
So there can be one or more features and each feature can contain one or more components. The rest of the task is just filling in the blanks. The remainder of this page will describe each field, button, check box in the dialog box. The description will go from top to bottom of the dialog box because that is the usual way of filling them in.
Product Section
Instller Name
Choose a name for the installer that will be produce by Wix, something meaningful to the user that is going to install your product (e.g. MyAppInstaller or just Installer).
Version Number
The version number that will be used in the installer to decide if this is a later version (or the same if this option is set) than the version already installed.
There are two methods for determining the version number.
- Editing the value in the version edit box
- Setting the "Version No. Available check box for an application
Company Name
Used to tell Wix what the company name is. Both the creator and the distributor company names will be the same.
Product Name
This is used by Wix to identify the product in Windows (the registry in particular).
Control Panel Icon
For most installations this will be the icon that appears in the Programs and Features page in the Control Panel. Command Line programs appear to not use this icon.
Feature Name
The Feature Name combo box represents a list of features. Each name in the combo box is a "Feature." Each Feature may contain one or more "Components." The edit (visible) portion of the combo box shows the current feature. The corresponding components are then shown in the Component portion of the dialog box with the "current" component shown in the Component/Shortcut Name edit portion of that combo box.
There are three controls: a combo box, a "New" button and a "Delete" button. Click (Select) on "New" to create a new feature. This is just a name for a group of components. Every time a feature is selected the entire Componet/Shortcut Name combo box is changed. This is what allows one to create a massive tree of Features and Components.
The Delete Button deletes the feature currently showing in the Feature Name combo box and all Components that are contained in that feature. Use the button carefully.
Components
The attributes of the "current" Component are shown in this section of the dialog box. Each time one selects a different component the attributes are changed that correspond to that Component.
Component/Shortcut Name
This name identifies the component within Wix but it will also be used for the shortcuts on the start menu and desktop. Choosing New adjacent to the combo box will create a new component. Delete will delete the component.
Component Path
The Browse button will bring up an open file dialog box. Choose the file to be associated with the component. Only the last little bit of the path will be shown giving confirmation of the file name chosen.
Description
This will be used in a popup when the mouse lingers over the icon for the application.
Start Menu Check Box
Check when an entry in the start menu is to be placed. The Start Menu directory name is identified below the check boxes.
Desk Top Check Box
Check when a shortcut is to be placed on the desk top. No provision is made for folders.
Version No. Available Check Box
Some windows applications use resources and contain a version number. This is compiled into the application and can be used for the version number of the installation. However, some applications do not have a version number built in (or they are just additional files not applications). Only check this item when a version number can be extracted from the app by Wix. If the application has a version number it will be displayed in the Version edit box in the Product Area.
If no application is chosen such that Wix can extract a version number then the alternative is to set the version number in the Version edit box in the Product area.
On Path
Some applications need a Command Line interface. One requirement is that it be easily found. Putting the path of the Program Files directory in which the application resides makes it possible for the user to invoke the app from the command line without knowing where it is. Check this box to place the directory on the System Environment variable Path.
OS Check Boxes
It is possible to limit the installation of a component by referring to the OS version number (i.e. VersionNT). This value is supported in Win 2000 through the near future. I've only allowed three different OSes to be selected.
If Win7 is checked then all installation on Win7 and all future operating systems will be allowed but older ones will not. If two OS check boxes are selected then the OSes inclusive between the two OSes will allow installation (note, Win7 always allows future OSes too). Selecting one check box will only allow installation to that OS.
Select Icon
Both a drop down and a Browse button. When a new Icon is to introduced to the installer use the Browse button to select the icon. Later, the drop down combo box will have a list of icons and one can be selected that way.
Program File Dir Name
This can be one or two levels of directories that will appear in the "Program Files (X86)" directory. The lowest level will contain the described by the component. Here are two examples:
- Manufacture Name\My App -- two levels, app is stored in "My App" directory
- My App -- one level, app is stored in "My App"
Each component may share the same directory or they may be in entirely separate directories. Be sure to spell the names correctly (Cut and Paste may take the chance out of the spelling issue).
Start Menu Name
This can be one or two levels of directories that will appear in the "All Programs" directory on the Start Menu. The lowest level will contain the described by the component. Here are two examples:- Manufacture Name\My App -- two levels, app is stored in "My App" Menu
- My App -- one level, app is stored in "My App" Menu


Match Match
i
Match Match
is a card game in which all of the cards are laid face down on a surface and two cards are flipped face up over each turn. The object of the game is to turn over pairs of matching cards.
Complexity
12
24
48
Card shirt


{% if lst_most_played_games == [] %}
{% for (app_id,name,price,image_url) in lst_popular_games %}
{% endfor %}
{%else%}
{% for (app_id,name,price,image_url) in lst_most_played_games %}
{% endfor %}
{% for (app_id,name,price,image_url) in lst_content_recommended %}
{% endfor %}
{% for (app_id,name,price,image_url) in lst_item_recommended %}
{% endfor %}
{% for (app_id,name,price,image_url) in lst_als_recommended %}
{% endfor %}
{% endif %}


tl;dr: Quickly looking for the right way to subclass delegates? Jump to the Summary and Code Snippets.
The delegation pattern is widely used throughout the Cocoa frameworks, for good reason. A delegate customizes another object’s behavior in a lightweight way, with less coupling than a subclass. A single object can be the delegate of several others, so a set of controls presented to the user as a logical group (say, a table and some text fields) can be managed by code that’s also kept together.1
Interface or implementation?
List delegate protocols in the public interface of a class.
Let’s consider a chain of three classes: SpecificViewController : BaseViewController : UIViewController
. BaseViewController
contains behavior common to several views of our app, and has several subclasses, including SpecificViewController
. This behavior includes managing a UITextField
, and so it conforms to the <UITextFieldDelegate>
protocol.
Objective-C 2.0 added the ability to declare protocols in a class’s implementation, like this:
BaseViewController.h
@interface BaseViewController : UIViewController
@end
BaseViewController.m
#import "BaseViewController.h"
@interface BaseViewController () <UITextFieldDelegate>
@property (nonatomic) IBOutlet UITextField *baseTextField;
@end
@implementation BaseViewController
- (void)textFieldDidEndEditing:(UITextField *)textField {
if (textField == self.baseTextField) {
NSLog(@"%s with %@", __func__, textField.text);
// ...
}
}
@end
At first glance this seems like a good approach, and makes our encapsulation senses tingle with joy. But it has problems.
Someone else on the team is working on a subclass, SpecificViewController
, which needs an extra text field. So it also conforms to <UITextFieldDelegate>
:
SpecificViewController.m
#import "SpecificViewController.h"
@interface SpecificViewController () <UITextFieldDelegate>
@property (nonatomic) IBOutlet UITextField *secondTextField;
@end
@implementation SpecificViewController
- (void)textFieldDidEndEditing:(UITextField *)textField {
if (textField == self.secondTextField) {
// ...
}
}
@end
Have you noticed the problem? The behavior attached to baseTextField
has silently been lost!
The subclass hasn’t called super, but calling super would generate a compiler warning, because the protocol was listed in the superclass’s private implementation and not public interface.
The solution to this is to declare delegate protocol conformance in a class’s public interface, and call super from overridden delegate methods. Declaring protocol conformance in the implementation is therefore often inappropriate.
However, there’s another complication.
Forwarding optional methods
Cocoa delegate protocols2 have the slightly unusual feature of optional methods. This speeds up prototyping and development — just implement what you want to use — but interacts with subclassing in a subtle way.
Many delegate methods are @optional
.
Before calling one, you’re responsible for checking whether it’s available; the compiler doesn’t do any checking.
(It’s similar to pointers: the compiler allows you to dereference any address, and your code is responsible for at least ensuring it’s not NULL
.)
We’ve moved the delegate protocol into the BaseViewController.h
header file.
Now subclasses can know that the superclass cares about the protocol, so they should forward the methods.
Let’s adjust SpecificViewController.m
:
SpecificViewController.m
@implementation SpecificViewController
- (void)textFieldDidEndEditing:(UITextField *)textField {
if (textField == self.secondTextField) {
// ...
} else {
[super textFieldDidEndEditing:textField];
}
}
@end
Looks good, works great.
But secondTextField
needs some more behavior, so it adds another delegate method:
@implementation SpecificViewController
- (void)textFieldDidEndEditing:(UITextField *)textField {
// ... as above
}
- (BOOL)textFieldShouldClear:(UITextField *)textField {
if (textField == self.secondTextField) {
return [self someConditionIsMet];
} else {
return [super textFieldShouldClear:textField];
}
}
@end
We try it out, it works as expected.
But then someone presses the “clear” button in baseTextField
, and… crash!
-textFieldShouldClear:
is an optional method in <UITextFieldDelegate>
, so the onus is on the caller to check if it’s safe to call — in this case, it’s not.
So how can we check?
1. Look through the source code of all superclasses.
We see that BaseViewController
doesn’t implement this method, so we can remove the call to super.
But then if it ever adds it, the behavior will be silently lost again!
With this approach, adding or removing a delegate method requires auditing all superclasses and all subclasses — not a scalable solution.
Worse: If you miss something, the code will still compile and run. Everything will likely look fine on the surface, but have a bug lurking below.
2. -respondsToSelector:
, of course
When we implement an object that takes its own delegate, the rule is easy:
Guard all calls to @optional
methods by -respondsToSelector:
, like this:
- (BOOL)canBecomeFirstResponder {
if ([self.delegate respondsToSelector:@selector(myControlCanBecomeFirstResponder:)]) {
return [self.delegate myControlCanBecomeFirstResponder:self];
} else {
return YES;
}
}
Inside -[SpecificViewController textFieldShouldClear:]
the selector we want to check is the same as the method we’re in, so we can just refer to it as _cmd
.3
Now obviously [self respondsToSelector:_cmd]
will return true, because we’re in that very method right now.
So does [super respondsToSelector:_cmd]
perform the check we want?
No.
The quick-and-dirty translation into English reads, “ask super if it responds to the selector in _cmd
,” but that’s wrong and misleading.
We need to think more precisely about what the super
call actually means.
Which is, “call the superclass’s implementation of -respondsToSelector:
, passing it _cmd
”.
Laid out like this, the behavior is clear:
SpecificViewController
hasn’t overridden -respondsToSelector:
, which means [self respondsToSelector:]
and [super respondsToSelector:]
are exactly equivalent, both most likely using NSObject
’s implementation.
3. +instancesRespondToSelector:
, nice to meet you
This is an oft forgotten NSObject
method which does exactly as it sounds: performs -respondsToSelector:
, but at the class level.
So we can do this:
@implementation SpecificViewController
- (BOOL)textFieldShouldClear:(UITextField *)textField {
if (textField == self.secondTextField) {
return [self someConditionIsMet];
} else if ([BaseViewController instancesRespondToSelector:_cmd]) {
return [super textFieldShouldClear:textField];
} else {
return YES;
}
}
@end
This is correct! Note that we also had to hard-code the default value in case the superclass doesn’t respond.4
It’s ugly to hard-code the superclass in there like that, so how about [[[self class] superclass] instancesRespondToSelector:_cmd]
?
This feels like it should be the same, but it’s booby-trapped!
When someone else comes along and declares another subclass, EvenMoreSpecificViewController : SpecificViewController
, then [self class]
is EvenMoreSpecificViewController
and [[self class] superclass]
is SpecificViewController
.
That check would succeed, there’d be a call to super (BaseViewController
), then a crash.
So nope.
It’s slightly cleaner is to use the defining class: [[SpecificViewController superclass] instancesRespondToSelector:_cmd]
.
This adds safety in case a class ever gets split into two, changing the superclass.
But it’s still an absolute reference, so the code isn’t a nice cut-and-paste snippet.
It’d be nice to have a way to reference the defining class — like the __FILE__
macro but for the current @implementation
.
Although there’s nothing built-in, it’s possible to add with a macro (though less efficiently than one provided by the compiler).
I’ve done that in the MyLilKeyPathHelpers project, called _definingClass
.
Using this, we can create a copy-and-paste snippet:
[[_definingClass superclass] instancesRespondToSelector:_cmd]
.
Congratulations, you now know how to subclass a delegate properly!
Curious UITableViewController behavior
I was initially planning on using UITableViewController
for this posts’s example because it’s commonly subclassed, and conforms to both <UITableViewDataSource>
and <UITableViewDelegate>
.
I fired up the inquisitive Cocoa developer’s best friend, Hopper Disassembler, to see which of the optional methods it implemented.
To my surprise, I found it’s more complicated than that.
UITableViewController
plays games with respondsToSelector:
.
SEL sel = @selector(tableView:heightForRowAtIndexPath:);
[UITableViewController instancesRespondToSelector:sel]; // returns YES
[[[UITableViewController alloc] init] respondsToSelector:sel]; // returns NO!
Since this is exceptional behavior, I had to choose a different example.
The reason for this is to support the “static data source” mode which can be set up in storyboards5 with Interface Builder.
UITableViewController
overrides -respondsToSelector:
to check if its _staticDataSource
is nil, and then returns NO
for a bunch of these methods — but only if they haven’t been overridden by a subclass.
As of the iOS 7.0 SDK these methods are:
tableView:titleForHeaderInSection:
tableView:titleForFooterInSection:
tableView:heightForHeaderInSection:
tableView:heightForFooterInSection:
tableView:viewForHeaderInSection:
tableView:viewForFooterInSection:
tableView:heightForRowAtIndexPath:
tableView:indentationLevelForRowAtIndexPath:
There’s actually no harm in calling these without a static data source, so the above solutoion still works for UITableViewController
subclasses.
But what if you really really want to know what UITableViewController
would have returned?
If this were Mike Ash’s blog, there’d now be an in-depth examination on using object_setClass
.
But it’s not, so I’ll just leave you with this snippet:6
- (NSString *)tableView:(UITableView *)tableView titleForHeaderInSection:(NSInteger)section {
Class orig = object_getClass(self);
object_setClass(self, [UITableViewController class]);
BOOL superWouldRespond = [self respondsToSelector:_cmd];
object_setClass(self, orig);
// ...
}
I love the power of Objective-C. Just to be clear, don’t do this! Here be dragons.
Summary and Code Snippets
Specify delegate protocols in the public interface of a class.
@interface MyViewController : UIViewController <UICollectionViewDataSource>
@end
If you implement a method from a protocol conformed to by a superclass, call super.
@interface MyViewControllerSubclass : MyViewController
// adds a section to the collection view
@end
@implementation MyViewControllerSubclass
- (NSInteger)numberOfSectionsInCollectionView:(UICollectionView *)collectionView {
// Optional method. Super may not implement, must check.
NSInteger baseSections = 1;
if ([[MyViewControllerSubclass superclass] instancesRespondToSelector:_cmd]) {
baseSections = [super numberOfSectionsInCollectionView:collectionView];
}
return baseSections + 1;
}
- (NSInteger)collectionView:(UICollectionView *)collectionView numberOfItemsInSection:(NSInteger)section {
// Required method, just call super.
if (section == (collectionView.numberOfSections - 1)) {
return 1;
} else {
return [super collectionView:collectionView numberOfItemsInSection:section];
}
}
@end
With the _definingClass
macro, checking super doesn’t require specifying the class:
@implementation MyViewControllerSubclass
- (NSInteger)numberOfSectionsInCollectionView:(UICollectionView *)collectionView {
// Optional method. Super may not implement, must check.
NSInteger baseSections = 1;
if ([[_definingClass superclass] instancesRespondToSelector:_cmd]) {
baseSections = [super numberOfSectionsInCollectionView:collectionView];
}
return baseSections + 1;
}
@end
-
Of course, over time a controller can get overgrown with many independent delegate responsibilities. For practical advice on managing this in iOS, see the objc.io issue on Lighter View Controllers. ↩
-
Delegates have almost all switched over to fully-fledged Objective-C
@protocol
s. In the olden days they were declared as un-implemented categories onNSObject
, but still called “informal protocols”. ↩ -
_cmd
is likeself
, but references the selector instead of the receiver. See Objective-C Runtime Programming Guide: Using Hidden Arguments. ↩ -
The default for delegate methods that have a return value is usually in the documentation. However some methods, like
-[<UITextFieldDelegate> textFieldShouldReturn:]
, have a complicated set of behavior that’s difficult and fragile to replicate if super doesn’t respond. I consider this bad API design. ↩ -
I don’t know why this requires a storyboard instead of a xib. You can drag out a table view controller and a table view, add
dataMode="static"
in the XML, and Xcode will display and edit it fine. But when it’s compiled, you’ll get error: Table views with embedded sections and cells are only supported in storyboard documents. Anyway, this also requires having the nib load the view controller, instead of the (usual) other way around. ↩ -
You will need to
#import
a particular header for this hackery to compile. If you don’t know which one, using this technique will cause you too much trouble. ↩


Working At TheThemeio
We want people to thrive at TheThemeio; we believe you do your best work when you feel your best. Here are a just a few benefits we’re proud to provide our team.
Health & Wellness
Full health coverage for you and your family, and a $150 monthly fitness reimbursement.
Learning & Development
Receive $500 annually towards a personal development opportunity of your choice.
Parental Leave
Full health coverage for you and your family, and a $150 monthly fitness reimbursement.
Work Remotely
Receive $500 annually towards a personal development opportunity of your choice.
Open Positions
Following list displays our current required positions. This list will update regularly.
Senior Analyst, Data Analytics
Use your extensive knowledge of JavaScript, HTML, and CSS to improve our web client, a complex in-browser application relied upon by millions of users every day. You will work with real-time data streams, engineer for performance across browsers, and delight people by making the best software we can imagine. You will collaborate closely with Product, Design, Application Engineering, and QA to spec, build, test and deploy new features. This position reports directly to the Director of Front-end Engineering.
Senior Front-end Engineer
Use your extensive knowledge of JavaScript, HTML, and CSS to improve our web client, a complex in-browser application relied upon by millions of users every day. You will work with real-time data streams, engineer for performance across browsers, and delight people by making the best software we can imagine. You will collaborate closely with Product, Design, Application Engineering, and QA to spec, build, test and deploy new features. This position reports directly to the Director of Front-end Engineering.
Lead Product Designer
Use your extensive knowledge of JavaScript, HTML, and CSS to improve our web client, a complex in-browser application relied upon by millions of users every day. You will work with real-time data streams, engineer for performance across browsers, and delight people by making the best software we can imagine. You will collaborate closely with Product, Design, Application Engineering, and QA to spec, build, test and deploy new features. This position reports directly to the Director of Front-end Engineering.
Online Advertising Manager
Use your extensive knowledge of JavaScript, HTML, and CSS to improve our web client, a complex in-browser application relied upon by millions of users every day. You will work with real-time data streams, engineer for performance across browsers, and delight people by making the best software we can imagine. You will collaborate closely with Product, Design, Application Engineering, and QA to spec, build, test and deploy new features. This position reports directly to the Director of Front-end Engineering.
Customer Support Manager
Use your extensive knowledge of JavaScript, HTML, and CSS to improve our web client, a complex in-browser application relied upon by millions of users every day. You will work with real-time data streams, engineer for performance across browsers, and delight people by making the best software we can imagine. You will collaborate closely with Product, Design, Application Engineering, and QA to spec, build, test and deploy new features. This position reports directly to the Director of Front-end Engineering.


Hi, my name is Nicola Pisanti.
By writing code i craft devices that generate graphics, sounds and sense the world around them.
It's really important for me to keep my tools free (as in "freedom"), so I mostly work with libraries like openFrameworks and run everything on Linux ( both on desktop and raspberry pi ). I'm also skilled at overusing duct tape and asemic writing.
To see what i'm working on at the moment, check out my journal.
To mail me, click here.
[ the journal ]: my latest experiments
SELECTED WORKS
Synchron: A/V performance for three percussionists
artefatto::cinque: generative music for relays and micro televisions
cento sigilli: asemic writing lab
A man in a park is looking for something: sensors and music for a performance by Antonio Pipolo
scrittura generates music and alphabeths
Junk Rituals is a pseudo-ritual instrument (sawblade + projections)
Hexaglyphics was a digital noise instrument based on an invented hexadecimal alphabeth
artefatto::quattro: stone+monitor pseudosanctuary
Outpost is an installation i made at a Toolkit festival workshop
Profeticomatico: come se fosse un oracolo
Mechanical Pizzica is a mechanical rhytmical instrument (build with a bike and an hand drum)
Cellular Shapes: shaped cellular automata in processing 2
OPEN SOURCE
ofxPDSP: an openFrameworks addon for audio synthesis and generative music
np-samples: an archive of the audio samples i've recorded in the past, free to use
ofxClayblocks: sensors and actuators devices running on OSC networks, with client classes for openFrameworks
ofxDotFrag: cross compatible GL/GLES shaders to be tweaked in realtime
ofxLaunchControllers: binds ofParameters to a Novation Launch Control / XL
ofxFixture: addons to manage multiple DMX fixtures
ofxAudioFile: loads .wav, .ogg, .flac and .mp3 to a float buffer in all the oF platforms
np-extra: extra utility classes for openFrameworks
np-patches: sequencers, synths and effects for ofxPDSP
grimorio: setup scripts for my Debian GNU/Linux systems
<<selfteaching>>: resources for learning


Sign Up
Login
Questions?
Secure Login
Username:
Password:
Log in
Not enrolled? Sign up now. >
Log-in help options
Forgot password?
×
Forgot Password
Username
Submit
×
Forgot Password
Question 1:
Question 2:
Submit
×
Enter OTP
OTP
New Password
Confirm Password
Submit
×
Contact Helpdesk Support:
Mihir Thakkar
for help.


The XML Schema Instance namespace defines two attributes that declare schema location hints that can be used by an XML processor to locate the schema.
Gloze may be be supplied with user-defined namespace/schema-location hints (from the command line or through the API), or it looks for xsi:schemaLocation or xsi:noNamespaceSchemaLocation on the document element.
The following simple XML may be lifted into RDF either by supplying the schema location on the command line (eg. "schemaLocation.xml http://example.org/ mySchema.xsd"), or as shown in this case, using an explicit xsi:schemaLocation. Note that the xsi namespace must be declared.
<?xml version="1.0" encoding="UTF-8"?> <foobar xmlns="http://example.org/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://example.org/ mySchema.xsd" />
When dropping a document into XML, a schema location can be added by supplying the full (base) path of the schema using the schemaLocation parameter (e.g. "-Dgloze.schemaLocation=file:/C:/myExamples/mySchema.xsd" ). The schema used in the mapping are relativized against this base and added to the xsi:schemaLocation or xsi:noNamespaceSchemaLocation attribute on the document element.


---
layout: post
title: turned out pretty well i had some errands to
date: '2011-08-30T09:11:00+02:00'
tags:
- kitchen
- bread
tumblr_url: http://dotmr.tumblr.com/post/9585606742/turned-out-pretty-well-i-had-some-errands-to
---
Turned out pretty well. I had some errands to run, so i let it rise for 2 hours before turned on the oven.


PART 1 - Which film/ TV show features these famous houses?
PART 2 - Reality TV
What was the profession of Gemma Collins when we first met her in TOWIE?.
This Morning presenter Alison Hammond shot to fame on which reality show?
Who did Susan Boyle lose to when she reached the final of Britain’s Got Talent?
Name the 2002 BBC singing competition hosted by Cat Deeley and Patrick Kielty.
What is the name of the medic on I’m A Celebrity...?
PART 3 - Which band member is missing?
Agnetha Fältskog, Björn Ulvaeus, Benny Andersson
Perrie Edwards, Leigh-Anne Pinnock, Jesy Nelson
Tom Fletcher, Danny Jones, Dougie Poynter
Nicole Scherzinger, Carmit Bachar, Ashley Roberts, Jessica Sutta, Melody Thornton
Lee Latchford-Evans, Lisa Scott-Lee, Faye Tozer, Ian "H" Watkins


This test case demonstrates functionality of the Web Forms 2.0 specification and tests the compliance of this cross-browser Web Forms 2.0 implementation.
Regarding fetching data from external resources, the specification states:
There are two scenarios where authors may wish data to be fetched from an external file to fill forms. In the first, a select's options are replaced by options from an external file. In the second, a form's values are prefilled from an external data source.
In both cases, the prefilling may either be full, in which case the previous contents are removed first, or incremental, in which case the fetched data is in addition to the data already in the form.
select
elementsThe formdata
XML file associated with the following form first
clears all of the existing repetition blocks. Then it adds two repetition blocks
with indexes 100 and 101. Then it prefills the values of these two repetition blocks.
data
DOM property dynamically on select
elementsThe following tests are adapted from an example in the Filling select
elements section.
The eight following select
elements should have exactly the same contents. Note that only the first four will work in Internet Explorer 7.
data
DOM attribute set on statically created element:
data
content attribute set on statically created element:
select
element dynamically created; data
DOM attribute set:
select
element dynamically created; data
content attribute set with setAttribute(...)
:
data
DOM attribute set on statically created element:
data
content attribute set on statically created element:
select
element dynamically created; data
DOM attribute set:
select
element dynamically created; data
content attribute set with setAttribute(...)
:


| Central - Yung Shue Wan |
| Effective 1st July 2011 | Updated on 22nd June 2011 |
Mondays to Saturdays |
Sundays and Public Holidays |
||||||||||||||||||||||
From Central |
From Yung Shue Wan |
From Central |
From Yung Shue Wan |
||||||||||||||||||||
|
|
|
|
|
|
|
|
||||||||||||||||
| Remarks: | |||||||||||||||||||||||
| { } Only available on Saturdays, no freight service * Only small quantity of freight can be carried # Advanced booking is required for carriage of large quantity of freight @ Optional Service ^ Last sailing |
|||||||||||||||||||||||
Fare Table ($HKD) |
Mondays to Saturdays |
Sundays and Public Holidays |
|||||||||||||||||||||
| Adult | $16.10 |
$22.30 |
|||||||||||||||||||||
| Children (Under 12) | $8.10 |
$11.20 |
|||||||||||||||||||||
| Elderly (Aged 65 or above) / Disable | $8.10 |
$11.20 |
|||||||||||||||||||||
| Children under 3 (Accompanied) | Free |
Free |
|||||||||||||||||||||
|
Aberdeen - Yung Shue Wan via Pak Kok Tsuen
| Operator: | Tsui Wah Ferry Service (H.K.) Limited |
| Boarding location: | Aberdeen: Pontoon at Aberdeen Promenade Pak Kok Tsuen: Pak Kok Tsuen Public Landing Steps at Lamma Island Yung Shue Wan: Yung Shue Wan Ferry Pier at Lamma Island |
Adult fare per trip: Full Fare HK$17.5
Section Fare (Pak Kok Tsuen - Yung Shue Wan) HK$7.5
Journey time: about 35 minutes
Frequency:
| Mondays to Saturdays (except public holidays) | ||||
| From Aberdeen | From Pak Kok Tsuen | From Yung Shue Wan | ||
| 6.00 a.m. | → | 6.25 a.m. | → | |
| ← | 6.50 a.m. | ← | 6.40 a.m. | |
| 7.20 a.m. | → | 7.45 a.m. | → | |
| ← | 8.10 a.m. | ← | 8.00 a.m. | |
| 8.40 a.m. | → | 9.05 a.m. | → | |
| ← | 9.30 a.m. | ← | 9.20 a.m. | |
| 11.15 a.m. | → | 11.40 a.m. | → | |
| ← | 12.10 p.m. | ← | 12.00 noon | |
| 1.45 p.m. | → | 2.10 p.m. | → | |
| ← | 2.30 p.m. | ← | 2.20 p.m. | |
| 3.20 p.m. | → | 3.45 p.m. | → | |
| ← | 4.05 p.m. | ← | 3.55 p.m. | |
| 4.30 p.m. | → | 4.55 p.m. | → | |
| ← | 5.15 p.m. | ← | 5.05 p.m. | |
| 5.50 p.m. | → | 6.15 p.m. | → | |
| ← | 6.50 p.m. | ← | 6.35 p.m. | |
| 7.15 p.m. | → | 7.40 p.m. | → | |
| ← | 8.10 p.m. | ← | 8.00 p.m. | |
| 8.35 p.m. | → | 9.00 p.m. | → | |
| ← | 9.20 p.m. | ← | 9.10 p.m. | |
| 10.00 p.m. | → | 10.25 p.m. |
| Sundays and public holidays | ||||
| From Aberdeen | From Pak Kok Tsuen | From Yung Shue Wan | ||
| 7.30 a.m. | → | 7.55 a.m. | → | |
| ← | 8.15 a.m. | ← | 8.05 a.m. | |
| 8.40 a.m. | → | 9.05 a.m. | → | |
| ← | 9.25 a.m. | ← | 9.15 a.m. | |
| 9.50 a.m. | → | 10.15 a.m. | → | |
| ← | 10.35 a.m. | ← | 10.25 a.m. | |
| 11.00 a.m. | → | 11.25 a.m. | → | |
| ← | 11.45 a.m. | ← | 11.35 a.m. | |
| 12.10 p.m. | → | 12.35 p.m. | → | |
| ← | 12.55 p.m. | ← | 12.45 p.m. | |
| 2.00 p.m. | → | 2.25 p.m. | → | |
| ← | 2.45 p.m. | ← | 2.35 p.m. | |
| 3.15 p.m. | → | 3.40 p.m. | → | |
| ← | 4.00 p.m. | ← | 3.50 p.m. | |
| 4.25 p.m. | → | 4.50 p.m. | → | |
| ← | 5.10 p.m. | ← | 5.00 p.m. | |
| 5.35 p.m. | → | 6.00 p.m. | → | |
| ← | 6.20 p.m. | ← | 6.10 p.m. | |
| 6.45 p.m. | → | 7.10 p.m. | → | |
| ← | 7.30 p.m. | ← | 7.20 p.m. | |
| 7.55 p.m. | → | 8.20 p.m. | → | |
| ← | 8.40 p.m. | ← | 8.30 p.m. |
Effective day : 20.2.2012


A couple more inches of rain a year or a few degrees warmer may not seem like a big deal. But these weather changes, already underway in the Midwest, could be devastating for the health of Lake Michigan. When city sewers overflow, rainwater runs into the Chicago River and the lake, carrying pollutants with it. As heavy rainstorms become more frequent, more gasoline, trash, fertilizers, pesticides, and other contaminants will degrade Chicago's water quality and harm fish and other aquatic wildlife. And the force of the stormwater flowing into the lake or river can erode shorelines and disrupt fish spawning, development, and migration.
Warmer-than-normal lake temperatures could also wreak havoc on the food web. Higher water temperatures block nutrients from spreading throughout the lake, making it hard for organisms at the bottom of the food chain to find food. As a result, everything from tiny crustaceans to large predatory fish are left with less to eat. And because warmer water has less oxygen, rising temperatures could leave fish without the oxygen they need to grow and reproduce. Long-term increases in water temperature can even cause a massive die-off of fish.
Unusually high temperatures, especially during winter months, also contribute to Lake Michigan's record-low water levels. Large sheets of ice that cover the lakes in winter and slow down evaporation are becoming a thing of the past. Large portions of the lake have been ice-free for years, causing more water to evaporate than spring and summer rainfall can replace.