# Machine Learning, Deep Learning, and Hedonic Methods for Real Estate Price
Prediction

Mahdieh Yazdani Department of Economics, Colorado University at Boulder. Email
contact<EMAIL_ADDRESS>
###### Abstract

In recent years several complaints about racial discrimination in appraising
home values have been accumulating. For several decades, to estimate the sale
price of the residential properties, appraisers have been walking through the
properties, observing the property, collecting data, and making use of the
hedonic pricing models. However, this method bears some costs and by nature is
subjective and biased. To minimize human involvement and the biases in the
real estate appraisals and boost the accuracy of the real estate market price
prediction models, in this research we design data-efficient learning machines
capable of learning and extracting the relation or patterns between the inputs
(features for the house) and output (value of the houses). We compare the
performance of some machine learning and deep learning algorithms,
specifically artificial neural networks, random forest, and $k$ nearest
neighbor approaches to that of hedonic method on house price prediction in the
city of Boulder, Colorado. Even though this study has been done over the
houses in the city of Boulder it can be generalized to the housing market in
any cities. The results indicate non-linear association between the dwelling
features and dwelling prices. In light of these findings, this study
demonstrates that random forest and artificial neural networks algorithms can
be better alternatives over the hedonic regression analysis for prediction of
the house prices in the city of Boulder, Colorado.

keywords: Housing price prediction model, Artificial neural network, Random
Forest, K nearest neighbor, regression analysis.

###### Contents

  1. 1 Introduction
  2. 2 Modeling the Housing Market
    1. 2.1 Hedonic Pricing Model
    2. 2.2 Artificial Neural Network Model
    3. 2.3 Random Forest Model
    4. 2.4 K Nearest Neighbors Model
  3. 3 Data and Functional Forms
  4. 4 Results
  5. 5 Conclusion
  6. A Appendix

##  1 Introduction

In most families dwelling is one of the most important components of a
household’s wealth (see e.g., Arvanitidis, (2014)). Consequently, house prices
are of great interest to actual and potential homeowners. The property prices
are not only important to stakeholders, but also to insurance companies,
property developers, appraisers, tax assessors, brokers, banks, mortgage
lenders, and policy makers (see e.g., Frew and Jud, (2003)). The price of
residential properties is affected directly by their attributes such as
structural factors, socio-economic status of the neighborhood, environmental
amenities, and location. For several decades, the estimation of real estate
assets have relied on hedonic pricing (HP) models. In this method, to estimate
the sale price of the residential properties, appraisers usually walk through
the properties, observe the house, collect data, and make use of the HP
models. However, this method bears some costs and by nature is subjective and
biased. Biased real estate appraisal is a common and longstanding issue in the
United States. For example, in the recent years several complaints about
racial discrimination in estimating home values have been accumulating (see
e.g., Bloomberg CityLab). Consequently, to obtain a fair home assessment and
avoid subjectivity and biases in the dwelling appraisals, the need for
development of accurate housing valuation and property price prediction models
has been growing (see e.g., Fan et al., (2018)).

As alternative tools, machine learning and deep learning algorithms such as
artificial neural networks (ANN) , $k$ nearest neighbor (kNN), bounded fuzzy
possibilistic method (BFPM), random forest (RF), and support vector regression
(SVR) have been offered for the house price valuation and real estate property
price prediction model. Some studies such as Bigus, (1996), Lenk et al.,
(1997), Kauko et al., (2002), Curry et al., (2002), Pagourtzi et al., (2003),
Limsombunchai, (2004), Peterson and Flanagan, (2009), Islam and Asami, (2009),
Selim, (2011), Morano and Tajani, (2013), Ali et al., (2015), Park and Bae,
(2015), Yazdani and Choroś, (2018), Čeh et al., (2018), Poursaeed et al.,
(2018), Hong et al., (2020), and Pai and Wang, (2020) utilized deep learning
and machine learning techniques on real estate markets. Most of these studies
found that deep learning and machine learning algorithms perform better than a
hedonic regression method for real estate market appraisal. Namely, Selim,
(2011) compared the prediction performances of HP regression and ANN model for
prediction of dwelling prices in Turkey. It demonstrated that an ANN performs
better. Yao et al., (2018) integrated a convolutional neural network with RF
inside Shenzhen’s housing market. Park and Bae, (2015) has compared the
performance among several machine learning algorithms such as Repeated
Incremental Pruning to Produce Error Reduction, Naïve Bayesian, and AdaBoost
to identify better forecasting models. These studies have shown promising
application of machine learning and deep learning algorithms in housing
markets.

In order to minimize human involvement and the biases in the real estate
appraisals and boost the accuracy of the real estate market price prediction
models, in this paper we design data-efficient learning machines capable of
learning and extracting the relation or patterns between the inputs (features
for the house) and output (value of the houses). In this study we compare the
performance of some machine learning and deep learning algorithms,
specifically ANN, RF, and kNN approaches to that of HP method on house price
prediction to develop property appraisal models in the city of Boulder,
Colorado. In the city of Boulder, a popular mountain city, buying a house is
considered as one of the most profitable investments and most people in
Boulder know the benefit of owning a house. Even though this study has been
done over the houses in the city of Boulder it can be generalized to the
housing market in any cities.

The objective of this study is to compare the predictive power of HP
regression methods with machine learning and deep learning algorithms on house
price prediction. Development of an accurate dwelling price prediction method
can significantly improve the efficiency of the housing market. Having access
to efficient and accurate house price prediction models can be important for
actual and potential homeowners, investors, property developers, appraisers,
tax assessors, brokers, banks, mortgage lenders, and policy makers and affect
their decisions and policies (see e.g., Frew and Jud, (2003) and Yazdani,
(2020)).

The contributions of this paper are as follows:

  * •

Collecting real estate datasets from different resources consisting of
Multiple Listing Service (MLS) database, Public School Ratings, Colorado Crime
Rates and Statistics Information, CrimeReports, WalkScore, and US Census
Bureau.

  * •

Screening the collected data set, data cleansing, and applying imputation and
winsorization methods.

  * •

Detecting multicollinearity problems, standardizing the numerical variables,
and applying the one-hot encoding method over the categorical variables.

  * •

Applying the ANN, RF, and kNN approaches to develop property appraisal models
and comparing these methods with HP model.

The rest of the paper is organized as follows. Section 2 provides an overview
of the hedonic price, deep learning, and machine learning approaches.
Following this, section 3 explains the empirical models and presents the data
set, variables and methodology used in this paper. The estimation results
obtained by these techniques described in section 4. Finally, conclusions and
implications are provided in section 5.

##  2 Modeling the Housing Market

We collected a labelled data set composed of input and target variables
$(X,Y)$, where in this study the residential property features are the input
variables $X$ and the vector $Y$ is the output variable consisting of
logarithm of house values. Our goal is therefore to learn a function

$F:X\rightarrow Y$ (1)

such that given new residential property features $X$, $F(X)$ can predict the
house value $Y$.

In this study we first briefly introduce HP, ANN, RF, and kNN techniques, and
next apply these methods to our data set and compare their performance.

### 2.1. Hedonic Pricing Model

Hedonic price analysis originates from Waugh, (1928), Court, (1939), Stone,
(1954), Adelman and Griliches, (1961), and Lancaster, (1966). The literature
on HP methodology is broad. For example, Rosen, (1974) used the hedonic
regression models in the field of real estate and urban economics. Thereafter,
residential hedonic analysis has become widely used as an assessment tool to
model real estate markets (see e.g., McMillan et al., (1980), Blomquist and
Worley, (1981), Milon et al., (1984), Haase et al., (2013), Jiang et al.,
(2014), Del Giudice et al., (2017), Król, (2020), and Yazdani, (2021)). In the
housing market the HP regression model is essentially a function of a bundle
of property characteristics such as structural attributes, socio-economic
status of the neighborhood, environmental amenities, and location. This model
is constructed by regressing the price of the dwellings on a vector of housing
features. The HP models have been used to explain variations in dwelling
prices and to determine the impact of specific attributes on house values (see
e.g., Geng et al., (2015)). By differentiating the HP model with respect to
each feature we can easily find the marginal contribution or implicit
valuations of the house features (see McMillan et al., (1980)). The simplicity
in estimating and interpreting the regression coefficients is the major
advantage of the HP models.

Even though the HP regression approach has been extensively used to analyze
the market price of houses, it is limited in its ability to uncover the non-
linear relationships between housing price and its characteristics. Factors
such as model specification procedures, heteroscedasticity, and outlier data
points are other issues that can impede the performance of hedonic regression
models. In general, HP method needs feature engineering which involves human
and consequently, subjectivity and biases in the real estate appraisals. There
are, however, some alternative techniques, which are better suited to deal
with these problems. As alternative tools, machine learning and deep learning
algorithms are more flexible than the hedonic method. Unlike the HP regression
model, there are neither strict assumptions nor predefined functional form to
be employed in the deep learning and machine learning algorithms and we do not
need to specify an explicit function between input and target variables.
Instead, these algorithms learn from the observed data set and explore the
linear and non-linear relationship between attributes of the properties and
the property values. Some studies such as (Lenk et al., (1997), Kauko et al.,
(2002), Curry et al., (2002), Pagourtzi et al., (2003), Limsombunchai, (2004),
Yu and Wu, (2006), Islam and Asami, (2009), Selim, (2011), Kontrimas and
Verikas, (2011), Morano and Tajani, (2013), Ali et al., (2015), Park and Bae,
(2015), Čeh et al., (2018), and Hong et al., (2020)) utilized deep learning
and machine learning techniques on real estate markets to develop housing
price prediction models. Most of these studies found that deep learning and
machine learning algorithms perform better than a hedonic regression method
for estimating dwelling values.

### 2.2. Artificial Neural Network Model

The first artificial neural network was designed by psychologist Frank
Rosenblatt (see Rosenblatt, (1958)). Artificial neural network is a deep
learning technique that was inspired by the human brain (see e.g., Bengio et
al., (2021)). ANN is designed to simulate the learning process in the human
brain (see e.g., Collins et al., (1998), Agatonovic-Kustrin and Beresford,
(2000), Vazirizade et al., (2017), and Fleuret, (2020)). In particular, the
objective of designing a neural network is to learn the association between
inputs and outputs in the given data set, so that for any new inputs, the
corresponding output could be determined.

In ANN there is no need to assume explicit functions between input and output
of the studies. ANN learns directly from observed data and models non-linear
and compound relationships. An ANN requires a number of processing units,
called neurons (nodes), to be connected together into a neural network (see
e.g., Heiat, (2002) and Yegnanarayana, (2009)). The ANN method consists of
multiple layers: input, hidden, and output layers. Each layer contains one or
more neurons. Every neuron in one layer is connected to each single node at
the neighbor layer(s), and each connection is associated with a weight, which
determines how strongly each of the neurons affects the others (see e.g.,
Haykin and Network, (2004) and Kassambara, (2018)). The number of neurons and
layers in different artificial neural networks can vary. The number of neurons
in the hidden layers determines the width of the model and the number of
layers determines the depth of the model. Figure 1 exhibits a simplified
feedforward neural network model. This network consists of three layers of
neurons in which each neuron is connected to all the neurons in the next
layer. The input enters the network through the input layer, it is multiplied
by a weight matrix and the result passes through an activation function
(transfer function) to the intermediate hidden layer, which is located between
the input and output layer. An activation function defines how the weighted
sum of the input is transformed into an output from a neuron or neurons in a
layer of the network. Usually, the purpose of an activation function is to
introduce non-linearity to help identify the underlying patterns which are
complex (see e.g., Hopfield, (1982)). Similarly, the hidden layer through an
activation function performs linear or non-linear transformations over the
output of the previous layer (the input layer in this figure) and directs them
to the output layer. Finally, the output layer performs the calculations and
then the outputs or solutions are computed.

In an ANN, during the learning process, the values of the weight vector are
adjusted to minimize the loss function. The most often loss function used in a
supervised ANN is defined as

$L={\frac{\sum_{i=1}^{n}{(y_{i}-\hat{y_{i}})}^{2}}{n}}$

where L is the mean squared errors, where error is the difference between
actual output ${y_{i}}$ and estimated output $\hat{y_{i}}$ for a given input
and $n$ is the number of samples.

To minimize the loss function all of the connection weights in the network are
gradually adjusted, using the backpropagation method to do gradient descent
optimization. In backpropagation we start from the output layer through the
hidden layer(s), and to the input layer, by adjusting weights for each layer
throughout the network. Computing the sensitivity of a loss function with
respect to each weight in the network, will eventually lead to minimization of
error between the predicted and actual outputs. We update each weight until we
get estimated outputs which are close to the actual outputs for a particular
input. This process of adjusting the weights is referred to as learning. In
the learning process the weights of the connecting links in a network are
adjusted so as to enable the network to learn the pattern between the given
input and output data set. As the learning process proceeds, the connections
which lead to the correct solutions are strengthened, and the incorrect
connections are weakened.

Figure 1: An Artificial Neural Network Model.

In real life almost all of our data is non-linear. ANNs are usually efficient
in predicting complex systems that demonstrate non-linear behavior. ANN has
applications in many different fields, namely medical diagnosis, image
processing, voice recognition, credit scoring, evaluating loan applications,
forecasting, fraud detection, and making decisions. By changing the activation
functions of the output layer, ANN can be applied to both classification and
regression problems.

In this study, input and output variables in the ANN are the same as in the HP
regression model. The inputs are features for the house, and the predicted
outputs are the estimated price of the houses. In this study we employ the
feed forward network to estimate the sale price of the residential properties.
We use the Rectified Linear Unit (ReLU) as the activation function (see e.g.,
LeCun et al., (1998)). The ReLU activation function is easy to optimize
because it is similar to linear function. A ReLU activation function will
output the input directly if it is positive, otherwise, it will output zero.

### 2.3. Random Forest Model

Random forest, a supervised learning technique, is one of the most robust
machine learning algorithms that can be used for both classification and
regression problems (see Breiman, (2001)). RF is a special type of bootstrap
aggregation called bagging that combines many decision trees (see e.g.,
Breiman, (1996) and Ho, (1998)). The idea is to train several different trees
separately, and then integrate the predictions from multiple decision trees to
make more reliable predictions than that of any single decision tree. In RF
each decision tree is grown independently on a separate bootstrap sample
derived from the training data set. Figure 2 illustrates an example of RF.

In general, to make the RF predictions more reliable, more trees are required.
However, increasing the number of trees slows down the prediction process. To
limit overfitting and avoid high correlation amongst the decision trees the RF
technique uses bagging (see e.g., Breiman, (1996)). In addition, each decision
tree is grown on only a random selection of attributes (predictors). To
construct each single decision tree, increasing the number of variables, when
deciding to split a node, increases both the internal correlation and the
predicted power. Reducing it decreases both (see e.g., Strobl et al., (2008)).
Somewhere in between is an optimal range for the number of variables. The
optimal number of features (also known as mtry in R) is normally around the
square root of the number of features in the classification method. In the
regression approach, the optimal number of variables is usually around the
$\frac{1}{3}$ of the number of characteristics. Reducing the maximum depth
will also help reduce the complexity of the model and avoid overfitting.

StatementStatementStatementStatementLeafLeafStatementTrueFalseTrueFalseTrueFalseStatementStatementStatementLeafStatementLeafStatementTrueFalseTrueFalseTrueFalseLeafLeafTrueFalseLeafLeafTrueFalseLeafLeafTrueFalseLeafLeafTrueFalseStatementStatementStatementStatementLeafLeafLeafTrueFalseTrueFalseTrueFalseStatementStatementStatementStatementLeafStatementLeafTrueFalseTrueFalseTrueFalseLeafLeafTrueFalseLeafLeafTrueFalseLeafLeafTrueFalse
Figure 2: A Random Forest Model.

RF is an ensemble technique (see e.g., Breiman, (2001)) that integrates the
predictions from multiple decision trees to make more reliable predictions
than that of any single decision tree. After a large number of trees are
generated, depending on the task, whether it is classification or regression
problem RF aggregates the results from the decision trees by voting or
averaging respectively. In the classification problem the RF method gives a
prediction based on a majority vote. The random forest regression model
produces the final prediction by averaging the predictions made by each of its
decision trees (see e.g., Antipov and Pokryshevskaya, (2012)). Merging
decision trees’ predictions, leads to more robust overall predictions and the
predictions will be closer to the true value on average.

RF has many advantages. It is relatively easy to build and does not require
too much computational time and the hyperparameters tuning is not too
expensive. Since the RF algorithm is based on the bootstrapping, it performs
well on even small data sets. Moreover, RF has few, if any, statistical
assumptions. It does not assume that data is normally distributed, the
relationship is linear, or there are specified interactions. Interpretability
of the trained model is another major advantage of the RF method.
Additionally, it is very easy to measure the relative importance of each
feature on the output variable.

The RF algorithm is used in many different fields, namely stock market,
banking, health care, and housing market. A number of researchers applied the
RF technique in the real estate market (see e.g., Antipov and Pokryshevskaya,
(2012), Yoo et al., (2012), Hong et al., (2020), and Levantesi and Piscopo,
(2020)).

### 2.4. K Nearest Neighbors Model

Another machine learning method for housing market valuation is the k nearest
neighbors approach. kNN is a supervised learning technique that has been used
in both regression and classification problems. The kNN method uses attributes
similarity to predict the values of any new data points. It checks how alike
is a data point (a vector of features) in the test data set from other
available data points in the training data set. The kNN method is similar to
what the appraisers do in the real estate market. To determine a house price,
realtors or appraisers search for dwellings with akin features, namely square
footage, number of rooms, property type, location, neighborhood, and socio-
economic attributes. They would value the new house around the average price
that these properties sold for. Likewise, in the kNN approach the estimated
property value is the average of the values of its $k$ nearest neighbors (see
e.g., Kassambara, (2018)).

KNN is a distance based model. In both regression and classification problems
kNN approach generally measures the similarity between the available
transactions and the new house by adopting a distance function. There are
various methods such as Euclidean, Manhattan, Minkowski, Chebychev,
Mahalanobis, and Hamming distance that we could use, to measure the distance
between the target point and each training point and calculate the alikeness
of data points based on all factors. Once the distance of a new observation
from the current observations has been measured kNN finds the $k$ data points
that are alike or nearest to the new record. In the kNN regression approach,
the value of the new numerical data point is predicted by calculating the
average of the observations in the $k$ closest neighborhood. The average of
the $k$ nearest neighbors is the prediction for the numerical target. In
contrast to the kNN regression method, in kNN classification problem, to
classify a new data point in the test data set, kNN looks for its $k$ nearest
points in the training samples and predicts the class of the target point in
the test sample based on the mode of its $k$ nearest neighbors or the majority
of classes among the $k$ closest imminent. It then assigns the corresponding
label to the observation (see e.g., Larose and Larose, (2005) and Domeniconi
et al., (2002)).

KNN method is an efficient and accurate technique that can be used in a
variety of applications such as finance, economic forecasting or prediction,
climate forecasting, text mining, and genetics (see e.g., Larose and Larose,
(2014)). One major advantage of the kNN approach is that it is a non-
parametric method, which means there are no assumptions about how the data set
has been distributed. Consequently, if we do not have enough information about
the distribution of our data, the kNN method might be the right model to pick
(see e.g., Devroye et al., (1994) and Devroye and Wagner, (1982)).

kNN is a lazy learning or an instance-based method (see e.g., Aha, (2013)).
This means kNN does not explicitly learn a model. kNN memorizes the training
data set and stores the training data set for the prediction. When a new data
point is given the kNN algorithm uses the training data set for the
prediction. For every point in the test data set, the model has to compute the
distance between each data point in the test data set and all data points in
the training data set and then find the nearest neighbors. Next, it returns
the average of the corresponding output values in the training set. Therefore,
it needs high memory to store all this information. As such, the computation
and memory costs are quite high. When there are many independent variables the
kNN method becomes impractical. In addition, kNN is not robust against
irrelevant factors since in distance metrics all factors contribute to the
similarity. However, by careful feature selection or weighting the feature,
this can be avoided.

Figure 3: A K Nearest Neighbors Model.

In this study the target variable is the house value. Since the house value is
a continuous variable we will focus on the kNN regression approach. Figure 3
shows a kNN algorithm for $k=5$. The predicted outcome ($\widehat{Y^{*}}$) for
the new data point (the star with the red color) would be the average of the
outcomes of its $5$ closest neighbors.

This paper examines the relative influence of property characteristics, socio-
economic factors, and the impact of accessibility on the price of residential
properties in the city of Boulder, Colorado. In this study we use HP analysis,
ANN, RF, and kNN approaches, to develop housing price prediction models in the
city of Boulder, Colorado. Before proceeding with these models, we are
interested in getting some insights about the available data set.

##  3 Data and Functional Forms

In general, machine learning and deep learning techniques are used over large
data sets to recognize hidden patterns that are difficult to detect through
regression models. However, it is also applicable in small data sets (see
e.g., Steffensen, (2021) and Levantesi and Piscopo, (2020)). The data set we
collected in this study consists of $1061$ residential properties sold in the
city of Boulder, Colorado. Figure 4 plots the city of Boulder on the map. To
isolate the influence of time on property prices, the data used in this study
is restricted to houses sold in a single year between January 1, 2019 and
December 31, 2019 (see Eckert et al., (1990)). The samples used for housing
market estimation are not necessarily random draws from the population of
dwellings, but are all the properties recently sold in the city of Boulder.
Robinson, (1979) states that recent sales are not necessarily a random draw
from the total housing stock. If the purpose is to index the market of
available units, this may not be of great concern, but if the purpose is to
index the total stock, we must concern ourselves with possible selection bias.
Several studies have tested the existence of such biases. So far most studies
have found the magnitude of the bias to be modest (see e.g., Gatzlaff and
Haurin, (1998)).

Figure 4: City of Boulder.

We collected real estate datasets from different resources consisting of
Multiple Listing Service databases222https://realtyna.com/blog/list-mls-us,
Public School Ratings333https://www.greatschools.org, Colorado Crime Rates and
Statistics Information444https://www.neighborhoodscout.com/co/crime,
CrimeReports555https://www.cityprotect.com,
WalkScore666https://www.walkscore.com, and US Census
Bureau777https://data.census.gov/cedsci. Before proceeding with the estimation
of our models we merged all data sets obtained from these websites. We
screened our collected data set. Among these transactions 4 houses were in
rough shape and needed to be rebuilt. Consequently, we deleted those 4
observations. All houses except one; which has lots of luxury furniture, have
been sold unfurnished. So, we dropped the only furnished property. 30
properties were reported as horse properties. We eliminated those associated
records. We checked for duplicate records. We found 4 duplicate transactions
and eliminated those. In a number of observations we faced missing data points
for some characteristics. For example, the number of bedrooms, bathroom,
parking, Lot Area, HOA fees, Solar Power and Pool, Bath tub, Sauna, or Jacuzzi
were missing in some records. We updated some of these missing data points by
visiting different websites and looking up for the correct information.
Finally, a few observations left with missing data points for the continuous
variable Lot Area and the dummy variables Solar Power and Pool, Bath tub,
Sauna, or Jacuzzi. To deal with missing data points we could eliminate from
the model any attributes that have missing values, or alternatively we could
drop all those properties with missing attributes. However, deleting
observations due to incomplete list of attributes can cause sample size
reduction and sample selection bias (see e.g., Hill, (2013)). Instead, to
avoid losing the samples from analysis we imputed missing values in the lot-
size variable with its mean. Similarly, we imputed missing values for the
dummy variables Solar Power and Pool, Bath tub, Sauna, or Jacuzzi with their
mode. Another problem we encountered is the presence of outliers. The outliers
were identified by calculating the $95\%$ percentile. To mitigate the
influence of outliers on the analysis, we applied one-sided $95\%$ or two-
sided $90\%$ Winsorization. That means outliers were set to the value of the
$95\%$ percentile. The data cleaning process left us with a final sample of
$1018$ observations. Information about the descriptive statistics of the
variables in this data set can be found in Table 1 and 2. In addition to the
mean, standard deviation, minimum and maximum, information about the relative
standard deviation (the coefficient of variation ($CV=\frac{\sigma}{\mu}$),
which represents the extent of variability in relation to the average of the
variable has been reported in Table 1.

Table 1: Descriptive Statistics for Numerical Variables.

Variables Citywide Level Mean St. Dev. Min Max Coefficient of Variation (CV)
House Price ($\$$) $896,332$ 679,195.8 112,897 7,200,000 $76\%$ Lot Area
(SqFt) 18,367 84,909.71 0 1,577,744 462.29% Living Area (SqFt) 2,264 1,398.79
416 10,354 61.78% Age (year) 43.12 21.46 1 98 50% Number of Full Bathrooms
1.55 0.76 0 3 49% Number of Half Bathrooms 0.41 0.51 0 2 124% Number of
$\frac{3}{4}$ Bathrooms 0.64 0.69 0 2 108% Parking 1.68 0.71 0 3 42% HOA Fees
(annually) ($\$$) 1,693.32 2,033.16 0 7,113 120% Drive to CBD (minutes) 11.42
6.91 1 26 61% Walk to E.School (minutes) 21.37 17.31 2 68 81% Walk to M.school
(minutes) 33.21 27.72 2 96 83% Walk to H.school (minutes) 46.94 32.92 4 122
70% Married ($\%$) 42.97 16.87 9.90 70.30 40% Median Household Inc. ($\$$)
61,137.44 20,891.56 19,985 96,406 34% Neighborhood’s Population 43,641.85
46,872.42 888 99,081 107% Sample size 1018

Note: Homeowner’s association (HOA). Central business district (CBD).  Table
2: Descriptive Statistics for Categorical Variables.

Variables Citywide Level Levels Description Frequency Percent 0 No 658 64.64
Pool, Bath tub, Sauna, or Jacuzzi 1 Yes 360 35.36 0 No 724 71.12 Solar Power 1
Yes 294 28.88 1 A 334 32.81 Nearest E.School Rank 2 B 548 53.83 3 C 136 13.36
1 A 125 12.28 Nearest M.School Rank 2 B 633 62.18 3 C 260 25.54 Nearest
H.School Rank 1 A 1018 100 1 Central 230 22.59 2 North 238 23.38 3 South 143
14.05 Region 4 East 200 19.65 5 Gunbarrel 125 12.28 6 Rural 82 8.06 0 0
bedroom 3 0.29 1 1 bedroom 72 7.07 2 2 bedrooms 253 24.85 Number of Bedrooms 3
3 bedrooms 285 28.00 4 4 bedrooms 249 24.46 5 5 bedrooms 127 12.48 6 6
bedrooms 27 2.65 7 7 bedrooms 2 0.20 1 Condominum 324 31.83 Property Type 2
Town - Home 90 8.84 3 Single Family 604 59.33 1 Highest crime rate 82 8.06
Neighborhood’s Crime Level 2 Middle crime rate 505 49.61 3 Lowest crime rate
431 42.34 Sample size - 1018 100 -

Before performing our models we should test for multicollinearity problem.
Figure 5 summarizes the Pearson correlation coefficients. Positive
correlations are shown in red color and negative correlations in dark blue
color. The color intensity and the size of the circle are proportional to the
correlation coefficients. We drop variables which generate pairwise
correlation coefficients greater (smaller) than the common thresholds $0.8$
(-$0.8$). We also calculate the generalized variance inflation factors (GVIF).
The accepted GVIF cutoff is $\sqrt{VIF}=\sqrt{5}=2.24$.

Figure 5: Correlation Plot.

We standardize the numerical variables in our data set. In order to
standardize the variables we first compute the average value and the standard
deviation of each variable in our data set. Next, we subtract the mean from
each variable and divide it by its standard deviation. As a result, the mean
value of each numerical variable is approximately zero, and the standard
deviation is almost one. Standardizing the data usually speeds up learning and
leads to more efficient, faster convergence, and better predictions.

The ANN and KNN algorithms would accept only numeric data and cannot directly
work with categorical variables. Consequently, we apply the one-hot encoding
method to convert the categorical variables into numbers. The one-hot encoding
for a variable with $n$ categories creates $n$ new binary variables, one for
each possible category and assigns values of one and zero.

Usually, we are interested in how well the deep learning and machine learning
algorithms perform on data that it has not seen before, since this determines
how well it will work when deployed in the real world. We therefore evaluate
these models’ performance using a test data which is separate from the data
used for training the deep learning and machine learning models.

To evaluate the predictive accuracy and reliability of our models, we split
our sample into training, validation, and test data sets. In this study, the
training data set is a sample of $80\%$ random observations from the $1018$
total transactions. We train the model over the training data set. These
transactions are used to estimate the parameters of the models. The remaining
$20\%$ random samples are kept out from the training data set and are evenly
divided into validation and test data sets. Validation data set is used to
tune the hyperparameters and the test data set is used to assess prediction
precision of the models.

To process the data, we are doing feature engineering. We apply the log-
transformation on the house prices. The log-transformation of the dependent
variable (house prices) often mitigates the problem of heteroskedasticity
associated with the use of the highly skewed sales price variable. Figure 6
plots the distribution of house prices before and after applying the log-
transformation. In addition, we are adding categorical variables in our models
to control for location and property differences.

Figure 6: Distribution of House Prices before and after Applying the Log-
transformation.

##  4 Results

In this study, all of the models were implemented using the open-source R
statistical computing environment with Keras, randomForest, and Caret packages
(see e.g., RColorBrewer and Liaw, (2018)). We run these models separately to
develop housing price prediction models. To evaluate the performance of these
models we computed the performance metrics, namely root mean squared error
(RMSE), mean absolute percentage error (MAPE), and ${\text{R}^{2}}$.

We first make use of linear regression model to estimate the hedonic price
equation over the training data set, at the citywide market level. The housing
market in the aggregate market level contains high variation in the houses
attributes. We both graphically and statistically detected heteroskedasticity.
In the presence of heteroskedasticity OLS estimators are unbiased, while the
standard error and the statistics we used to test hypotheses based on ordinary
least square estimation method will be inefficient. Following Fletcher et al.,
(2000) we use White’s standard errors test to correct the issue of incorrect
standard errors and test statistics. Moreover, to control for property type
and location differences we are adding categorical variables into the hedonic
pricing model. The robust standard errors and heteroskedasticity-consistent
estimates are reported in Table A.6.

Table 3: Hedonic Model Estimates - White Correction.

Independent Variables Citywide Level Coeff. White’s Std. Error White’s
T-Statistics (Intercept) $1.21$ $0.25$ $4.93^{***}$ Ln(Lot Area) $0.81$ $0.15$
$5.38^{***}$ Ln(Living Area) $0.56$ $0.04$ $13.89^{***}$ Age -$0.05$ $0.02$
-$2.37^{**}$ Number of Bedrooms (Bedrooms)2 $0.18$ $0.07$ $2.61^{***}$
(Bedrooms)3 $0.13$ $0.08$ $1.55$ (Bedrooms)4 $0.04$ $0.09$ $0.42$ (Bedrooms)5+
-$0.04$ $0.10$ -$0.34$ Number of Full Bathrooms $0.11$ $0.02$ $4.48^{***}$
Number of Half Bathrooms -$0.002$ $0.02$ -$0.13$ Number of $\frac{3}{4}$
Bathrooms $0.07$ $0.02$ $3.42^{***}$ Parking $0.04$ $0.02$ $2.17^{**}$ HOA
Fees $0.04$ $0.04$ $0.94$ School Ranking (Nearest E.School Rank)2 -$0.06$
$0.04$ -$1.33$ (Nearest E.School Rank)3 -$0.36$ $0.06$ -$5.64^{***}$ (Nearest
M.School Rank)2 -$0.32$ $0.09$ -$3.64^{***}$ (Nearest M.School Rank)3 -$0.21$
$0.11$ -$1.98^{**}$ (Pool, Bath tub, Sauna, or Jacuzzi)1 $0.004$ $0.04$ $0.11$
(Solar Power)1 $0.18$ $0.07$ $2.60^{***}$ Drive to CBD -$0.26$ $0.04$
-$6.06^{***}$ Walk to E.School -$0.0001$ $0.02$ -$0.01$ Walk to M.school
-$0.01$ $0.04$ -$0.30$ Median Household Inc. -$0.01$ $0.03$ -$0.22$
Neighborhood’s Population -$0.02$ $0.02$ -$0.97$

Table 4: (Continued.)

Independent Variables Citywide Level Coeff. White’s Std. Error White’s
T-Statistics Crime Levels (Base: Level 3) Neighborhood’s Crime Level: 1
-$0.20$ $0.07$ -$2.88^{***}$ Neighborhood’s Crime Level: 2 -$0.06$ $0.04$
-$1.55$ Property Type Single Family Houses -$0.97$ $0.31$ -$3.14^{***}$ Town-
Home Houses -$1.04$ $0.27$ -$3.80^{***}$ Regions (Base: Central) (North)2
-$0.30$ $0.07$ -$4.11^{***}$ (South)3 -$0.30$ $0.10$ -$3.11^{***}$ (East)4
-$0.44$ $0.08$ -$5.53^{***}$ (Gunbarrel)5 -$0.43$ $0.14$ -$3.16^{***}$
(Rural)6 -$0.64$ $0.14$ -$4.55^{***}$ Chi-squared ($p-$value) $105$ ($0.0009$)
Dependent variable: Ln(Price)

Note: $***,~{}**$, and $*$ indicate $p<0.01$, $p<0.05$, and $p<0.1$
respectively.
Base group in the citywide housing market: condominium houses, with one
bedroom or less, no pool, bath tub, sauna, or jacuzzi, no solar power, in
central Boulder neighborhood with crime level 3.

Next, we perform the deep learning and machine learning algorithms, ANN, RF,
and kNN, to predict the house prices in the city of Boulder, Colorado. The
structure of deep learning and machine learning algorithms, will substantially
affect the performance of the models’ predictions. The optimal structure is
the one that corresponds to the lowest test error rate. However, using the
test data set for hyperparameter tuning can lead to overfitting. As such, to
evaluate our model performance we use validation data set to estimate the test
error. In order to find an accurate and efficient estimation for the target
variable we find the values for the hyperparameters by minimizing the mean
squared error (MSE) for our validation data sets. This allows us to obtain a
high percentage of variance explained and avoid model’s over-fitting. We
determine the best structure of our models using the validation data by a
random (grid) search method. Following a random search method we test
different model structures on the validation data set in order to select the
best performing model.

For instance in the ANN model, depth and width; the number of layers (depth)
and the number of nodes (width) that are included in each layer will
substantially affect the performance of learning and prediction of the
network. In RF, different values of the hyperparameters namely, number of
trees, number of input variables drawn randomly for each node (mtry), the
splitting rule, and the maximum depth provide different outcomes. In the kNN
approach, the predicted value for the target variable depends on the value of
the hyperparameter $k$; the number of neighbors. The optimal hyperparameters
are those that result in the lowest test error rate. By holding out the
validation data set we are estimating the test error rate. As such, we tune
the hyperparameters in these models.

To find the optimal values of the hyperparameters we randomly selected
training, validation, and test data sets for $20$ times. We ran ANN, RF, and
kNN models over all these samples and found the optimal hyperparameters over
all these 20 samples separately. Next, we averaged all these optimal
hyperparameters. After tuning the hyperparameters we fixed our model
structures by the optimal values of the hyperparameters. More details on the
optimal values of these hyperparameters over our data set are shown in the
Appendix A.

To appraise the predictive performance of all these models in predicting
housing prices on the unseen (test) data set, given the optimal values of the
hyperparameters over these $20$ samples we used some performance metrics such
as root mean squared error (RMSE) that capture the distance between predicted
and observed transaction price, mean absolute error (MAE), mean absolute
percentage error (MAPE), that captures the average percentage deviation of
predictions from the actual transaction prices, and ${\text{R}^{2}}$ that
measures the explanatory power of these models. These performance metrics are
defined as:

$RMSE=\sqrt{{\frac{\sum_{i=1}^{n}{(y_{i}-\hat{y_{i}})}^{2}}{n}}}$

$MAPE=\frac{100}{n}\times\sum_{i=1}^{n}|{\frac{y_{i}-\hat{y_{i}}}{{y_{i}}}}|$

$R^{2}=1-\frac{\sum_{i=1}^{n}{(y_{i}-\hat{y_{i}})}^{2}}{\sum_{i=1}^{n}{(y_{i}-\bar{Y})}^{2}}$

where ${y_{i}}$ is the actual price value, $\hat{y_{i}}$ is the predicted
value, n is the number of samples, and $\bar{Y}$ is the average of the target
variables.

We calculated all these performance metrics (RMSE, MAE, and ${\text{R}^{2}}$)
over all these 20 samples and averaged these values. The results are reported
in 5. Among HP, ANN, RF, and KNN models, we observe that RMSE and MAPE are the
smallest in the RF model. The performance measures RMSE and MAPE in the ANN
model are larger than in RF but smaller than in kNN and HP models. RMSE and
MAPE are lower in HP model than in the kNN model. To examine whether or not
these differences are statistically significant we performed a paired T-test
over these 20 samples. We observed that these differences are statistically
significant at $1\%$ significance level. See Table 5 for the details.

Table 5: Comparison of the performance of HP model with ANN, RF, and kNN.

Measurement HP ANN RF kNN Training Validation Test Training Validation Test
Training Validation Test Training Validation Test RMSE $0.22$ $(0.003)$ $0.23$
$(0.02)$ $0.24$ $(0.01)$ $0.04$ $(0.02)$ $0.21$ $(0.02)$ $0.22$ $(0.02)$
$0.09$ $(0.002)$ $0.21$ $(0.02)$ $0.21$ $(0.03)$ $0.22$ $(0.03)$ $0.27$
$(0.03)$ $0.28$ $(0.03)$ MAEP $1.21\%$ $1.26\%$ $1.36\%$ $0.2\%$ $1.12\%$
$1.21\%$ $0.48\%$ $1.08\%$ $1.14\%$ $1.2\%$ $1.44\%$ $1.58\%$ $\text{R}^{2}$
$0.82$ $0.85$ $0.86$ $0.75$

Note: The standard deviations of these performance metrics are represented in
the parenthesis.

To compare the explanatory power of these models we also compare the
${\text{R}^{2}}$ values of these models. Among these models the
${\text{R}^{2}}$ values ranged between $(0.75,0.86)$. The ${\text{R}^{2}}$ of
the RF model is $0.86$, which implies that $86\%$ of the variation of the
dependent variable has been explained by the available features in the model.
This value is the highest amongst these models.

Additionally, for the purpose of comparison between different models
performance the predicted prices of the first 20 properties obtained by these
models were compared with the actual prices. Figure 7 depicts the estimated
house prices obtained by these models versus the actual values. This graph
represents the RF price predictions are closer to the purchase prices than
other models’ predictions.

Figure 7: Comparison of the Performance of the HP, ANN, RF, and kNN Models.

Given the value of RMSE, MAPE, and ${\text{R}^{2}}$ and visualizing the actual
and predicted prices through differen models we realize that RF model
outperformed all other models in this housing market. ANN is the second best
performing method. HP model performed better than the kNN model.

Now we aim to learn what features have the biggest impact on house prices in
the city of Boulder, Colorado. We investigate the $22$ variables in Table 1
and 2 to determine which of them have a significant or dominant impact on the
house prices. We rely on RF model which is a popular method for feature
ranking. We measure the importance of a feature by calculating the increase in
the model’s prediction error after permuting the feature (see e.g., Breiman,
(2001) and Fisher et al., (2019)).

Figure 8: Feature Importance - Random Forest Model.

Figure 8 shows the importance of the variables in the trained RF model. The
increase in the values of the MSE, are depicted in this figure. The RF model
selects the $\text{Ln}(\text{Living Area})$ as the most important predictive
variable for the property prices in the city of Boulder. The Drive to CBD is
the second most important variable followed by Region and Age. The variable
Bath.Half (the half of bathroom) is the least important feature.

Figure 9: Marginal Effect of the Three Most Important Numerical Features on
the House Value from the Random Forest Model.

Figure 9 shows the marginal effect of the three most important features on the
property values in the city of Boulder, Colorado. All these variables clearly
show non-linear association with the target variable. We observe that the
feature $(\text{Living Area})$ has positive marginal effect on the property
values. As $(\text{Living Area})$ increase the house price increases at a
different rate. As the residential property distance to CBD increases the
house value decreases at a non-constant rate. Dwellings built in during the
last $50$ years have negative marginal effect on house prices. However, houses
with the age above $50$ years have positive marginal effect on property
values.

Since the marginal effects in Figure 8 depicted non-linear patterns, the
machine learning and deep learning techniques such as RF and ANN can be
considered as more accurate and efficient prediction methods for the real
estate market prices in the city of Boulder instead of hedonic price
regression model.

##  5 Conclusion

In this paper we implemented hedonic price regression model, deep learning and
machine learning techniques to develop property appraisal models in the city
of Boulder, Colorado. In this study we compared the predictive power of the HP
model with ANN, RF, and kNN models for the house values. We observed non-
linear association between the house features and house prices. As such, the
machine learning and deep learning techniques can be considered as more
accurate and efficient prediction methods for the real estate market prices in
the city of Boulder than hedonic price regression models.

We compared several performance measurements for the predictions of the house
prices in the unseen (test) data set. Considering the ${R^{2}}$, RMSE and MAPE
values for these approaches, it can be concluded that the RF outperforms the
other methods in this housing market. It seems the RF method captures the
complexity or non-linearity of the housing markets in the city of Boulder more
than other models.

In general, when there is small data set random forest is a good algorithm,
since it is a bagging of decision trees with bootstrap. Each decision tree is
fed with a sample of data with replacement, in this way even if the data set
is small there are bigger chances of making a good model. Moreover, RF can
handle categorical variables with many levels. In the case of ANN method, for
categorical variables with many levels the one-hot encoding creates a large
number of binary variables which lead to a larger number of estimated
parameters and this usually results in overfitting.

The most important aspect of the deep learning and machine learning approaches
are that, they learn directly from observed data and there is no need to
assume explicit functions between the inputs and output of the studies. They
have the ability to rearrange their structure by adjusting the values of
parameters and hyperparameters.

As dwelling price prediction involves many non-linear variables, in order to
uncover non-linear relationships between the property attributes and predict
the value of a house the deep learning and machine learning algorithms should
be considered. Deep learning and machine learning algorithms such as ANN and
RF are accurate and efficient prediction methods. They have however, a serious
limitation and that is they are not interpretable. In contrast, the
coefficients in the hedonic models are easily interpretable.

There are however, some methods such as Lime, Shap and Eli5 that make the deep
learning and machine learning algorithms interpretable. In future studies it
would be useful to use those methods. It would be also interesting to expand
our study and in addition to the micro factors such as neighborhood, location,
and structure characteristics consider macroeconomic variables such as
unemployment rate. A more accurate forecast of the real estate market prices
must exploit not only variables relating to characteristics of the market, but
also combine them with different information sources such as macroeconomic
factors.

## Acknowledgements

Thanks goes to Dr. Nicholas E. Flores888Department Chair of Economics,
Colorado University at Boulder for his comments.

## References

  * Adelman and Griliches, (1961) Adelman, I. and Griliches, Z. (1961).  On an index of quality change.  Journal of the American Statistical Association, 56(295):535–548.
  * Agatonovic-Kustrin and Beresford, (2000) Agatonovic-Kustrin, S. and Beresford, R. (2000).  Basic concepts of artificial neural network (ann) modeling and its application in pharmaceutical research.  Journal of pharmaceutical and biomedical analysis, 22(5):717–727.
  * Aha, (2013) Aha, D. W. (2013).  Lazy learning.  Springer Science & Business Media.
  * Ali et al., (2015) Ali, G., Bashir, M. K., Ali, H., et al. (2015).  Housing valuation of different towns using the hedonic model: A case of faisalabad city, pakistan.  Habitat International, 50:240–249.
  * Antipov and Pokryshevskaya, (2012) Antipov, E. A. and Pokryshevskaya, E. B. (2012).  Mass appraisal of residential apartments: An application of random forest for valuation and a cart-based approach for model diagnostics.  Expert Systems with Applications, 39(2):1772–1778.
  * Arvanitidis, (2014) Arvanitidis, P. A. (2014).  The economics of urban property markets: an institutional economics analysis.  Routledge.
  * Bengio et al., (2021) Bengio, Y., Lecun, Y., and Hinton, G. (2021).  Deep learning for ai.  Communications of the ACM, 64(7):58–65.
  * Bigus, (1996) Bigus, J. P. (1996).  Data mining with neural networks: solving business problems from application development to decision support.  McGraw-Hill, Inc.
  * Blomquist and Worley, (1981) Blomquist, G. and Worley, L. (1981).  Hedonic prices, demands for urban housing amenities, and benefit estimates.  Journal of Urban Economics, 9(2):212–221.
  * Breiman, (1996) Breiman, L. (1996).  Bagging predictors.  Machine learning, 24(2):123–140.
  * Breiman, (2001) Breiman, L. (2001).  Random forests.  Machine learning, 45(1):5–32.
  * Čeh et al., (2018) Čeh, M., Kilibarda, M., Lisec, A., and Bajat, B. (2018).  Estimating the performance of random forest versus multiple regression for predicting prices of the apartments.  ISPRS international journal of geo-information, 7(5):168.
  * Collins et al., (1998) Collins, D. L., Zijdenbos, A. P., Kollokian, V., Sled, J. G., Kabani, N. J., Holmes, C. J., and Evans, A. C. (1998).  Design and construction of a realistic digital brain phantom.  IEEE transactions on medical imaging, 17(3):463–468.
  * Court, (1939) Court, A. (1939).  Hedonic price indexes with automotive examples.
  * Curry et al., (2002) Curry, B., Morgan, P., and Silver, M. (2002).  Neural networks and non-linear statistical methods: an application to the modelling of price–quality relationships.  Computers & Operations Research, 29(8):951–969.
  * Del Giudice et al., (2017) Del Giudice, V., Manganelli, B., and De Paola, P. (2017).  Hedonic analysis of housing sales prices with semiparametric methods.  International Journal of Agricultural and Environmental Information Systems (IJAEIS), 8(2):65–77.
  * Devroye et al., (1994) Devroye, L., Gyorfi, L., Krzyzak, A., and Lugosi, G. (1994).  On the strong universal consistency of nearest neighbor regression function estimates.  The Annals of Statistics, pages 1371–1385.
  * Devroye and Wagner, (1982) Devroye, L. and Wagner, T. J. (1982).  8 nearest neighbor methods in discrimination.  Handbook of Statistics, 2:193–197.
  * Domeniconi et al., (2002) Domeniconi, C., Peng, J., and Gunopulos, D. (2002).  Locally adaptive metric nearest-neighbor classification.  IEEE transactions on pattern analysis and machine intelligence, 24(9):1281–1285.
  * Eckert et al., (1990) Eckert, J. K., Gloudemans, R. J., and Almy, R. R. (1990).  Property appraisal and assessment administration.  International Assn of Assessing Office.
  * Fan et al., (2018) Fan, C., Cui, Z., and Zhong, X. (2018).  House prices prediction with machine learning algorithms.  In Proceedings of the 2018 10th International Conference on Machine Learning and Computing, pages 6–10.
  * Fisher et al., (2019) Fisher, A., Rudin, C., and Dominici, F. (2019).  All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously.  J. Mach. Learn. Res., 20(177):1–81.
  * Fletcher et al., (2000) Fletcher, M., Gallimore, P., and Mangan, J. (2000).  Heteroscedasticity in hedonic house price models.  Journal of Property Research, 17(2):93–108.
  * Fleuret, (2020) Fleuret, F. (2020).  Deep learning 1.1. from neural networks to deep learning.
  * Frew and Jud, (2003) Frew, J. and Jud, G. (2003).  Estimating the value of apartment buildings.  Journal of Real Estate Research, 25(1):77–86.
  * Gatzlaff and Haurin, (1998) Gatzlaff, D. H. and Haurin, D. R. (1998).  Sample selection and biases in local house value indices.  Journal of Urban Economics, 43(2):199–222.
  * Geng et al., (2015) Geng, B., Bao, H., and Liang, Y. (2015).  A study of the effect of a high-speed rail station on spatial variations in housing price based on the hedonic model.  Habitat International, 49:333–339.
  * Haase et al., (2013) Haase, D., Piorr, A., Schwarz, N., Rickebusch, S., Kroll, F., van Delden, H., Zuin, A., Taylor, T., Boeri, M., Zasada, I., et al. (2013).  Tools for modelling and assessing peri-urban land use futures.  In Peri-urban futures: Scenarios and models for land use change in Europe, pages 69–88. Springer.
  * Haykin and Network, (2004) Haykin, S. and Network, N. (2004).  A comprehensive foundation.  Neural networks, 2(2004):41.
  * Heiat, (2002) Heiat, A. (2002).  Comparison of artificial neural network and regression models for estimating software development effort.  Information and software Technology, 44(15):911–922.
  * Hill, (2013) Hill, R. J. (2013).  Hedonic price indexes for residential housing: A survey, evaluation and taxonomy.  Journal of economic surveys, 27(5):879–914.
  * Ho, (1998) Ho, T. K. (1998).  The random subspace method for constructing decision forests.  IEEE transactions on pattern analysis and machine intelligence, 20(8):832–844.
  * Hong et al., (2020) Hong, J., Choi, H., and Kim, W.-s. (2020).  A house price valuation based on the random forest approach: the mass appraisal of residential property in south korea.  International Journal of Strategic Property Management, 24(3):140–152.
  * Hopfield, (1982) Hopfield, J. J. (1982).  Neural networks and physical systems with emergent collective computational abilities.  Proceedings of the national academy of sciences, 79(8):2554–2558.
  * Islam and Asami, (2009) Islam, K. S. and Asami, Y. (2009).  Housing market segmentation: A review.  Review of Urban & Regional Development Studies: Journal of the Applied Regional Science Conference, 21(2-3):93–109.
  * Jiang et al., (2014) Jiang, L., Phillips, P. C., and Yu, J. (2014).  A new hedonic regression for real estate prices applied to the singapore residential market.
  * Kassambara, (2018) Kassambara, A. (2018).  Machine learning essentials: Practical guide in R.  Sthda.
  * Kauko et al., (2002) Kauko, T., Hooimeijer, P., and Hakfoort, J. (2002).  Capturing housing market segmentation: An alternative approach based on neural network modelling.  Housing Studies, 17(6):875–894.
  * Kontrimas and Verikas, (2011) Kontrimas, V. and Verikas, A. (2011).  The mass appraisal of the real estate by computational intelligence.  Applied Soft Computing, 11(1):443–448.
  * Król, (2020) Król, A. (2020).  The Application of Hedonic Methods in Quality-Adjusted Price Indices.  Wydawnictwo Uniwersytetu Ekonomicznego we Wrocławiu.
  * Lancaster, (1966) Lancaster, K. J. (1966).  A new approach to consumer theory.  Journal of political economy, 74(2):132–157.
  * Larose and Larose, (2005) Larose, D. T. and Larose, C. D. (2005).  K-nearest neighbor algorithm.  Discovering knowledge in data: An introduction to data mining, 90\.
  * Larose and Larose, (2014) Larose, D. T. and Larose, C. D. (2014).  Discovering knowledge in data: an introduction to data mining, volume 4.  John Wiley & Sons.
  * LeCun et al., (1998) LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998).  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324.
  * Lenk et al., (1997) Lenk, M. M., Worzala, E. M., and Silva, A. (1997).  High-tech valuation: should artificial neural networks bypass the human valuer?  Journal of Property Valuation and Investment.
  * Levantesi and Piscopo, (2020) Levantesi, S. and Piscopo, G. (2020).  The importance of economic variables on london real estate market: A random forest approach.  Risks, 8(4):112.
  * Limsombunchai, (2004) Limsombunchai, V. (2004).  House price prediction: hedonic price model vs. artificial neural network.  In New Zealand agricultural and resource economics society conference, pages 25–26.
  * McMillan et al., (1980) McMillan, M. L., Reid, B. G., and Gillen, D. W. (1980).  An extension of the hedonic approach for estimating the value of quiet.  Land economics, 56(3):315–328.
  * Milon et al., (1984) Milon, J. W., Gressel, J., and Mulkey, D. (1984).  Hedonic amenity valuation and functional form specification.  Land Economics, 60(4):378–387.
  * Morano and Tajani, (2013) Morano, P. and Tajani, F. (2013).  Bare ownership evaluation. hedonic price model vs. artificial neural network.  International Journal of Business Intelligence and Data Mining, 8(4):340–362.
  * Pagourtzi et al., (2003) Pagourtzi, E., Assimakopoulos, V., Hatzichristos, T., and French, N. (2003).  Real estate appraisal: a review of valuation methods.  Journal of Property Investment & Finance.
  * Pai and Wang, (2020) Pai, P.-F. and Wang, W.-C. (2020).  Using machine learning models and actual transaction data for predicting real estate prices.  Applied Sciences, 10(17):5832.
  * Park and Bae, (2015) Park, B. and Bae, J. K. (2015).  Using machine learning algorithms for housing price prediction: The case of fairfax county, virginia housing data.  Expert systems with applications, 42(6):2928–2934.
  * Peterson and Flanagan, (2009) Peterson, S. and Flanagan, A. (2009).  Neural network hedonic pricing models in mass real estate appraisal.  Journal of real estate research, 31(2):147–164.
  * Poursaeed et al., (2018) Poursaeed, O., Matera, T., and Belongie, S. (2018).  Vision-based real estate price estimation.  Machine Vision and Applications, 29(4):667–676.
  * RColorBrewer and Liaw, (2018) RColorBrewer, S. and Liaw, M. A. (2018).  Package ‘randomforest’.  University of California, Berkeley: Berkeley, CA, USA.
  * Robinson, (1979) Robinson, R. (1979).  Housing economics and public policy.  Springer.
  * Rosen, (1974) Rosen, S. (1974).  Hedonic prices and implicit markets: product differentiation in pure competition.  Journal of political economy, 82(1):34–55.
  * Rosenblatt, (1958) Rosenblatt, F. (1958).  The perceptron: a probabilistic model for information storage and organization in the brain.  Psychological review, 65(6):386.
  * Selim, (2011) Selim, S. (2011).  Determinants of house prices in turkey: A hedonic regression model.  Doğuş Üniversitesi Dergisi, 9(1):65–76.
  * Steffensen, (2021) Steffensen, M. (2021).  Risks: Feature Papers 2020.  MDPI.
  * Stone, (1954) Stone, R. (1954).  The measurement of consumer behavior and expenditure in the united kingdom, 1920-1938.  Studies in the National Income and Expenditure of the United Kingdom, 1.
  * Strobl et al., (2008) Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., and Zeileis, A. (2008).  Conditional variable importance for random forests.  BMC bioinformatics, 9(1):1–11.
  * Vazirizade et al., (2017) Vazirizade, S. M., Nozhati, S., and Zadeh, M. A. (2017).  Seismic reliability assessment of structures using artificial neural network.  Journal of Building Engineering, 11:230–235.
  * Waugh, (1928) Waugh, F. V. (1928).  Quality factors influencing vegetable prices.  Journal of farm economics, 10(2):185–196.
  * Yao et al., (2018) Yao, Y., Zhang, J., Hong, Y., Liang, H., and He, J. (2018).  Mapping fine-scale urban housing prices by fusing remotely sensed imagery and social media data.  Transactions in GIS, 22(2):561–581.
  * Yazdani and Choroś, (2018) Yazdani, H. and Choroś, K. (2018).  Comparative analysis of accuracy of fuzzy clustering methods applied for image processing.  In International Conference on Multimedia and Network Information System, pages 89–98. Springer.
  * Yazdani, (2020) Yazdani, M. (2020).  Individuals’ Decision Making about Risk under Both Description and Experience.  PhD thesis, State University of New York at Buffalo.
  * Yazdani, (2021) Yazdani, M. (2021).  House price determinants and market segmentation in boulder, colorado: A hedonic price approach.  arXiv preprint arXiv:2108.02442.
  * Yegnanarayana, (2009) Yegnanarayana, B. (2009).  Artificial neural networks.  PHI Learning Pvt. Ltd.
  * Yoo et al., (2012) Yoo, S., Im, J., and Wagner, J. E. (2012).  Variable selection for hedonic model using machine learning approaches: A case study in onondaga county, ny.  Landscape and Urban Planning, 107(3):293–306.
  * Yu and Wu, (2006) Yu, D. and Wu, C. (2006).  Incorporating remote sensing information in modeling house values.  Photogrammetric Engineering & Remote Sensing, 72(2):129–138.

##  Appendix A Appendix

Table A.6: Description of Models’ Parameter Tuning.

Models Parameter Tuning on average Reference ANN Units: 192 Grid search
Activation: RELU Grid search Layer Dense: 2 Grid search Kernel Regularizer:
regularizer_l2 (0.001) Grid search Epochs: 100000 Grid search Batch Size: Grid
search Learning Rate: 0.001 Grid search Optimizer: Adam optimizer Loss
Function: MSE RF Ntree:250 Grid search Mtry:7 Grid search kNN K:7 Grid search