INTRODUCTION
The authors introduce hybrid free-space optical and RF wireless links as potential technology for designing next-generation broadband wireless networks.
There is a surge in demand for high-data-rate connectivity among users in metropolitan area networks. This high bandwidth demand is driven by the increasing commercial use of the Internet, private intranets, videoconferencing, and voice over IP. Providing high-speed connections to the fiber optics backbone (last mile problem) is the key challenge in realizing this objective. Wireless technologies appear to be far more attractive than wired technologies due to low deployment costs, fewer regulatory restrictions, and configurable network topology. However, classical radio frequency (RF) bandwidth is limited and cannot fully utilize the high bandwidth offered by the fiber optics backbone. Free space optical wireless (FSOW), on the other hand, is one promising line-of-sight, high-speed, secure wireless technology that can facilitate the realization of next-generation carrier-grade high-reliability backbone and last-mile network access in an inexpensive and timely manner. For instance, the broadband wireless backbone architecture, based on FSOW links, has recently been introduced in the literature [1] [2] [3]. The multigigabit-per-second data carrying capacity of FSOW systems render them the most economically viable solution for point-to-point and point-to-multipoint deployments. Rapid deployment is another important advantage of FSOW systems that facilitates the realization of high-speed mesh-like networks in urban and semi-urban environments. However, the question that arises next is whether the FSOW technology is mature enough to guarantee high-grade link and network availability figures. Unfortunately, there has been much practical evidence in the literature that the performance of FSOW links severely deteriorates under adverse weather conditions [4] [5] [6]. Despite the fact that excessive scattering due to dust particles, heavy rain, or snow may possibly degrade the quality of FSOW links, moderate to dense fog conditions turn out to be the most challenging factor. This is attributed to the fact that tiny fog particles scatter and distort the FSOW signal in conjunction with absorption of the radiated energy significantly, which causes atmospheric attenuation to reach high levels resulting in link failure. In some scenarios, the overall signal attenuation can be as high as 300 dB/km. Experimental results have recently shown that optical wireless links alone cannot achieve required availability figures over long distances and high data rates [4], [5]. Natural movement of buildings is another challenge. The FSOW beams tend to lose alignment due to the relative movement of buildings [7] and therefore may lead to link outages. This problem can be taken care of via:
- Slightly increasing the footprint of the coverage
- Employing auto-beamtracking capabilities in the system
In addition to adverse weather conditions, random air turbulence due to temperature variation in the atmospheric layers can also affect the performance in some cases, but not significantly.
Several methods have been introduced for improving the FSOW link availability. The following two technologies are among those proposed:
- Providing hybrid link protection [4] [5] [6] via an RF (micro-/millimeter-wave) link complementary to FSOW with respect to weather sensitivity
- Scaling down the distance between transmitter-receiver pairs via multihop routing [3], [5], [6]
Our main contribution in this article is to introduce novel architecture and networking algorithms that efficiently exploit the advantages of both technologies. To this end, we first introduce a link monitoring scheme that provides real-time characterization of the link performance under various weather conditions. We introduce a simple yet efficient algorithm for dynamically dcciding the link status based on measured bit error rate (BER) data. Next, we propose a dynamic load switching (DLS) algorithm that partitions the traffic between the two types of links depending on weather conditions, desired link quality, and transmission delay constraints. We present experimental and simulation results that emphasize the significant role DLS plays in utilizing the hybrid architecture and, hence, improving the quality of service (00s). Finally, we shed some light on the trade-off between multihop path availability and delay. This trade-off is attributed to the fact that going over a multihop path, consisting of short-length links, is expected to achieve higher availability than going from source to destination over a long single hop. On the other hand, sending packets over large number of hops increases the end-to-end delay, which in turn leads to QoS degradation for delaysensitive applications. In addition, we provide insights about the relationship between topology discovery and routing criteria on one hand and link availability on the other.
The article is organized as follows. We introduce the hybrid wireless architecture. We devote a section to proposing novel FSOW link mvnitoring and restoring techniques. Afterword, the hybrid wireless testbed and experimrntal results are presnted. Finally, conclustion are drawn.
HYBRID WIRELESS ARCHITECTURE
In view of the aforementioned challenges, the current trend is to adopt novel hybrid architectures that exploit the complementary nature of FSOW and RF links with respect to their individual weather sensitivities. As mentioned above, FSOW links arc highly susceptible to dense fog, mist, and dust particles, hut relatively less vulnerable to rain conditions. On the contrary, the performance of RF systems degrades significantly during rain events (especially at frequencies above 10 GHz), but they are less susceptible to dense fog particulates. The hybrid architecture combines both technologies to improve the vverall wireless link andlor network availability. Figures 1 and 2 depict several application scenarios for protected high-availability access and distribution networks based on the RFIFSOW hybrid concept. These scenarios include protection for single and multichannel multiservice links, broadband network extension, wavelength-division multiplexed (WDM) synchronous optical network (SONET) ring protection, and terrestrialiairhorncisatelite link protection architectures.
The FSOW and RF subsystems are placed parallel to each other for 1:l protection. Currcnt state-of-the-art FSOW systems achieve data rates on the order of multiple gigabits per second using single or multiple WDM channels. On the other hand, RF systems operating in the 5.4 GHz, 28 GHz, or 38 GHz licensed frequency bands can sustain up to OC-24 (1200 Mbls) data rates and 2.5 Chis with polarization diversity. During clear sky conditions, RF channels can be used to augment the overall data capacity of FSOW channels to take over a portion or all of the traffic when FSOW cxperiences total outage. System-relatcd concepts and implcmentation details are prcsented in later sections.
ADAPTIVE OPTICAL WIRELESS LINKS
In this section wc investigate candidate techniques for maintaining the quality of FSOW links at an acceptable level under severe weather conditions. The system must not only be able to adapt to changing weather conditions and maintain sustained cud-to-cnd connectivity, but also to meet the desired level of QoS.
FSOW LINK MONITOR
Thc development of adaptive FSOW links requires implementation of a performance monitoring subsystem that can probe and accurately characterize the performance of the wireless channel in real timc. Essentially, the data recorded by the monitoring subsystem plays a crucial role in governing the dynamic link restoration techniques and, consequently, the link performance metrics. In this section we discuss challenges and proposed techniques to characterize the dynamic behavior of FSOW links in response to time-valying weather conditions. To this end, we go through two phases. First, we introduce candidate metrics for judging whether the performance of an FSOW link is satisfactory or not. Second, we introduce a procedure for mapping the gathered measurements to QoS paramcters.
Performance Mefricr
The performance of wireless links is normally characterized by the signal detection threshold which is defined as a lowcr bound on the signal-lo-noise-ratio (SNR) (or equivalently an upper bound on the BER) at the rcceiver [8]. Therefore, the BER threshold is normally adopted as the QoS parameter for wireless links. In this work, the real-time BER parameter is used to characterize the FSOW channel performance and initiate link restoration techniques whenever adverse weather conditions arise. However, although instantaneous (real-time) BER measurements may be good indicators of the performance of the wireless channel at a particular instant of time, they may not accurately reflect the cause of link deterioration. For instance, link deterioration due to temporary line-of-sight obstructions, which are unavoidable, may erroneously activate expensive link restoration techniques. Thus, the software supporting the adaptive feature of FSOW links must effectively prevent such momentary events and small-scale weather fluctuations from initiating link restoration. This challenge is discussed in more detail in the next section. Furthermore, the FSOW system may utilize multiple environmental and system parameters to define a comprehensive and effective performance metric. For example, received signal strength (RSS), humidity, and temperature, can be recorded and used individually or cooperatively in the decision making process. It is shown later that, unlike optical fibers, the RSS performance parameter alone is not an accurate indicator of FSOW link quality. Average BER turns out to be more accurate in determining FSOW link status.
Sliding Window Averaging
The objective of the proposed algorithm is to eliminate the impact of temporary line-of-site obstructions, since they may unnecessarily activate expensive link restoration algorithms. Accordingly, we propose to smooth out the sudden fluctuations in the measured BER data via a sliding window averaging mechanism.
Given the instantaneous BER measurements of an FSOW link recorded every minute over a period of several hours, we noticed that the measured BER values may oscillate between low and highly unacceptable values over a short period of time. The objective of the proposed algorithm is to eliminate the impact of temporary line-of-site obstructions, since they may unnecessarily activate expensive link restoration algorithms. Accordingly, we propose to smooth out the sudden fluctuations in the measured BER data via a sliding window averaging mechanism. Two parameters need to be specified for this algorithm: window.shape and window length (W) selected according to application required QoS. For simplicity, we employ a rectangular window where all samples are given equal weight in the averaging operation. On the other hand, we examine a wide range of window lengths, W = 5?100 min. It is worth noting that the optimum window length, which strikes a balance between filtering out a high rate of temporary link state changes and maintaining necessary information, depends solely on the rate of weather changes and intermittent BER variations. According to the proposed scheme, the system monitors the BER every single minute. However, the link restoration decisions are based on the averaged BER and recur only once in an interval greater than or equal to $W$ min.
From Fig. 3, it is straightforward to notice that short window lengths ($W<5$ min) suffer the problem of high-rate BER changes caused by temporary sudden fluctuations. On the other hand, large window lengths ($W=100$ min) filter out important information and lead to inaccurate representation of the link status (link outage intervals become much longer than reality). Finally, intermediate window lengths ($W=5-20$ min) provide the best balance between filtering out high rates of sudden state changes and preserving necessary information.
FSOW LINKRESTORATION
In this section we investigate candidate techniques for FSOW link restoration. To this end, we go through two phases. First, we examine the known techniques of dynamic power and rate control. The transmission power can be increased to mitigate atmospheric attenuation during high attenuation conditions. This can be implemented such that the transmission power increases incrementally in moderate steps to ensure acceptable link quality and energy efficiency. However, boosting the transmitted power is constrained by maximum power, eye safety standards, and health considerations. An alternative approach is to dynamically adapt the data rate of the FSOW link in response to weather conditions. During adverse conditions, the data rates can be reduced in predefined steps to improve/maintain the link QoS. For example, FSOW data rates can be reduced from an optical channel rate of OC-48 to OC-24 and further down to OC-12 with successive increase in the permissible atmospheric attenuation levels, as shown by the FSOW link budget equation in [4], [6]. Second, we examine two novel techniques for FSOW availability enhancement at the link and network levels: dynamic load switching and multihop routing.
Dynamic Load Switching
The hybrid architecture must be designed such that in the event of adversities, it automatically regulates all RF links into a standby mode followed by automatic switching of data traffic from FSOW to RF as soon as the FSOW link experiences outage. This can easily be achieved using requisite hardware and appropriate software. The system hardware can be configured to monitor the performance of the FSOW link in real time. On the other hand, the system software can be customized to map the measurements to QoS parameters such as average BER or average RSS, as described in the previous section. These metrics can then be used to make necessary link restoration decisions depending on the desired QoS.
In this section we present a novel algorithm for utilizing "all-weather" hybrid RF/FSOW wireless links. It partitions the load in the form of incremental load shifting from one link to the other as atmospheric attenuation becomes significant on one of them. The size of these increments certainly affects link utilization and availability; finer increments yield more efficient link utilization and higher availability at the expense of circuit complexity. On the other hand, larger increments simplify circuit design and operation at the expense of poor link utilization. Therefore, the increment size should be chosen carefully in order to balance the aforementioned trade-off. In the simulation study discussed later, we chose the increment size to be $R/4$, where $R$ is the maximum FSOW link bit rate. Finally, it is worth noting that the DLS algorithm is expected to be activated periodically at a rate that can be adapted depending on the rate of weather changes. It should not be activated too frequently to avoid processing overhead, and at the same time we cannot tolerate link outage due to long periods between decision epochs.
It is worth noting that the DLS algorithm is expected to be activated periodically at a rate that can be adapted depending on the rate of weather changes. It should not be activated too frequently to avoid processing overhead, and at the same time we cannot tolerate link outage due to the long periods between decision epochs.
In Fig. 4, we illustrate the operation of the proposed algorithm. The FSOW link is configured as the primary link and the RF link(s) as a backup that should take over part (or all) of the load whenever necessary. Furthermore, we assume that for each FSOW link there is one (or a set) of RF backup link(s) able to accommodate the maximum transmission rate $R$ (i.e., full load) carried by the FSOW link if needed. For instance, for the OC-12 FSOW link supporting $R =622$ Mb/s rate, we would need four RF backup links each operating at $R/4=155$ Mb/s. Throughout this article, we refer to the set of RF backup links as one RF link in order to simplify notations. Thus, we assume that initially all traffic is transmitted over the FSOW link (i.e. the FSOW bit rate is $R_{\rm FSOW}=R$ and the RF bit rate $R_{\rm RF}=0$. As illustrated in the previous section, the system records BER on both links every minute. To eliminate unnecessary link state alterations, the DLS algorithm averages the actual atmospheric attenuation over a rectangular window of length $W$. Afterward, the algorithm computes the permissible atmospheric attenuation on both links, as illustrated in [5], [6]. The last step is to compare actual to permissible atmospheric attenuation for each link in order to identify the dynamic status of both links and partition the traffic accordingly. Load switching can be realized using a switch controlled by a feedback mechanism discussed in [2].
Multihop Routing
In this section we extend the single link availability study, discussed in the previous section, to multihop networks. More specifically, we focus on improving the availability of FSOW networks connecting several establishments in an urban environment. A mesh-like design that includes long links with RF protection [2] provides the desired redundancy and allows multihop routing over short-length links via the following techniques:
- Reducing the distance between the transmitter and the receiver
- Modifying the routing criteria in order to take link availability into consideration
It is evident that going from source to destination over multihop paths consisting of short-length links would achieve higher end-to-end availability than going from source to destination over a long single hop. This is attributed to the fact that shorter FSOW links are less prone to outage due to hefty built-in margins and thus can continue to function satisfactorily even when longer FSOW links experience outage. However, sending packets over a large number of hops would increase the end-to-end delay, which in turn leads to QoS degradation for real-time applications. Therefore, there is a fundamental trade-off between end-to-end path availability and end-to-end path delay. Our main objective in this section is to shed some light on this tradeoff and point out guidelines for striking a balance between availability and delays depending on the traffic type and QoS constraints.
Shortest path (SP) routing is widely used in wired as well as multihop wireless networks [9]. It relays packets through the shortest path, where the link metric could be physical length, transmission delay, or load, depending on the QoS parameter of interest. For optical wireless networks, we introduce a new criterion for routing information: link availability. More specifically, we focus on how to make packet routing decisions under severe weather conditions where many links suffer outage. In addition, the routing algorithm should be designed to cope with temporary line-of-site obstructions causing intermittent failure of the optical wireless links. Thus, the impact of FSOW link availability on the following aspects of the routing problem are investigated.
Topology Discovery
The objective of this phase is to define, under adverse weather conditions, the network connectivity (topology). This depends solely on the status of various links and the transmission parameters associated with active links (bit rate, power, etc.). To this end, the topology discovery algorithm goes through three phases in an attempt to restore broken links. First, it boosts the transmitted power to the maximum value that does not violate eye safety standards or cause potential health risks. If this fails to restore the link, the algorithm attempts to reduce the link bit rate since losing a connection completely is certainly much worse than communicating at a lower speed for a short period of time. As the last resort, the DLS algorithm is activated to restore the disrupted link. The output of this phase includes the following information:
- State of each link
- Link transmission parameters (bit rate and power)
- Atmospheric attenuation over each link
This information is to be fed to the routing algorithm in order to make the best decision depending on the QoS requirements and weather conditions.
Routing Criteria
Given the set of available links along with their bit rates and link qualities (measured by the atmospheric attenuation), we attempt to answer the question of how to route a packet from source to destination. As pointed out earlier, the speed of wireless links can be traded for their quality; that is, the bit rate of an optical wireless link can be reduced in order to increase permissible atmospheric attenuation and hence improve its quality [2]. Under the proposed scheme, the network is expected to consist of a number of heterogeneous links ranging from high-speed vulnerable links to low-speed reliable links. A major challenge is how to route information packets efficiently in this heterogeneous environment. For example, loss-tolerant real-time traffic should be transferred over high-speed paths, while delay-tolerant data traffic should be transferred over reliable paths. Accordingly, the proposed routing algorithm plugs the link quality criterion, along with the classical link delay and link load criteria, into the link metric depending on the QoS requirements. More specifically, for real-time traffic more weight should be given to minimizing link delays, while for data traffic more weight should be given to minimizing link attenuation. In addition, the interaction between the DLS decision and the routing decision should be examined. For instance, link failures might be tolerated if there is an alternate route, via other available links, that leads from source to destination. More precisely, the problem of path availability could be solved via either DLS, alternate path routing, or both techniques.
PERFORMANCE EVALUATION AND DISCUSSION
In this section we present performance evaluation results of an experimental hybrid testbed. In addition, we discuss simulation results of the DLS algorithm.
FIELD EXPERIMENTAL SETUP
Figure 5 shows a schematic diagram of the field experimental testbed shown in Fig. 1b. A general-purpose computer is used to control the wireless channel monitoring subsystem, traffic switch, BER tester, and link data rate controller. The FSOW transceivers are capable of carrying up to OC-12 data rates (622 Mb/s). The multichannel RF transceivers operate at 38 GHz and are capable of carrying OC-3 data rates on each channel. During clear sky conditions, the FSOW and RF subsystems can provide BER performance on the order of 10?13 and 10?12, respectively, for the ~500 m link length. A pair of optical fibers connects the FSOW transceivers to the BER test (BERT) control panel in the laboratory network center.
As part of the hybrid testbed, two wireless channel monitoring subsystems are developed and compared:
- Real-time RSS
- Real-time BER
The RSS-based system involves monitoring and recording real-time RSS values at the receiver end of the system. The experimental results from the RSS- and BER-based monitoring subsystems are compared to ascertain their credibility in judging the time-varying channel conditions. Figures 6a and b present typical BER/RSS measured performance results for comparison.
The instantaneous RSS and BER values were recorded over 24- and 12-hour periods. From Fig. 6a, it is easily noticed that there is a strong correlation between the real-time recorded values of RSS and BER. During the first six hours, a relatively low RSS level translates very well to the corresponding high BER values. In the next four hours, as the RSS tends to improve slightly, a corresponding slight improvement in the BER values can also be noticed. Finally, during the last two hours (dense fog event), the correlation between extremely low RSS and high BER values can be noticed again. On the other hand, Fig. 6b shows a distinct lack of correlation between the RSS and BER values due to fog scattering and other atmospheric effects. During the first few hours, the RSS level is fairly constant or fluctuates in a very narrow range. Based on the observations from Fig. 6a, one would expect the BER statistics to show a similar trend (i.e., stay fairly constant or fluctuate slightly around the mean value). On the contrary, for the first hour the BER remains high and then abruptly drops to zero (error-free performance). The error-free performance continues for a few hours, during which the RSS values increase significantly. Again, one would expect continued error-free performance since higher power is being received at the FSOW receiver [2]. However, the BER fluctuates significantly despite the consistently high RSS levels during that interval. Thus, we conclude that the correlation between real-time recorded RSS and BER values may vary over time for FSOW links. RSS-based monitoring systems are more favorable due to ease of implementation and integration into the hybrid architecture than other possible schemes. However, unlike optical fiber transmission, RSS is not an accurate indicator of FSOW link performance, and hence cannot be considered a reliable criterion for activating link restoration schemes.
EXPERIMENTAL AND SIMULATION RESULTS
The lack of consistency between real-time RSS values and FSOW system performance necessitated the development of a BER-based monitoring subsystem. The real-time BER statistics measured by the BERT data controller were recorded and processed by a computer running a comprehensive connectivity management and link restoration software algorithm. A rectangular window size of 5 min was chosen for the calculation of the desired performance metric.
The dynamic rate control scheme for link restoration has been examined at a fixed BER threshold level of 10?7 [2]. However, the FSOW link performance deteriorates significantly during moderate to heavy fog and mist conditions. Despite the fact that power and rate control can restore the FSOW link and mitigate moderate performance deteriorations, the traffic must be dynamically switched from FSOW to RF during extreme weather cases. An automatic feedback-controlled optical switch is used to carry out the traffic partitioning task. The switching decision is based on the average BER calculated from realtime BER values recorded by the link monitoring subsystem. During this experiment, the QoS threshold was preset to 10?5 under the assumption that channel coding can easily improve the QoS performance to less than 10?9. The experimental behavior of the hybrid architecture is shown in Fig. 7.
It is evident from the figure that the performance of FSOW link starts deteriorating due to the gradual buildup of fog. The traffic continues to flow through the FSOW link despite performance degradation until the threshold of 10?5 is breached. As soon as the predefined QoS threshold level is reached, the data traffic is promptly switched over to the RF link (complete switchover). The control signal and corresponding event A indicate this transition. Error-free data transmission during event A in the figure clearly shows that RF link is not affected by fog. Furthermore, note that as fog fades away, the performance of the FSOW link improves and the link is automatically restored. During this event (between A and B), although the RF link has better QoS, the data traffic is switched back to FSOW to facilitate the possibility of data transmission at higher rates and capacity. During event B, when dense fog rolls in again, the data traffic is again routed through the RF for a few minutes before automatic switch back to FSOW. Thus, Fig. 7 successfully demonstrates the effectiveness of load switching on top of the hybrid architecture. Next, we demonstrate simulation results that reflect the behavior of incremental load switching. Figure 8 shows the dynamic behavior of the actual and permissible atmospheric attenuation measured over time. The permissible atmospheric attenuation of FSOW at maximum bit rate (622 Mb/s) is given by 43.5 dB for the simulated link parameters [6]. The role of DLS appears at the 13th hour, where the FSOW experiences outage. Thus, the transmission rate over the FSOW link is reduced to 3R/4 via switching a quarter of the load to the RF link. This in turn increases the permissible attenuation of the FSOW to 44.749 dB and hence restores it. When weather conditions improve and the attenuation over the FSOW link shrinks, the full load is switched back to the FSOW in order to resume operation at maximum rate. Finally, it is worth noting that the advantages of partial load switching compared to complete switchover prevail under practical scenarios where the FSOW bit rate is greater than the RF bit rate. This is attributed to the fact that when the FSOW link fails, partial load switching utilizes both links in order to support the FSOW bit rate. On the other hand, complete switchover would utilize only the RF link under a reduced bit rate.
Propagation measurements to evaluate the performance of FSOW, alone and as a part of the hybrid architecture, have been consistently performed at our research facility. Table 1 presents some of the consolidated performance statistics recorded over a period of four months.
The data in the available hours column represent the number of hours during which the FSOW link operated within the desired QoS requirements irrespective of prevailing weather conditions. The outage hours statistics indicate the number of hours for which the traffic was routed through the RF link when the performance of the FSOW link deteriorated below acceptable QoS levels (BER < 10?6). It can be clearly seen that the hybrid architecture significantly improves the overall link availability statistics from 68.4, 88.2, 96.5, and 89.1 percent to as high as 99.998 percent.
CONCLUSIONS
In this article we present design challenges and potential solutions for carrier-grade availability over FSOW links during adverse weather conditions. We demonstrate a link monitoring scheme that accurately reflects the performance of optical wireless links under various weather conditions. We noticed that, unlike optical fibers, RSS alone does not provide accurate characterization of FSOW link performance. Therefore, we relied on both the RSSI and average BER measurements as the criteria for link adaptation. In addition, we introduced two novel link restoration schemes that efficiently utilize the hybrid architecture:
- Dynamic load switching
- Multihop routing
The extensive long term results, recorded from the experiments during extreme weather conditions, validate the hybrid architecture concept and prove the significant DLS role in achieving sustained high-speed connectivity.
We notice that DLS improves the FSOW link availability considerably. Finally, we present an elaborate field testbed based on the hybrid architecture and various link adaptation techniques. Extensive long-term results, recorded from the experiments during extreme weather conditions, validate the hybrid architecture concept and prove the significant role of DLS in achieving sustained high-speed connectivity.
Acknowledgment
The authors would like to acknowledge the partial support of the DARPA NGI Program for this project.


Managing microservices with the "batteries included"
Distributed systems are required for microservices and they can benefit a business greatly given their potential for increased availability. However writing distributed systems is hard and managing them is even harder. When you can no longer name your machines or count them on one hand then you need something to help you.
ConductR is a "batteries included" approach to managing distributed systems. No more cobbling together of service gateways, service locators, consolidated logging, monitoring and so forth. All of these essential items and more are included with ConductR. In fact we want ConductR to be to operations what Play and Lagom are to developers; we want operations to be productive so that they can concentrate on keeping their business customers happy.
ConductR is also the easiest way to manage Akka cluster based applications, and laser-focused on ensuring that Lightbend Reactive Platform applications can be managed with ease. ConductR guides you regarding distributed concerns when embarking on developing a project; not as an after-thought when you near that release deadline.
Please refer to the getting started page if you simply want to download ConductR and have it run on your local machine for non-production purposes. Otherwise head on over to the documentation in order to find out all that you need to know.
Welcome to ConductR,
The Global ConductR Team


ABOUT
HOW IT WORKS
CONTACT
Riskars analyzes big data to visualize
your health risks through user friendly
and smart platforms
Select location or just manuver through map
Check various filters and health issues
See details of health statistics and graphs
Name:
Email:
Message:
MD, PhD Candidate
BSE MIT
Emory University
School of Medicine
MD, PhD
Associate Professor
Emory University
School of Medicine
MD, PhD Candidate
BS MIT
Emory University
School of Medicine
BSN Candidate
Emory University
School of Nursing


Official Statistics
Number of entries by age group and subject for GCSEs, level 1 / 2 certificates, AS and A levels in England for the summer 2015 exam series.
Ref: Ofqual/15/5690 PDF , 469KB , 14 pages
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
MS Excel Spreadsheet , 84.5KB
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
MS Word Document , 12.5KB
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
the overall entry for GCSEs in England has decreased from 2014 to 2015 from 5,085,000 to 4,916,000
the overall entry for students in year 10 and below for GCSE has decreased by 32%, from 504,000 to 344,000
one notable exception is computing where the entry increased by 83%
the year 11 entry in most EBacc GCSE subjects has decreased, ranging from 3% to 14%, whereas there were some significant increases in equivalent subjects for level 1 / 2 certificates (commonly known as IGCSEs), ranging from 78% to 447%, but overall entries for the EBacc subjects has increased
overall entries for AS in England have decreased by 1%, while overall A level entries have increased by 1%
computing and Spanish both saw large increases at AS and A level
subjects with large decreases at both AS and A level are critical thinking, general studies and ICT
Ofqual is running a rolling series of online surveys to make sure its statistical releases meet your needs.
Ofqual would like to invite you to take part in the online survey for this release
It will take about ten minutes to complete and your responses will remain entirely confidential in any reports published about the survey.
The survey can be found here
Published: 22 May 2015
Updated: 27 May 2015
From: Ofqual


Sans-Serif Experience Design Made Simple
for living design systems
Sans-Serif Experience Design Made Simple
for living design systems
Monospace Experience Design Made Simple
for living design systems
Cursive Experience Design Made Simple
for living design systems
Arial Experience Design Made Simple
for living design systems
Arial Black Experience Design Made Simple
for living design systems
Athelas Experience Design Made Simple
for living design systems
Avenir Experience Design Made Simple
for living design systems
Baskerville Experience Design Made Simple
for living design systems
Bodoni Experience Design Made Simple
for living design systems
Calisto Experience Design Made Simple
for living design systems
Consolas Experience Design Made Simple
for living design systems
Courier Experience Design Made Simple
for living design systems
Courier New Experience Design Made Simple
for living design systems
Didot Experience Design Made Simple
for living design systems
Garamond Experience Design Made Simple
for living design systems
Georgia Experience Design Made Simple
for living design systems
Gotham Experience Design Made Simple
for living design systems
Helvetica Experience Design Made Simple
for living design systems
Helvetica Thin Experience Design Made Simple
for living design systems
Menlo Experience Design Made Simple
for living design systems
Monaco Experience Design Made Simple
for living design systems
Times Experience Design Made Simple
for living design systems
Times New Experience Design Made Simple
for living design systems
Native Experience Design Made Simple
for living design systems
Native Bootstrap Experience Design Made Simple
for living design systems
Native Tachyons Experience Design Made Simple
for living design systems


By Fingerprinting, you can get the UNIQUE fingerprint of your browser or computer. It based on the basic browser information, GPU, fonts and so on
We are members of a security lab (SecLab) in Lehigh University
Leader of SecLab
Website Developer and Maintainer
Report bug to me (lisong1013@gmail.com)


MAP SKETCHES
Trying some things to work with our datasets, locations, industry sectors to provide users with clear information to help make investment decisions.
These are sample plots from the existing alpha, background map removed. I think this informed the following abstraction of the map...
These are the things we probably need to plot on a map (or provide in some way):
- Companies
- Infrastructure + Projects
- Transport
- Universities
- Workforce + Skills
- Success stories
- Supply chains
- ...likely more
×
- Location
- Industry
- Sector
- Value
This is starting to roughly lay out what a stylised abstracted map would look like with text-label UI as a priority. Note, the top (busiest? most investment?) areas are highlighted, the overlay data are not present.
I tried it, but at this point the added data layers (eg. transport) made it look really messy...
Based on the type of data we have - lots of locations in some of the sector/region intersections - I tried to get a list out of all of them. Here, trying to make the layout work, the list is quite long, and not immediately useful.
Hover effects (drawing a line connecting the place to label) may work.
In parallel to map-centric UI it is becoming clear that might not be the best way for users to get the most out of their visit to the INVEST site.
Here, I'm trying to look at a layout that is the equivilant for a region profile, with maps as part of the content but moving away from one map for all data.


---
layout: page
title: Programme
---
{% assign latest_programme = site.programmes | last %}
Programme (all times in CET)
Monday - Nov 9
| 9:30 |
Opening the Remo |
| 10:00 - 10:10 |
Welcome words and introductions for the week |
| 10:10 - 10:20 |
Toshiaki Katayama: BiohackrXiv |
| 10:20 - 10:30 |
Leyla Garcia: covid-19 Biohackathon outcomes |
| 10:30 - 10:40 |
Spanish Node welcome |
| 10:40 - 10:50 |
Coffee Break |
| 10:50 - 11:50 |
Project leads 1-20 projects (3 mins each) |
| 11:50 - 12:00 |
Coffee Break |
| 12:00 - 13:00 |
Project leads 21-40 projects |
| 13:00 - 13:45 |
Lunch and group photo |
| 13:45 - 17:00 |
Hacking |
| 17:00 - 18:00 |
Speed Networking (open) |
Tuesday - Nov 10
| 8:30 - 9:00 |
Breakfast + Fitness class |
| 9:00 - 10:30 |
Hacking |
| 10:30 - 11:00 |
Break |
| 11:00 - 12:00 |
Hacking |
| 12:00 - 13:00 |
Lunch |
| 13:00 - 14:30 |
Project reporting (1 slide per project) |
| 14:30 - 15:00 |
Break |
| 15:00 - 17:00 |
Hacking |
| 17:00 - 18:00 |
General knowledge Quiz (Prizes) |
Wednesday - Nov 11
| 8:30 - 9:00 |
Breakfast + Fitness class |
| 9:00 - 10:30 |
Hacking |
| 10:30 - 11:00 |
Break |
| 11:00 - 12:00 |
Hacking |
| 12:00 - 13:00 |
Lunch |
| 13:00 - 14:30 |
Project reporting |
| 14:30 - 15:00 |
Break |
| 15:00 - 18:00 |
Hacking |
| 17:00 - 18:00 |
Speed Networking (open) |
Thursday - Nov 12
| 8:30 - 9:00 |
Breakfast + Fitness class |
| 9:00 - 10:30 |
Hacking |
| 10:30 - 11:00 |
Break |
| 11:00 - 12:00 |
Hacking |
| 12:00 - 13:00 |
Lunch |
| 13:00 - 14:30 |
Project reporting |
| 14:30 - 15:00 |
Break |
| 15:00 - 17:00 |
Hacking |
| 17:00 - 18:00 |
Photo gallery (Prizes) |
Friday - Nov 13
| 8:30 - 9:00 |
Breakfast + Fitness class |
| 9:00 - 10:30 |
Hacking |
| 10:30 - 11:00 |
BioHackrXiv |
| 11:00 - 12:00 |
Final reporting |
| 12:00 - 12:15 |
Break |
| 12:15 - 13:15 |
Final report session |
| 13:15 - 13:25 |
Pacer - Step Challenge Team winner (Prizes) |
| 13:25 - 13:35 |
Wrap-up |


TOMORROW NEVER DIES (United Artists) Starring: Pierce Brosnan, Jonathan Pryce, Michelle Yeoh, Teri Hatcher. Screenplay: Bruce Feirstein. Producers: Michael G. Wilson and Barbara Broccoli. Director: Roger Spottiswoode.  Reviewed by Scott Renshaw.
Michelle Yeoh is ultimately the best thing about TOMORROW NEVER DIES, the 18th installment in the phenomenally popular series based on Ian Fleming's James Bond (Pierce Brosnan). And in a strange way, she also points out its most glaring weaknesses. You see, Bond may be a brand name, but as I noted in my 1995 review of GOLDENEYE, the franchise no longer has the market cornered on high-flying, globe-hopping action. When the alternatives to the latest Bond adventure include watching the dazzling Jackie Chan battle international espionage with considerably more imagination, TOMORROW NEVER DIES starts to look rather bland and uninspired. The presence of Yeoh, Chan's capable sidekick in SUPERCOP, only serves as a reminder that there are new kids on the block. Brosnan may look appropriately dapper, but the series just looks old, cruising on its place in cinematic history.
The premise has a kind of perfunctory Bond-ian grandiosity about it, despite an extremely lackluster villain. Jonathan Pryce plays Elliot Carver, an international media mogul just about to launch a new satellite news network. Operating on the principle that it's easier to create news than to hunt it down, Carver and his operatives begin making their own headlines by instigating a conflict between the British navy and China, one which results in a downed British battleship and talk of war. Enter Bond, assigned to determine the dirty details of Carver's operation. Ably assisted by Chinese government agent Wai Lin (Yeoh), Bond must stop the presses before Carver's meddling results in all-out war between England and China.
The ho-hum megalomania of Pryce and his obligatory, inhumanly resilient henchman (Gotz Otto) are only part of an extremely familiar package which prevents TOMORROW NEVER DIES from ever getting off the ground. Token attempts at adding depth to Bond's romantic conquests (he _really_ cares about old flame Teri Hatcher...no, really, he does) fall flat, but that's no great surprise. The real disappointment is the quality of the action set pieces -- a few flashes of creativity chaotically directed by Roger Spottiswoode (AIR AMERICA, TURNER AND HOOCH) to lend the illusion of real energy. The gunplay is routine, the chases are routine, even Q's gadgets are routine (a car that drops little spikey things to flatten the tires of the car behind you...what will he think of next?). Even peppering the action with traditional Bond quips doesn't help, especially when most of the audience is baffled into silence by a reference to Carver's "edifice complex."
The only real change of pace is Yeoh, who provides the Bond series with its best, most exciting leading lady in years (with all due respect to Maud Adams and Grace Jones). She's such a match for Bond -- just as resourceful and even more physically capable -- that it's a shame to see her reduced to damsel-in-distress status. Watching Yeoh fly through an acrobatic martial arts fight sequence conjures images of the more inventive stunt wizardry of Jackie Chan's films. Chan has been out-Bonding Bond for years now, a detail it's hard to overlook while TOMORROW NEVER DIES is punching and exploding its way through two hours on automatic pilot.
Pierce Brosnan once again provides Bond with an appropriate ruthlessness, but he's also burdened with the anachronistic he-man antics. Much of the continued appeal of the series can be attributed to the fact that the more things change, the more James Bond remains the same -- in the liberated, safe-sex, designated driver 90s, he's still an unabashed chauvinist who sleeps around, drinks his vodka martinis and kicks ass for queen and country. That sense of familiarity may be the series' greatest financial strength, but it's also its greatest impediment to quality. Once upon a time, the James Bond series was a bi-annual festival of "can you top this?" The goal is simpler now; the producers aren't even trying to compete with the big-budget big boys. They rely on one secret weapon not provided by Q: nostalgia. TOMORROW NEVER DIES never once tries to play "can you top this?" It's a film content to offer "do you remember this?"
On the Renshaw scale of 0 to 10 Bond futures: 4.
Visit Scott Renshaw's MoviePage http://www.inconnect.com/~renshaw/ *** Subscribe to receive new reviews directly by email! See the MoviePage for details, or reply to this message with subject line "Subscribe".
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


NATURES PURITY SCENTED- petrolatum ointment
AMERICAN EAGLE STAR IMPORTS INC.
Disclaimer: Most OTC drugs are not reviewed and approved by FDA, however they may be marketed if they comply with applicable regulations and policies. FDA has not evaluated whether this product complies.
Active IngredientPetrolatum 99.9%
Uses:
- Temporarily protects minor cuts, scrapes or burns.
- Temporarily protects and helps relieve chapped or cracked skin and lips.
- Helps protect from the drying effects of wind and cold weather.
WARNINGS
For external use only.
- When using this product do not get into eyes.
Stop use and ask a doctor if condition worsens symptoms last more than 7 days or clear up and occur again within a few days
Do not use on deep or puncture wounds animal bites serious burns
Keep out of reach of children. If swallowed get medical help or contact a Poison Control Center right away.
Directions:
Apply to a clean and dry diaper area as often as necessary
Inactive ingredients: Fragrance
AMERICAN EAGLE STAR IMPORTS INC.


Woman opens shop selling black dolls as a way to cope with grief
by Faima BakarSanda Monero started making dolls as a form of therapy, during a time she was struggling.
After being inspired by the woman in her life, Sandra made black dolls with different shades and hairstyles.
What started as a hobby grew and grew and last year Sandra, from Stoke Newington, decided to open her own store called Monero Kids Boutique in Balls Pond Road.
As well as her ‘gratitude dolls’, Sandra, who has also worked in social care, also sells eco-friendly children’s clothing and educational books.
Sandra takes inspiration from her St Lucian heritage and upbringing in London and uses Caribbean and African fabrics such as Kente cloth in her work.
Currently, she has a range of dolls clothing in production with one for every Caribbean island.
Of course, there are thousands of islands, which would take forever to create so Sandra is sticking to approximately 26 dolls.
Her idea started when she lost her parents and her brother – something she still finds difficult to talk about – but she says that having the focus of creating doll clothes has helped to bring her happiness back.
She hopes to introduce two male dolls – one to represent her father and one for her older brother.
‘It is very important as [these dolls] represents little girls and boys’ Sandra told Metro.co.uk.
‘Growing up, I only knew about Barbie and Sindy and of course, I wanted to be like them.
‘But being a black child it was impossible as it did not represent me at all. I am so lucky, now, I am in a position to be able to share and showcase dolls for boys and girls that represent their colour tone.’
There are three main dolls in the Monero Doll collection – Honey, Ellie and Harley D.
One of the dolls, Ellie, is named after Sandra’s mum who she describes as a ‘beautiful, graceful woman’.
She adds: ‘Honey, is named after a friend of mine called Yinka who has supported and helped me through my challenges.
‘Harley D is named after the Harley Davidson motorcycle as I have a passion for bikes. In addition, her character is partly built around myself and the daughter I would have had.’
She recalls one of her favourite memories in at the Boutique when a little white girl called Hannah came in.
‘She came in and bought the darkest doll,’ said Sandra. ‘When asked why she had chosen that one she said: “Mummy, she looks like my best friend Yemi”.’
If you want to visit Sandra’s store, you can find it in Dalston Junction at 26a Balls Pond Road, London N1 4AU.


New problems added, HTTP API addition 2021-07-10 00:00:00+00:00
We have added 19 new problems to poses.live.
We would like to remind everyone that the Lightning Division ends in twelve hours on Sat, 10 July 12:00 (noon) UTC.
To keep things exciting, we will freeze the scoreboard temporarily on Sat, 10 July from 11:30 to 12:30 UTC. We've also limited the scoreboard to teams with at least one valid solution.
Additionally, we have updated the specification with a new API endpoint to obtain some information about submitted poses.


A free test data generator and API mocking tool - Mockaroo lets you create custom CSV, JSON, SQL, and Excel datasets to test and demo your software.
mock lat/long
smk762 - dracocanis ominator10-Oct-18 12:21 PM
I'll have to take the dog on some epic walks so I can instagramize it and aviod PETA telling me off on twitter for not walking him when I"m called out to a fire.
blackjok3r10-Oct-18 12:21 PM
gotta shoot for a bit, daucters 2nd bfday wife finally got home
1
1
CrisF10-Oct-18 12:24 PM
How it came about will be a good part of the story
then right click to make the margins wider and left click and drag(edited)
ctrl a/b shift + " to split down and shift + % to split across
ctrl +a/b c to make a new window
some on the vms on my server have like 10 windows each ful of panes
lol
right now i just using tmux and a bash script to amke the TROLL box... because its the fastest for me to do(edited)
and I think everyone here will be able to use it
blackjok3r10-Oct-18 01:43 PM
I am nearly finished TROLL box btw if anyone is still around.
Mylo10-Oct-18 01:44 PM
yes stick to bash, my makefile suggestion is good for some things, not all i've discovered today getting cc_gui stuff going for conventions for others to use.
blackjok3r10-Oct-18 01:45 PM
I wanted to make somethig like Tonyx TUI ... but the more I looked at it, I figured it was just better with a tmux, start.sh STAKEDTROLL on one side and a couple of scripts.
one to get faucet ...
whcih tbh is hell annoying...
I should have made faucet 1000 instead of 100.
if you somehow manyage to spend 100 coins .... you need to make a new config.ini to get more coins
but you can claim 3 times on first load
I think at some point I might need to make a script to create a config.ini... getnewaddress, validate it get pubkey and dumppriv
good thing seems to be you can use the same username and password from dirrent wallets.. so its not really an issue
I mena 100 coins is like millions of messages though I think
random assortment of small scripts for use in the Komodo ecosystem - smk762/kmd_pulp
1
I used it for dice and pool
blackjok3r10-Oct-18 02:10 PM
mad
that will come in handy, although 1 faucet get is 100,000 messages
currently the user needs an address already, it would be good to gen one then restart the daemon automatically
smk762 - dracocanis ominator10-Oct-18 02:12 PM
someehow the space in assetchains.json fro TROLL genproclimit broke my jq. Too fuzzy to fix it now tho
blackjok3r10-Oct-18 02:12 PM
dont want to go too far on a bash implemntation though... but I guess if people want to chat from terminal they can use this
yeah it might do that...
I assumed it was using listparams
I had to change listparams.py quite a lot with the new changes
your jq script ight not work any more
smk762 - dracocanis ominator10-Oct-18 02:13 PM
I was planning to nodejs some stuff, though might use something else.
bash is good for concept, but has its limit
I'll mod to listparams, keeps thinks standard
1
blackjok3r10-Oct-18 02:13 PM
yeah ... currently the whoe thing is a mix of python and bash, becasue noth have advantages ... I just use both for diff jobs... its not pretty but it works good
like there are a lot of files...
trying to work out how to get the ./send to worj without quotes atm ... I think I have an idea.
man its being a prick
it has quotes when i echo it, but trying to send to cli call its not working.
maybe I need to send with curl
blackjok3r10-Oct-18 02:30 PM
yeah send script should have been in python lol... bash one has gotten very messy fast
Mylo10-Oct-18 02:32 PM
go make a cup of tea, take 10 mins off and in 20 mins i'll let ya know if i've got interactive console for you to test
fark...bets are off. resync
blackjok3r10-Oct-18 02:42 PM
lol
Troll box is done though
ComputerGenie 👴👴👴10-Oct-18 02:44 PM
go make a cup of tea
awful bossy, ain't ya?
blackjok3r10-Oct-18 02:48 PM
lol at 11PM at night ...
genie you should use the tTROLLBOX to troll people LOL
I am getting mighty sick of taking to myself on it


class: center, middle, inverse, title-slide # Classification Problems ## Machine Learning in Economics ### Hüseyin Taştan ### Yildiz Technical University --- class: my-medium-font <style type="text/css"> .remark-slide-content { font-size: 28px; padding: 1em 4em 1em 4em; } .my-large-font { font-size: 40px; } .my-small-font { font-size: 20px; } .my-medium-font { font-size: 30px; } </style> # Lecture Outline - [Classification problems](#classprobs) - [Logistic regression](#logistic) - [Linear Discriminant Analysis (LDA)](#lda) - [Measuring classification performance](#classperformance) - [ROC curve and AUC](#roc) - [Quadratic Discriminant Analysis (QDA)](#qda) --- name: classprobs # Classification problems * In regression problems, the response variable `\(Y\)` is quantitative * If `\(Y\)` is categorical (qualitative) then we have classification problem * Classifier: a model that can be used to classify the category of `\(Y\)` given a set of `\(X\)` variables. * Example: a bank wants to estimate the probability of default for a credit applicant and then classify into default (reject) vs. no-default (accept) groups. * Data set: default (Yes/No), `\(X\)` variables: credit card balance (balance), income level of applicant (income), student dummy (student) --- # Classification: Example ![:scale 100%](img/default1.PNG) Blue: Default=NO, Orange: Default=YES (ISLR Fig-4.1, p.129) --- # Can we use linear regression? * Can we classify observations using the standard regression analysis? Suppose that there are two response categories: `$$default = \beta_0 + \beta_1 income + \beta_2 balance + \epsilon$$` where default has binary (0/1) values. * Classification rule: If the OLS prediction is larger than `\(0.5\)` then assign default=YES (1) group, otherwise assign default=NO (0) group. * But there is no guarantee that predicted values lie between 0 and 1. They can be any value, even negative or larger than one. * Also the error term will not have constant variance. And it cannot be applied to multiple categories. --- name: logistic # Logistic regression * Instead of directly modeling the response variable, we may try to model the classification probability, that is, `$$p(X) = \beta_0 + \beta_1 X$$` where `\(p(X)=Pr(Y=1|X)\)` is the probability of classifying the response variable into group =1, conditional on `\(X\)`. * By definition `\(0\leq p(X)\leq 1\)`. For example, the logistic function is defined as `$$p(X)=\frac{e^{\beta_{0}+\beta_{1} X}}{1+e^{\beta_{0}+\beta_{1} X}}$$` * The Logit model can be written as `\(\log \left(\frac{p(X)}{1-p(X)}\right)=\beta_{0}+\beta_{1} X\)` --- # Logistic Regression vs Linear Probability Model (LPM) ![:scale 110%](img/default2.PNG) **Left**: conditional probabilities from LPM, **Right**: conditional probabilities from logistic regression (ISLR Fig-4.2, p.131) --- # Estimation of Logistic Regression * We cannot use OLS to estimate the logistic regression * Maximum Likelihood estimation method gives consistent and efficient estimators. It finds the coefficients by maximizing the likelihood: `$$\ell\left(\beta_{0}, \beta_{1}\right)=\prod_{i: y_{i}=1} p\left(x_{i}\right) \prod_{i^{\prime}: y_{i^{\prime}}=0}\left(1-p\left(x_{i^{\prime}}\right)\right)$$` * After estimating the coefficients of the logistic regression we can compute conditional probabilities. * The classification is based on the conditional probabilities. --- # Multiple Logistic Regression * The model can easily be extended to include quantitative or qualitative predictors: `$$p(X)=\frac{e^{\beta_{0}+\beta_{1} X_1 + \ldots +\beta_{p} X_p}}{1+e^{\beta_{0}+\beta_{1} X_1 + \ldots +\beta_{p} X_p}}$$` * Because the model is nonlinear, the coefficients cannot be interpreted as changes in conditional probabilities. But we can interpret their signs. * After finding `\(\hat{\beta}_j\)`s we can calculate conditional probabilities by plugging `\(X_j\)` values in the equation above. --- name: lda # Linear Discriminant Analysis * Linear Discriminant Analysis (LDA) attempts to model the distribution of `\(X\)` variables for each category. * Then, using the Bayes Theorem we can estimate the conditional probability `\(Pr(Y=k|X=x)\)`. * If the classes are well-separated, the logistic regression can be unstable. LDA does not suffer from that problem. * Even if `\(n\)` is small, LDA is more stable if the `\(X\)` variables are approximatley normal. * Unlike the logistic regression, LDA can be applied to cases where there are more than two response categories. --- # Using the Bayes Theorem in Classification * We want to classify observations into `\(K\geq 2\)` groups. The response variable is `\(Y_i =k,~k=1,2,\ldots,K\)` (order is not important) * Prior = `\(\pi_k\)`, probability that randomly drawn observation comes from class `\(k\)` - The density of `\(X\)` for an observation that comes from the `\(k\)`th class: `\(f_k(x)\equiv Pr(X=x|Y=k)\)` (For simplicity assume `\(X\)` is discrete). `\(f_k(x)\)` takes larger values as the probability of `\(X\)` from class `\(k\)` increases. * Now we can apply the Bayes Theorem to obtain the posterior probability: `$$p_k(X)\equiv \operatorname{Pr}(Y=k \mid X=x)=\frac{\pi_{k} f_{k}(x)}{\sum_{l=1}^{K} \pi_{l} f_{l}(x)}$$` * Instead of directly estimating `\(p_k(X)\)`, we can estimate its components. --- # Using the Bayes Theorem in Classification Posterior probability function: `$$p_k(X)\equiv \operatorname{Pr}(Y=k \mid X=x)=\frac{\pi_{k} f_{k}(x)}{\sum_{l=1}^{K} \pi_{l} f_{l}(x)}$$` * `\(\pi_{k}\)` can easily be estimated from data. We just need to calculate the proportion of observations in class `\(k\)`. * But estimating `\(f_{k}(x)\)` is more difficult. We need to make some distributional assumptions. * LDA: assumes normal distribution. For example, for `\(p=1\)` `$$f_{k}(x)=\frac{1}{\sqrt{2 \pi} \sigma_{k}} \exp \left(-\frac{1}{2 \sigma_{k}^{2}}\left(x-\mu_{k}\right)^{2}\right)$$` --- # Normal Distribution .pull-left[ Different means, same variance ![](img/norm1.PNG) ] .pull-right[ Different variances, same mean ![](img/norm2.PNG) ] --- # LDA, conditional probabilities for p=1 * For a single predictor, `\(p=1\)`, and same variance across groups ( `\(\sigma^2\)` ) the normality implies that `$$p_{k}(x)=\frac{\pi_{k} \frac{1}{\sqrt{2 \pi} \sigma} \exp \left(-\frac{1}{2 \sigma^{2}}\left(x-\mu_{k}\right)^{2}\right)}{\sum_{l=1}^{K} \pi_{l} \frac{1}{\sqrt{2 \pi} \sigma} \exp \left(-\frac{1}{2 \sigma^{2}}\left(x-\mu_{l}\right)^{2}\right)}$$` * Bayes classifier assigns an observation `\(X=x\)` into the class for which `\(p_k(x)\)` is the largest. * Taking the natural logarithm of `\(p_k(x)\)`, we obtain the **discriminant** function: `$$\delta_{k}(x)=x \cdot \frac{\mu_{k}}{\sigma^{2}}-\frac{\mu_{k}^{2}}{2 \sigma^{2}}+\log \left(\pi_{k}\right)$$` * Observations can be assigned to groups for which `\(\delta_{k}(x)\)` is the largest. --- # Estimating LDA coefficients `$$\hat{\mu}_{k}=\frac{1}{n_{k}} \sum_{i: y_{i}=k} x_{i}$$` `$$\hat{\sigma}^{2}=\frac{1}{n-K} \sum_{k=1}^{K} \sum_{i: y_{i}=k}\left(x_{i}-\hat{\mu}_{k}\right)^{2}$$` `$$\hat{\pi}_k = n_k/n$$` Substituting these estimates into the discriminant function we obtain `$$\hat{\delta}_{k}(x)=x \cdot \frac{\hat{\mu}_{k}}{\hat{\sigma}^{2}}-\frac{\hat{\mu}_{k}^{2}}{2 \hat{\sigma}^{2}}+\log \left(\hat{\pi}_{k}\right)$$` --- # LDA: Example .center[![:scale 110%](img/lda1.PNG)] Note: (Left) vertical dashed line is the Bayes boundary (known because the data is simulated), (Right) vertical line is the LDA decision boundary. (ISLR, Fig. 4.4, p.140) --- # LDA with multiple predictors * Predictors: `\(\mathbf{X} = (X_1,X_2,\ldots,X_p)\)` * Distributional assumption (Multivariate Normal - or Gaussian - distribution): `\(\mathbf{X}\sim~N(\mathbf{\mu},~\mathbf{\Sigma})\)` ![:scale 90%](img/mnorm1.PNG) Left: `\(X_1\)` and `\(X_2\)` uncorrelated, Right: correlated with covariance = 0.7 --- # Multiple LDA Multivariate Gaussian density function: `$$f(x)=\frac{1}{(2 \pi)^{p / 2}|\mathbf{\Sigma}|^{1 / 2}} \exp \left(-\frac{1}{2}(x-\mu)^{T} \mathbf{\Sigma}^{-1}(x-\mu)\right)$$` Following similar steps to the case `\(p=1\)` we can obtain the discriminant function: `$$\delta_{k}(x)=x^{T} \mathbf{\Sigma}^{-1} \mu_{k}-\frac{1}{2} \mu_{k}^{T} \mathbf{\Sigma}^{-1} \mu_{k}+\log \pi_{k}$$` An observation is classified into the group for which `\(\delta_k(x)\)` is the largest. --- # Illustration .pull-left[ ![:scale 100%](img/mlda1.PNG) ] .pull-right[ * 2 predictors `\(X_1,X_2\)` * Number of groups is `\(K=3\)` * Number of observations in each group is 20 * Dashed lines are LDA decision boundaries * Smooth lines are Bayes boundaries * LDA error rate = 0.0770 * Bayes error rate = 0.0746 ] --- # Example: Default data .pull-left[ ![:scale 100%](img/defaultdata.PNG) ] .pull-right[ * n=10000 * Balance = credit card balance * income = credit applicant's income level * Student = dummy =1 for students * default = default status (Yes=1, No=0) ] --- # Example: Default data * LDA estimation with balance and student dummy gives a training error rate of 2.75%. This seems to be a very small error rate. * In the training data, the proportion of default=YES is 3.33%. If an observation is classified as default=NO regardless of balance and student status, the error rate will be 3.33% (this is called **null classifier**). * Now the LDA does not seem to be successful. --- class: my-small-font name: classperformance # Measuring Classification Performance Confusion Matrix .pull-left[ | | | Truth | Truth | | |--------|:------:|--------|-----|---------| | | | `\(-\)` | + | Total | | Prediction | `\(-\)` | A | B | A + B | | Prediction | + | C | D | C + D | | | Total | A + C | B + D | A + B + C + D | ] .pull-right[ * Number of observations in cells * A = Number of negatives predicted correctly * D = Number of positives predicted correctly * C = Number of false positives * B = Number of false negatives ] * False positive rate: `\(FP=C/(A+C)\)` * True positive rate (sensitivity): `\(TP=D/(B+D)\)` * True negative rate (specificity): `\(NP=A/(A+C)\)` * Positive Predictive value: PP = D/(C+D) * Negative predictive value: NP = A/(A+B) --- # Measuring Classification Performance Confusion Matrix .pull-left[ ![:scale 100%](img/confusion1.PNG) ] .pull-right[ * Default = YES (Positive group) * Default = NO (Negative group) ] * Number of obs. classified correctly = 9644+81=9725 with rate 97.25%. * Incorrectly classified number of obs. = 252+23=275. (2.75%) * The proportion that are classified incorrectly in default=YES group is 23/9667 = 0.00238, or 0.238%, a very small error. * But those who default are incorrectly classified into NO-default with the proportion of 75.7% (=252/333) --- # Measuring Classification Performance Confusion Matrix .pull-left[ ![:scale 100%](img/confusion1.PNG) ] .pull-right[ * Default = YES (+) * Default = NO (-) ] * Specificity = `\((1-23/9667)\times 100\)` = 99.8% * TP rate or Sensitivity = 24.3% = `\((=100\times 81/333)\)` * False negative rate `\(=75.7\% =100\times 252/333=1-Sensitivity\)`. For a bank that wants to identify high-risk individuals, this is too high * In the classification above, we used the threshold 0.5. That is we classified an observation as default=YES if the posterior probability is larger than 0.5. --- # Different threshold values A bank may use 0.2 instead of 0.5 in order to identify high-risk customers `$$Pr(default=YES|X)>0.2$$` .pull-left[ ![:scale 100%](img/confusion2.PNG) ] .pull-right[ * Sensitivity `$$=100\times 195/333 = 58.6\%$$` * False negative rate `$$41.44\% = 100\times 138/333$$` ] * As expected, sensitivity increased but the rate of false positives also increased: `\(2.43\% = 100\times 235/9667\)`. Overall error rate is 3.73% * There is a trade-off between error rate and threshold --- # Threshold and error rate .pull-left[ ![:scale 100%](img/lda2.PNG) ] .pull-right[ * Black: overall error rate, orange dashed: false positives * Blue dashed line: false negative rate (defaulters misclassified as default=NO) ] * When threshold = 0.5, error rate is minimum. But false negative rate is the highest. * As threshold decreases, false negative rate also decreases * At the same time, false positive rate also increases. * How to choose the appropriate threshold value? Field knowledge is important. Also ROC curve may be helpful. --- name: roc # ROC curve ROC (Receiver Operating Characteristics) curve shows true and false positive rates for all threshold values: .pull-left[ ![:scale 90%](img/roc1.PNG) ] .pull-right[ * Overall performance measure = AUC * AUC = Area Under the Curve * Maximum AUC = 1, high AUC is preferred * Random classification: AUC = 0.5 * For the default example LDA gives AUC = 0.95. The same as the logistic regression. ] --- class: my-medium-font name: qda # Quadratic Discriminant Analysis (QDA) * LDA assumes the variances are the same in each group * If this assumption fails LDA performance worsens * QDA (quadratic discriminant analysis) is similar to LDA but assumes different covariance matrices in each group instead of common covariance matrix in the multivariate normal distribution. * The resulting discriminant function is not linear but quadratic. * If the variances are different QDA performs better than LDA. But there must be sufficient number of observations in each group. * If the group variances are the same we may prefer LDA. --- # LDA vs. QDA .center[![](img/qda1.PNG)] **Left**:common covariance matrix. LDA (black dashed) is more successful than QDA (green) (purple dashed line is Bayes boundary); **Right**: Different group variance, QDA performs better (source: James et al., ISLR, Fig-4.9, p.150)


Jesse Wolgamott has launched a training course called Ruby Off Rails, which is geared toward helping existing programmers learn Ruby first, and not Rails.
I heartily endorse this and highly recommend this approach if you want to “get into Rails”.
Jesse Wolgamott has launched a training course called Ruby Off Rails, which is geared toward helping existing programmers learn Ruby first, and not Rails.
I heartily endorse this and highly recommend this approach if you want to “get into Rails”.
Anyone have a (formal / step-by-step) protocol for code reviews?
Yes, I do. At Opower, we were pretty serious about code reviews. We didn’t review every commit, but we did review a lot, and, after a while, fell into a pretty decent routine. This is an adaptation of that that I think is pretty decent.
When creating the outline for my book (now officially published and in print!), I decided to organize it around the nine facets of an awesome command-line app. Each chapter focuses on one of these facets. They state that an awesome command-line app should:
In this post, I’ll illustrate each of these facets (along with a test of the tenth chapter on color and formatting), via a code walkthrough of a simple command-line app I created for work.
Here’s a link to download from my talk on JRuby and Threads.
If you just want the “exercises”, here they are:
The trailer for my book, check it out:
It’s a nonsensical journey through the computer, ending on the command-line. It’s got drama, special effects, and a stern soundtrack :)
When mocking and dynamic typing travel in the same cart, the riders believe nothing can stand in their way. Their movements become headlong - faster and faster and faster. They put aside all thoughts of obstacles and forget the precipice does not show itself to the man in a blind rush until it’s to late.
Apple has just announced Mountain Lion, the next version of OS X, due out in the summer. Although it’s more of an incremental upgrade than Lion was1, one of the most interesting features is Gatekeeper. Gatekeeper, in brief, prevents unsigned code from running on OS X. Although it can be disabled, it’s on by default, and many feel this is the beginning of the end for users having full control of their computers. I feel the opposite: it’s not a slippery slope, and it’s actually a great feature.
In the last two posts about “clean tests”, we talked about the structure of a test, how to eliminate duplication, and how to make intent clear when using mocks. We left off with a question of magic values: Why do we seem to use them in our tests, when we know they are wrong in production code? Let’s explore that and see how to eliminate their use in our tests without making the tests hard to understand.
Hopefully you’ve read Steve Yegge’s excellent kingdom of nouns essay, in which he bemoans a pattern that exists in a lot of Java-base systems. The tell-tale sign is a class named ThingDoer
with a single method doThing()
. Systems like this don’t arise simply because Java is the way it is, but when you follow SOLID principles (particularly the single responsibility and dependency inversion principles), your code ends up with lots of small classes that do one thing only.
In Java, you are basically stuck with this, but in Ruby (or any OO language that supports closures/blocks/functions), we can fight this by using Procs instead of classes.


Floyd Mayweather in UFC talks to 'come out of retirement' in 2020
Former five-weight boxing world champion Floyd Mayweather says he is "coming out of retirement in 2020".
Mayweather, 42, made the announcement on social media after an earlier message that he was in talks with UFC president Dana White.
"Dana White and I are working together again to bring the world another spectacular event in 2020," he said.
Mayweather's last outing saw him win an exhibition boxing match against Japanese kickboxer Tenshin Nasukawa.
That fight took place in December 2018 and the American needed just 140 seconds to beat his opponent in a bout worth a reported $9m (£7m).
Mayweather also came out of retirement in 2017 to beat UFC fighter Conor McGregor in the Irishman's first ever boxing match.


EXO Dynamics Power Consumption Calculator
Production
Shredders
Printers
Vehicles
Backpack Items
Miscellaneous
0.00 U/s
0 RTGs
0 small generators
0 medium generators
0 small solar panels
0 medium solar panels
0 solar arrays
0 small wind turbines
0 medium wind turbines


Education
University of Rhode Island (1994-1998)
Bachelor's Degree, Political Science and Government
Activities and Societies: College Democrats, Phi Kappa Phi Honor Society, Phi Beta Kappa Honor Society, Pi Sigma Alpha Political Science Honor Society, Phi Eta Sigma Freshman Honor Society, Golden Key National Honor Society
- Completed Calculus courses: MTH 141 (with AP credits) and MTH 142
- Completed HPR 319: The Technology and Social Impact of the Internet, Spring 1997
- Completed Honors project in Advanced Web Design and Implementation, Spring 1998
- Gathered requirements, coded and installed a new homepage for the University of Rhode Island Economics department
Additional Training - Coursera (2019-2020)
- Introduction to Front-end Development with ReactJS, completed August 2020
- Introduction to Genomic Technologies, completed May 2019
- Data Science Math Skills, completed October 2019


When I was young and free and my imagination had no limits,I dreamed of changing the world. As I grew older and wiser, I discovered the world would not change, so I shortened my sights somewhat and decided to change only my country. But it, too, seemed immovable. As I grew into my twilight years, in one last desperate attempt, I settled for changing only my family, those closest to me, but alas, they would have none of it. And now, as I lie on my death bed, I suddenly realize: If I had only changed myself first, then by example I would have changed my family.From their in spiration and encouragement, I would then have been able to better my country, and who knows, I may have even changed the world.
当我年轻的时候，我的想象力从没有受到过限制，我梦想改变这个世界。 当我成熟以后，我发现我不能改变这个世界，我将目光缩短了些，决定只改变我的国家。 当我进入暮年后，我发现我不能改变我的国家，我的最后愿望仅仅是改变一下我的家庭。但是，这也不可能。 当我躺在床上，行将就木时，我突然意识到： 如果一开始我仅仅去改变我自己，然后作为一个榜样，我可能改变我的家庭；在家人的帮助和鼓励下，我可能为国家做一些事情。然后谁知道呢？我甚至可能改变这个世界。
北国风光，千里冰封，万里雪飘。望长城内外，惟余莽莽；大河上下，顿失滔滔。山舞银蛇，原驰蜡象，欲与天公试比高。须晴日，看红装素裹，分外妖娆。 江山如此多娇，引无数英雄竞折腰。惜秦皇汉武，略输文采；唐宗宋祖，稍逊风骚。一代天骄，成吉思汗，只识弯弓射大雕。俱往矣，数风流人物，还看今朝。


Lorem ipsum dolor sit amet, conse adipiscing eli... :)
1 minutes ago
15 News
12:18
pm
Bootstrap 3 releasedJohn SmithCircle
Sleek, intuitive, and powerful mobile-first front-end framework for faster and easier web development.
17
feb
Bootstrap documentsJohn SmithFriends
There are a few easy ways to quickly get started with Bootstrap, each one appealing to a different skill level and use case. Read through to see what suits your particular needs.


Anomaly
Prediction
Clustering Algorithm
Real Time
LSTM
Maps
More
Data Trainer
Vessel finder
Web Genome
Broadcasting
Setings
Your browser doesn't support this


|
Class Summary |
| ClassMap |
Taken from the Velocity tree so we can be self-sufficient
A cache of introspection information for a specific class instance. |
| Info |
Little class to carry in info such as template name, line and column for
information error reporting from the uberspector implementations
Taken from velocity for self-sufficiency. |
| Introspector |
This basic function of this class is to return a Method
object for a particular class given the name of a method
and the parameters to the method in the form of an Object[]
The first time the Introspector sees a
class it creates a class method map for the
class in question. |
| IntrospectorBase |
This basic function of this class is to return a Method object for a
particular class given the name of a method and the parameters to the method
in the form of an Object[]
The first time the Introspector sees a class it creates a class method map
for the class in question. |
| MethodMap |
|
| UberspectImpl |
Implementation of Uberspect to provide the default introspective
functionality of Velocity. |


{{examDetails.examName}} Exam
| Grade: | {{examDetails.grade}} Points |
|---|---|
| Student Answered: | {{examDetails.numberofStudentsAnswers == null ? 0 : examDetails.numberofStudentsAnswers}} Student |
| Grade: | {{examDetails.grade}} Points |
|---|---|
| Student Answered: | {{examDetails.numberofStudentsAnswers == null ? 0 : examDetails.numberofStudentsAnswers}} Student |


{% extends 'base.html' %} {% block title %}Confirmation{% endblock %} {% block content %}
The assignment has been successfully created/updated.
Back


SOCIETIES Platform Distribution
The latest SOCIETIES Virgo releases can be downloaded from our downloads site (http://www.ict-societies.eu/open-source/downloads/). Each release includes a custom distribution of the Virgo application server and the SOCIETIES 'cloud' and 'domain authority' software bundles and configuration files. Android client application is also provided as of Release 1.0.
This steup guide is based on the latest SOCIETIES platform release, version 1.0, where the default configuration is a multi-user container configuration of one combined node (Domain Authority + CSS User cloud) and one CSS user container.
It is recommended that you read the Virgo user guide (http://www.eclipse.org/virgo/documentation/virgo-documentation-3.5.0.RELEASE/docs/virgo-user-guide/html/index.html) before installing the distribution. Also make sure to review other setup guides, such as Developer_Environment_and_Toolkit_Setup (https://redmine.ict-societies.eu/projects/sp/wiki/Developer_Environment_and_Toolkit_Setup) for installing a Java runtime.


Chapter 7 Getting help
7.1 Introduction
R has a comprehensive built-in help system. This system is orientated around the base R functions and packages. Every good package comes with a set of help files. At a minimum these should provide information about the individual package functions and summaries of the data included with the package. They sometimes give descriptions of how different parts of the package should be used, and if we’re lucky, one or more “vignettes” that offer a practical demonstration of how to use a package. Other files are sometimes shipped with packages. For example, these might give an overview of the mathematical or computational theory a package relies on. We will not worry about these in this course.
We may as well get something out of the way early on. The word “help” in the phrase “help file” is a bit of a misnomer. It is probably more accurate to say R has an extensive documentation system. The reason we say this is that the majority help files are associated with functions, and these kinds of files are designed first and foremost to document how a particular function or group of functions are meant to be used. For example, they describe what kinds of arguments a function can take and what kind of objects it will return to us. Help files are also written in a very precise, painful-to-read manner. They contain a lot of jargon which can be hard to decipher unless you already know a lot about R or you have a computer science background.
The take-home message is that R help files are aimed more at experienced users than novices. Their primary purpose is to carefully document the different elements of a package, rather than explain how a particular function or the package as whole should be used to achieve a given end. That said, help files often contain useful examples, and many package authors do try to make our life easier by providing functional demonstrations of their package (those “vignettes” we mentioned above are a vehicle for doing this). It is very important to try to get to grips with the built in help system. It contains a great deal of useful information which you need to learn how to decipher in order to start using R effectively. The road to enlightenment is bumpy though.
7.2 Browsing the help system
How do we access the help system? Help files are a little like mini web pages, which means we can navigate among them using hyperlinks. This makes it very easy to explore the help system. One way to begin browsing the help system uses the help.start
function:
help.start()
If you type this now at the Console you will see the Package Index page open up in the Help tab of the bottom right pane in RStudio. This lists all the packages currently installed on your computer. We can view all the help files associated with a package by clicking on the appropriate link. For example, the functions that come with the base installation of R have a help file associated with them – click on the link to the R base package (base
) to see these. Though you know about a few of these already, you will see a lot of functions listed here. R is huge.
The packages that come with the base R installation and those that we install separately from base R have their own set of associated help files. These can be viewed by following the appropriate link on the Package Index page. We will learn how to navigate these in a moment. Take note: it is up to the developer of a package to produce usable help files. Well-designed packages like dplyr and ggplot2 have an extensive help system that covers almost everything the package can do. This isn’t always the the case though, particularly with new or packages or packages that are not widely used. We will only ever use well-documented packages.
Notice that the help browser has Forward, Back, and Home buttons, just like a normal web browser. If you get lost in the mire of help pages you can always navigate backward until you get back to a familiar page. Annoyingly, the Home button does not take you to the same page as help.start
. Click the home button now if you have not already done so. You will see three sections:
The Manuals section looks like it might be useful for novice users. Unfortunately, it is not. Even the “Introduction to R” manual is only helpful if you already have some programming experience, since it assumes you understand what terms like “data structure” and “data type” mean. It is worth reading this manual once you have gained a little more experience though. The others manuals. are more or less impenetrable unless you already know quite a bit about computing in general.
The Reference section is a little more helpful. The “Packages” link just takes you to the same page opened by
help.start
so that you can browse help pages on a package-specific basis. The “Search Engine & Keywords” link takes you to a search engine page (no surprises there). You can use this to search for specific help pages, either by supplying a search term or by navigating through the different keywords. We will discuss the built-in search engine in the next subsection.The Miscellaneous Material section has a couple of potentially useful links. The “User Manuals” link lists any user manuals supplied by package authors. These tend to be aimed at more experienced users and the packages we will learn to use in this course do not provide them. However, it is worth knowing these exist as they are occasionally useful. The “Frequently Asked Questions” link is worth reviewing once you have gained a little experience of R, but again, most of the FAQs are a little difficult for novice users to fully understand.
7.3 Searching for help files
If you spend a short amount of time browsing help files via help.start
it will quickly become obvious that this way of searching for help is not very efficient. Quite often we know the name of the function we need to use and all we want to do is open its associated help file. We do this using the help
function:
help(topic = Trig)
If you run this simple example you should see RStudio open up the help file for the trigonometry topic in the Help tab. This provides information about the various trigonometric functions such as sin
or cos
. We will learn how to make sense of help pages in the next subsection. For now we just need to understand how to use help
.
The help
function needs a minimum of one argument: the name of the topic or function of interest. When we use it like this the help function searches across packages, looking for a help file whose name gives an exact match to the name we supplied. In this case, we opened the help file associated with the Trig
topic. Most of the time we use the help
function to find the help page for a specific function, rather than a general topic. This is fine if you can remember the name of the topic associated with different functions. Most of us cannot. Luckily, the help function will also match help pages by the name of the function(s) they cover:
help(topic = sin)
Here we searched for help on the sin
function. This is part of the Trig
topic so help(topic = sin)
brings up the same page as the help(topic = Trig)
.
There are several arguments of help
that we can set to alter its behaviour. We will just consider one of these. By default, the help
function only searches for files associated with the base functions or with packages that we have loaded in the current session with the library
function. If you wanted to search for help on the mutate
function – which is part of the dplyr
package – but you haven’t run library(dplyr)
yet this will not work:
help(mutate)
## Help on topic 'mutate' was found in the following packages:
##
## Package Library
## dplyr /Library/Frameworks/R.framework/Versions/3.3/Resources/library
## plyr /Library/Frameworks/R.framework/Versions/3.3/Resources/library
##
##
## Using the first match ...
Instead, you need tell help
where to look by setting the package
argument:
help(mutate, package = dplyr)
You should try to start using help
every time you get stuck because you cannot get a function to work. It doesn’t matter how experienced an R user you become, you will occasionally forget how to use the odd function. It is for this reason that R has a built in shortcut for help
. This is accessed via ?
. For example, instead of typing help(topic = sin)
(or just help(sin)
) at the Console, we can bring up the help page for the sin
function by using ?
like this:
?sin
This is just a convenient shortcut that does the same thing as help
. The only difference is that ?
does not allow us to set arguments such as package
.
7.5 Vignettes and demos
The Oxford English Dictionary defines a vignette as, “A brief evocative description, account, or episode.” The purpose of a package vignette in R is to provide a relatively brief, practical account of one or more of its features. Not all packages come with vignettes, though many of the best thought out packages do. You can use the vignette
function to view all the available vignettes in Rstudio. This will open up a tab that lists each vignette under their associated package name along with a brief description. A package will often have more than one vignette. If you just want to see the vignettes associated with a particular package, you should set the package
argument. For example, to see the vignettes associated with dplyr you would use:
vignette(package = "dplyr")
Each vignette has a name (the “topic”) and is available either as a PDF or HTML file (or both). We can view a particular vignette by passing the vignette
function the package
and topic
arguments. For example, to view the “data_frames” vignette in the dplyr
package we would use:
vignette(topic = "data_frames", package = "dplyr")
The vignette
function is fine, though it is usually more convenient to browse the list of vignettes inside a web browser. This allows you to open a particular vignette directly by clicking on its link, rather than working at the Console. We can use the browseVignettes
function to do this:
browseVignettes()
This will open a page in your browser showing the vignettes you can view. As you might expect by now, you can narrow your options to a specific package by setting the package
argument.
In addition to vignettes, some packages also include one or more demos (i.e. demonstrations). Demos are a little like vignettes, but instead of just opening a file for you to read, the demo function can actually runs a demonstration R scripts for you. We use the demo
function without any arguments to list the available demos:
demo()
When we use the demo
function like this it only lists the demos associated with packages that have been loaded in the current session (via library
). If we want to see all the demos you can run you need to use the somewhat cryptic demo(package = .packages(all.available = TRUE))
.
In order to actually run a demo we use the demo
function, setting the topic
and package
arguments. For example, to run the “colors” demo in the grDevices package we would use:
##
##
## demo(colors)
## ---- ~~~~~~
##
## > ### ----------- Show (almost) all named colors ---------------------
## >
## > ## 1) with traditional 'graphics' package:
## > showCols1 <- function(bg = "gray", cex = 0.75, srt = 30) {
## + m <- ceiling(sqrt(n <- length(cl <- colors())))
## + length(cl) <- m*m; cm <- matrix(cl, m)
## + ##
## + require("graphics")
## + op <- par(mar=rep(0,4), ann=FALSE, bg = bg); on.exit(par(op))
## + plot(1:m,1:m, type="n", axes=FALSE)
## + text(col(cm), rev(row(cm)), cm, col = cl, cex=cex, srt=srt)
## + }
##
## > showCols1()
##
## > ## 2) with 'grid' package:
## > showCols2 <- function(bg = "grey", cex = 0.75, rot = 30) {
## + m <- ceiling(sqrt(n <- length(cl <- colors())))
## + length(cl) <- m*m; cm <- matrix(cl, m)
## + ##
## + require("grid")
## + grid.newpage(); vp <- viewport(w = .92, h = .92)
## + grid.rect(gp=gpar(fill=bg))
## + grid.text(cm, x = col(cm)/m, y = rev(row(cm))/m, rot = rot,
## + vp=vp, gp=gpar(cex = cex, col = cm))
## + }
##
## > showCols2()
## Loading required package: grid
##
## > showCols2(bg = "gray33")
##
## > ###
## >
## > ##' @title Comparing Colors
## > ##' @param col
## > ##' @param nrow
## > ##' @param ncol
## > ##' @param txt.col
## > ##' @return the grid layout, invisibly
## > ##' @author Marius Hofert, originally
## > plotCol <- function(col, nrow=1, ncol=ceiling(length(col) / nrow),
## + txt.col="black") {
## + stopifnot(nrow >= 1, ncol >= 1)
## + if(length(col) > nrow*ncol)
## + warning("some colors will not be shown")
## + require(grid)
## + grid.newpage()
## + gl <- grid.layout(nrow, ncol)
## + pushViewport(viewport(layout=gl))
## + ic <- 1
## + for(i in 1:nrow) {
## + for(j in 1:ncol) {
## + pushViewport(viewport(layout.pos.row=i, layout.pos.col=j))
## + grid.rect(gp= gpar(fill=col[ic]))
## + grid.text(col[ic], gp=gpar(col=txt.col))
## + upViewport()
## + ic <- ic+1
## + }
## + }
## + upViewport()
## + invisible(gl)
## + }
##
## > ## A Chocolate Bar of colors:
## > plotCol(c("#CC8C3C", paste0("chocolate", 2:4),
## + paste0("darkorange", c("",1:2)), paste0("darkgoldenrod", 1:2),
## + "orange", "orange1", "sandybrown", "tan1", "tan2"),
## + nrow=2)
##
## > ##' Find close R colors() to a given color {original by Marius Hofert)
## > ##' using Euclidean norm in (HSV / RGB / ...) color space
## > nearRcolor <- function(rgb, cSpace = c("hsv", "rgb255", "Luv", "Lab"),
## + dist = switch(cSpace, "hsv" = 0.10, "rgb255" = 30,
## + "Luv" = 15, "Lab" = 12))
## + {
## + if(is.character(rgb)) rgb <- col2rgb(rgb)
## + stopifnot(length(rgb <- as.vector(rgb)) == 3)
## + Rcol <- col2rgb(.cc <- colors())
## + uniqC <- !duplicated(t(Rcol)) # gray9 == grey9 (etc)
## + Rcol <- Rcol[, uniqC] ; .cc <- .cc[uniqC]
## + cSpace <- match.arg(cSpace)
## + convRGB2 <- function(Rgb, to)
## + t(convertColor(t(Rgb), from="sRGB", to=to, scale.in=255))
## + ## the transformation, rgb{0..255} --> cSpace :
## + TransF <- switch(cSpace,
## + "rgb255" = identity,
## + "hsv" = rgb2hsv,
## + "Luv" = function(RGB) convRGB2(RGB, "Luv"),
## + "Lab" = function(RGB) convRGB2(RGB, "Lab"))
## + d <- sqrt(colSums((TransF(Rcol) - as.vector(TransF(rgb)))^2))
## + iS <- sort.list(d[near <- d <= dist])# sorted: closest first
## + setNames(.cc[near][iS], format(d[near][iS], digits=3))
## + }
##
## > nearRcolor(col2rgb("tan2"), "rgb")
## 0.0 21.1 25.8 29.5
## "tan2" "tan1" "sandybrown" "sienna1"
##
## > nearRcolor(col2rgb("tan2"), "hsv")
## 0.0000 0.0410 0.0618 0.0638 0.0667
## "tan2" "sienna2" "coral2" "tomato2" "tan1"
## 0.0766 0.0778 0.0900 0.0912 0.0918
## "coral" "sienna1" "sandybrown" "coral1" "tomato"
##
## > nearRcolor(col2rgb("tan2"), "Luv")
## 0.00 7.42 7.48 12.41 13.69
## "tan2" "tan1" "sandybrown" "orange3" "orange2"
##
## > nearRcolor(col2rgb("tan2"), "Lab")
## 0.00 5.56 8.08 11.31
## "tan2" "tan1" "sandybrown" "peru"
##
## > nearRcolor("#334455")
## 0.0867
## "darkslategray"
##
## > ## Now, consider choosing a color by looking in the
## > ## neighborhood of one you know :
## >
## > plotCol(nearRcolor("deepskyblue", "rgb", dist=50))
##
## > plotCol(nearRcolor("deepskyblue", dist=.1))
##
## > plotCol(nearRcolor("tomato", "rgb", dist= 50), nrow=3)
##
## > plotCol(nearRcolor("tomato", "hsv", dist=.12), nrow=3)
##
## > plotCol(nearRcolor("tomato", "Luv", dist= 25), nrow=3)
##
## > plotCol(nearRcolor("tomato", "Lab", dist= 18), nrow=3)
This particular demo shows off some of the pre-defined colours we might use to customise the appearance of a plot. We’ve suppressed the output though because so much is produced.