spu_create - create a new spu context
#include <sys/types.h>
#include <sys/spu.h>
int spu_create(const char *pathname, int flags, mode_t mode);
int spu_create(const char *pathname, int flags, mode_t mode,
int neighbor_fd);
Note: There is no glibc wrapper for this system call; see
NOTES.
The spu_create() system call is used on PowerPC machines that implement
the Cell Broadband Engine Architecture in order to access Synergistic
Processor Units (SPUs). It creates a new logical context for an SPU in
pathname and returns a file descriptor associated with it.
pathname must refer to a nonexistent directory in the mount point of
the SPU filesystem (spufs). If spu_create() is successful, a
directory is created at pathname and it is populated with the files
described in spufs(7).
When a context is created, the returned file descriptor can only
be passed to spu_run(2), used as the dirfd argument to the
*at family of system calls (e.g., openat(2)), or closed; other
operations are not defined. A logical SPU context is destroyed (along with
all files created within the context's pathname directory) once the
last reference to the context has gone; this usually occurs when the file
descriptor returned by spu_create() is closed.
The flags argument can be zero or any bitwise OR-ed
combination of the following constants:
- SPU_CREATE_EVENTS_ENABLED
- Rather than using signals for reporting DMA errors, use the event
argument to spu_run(2).
- SPU_CREATE_GANG
- Create an SPU gang instead of a context. (A gang is a group of SPU
contexts that are functionally related to each other and which share
common scheduling parameters—priority and policy. In the future,
gang scheduling may be implemented causing the group to be switched in and
out as a single unit.)
- A new directory will be created at the location specified by the
pathname argument. This gang may be used to hold other SPU
contexts, by providing a pathname that is within the gang directory to
further calls to spu_create().
- SPU_CREATE_NOSCHED
- Create a context that is not affected by the SPU scheduler. Once the
context is run, it will not be scheduled out until it is destroyed by the
creating process.
- Because the context cannot be removed from the SPU, some functionality is
disabled for SPU_CREATE_NOSCHED contexts. Only a subset of the
files will be available in this context directory in spufs.
Additionally, SPU_CREATE_NOSCHED contexts cannot dump a core file
when crashing.
- Creating SPU_CREATE_NOSCHED contexts requires the
CAP_SYS_NICE capability.
- SPU_CREATE_ISOLATE
- Create an isolated SPU context. Isolated contexts are protected from some
PPE (PowerPC Processing Element) operations, such as access to the SPU
local store and the NPC register.
- Creating SPU_CREATE_ISOLATE contexts also requires the
SPU_CREATE_NOSCHED flag.
- SPU_CREATE_AFFINITY_SPU
- Create a context with affinity to another SPU context. This affinity
information is used within the SPU scheduling algorithm. Using this flag
requires that a file descriptor referring to the other SPU context be
passed in the neighbor_fd argument.
- SPU_CREATE_AFFINITY_MEM
- Create a context with affinity to system memory. This affinity information
is used within the SPU scheduling algorithm.
The mode argument (minus any bits set in the process's
umask(2)) specifies the permissions used for creating the new
directory in spufs. See stat(2) for a full list of the
possible mode values.
On success, spu_create() returns a new file descriptor. On error, -1 is
returned, and errno is set to one of the error codes listed below.
- EACCES
- The current user does not have write access to the spufs(7) mount
point.
- EEXIST
- An SPU context already exists at the given pathname.
- EFAULT
- pathname is not a valid string pointer in the calling process's
address space.
- EINVAL
- pathname is not a directory in the spufs(7) mount point, or
invalid flags have been provided.
- ELOOP
- Too many symbolic links were found while resolving pathname.
- EMFILE
- The per-process limit on the number of open file descriptors has been
reached.
- ENAMETOOLONG
- pathname is too long.
- ENFILE
- The system-wide limit on the total number of open files has been
reached.
- ENODEV
- An isolated context was requested, but the hardware does not support SPU
isolation.
- ENOENT
- Part of pathname could not be resolved.
- ENOMEM
- The kernel could not allocate all resources required.
- ENOSPC
- There are not enough SPU resources available to create a new context or
the user-specific limit for the number of SPU contexts has been
reached.
- ENOSYS
- The functionality is not provided by the current system, because either
the hardware does not provide SPUs or the spufs module is not loaded.
- ENOTDIR
- A part of pathname is not a directory.
- EPERM
- The SPU_CREATE_NOSCHED flag has been given, but the user does not
have the CAP_SYS_NICE capability.
pathname must point to a location beneath the mount point of
spufs. By convention, it gets mounted in /spu.
The spu_create() system call was added to Linux in kernel 2.6.16.
This call is Linux-specific and implemented only on the PowerPC architecture.
Programs using this system call are not portable.
Glibc does not provide a wrapper for this system call; call it using
syscall(2). Note however, that spu_create() is meant to be used
from libraries that implement a more abstract interface to SPUs, not to be
used from regular applications. See
http://www.bsc.es/projects/deepcomputing/linuxoncell/
for the recommended libraries.
See spu_run(2) for an example of the use of spu_create()
close(2), spu_run(2), capabilities(7), spufs(7)
This page is part of release 5.05 of the Linux man-pages project. A
description of the project, information about reporting bugs, and the latest
version of this page, can be found at https://www.kernel.org/doc/man-pages/.


HI,
I'm following that guide to setup an OEMgateway forwarding data to my EmonCMS.org account.
It works fine until I try to setup Raspbian to run in read-only mode for increased stability.
If I put the following lines in /etc/fstab as mentioned in the guide, the OEMgateway doesn't work anymore.
tmpfs /tmp tmpfs nodev,nosuid,size=30M,mode=1777 0 0
tmpfs /var/log tmpfs nodev,nosuid,size=30M,mode=1777 0 0
proc /proc proc defaults 0 0
/dev/mmcblk0p1 /boot vfat defaults 0 2
/dev/mmcblk0p2 / ext4 defaults,ro,noatime,errors=remount-ro 0 1
# /dev/mmcblk0p3 /home ext4 defaults,noatime
Before that step, here is my file system :
pi@raspberrypi ~ $ df -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
rootfs rootfs 2721336 2086676 476708 82% /
/dev/root ext4 2721336 2086676 476708 82% /
devtmpfs devtmpfs 215824 0 215824 0% /dev
tmpfs tmpfs 44820 208 44612 1% /run
tmpfs tmpfs 5120 0 5120 0% /run/lock
tmpfs tmpfs 89620 0 89620 0% /run/shm
/dev/mmcblk0p1 vfat 57288 19072 38216 34% /boot
Must I replace /dev/mmcblk0p2 with /dev/root in fstab ?
Bye
Re: Oemgateway in read only
Hi Eric.
No, the the "/dev/mmcblk0p2" in the fstab is the physical location of the root file system, without that it won't boot correctly either.
Have you added RAMPTMP=yes to the end of /etc/default/rc5 and replaced /etc/mtab with a symlink to /proc/self/mounts ?
I have written a guide to this in the emonhub documentation which isn't up and running just yet. But there is a copy in Trystan's documentation for his later, emonhub based version of the gateway you are installing.
Paul
Re: Oemgateway in read only
I followed each steps of this guide. So I made the modifications that you mentioned. So it 's not the problem.
I just understood that there is typo in the step a) :a) sudo nano /etc/default/rcS, add line RAMTMP=yes ---> a) sudo nano /etc/default/rc5, add line RAMTMP=yesMay I edit this guide ?EDIT : there is a typo in you message, it's /etc/default/rcS and not /etc/default/rc5
EDIT 2 : I will try with "RAMPTMP=yes" instaed of "RAMTMP=yes"
Re: Oemgateway in read only
Apologies for the typos in my post (the guides do have rcS & RAMTMP), seems the old grey matter isn't firing on all cylinders this morning. I've not used the guide you are working from, but I can't see any reason for not working. I regularly use the method in my guide as, just cutting and pasting this chunk of code does everything and eliminates typo errors etc.
sudo cp /etc/default/rcS /etc/default/rcS.orig
sudo sh -c "echo 'RAMTMP=yes' >> /etc/default/rcS"
sudo mv /etc/fstab /etc/fstab.orig
sudo sh -c "echo 'tmpfs /tmp tmpfs nodev,nosuid,size=30M,mode=1777 0 0' >> /etc/fstab"
sudo sh -c "echo 'tmpfs /var/log tmpfs nodev,nosuid,size=30M,mode=1777 0 0' >> /etc/fstab"
sudo sh -c "echo 'proc /proc proc defaults 0 0' >> /etc/fstab"
sudo sh -c "echo '/dev/mmcblk0p1 /boot vfat defaults 0 2' >> /etc/fstab"
sudo sh -c "echo '/dev/mmcblk0p2 / ext4 defaults,ro,noatime,errors=remount-ro 0 1' >> /etc/fstab"
sudo sh -c "echo ' ' >> /etc/fstab"
sudo mv /etc/mtab /etc/mtab.orig
sudo ln -s /proc/self/mounts /etc/mtab
it also keeps all 3 original files so reverting to standard is easy. (I think I will post the full instructions on forum as it also has shortcuts to lock and unlock filesystem)
Also just as a "heads up" I don't think the oem_gateway you are using has variable size buffers or bulk upload enabled as discussed in your other thread.
Paul
Re: Oemgateway in read only
Thank's for your chunk of code and for your support in general!
It doesn't fix the problem. As soon as I setup Raspbian to run in read-only mode, the oemgateway doesn't work anymore after a reboot. When trying to stop the oemgateway service, I gotr the following error :
pi@raspberrypi ~/oem_gateway $ sudo service oemgateway stop
[....] Stopping OpenEnergyMonitor Gateway: oemgatewaystart-stop-daemon: warning: failed to kill 2463: No such process
The only way to make it work, is to come back to the three original files (fstab, mtab and rcS). But then the file systeme is in RW mode ... As you can see below there is no difference between mtab and mtab.origin. Is it normal ?
pi@raspberrypi ~ $ diff /etc/mtab /etc/mtab.orig
pi@raspberrypi ~ $ diff /etc/default/rcS.orig /etc/default/rcS
24a25
> RAMTMP=yes
pi@raspberrypi ~ $ diff /etc/fstab /etc/fstab.orig
1,6c1,4
< tmpfs /tmp tmpfs nodev,nosuid,size=30M,mode=1777 0 0
< tmpfs /var/log tmpfs nodev,nosuid,size=30M,mode=1777 0 0
< proc /proc proc defaults 0 0
< /dev/mmcblk0p1 /boot vfat defaults 0 2
< /dev/mmcblk0p2 / ext4 defaults,ro,noatime,errors=remount-ro 0 1
<
---
> proc /proc proc defaults 0 0
> /dev/mmcblk0p1 /boot vfat defaults 0 2
> /dev/mmcblk0p2 / ext4 defaults,noatime 0 1
> # a swapfile is not a swap partition, so no using swapon|off from here on, use dphys-swapfile swap[on|off] for that
Eric
Re: Oemgateway in read only
Anyway, I will try to install the Emonhub on a SD card with a RO file system to get variable size buffers or bulk upload enabled.
Which guide should I follow ? This one ?
Eric
Re: Oemgateway in read only
Unfortunately I'm not able to access a pi right now and I don't know 100% what to expect as I'm not sure how diff handles symlinks and I don't know the exact content of your mtab.orig. It certainly doesn't look right to me but I cannot be 100% without trying it.
You could use the version that that guide installs, it does have variable buffer size but not bulk upload, I'm not overly familiar with that version so may be of limited help to you.
Personally I would recommend (and appreciate) you installing the dev version of emonhub as I am more familiar with it and actively developing it. It has both variable buffer size and bulk upload. It currently posts to the input module rather than the node module as I'm working on a solution that utilises both rather than only one or other.
One line will install and set-up emonhub
then the conf file needs to be edited, just add an api key for basic operation, then add node details to change data types.
For example to add the node data for the emonTxV3_continuous_kwhtotals_noeeprom.ino v3 sketch for node 10
and then use this guide I just posted to forum to switch to read-only.
Paul
Re: Oemgateway in read only
I did a try with that Emonhub installation guide
Firstly, I had to modify the emonhubNode.conf file in order to modify the log file path that wasn't correct (logfile = /var/log/emonhub.log --> /var/log/emonhub/emonhub.log)
Then, I got the problem described above : after configuring raspbian in RO mode, Emonhub isn't working ... But, I did some progress !
When restarting the emonhub service, I get the following messages :
Restarting OpenEnergyMonitor emonHub: emonhubstart-stop-daemon: warning: failed to kill 1962: No such process
Could not start EmonHub: [Errno 2] No such file or directory: '/var/log/emonhub/emonhub.log'
If I unlock the file system, I get the same errors.
The workaround is to execute the following command.
sudo mkdir /var/log/emonhub
sudo chown pi /var/log/emonhub
sudo chmod 750 /var/log/emonhub
rpi-ro
sudo service emonhub restart
Then Emonhub works until the next reboot ...
EDIT : after a reboot :
pi@raspberrypi ~ $ sudo ls /var/log/emonhub
ls: cannot access /var/log/emonhub: No such file or directory
Re: Oemgateway in read only
The original location for the logfile was going to be /var/log/emonhub/emonhub.log but this doesn't work with RO image as the /var/log/emonhub directory is lost every time the pi reboots since /var/log is relocated to RAM and therefore recreated at every boot.
emonhub's default was changed to /var/log/emonhub.log to be RO compatible. If you change the emonhub.conf file back to " logfile = /var/log/emonhub.log " this should no longer be an issue.
Paul
Re: Oemgateway in read only
Ok, it's now working in RO !
Unfortunately, as you said me, this version is limited. So I will install the dev version of emonhub.
Do you think this dev version is enough reliable to be installed on a site that I will not be able to access ? (neither physically speaking, nor by shh)
Eric
Re: Oemgateway in read only
I installed the dev version of Emonhub on a SD card and than switch to read-only. YESS !
Paul, I really like your installation-procedure (one line) !
Note that it was not working until I made some modifications :
1. At the bottom of the /etc/inittab file, I commented out the line, by adding a ‘#’ at beginning) and the cmdline.text file
2. Changes /boot/cmdline.txt file with those lines :
sudo cp /boot/cmdline.txt /boot/cmdline.txt.origin
sudo sh -c "echo 'dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait' > /boot/cmdline.txt"
It seemed to work only after the first modification, so I'm not sure that the second one is required.
Eric
Re: Oemgateway in read only
I have no reason to doubt it's reliability but as with all new software only time will tell. any recent version of emonhub or oem gateway will not have the time served reassurance of the original oem gateway but they are built on the same reliable solution.
I have been using emonhub for a while now without any issues under normal running conditions, you would be wise to "stress test" any software you decide to use before committing. for example the buffers, you should try changing the url to something it cannot find to max out the buffers to test the size, the ability to resume sending and what happens if the buffer size is exceeded as these characteristics may change with larger buffer sizes, especially if you are anywhere close to maxing out the RAM.
The "dev" status is just to denote that emonhub is being tested and things are subject to change before a release version is finalised. In this "dev" version the files are not put where they should be rather they are symlinked so that git pull can be used to update, this method will not be used instead it will be available as a debian package at release time, I hope.
Yes both files need to be edited for the rfm2pi to access the uart with any software or OS on the pi. I did include a little reminder "(note - if serial UART access required make those changes before restarting)" at the end of the RO guide.
Paul


{% extends "base.html" %} {% block title %}404 Page Not Found{% endblock %} {% block container %}
The page you are looking for could not be found. Check the address bar to ensure your URL is spelled correctly. If all else fails, you can visit our home page at the link below.
Visit the Home Page


Disaster Management Intelligence System
Specialized in Management of Disaster
Login
{% csrf_token %}
Username:
Password:
Register New User

Submit


I tried 2C-C at a small party that we held on a friend's birthday. There were approximately 10 people taking various substances in a relaxed and controlled atmosphere. I was the only one on 2C-C, though many of my friends were on 2C-I.
The 2C-C came on gradually after about an hour. The exact point at which I noticed the 2C-C was obscured by a fairly intense Salvia experience, but otherwise the onset of 2C-C was simliar to an experience I had on 2C-T-2: nausea, agitation, and eye-popping visual disturbances. Fortunately, I had learned from prior experience and fasted for 8 hours prior to taking the 2C-C, so the nausea was not unbearable and I didn't vomit.
The nausea subsided after half an hour or so, giving way to crazy open eye visuals. I wandered around the house wearing goggles & gloves and carrying an opened umbrella. The goggles somehow made the visuals that much more interesting.I had the vague impression that they had become the All Revealing Goggles of Truth and I tried to convince my friends of this. heh.
After another half hour or so, the little elves tap-dancing on my optic nerves took a bow and vanished, and the visuals became less pronounced. It was about then (T+2:00) that the trip entered a VERY good phase. It felt a little like the nice numb feeling your spine gets on dexadrine, times 10^3 or so. All my muscles felt very relaxed, very tactile. I took off my gloves (gloves?) at some point and discovered to my delight that I had nerves on the tips of my fingers.  We all sat on the floor talking and laughing as we came back down to earth.
Overall, I'd say that 40 mg was a little intense on the onset. 30 mg would probably be a better starting dose to calibrate with. I'd definitely recommend 2C-C to any enterprising experimenter.
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


cook smarter
live smarter
smARTfood
What's New?
7.6.2015 | Green Goodness: Cook Book Launch Session
Join us to find out about our new cookbook. Taste some of the cookbook recipes and buy the book @ITU, SJ.
7.20.2015 | Raw Foods & Summer Desserts Menu: Cooking Workshop
Join us to learn new healthy summer desserts and treat yourself to some @ITU, SJ.
7.27.2015 | Potluck and Food photography workshop
This is a fun event where you cook new healthy recipes from our site and get on the day for others to enjoy, right after you learn and do some food photography with your food @ITU, SJ.


Monday, June 6, 2016
There is no doubt that React is a hot library. I would even consider it
the hottest (fastest rising) technology ever in the history of
whoishiring. But, I would not have predicted that it would pass the
language it's developed in, JavaScript.
Full hiring trends post for charts and rankings >>
Friday, May 6, 2016
Hacker News Hiring trends began as a way to discover emerging technolgies used to
develop software. But, it has become useful for tracking non-technical
aspects of technical hiring as well. The first non-technical focus was
on the use of "passionate" after
Avdi wrote about the Passion Gospel.
Then came examining experience level (Senior vs Junior) a couple months
ago. Now, we see the full force of the non-technical terms this month
with the addition of "onsite" which we can use to compare with "remote."
Full hiring trends post for charts and rankings >>
Thursday, March 7, 2016
Software development and programming technologies move so fast that it's
not often that developers can slow down and take a breath. This is that
month, as nothing really jumps out at me as a big trend.
Full hiring trends post for charts and rankings >>
Monday, March 7, 2016
Welcome to web development in 2016, where the LAMP stack was mentioned
exactly 0 times. It wasn't too long ago when web developers would
often be referred to as LAMP developers, working the stack of Linux,
Apache, MySQL and PHP. While PHP itself had a decent March performance,
though still far behind Rails and Node.js, it seems to me that this is
a milestone and clear signal of how competitive every layer of the
software development stack has become.
Full hiring trends post for charts and rankings >>
Saturday, February 6, 2016
Python won for the third time in a row and had its largest margin of
victory to date. JavaScript and React retain their 2nd and third place
finishes respectively. Meanwhile, AWS (Amazon Web Services) cracks the
top 5 with its best showing ever and is perhaps the most interesting
change near the top. Rounding out the top 5 was iOS, which leapfrogged
Android as the most popular mobile development technology.
Full hiring trends post for charts and rankings >>
Sunday, January 17, 2016
Based on years past, whoishiring is traditionally slower in January, so
it's still early to make any assumptions about this data. But, seeing
January with 395 postings compared to the peak last July of 901 does
flag this as an economic and/or startup indicator to watch.
Full hiring trends post for charts and rankings >>
These charts show what's popular and trending in programming languagues,
frameworks and technologies used to develop modern software applications as
indicated by Hacker News activity.
Using the HN Search API and a
dictionary of software
terms, all of the the comments (job postings) in the
"whoishiring"
submissions from April 2011 to the latest month were processed to count the
number of times each software term was mentioned.


In addition to creating the Richter Earthquake scale, Richter also wrote a few books, which these days, are widely accepted as scientific sources.
Seismicity of the Earth and Associated Phenomena
The first book that Richter published was Seismicity of the Earth and Associated Phenomena, which he wrote in association with Beto Gutenberg. The revised edition, published in 1954, is widely accepted these days as a standard reference in the field. This book was revolutionary since it made concepts that were usually reserved for scholars to a more accessible format.
Elementary seismology
This was another book that Richter published, this time in 1958. However, this book was one that was written by Richter alone. Again, this book was amazing in that it allowed the common folk to learn about concepts that in this day and age, they usually would not learn about.
Alvin Paul - Final Earthquakes Project
Earth Science Pd. 1 - Mr. Kidonakis
This webpage is purely conceptual, and is not intended to represent or depict any actual event, person, or entity, and any such similarities are purely coincidental.


I Alone Level-up: Chapter 73
BOOM!
Volkhan fell . From the dust blown up by its giant body, Jinwoo stepped out . He looked around . He was about to go to the aid of his army, but it looked like they were finishing up as well .
“Kiiiiek!”
“Kekek!”
“Keeeek!”
With the combined might of the Shadow Army, Volkhan’s minions rapidly fell left and right .
‘Status screen . ’
Jinwoo looked over his status . Because he had just leveled up, he had recovered all of his health, mana and stamina .
‘With my mana refreshed, I don’t need to help them . ’
As long as there was mana, the Shadow Soldiers were immortal . That was the power of the undead army . Deciding to leave the trash mobs to the soldiers, Jinwoo turned back to Volkhan’s corpse to collect the drops .
“Hmm . ”
It felt like he was about to open his Christmas gifts . Jinwoo could not help but smile while looking at Volkhan’s body .
‘Wonder what I’ll get?’
There were multiple lights coming from the corpse . As always, Jinwoo stretched a hand towards the light . The item message appeared .
ring~!
| You’ve found the “Monarch of Demons’ Earrings”. |
| Would you like to acquire it?. |
. .
| You’ve found the “Marble of Avarice”. |
| Would you like to acquire it?. |
| You’ve found 2 “Volkhan’s Horns”. |
| Would you like to acquire it?. |
| You’ve found a crafting ingredient, “Fragment of the World Tree”. |
| Would you like to acquire it?. |
[1].
Jinwoo had a good feeling while fighting Volkhan . As expected, multiple items dropped from the boss’s corpse . A joy filled the Hunter’s face as he looked over the drops .
‘Of course, Volkhan’s Horns are probably junk items . ’
Just looking at the name and the amount acquired, Jinwoo was sure Volkhan’s Horns were the same as the horns dropped by the other demons: just junk items to be sold . That left the other three items… One of them grabbed Jinwoo’s attention .
| You’ve found a crafting ingredient, “Fragment of the World Tree”. |
| Would you like to acquire it?. |
‘Crafting ingredient?’
What could he craft with it? Or was it part of the quest? His curiosity peaked at the new type of item he had found . In order to read the item’s information, he had to first solidify the item from the light . And so, Jinwoo decided to pick up all the items .
“Acquire all . ”
The multiple lights disappeared from Volkhan’s corpse, and an equal amount of physical objects appeared in front of Jinwoo out of thin air . Two earrings, a red marble…
.
Jinwoo instantly knew which one of these was the Fragment of the World Tree . He stared at the giant piece of lumber, and the corresponding information popped up .
| Fragment of the World Tree. |
| Rarity: ??. Type: Ingredient. Volkhan’s Club was made out of wood from the World Tree . This piece of wood remained after the contaminated parts were removed from the club . The wood obtained from the World Tree contains a strong magic power and is the highest-class material for crafting magic items . |
‘The ingredient for the highest-class magic items . ’
Jinwoo wanted to know just what he could make with this thing . Unfortunately, that information was nowhere to be found . Still, it was certain this was not an ordinary item . Standing close to it, he was able to feel a powerful energy emanating from the piece of wood .
‘I have a feeling… this wood will be useful somehow . ’
Instead of selling it, Jinwoo put the Fragment of the World Tree into his inventory . He pored over the other items’ information .
| The Monarch of Demons’ Earrings. |
| Rarity: S. Type: Accessory. Strength +20, Constitution +20. Set: The Crown Jewels of the Monarch of Demons[2] 1 . The Monarch of Demons’ Earrings 2 . The Monarch of Demons’ Necklace 3 . The Monarch of Demons’ Ring. Set Bonus: 1 . ??? (2 Items) 2 . ??? (3 Items) |
‘A set item?’
While the base effects were similar to the A-rank Gatekeeper’s Necklace, the Monarch of Demons’ Earrings had an unknown set effect . Even without knowing the set effects, it was an amazing item, increasing both strength and constitution by 20 points .
‘Since one level-up is equivalent to five stat points . ’
In that line of thought, this was the same as leveling up eight times . With a happy heart, Jinwoo picked up the earrings .
| Would you like to equip the “Monarch of Demons’ Earrings”?. |
‘Equip . ’
Like always, the earrings disappeared and its effects were applied to Jinwoo . He smiled in satisfaction at his increased stats .
The Crown Jewels of the Monarch of Demons .
‘I’m sure the other pieces will drop here . ’
With a name that included the word “Demon”, it was to easy to understand that the full set would drop from a place called Demon Castle . Just what would the set effects be? Even though he had obtained only one part, Jinwoo’s expectations were great . However, his smile would soon disappear .
‘This is…?’
| Marble of Avarice. |
| Rarity: A. Type: Magic Tool. Effect ‘Desire for Destruction’: Your magic damage is doubled . . It’s a marble made from the blood of the archdemon Volkhan . The marble will empower the wielder’s magic and increase the destruction caused . |
It was a red ball the size of a billiard ball . Jinwoo held the Marble of Avarice in his hand . There was no change to his stats . He pulled up the skill information on the only thing he had that might qualify as magic, his class-specific shadow skills, but there were no changes . The limits on the number of shadows remained the same .
‘Is it not something I could benefit from?’
Jinwoo tilted his head .
Doubling the magic damage . Its effects were incredible .
Out in the real world, magic tools that empowered the user’s magic were prohibitively expensive . Because of that, they were only used by a handful of the Mage-class Hunters . However, Jinwoo had never heard of an artifact that increased your offensive power by a 100% . If something like this was found, it would’ve made waves across the entire world . There was no way someone had kept something like that hidden .
And here he was, holding an item that doubled one’s magic damage .
‘Still, it’s useless to me . ’
An item that simply increased magic damage did nothing for him . After all, the shadow skills technically did not do damage on their own . His offensive skills weren’t even magic . If only an item that increased intelligence appeared .
‘Tsk . ’
While he tossed the marble up and down disappointedly, the Shadow Soldiers had finished the trash mobs and gathered before him . The fastest among them was Ygritte .
Three in his right hand . Three in his left hand .
The knight walked up to Jinwoo and placed six demons’ heads in front of the Hunter .
“ . ”
Jinwoo scratched his temple while looking at the kneeling knight .
“Could you please stop doing that?”
Of course, the respectfully bowed head of the knight did not reply .
“Because when you do that…”
Jinwoo was not sure when the other knight managed to do it, but Iron had cut off Volkhan’s head and dragged it over to his master as well .
“ . He also does it . ”
‘I mean, I’m the one who took it down, right?’
Jinwoo fixed his gaze on Iron and sighed deeply . Looking at the kneeling giant knight, an interesting idea occurred to him .
‘What if I can give the Marble of Avarice to one of them like how I was able to use Iron’s shield?’
Right on cue, the Shadow Mages were slowly approaching Jinwoo behind the infantry . Compared to the knights or even the infantry, the mages were rather slow . Jinwoo called over the nearest one to him . When he gestured with his hand, the indicated mage slowly walked over to his master .
‘ . Maybe I should go to him . ’
Jinwoo shook his head then quickly walked over to the mage . He placed the Marble of Avarice onto the mage’s hand, then pointed to the building he had used to jump towards Volkhan . As the Shadow Soldiers moved according to his will, he did not particularly need to explain the details .
Jinwoo gave the signal,
“Fire . ”
With it, a fireball twice the size of what he was used to seeing appeared in the mage’s hands .
‘Whoa!’
Jinwoo’s eyes widened . Ignoring his shock, the fireball flew from the mage’s hands .
WOOOOOSH-!
BAAAAAANG!
“Wha-!”
Jinwoo’s mouth opened wide . The building was swallowed by the explosion and blew apart like a toy house . The fire consuming the building did not look like it could be put out easily .
Flaaaare!
Watching the fireworks caused by the mage’s spell, Jinwoo quickly took back the Marble of Avarice . He pored over the item information, looking for a specific phrase,
‘It’s not there . It’s really not there . ’
No matter how much he looked up and down on the description, the item did not say that it was untradable . Did that mean he could sell this?
gulp
Jinwoo looked back at the burning building .
flaaare!
Even few parts of the building that remained standing fell eventually with a crashing sound . Jinwoo smiled at the destruction .
‘This thing…’
It’ll drive the Mage-class Hunters crazy . His disappointment was nowhere to be seen, replaced by an anticipation .
A bright afternoon .
“The young man in 902?”
A middle-aged woman thought for a moment then replied,
“Hmm, I’m not sure . Haven’t seen him around lately . ”
“Ah, okay . Thank you . ”
Hyun Gicheol bowed his head . It had been four days since the deputy had been waiting for Jinwoo near his house, but the man had not ran into the Hunter . He had been making vegetable juice nonstop . With a sigh, he took out his phone .
ring~ ring~
Someone picked up .
- “Oy, Gicheol . ”
“Chief . ”
It was Chief Ahn Sangmin .
“I haven’t seen Hunter Sung Jinwoo-nim . The neighbors also said they haven’t seen him recently . ”
- “Really?”
“Yes, sir . ”
- “Well, can’t be helped . Start coming to the office tomorrow, then . ”
“Yes, sir . Understood . ”
click
Ahn Sangmin hung up and tilted his head . Where did Hunter Sung Jinwoo go? The man fell off the grid few days ago . Thinking that he might be purposely avoiding their calls, Hyun Gicheol was sent to the Hunter’s residence but could not find him . The Second Management Division Chief began to worry if something had gone wrong .
‘Well, not sure who’s worrying about whom . ’
Still, the man felt restless .
tap tap
Someone tapped him on the shoulder . Turning around, he saw Hunter Park Heejin behind to him .
“According to his little sister, he’s gone on a trip with a friend for about a week . ”
“Hunter Sung Jinwoo-nim?”
“Yes . ”
“If that’s the case… I don’t think the destination is on Earth . ”
“Huh?”
Park Heejin was taken aback . Ahn Sangmin quickly explained,
“There’s no record of him leaving the country . And there are also no records of him pulling funds or using his card from the bank . ”
“You can obtain that kind of information?”
“Well, it’s our job to keep an eye on Hunters . ”
“Oh my . ”
“On top of that, according to the Association’s tracking records, the last place he was noted in was in the middle of the city, five days ago . Isn’t it all too mysterious?”
“Do you think he got kidnapped?”
 The pair met each other’s eyes and laughed .
Just who was Sung Jinwoo? He was the man who solo cleared a high-level Dungeon, a Red Gate at that . Ahn Sangmin scratched his cheek and spoke,
“If you want to nab Hunter Sung Jinwoo-nim, you’ll need to call in that Chinese special forces unit . The one solely comprised of Hunters . ”
Park Heejin nodded in agreement . Stifling his laughter, Ahn Sangmin became curious and asked a question,
“By the way, how did you find out that he was going on a trip?”
“Oh, it was Han Songyi . We still keep in touch . She goes to the same school as Sung Jinwoo-ssi’s little sister, so I asked her to find out . ”
“Aha…”
With that, the pair’s conversation was over . But they were thinking the same thing,
‘Just where did the man go?’
.
Translator’s Notes:
[1]: Lots of item names
1 . “Monarch of Demons’ Earrings” (악마 군주의 귀고리): Pretty literal . The word used for “monarch” is the same words as “Monarch of Shadows” . “Demon” 악마 + “monarch/king/lord” 군주 + “earrings” 귀고리 . One thing to note here, just like “Shadow Monarch vs . Monarch of Shadows”, 악마군주 is more closely read “Demon Monarch or Demonic Monarch” . I am making a purposeful style choice to flip the words .
2 . “Marble of Avarice” (탐욕의 구슬): Pretty literal . Same word “Avarice” as Volkhan’s title . “greed/avarice” 탐욕 + “marble” 구슬
3 . “Fragment of the World Tree” (세계수의 파편): Also literal . “world tree” 세계수 + “fragment” 파편 . “World tree” can also refer to Yggdrasil from nordic mythology, but I don’t know if it fit the context of the story .
[2]: Original korean says the “Accessory Series of the Monarch of Demons” (악마 군주 장신구 시리즈) . I styled it up a bit .


XRD patterns of the four samples prepared in the second batch of experiments. All of the patterns have been spaced 400 units to facilitate their visualization and comparison. The different markers indicate the 2θ positions of the main peaks of the three phases.


{% extends "base.html" %} {% block title %}Desk Check Status {{ sprint }}{% endblock %} {% block content %}
| {{ statistics.total.count_sp }} Story Points | {{ statistics.total.count }} Stories | ||||||
|---|---|---|---|---|---|---|---|
| Passed | Ready | Failed | To Do | Passed | Ready | Failed | To Do |
| {{ statistics.passed.count_sp }} | {{ statistics.ready.count_sp }} | {{ statistics.failed.count_sp }} | {{ statistics.other.count_sp }} | {{ statistics.passed.count }} | {{ statistics.ready.count }} | {{ statistics.failed.count }} | {{ statistics.other.count }} |
| {{ statistics.passed.percent_sp }}% | {{ statistics.ready.percent_sp }}% | {{ statistics.failed.percent_sp }}% | {{ statistics.other.percent_sp }}% | {{ statistics.passed.percent }}% | {{ statistics.ready.percent }}% | {{ statistics.failed.percent }}% | {{ statistics.other.percent }}% |


Since many potential pandas users have some familiarity with SQL, this page is meant to provide some examples of how various SQL operations would be performed using pandas.
If you’re new to pandas, you might want to first read through 10 Minutes to pandas to familiarize yourself with the library.
As is customary, we import pandas and NumPy as follows:
In [1]: import pandas as pd In [2]: import numpy as np
Most of the examples will utilize the tips dataset found within pandas tests. We’ll read the data into a DataFrame called tips and assume we have a database table of the same name and structure.
tips
In [3]: url = ('https://raw.github.com/pandas-dev' ...: '/pandas/master/pandas/tests/data/tips.csv') ...: In [4]: tips = pd.read_csv(url) In [5]: tips.head() Out[5]: total_bill tip sex smoker day time size 0 16.99 1.01 Female No Sun Dinner 2 1 10.34 1.66 Male No Sun Dinner 3 2 21.01 3.50 Male No Sun Dinner 3 3 23.68 3.31 Male No Sun Dinner 2 4 24.59 3.61 Female No Sun Dinner 4
In SQL, selection is done using a comma-separated list of columns you’d like to select (or a * to select all columns):
*
SELECT total_bill, tip, smoker, time FROM tips LIMIT 5;
With pandas, column selection is done by passing a list of column names to your DataFrame:
In [6]: tips[['total_bill', 'tip', 'smoker', 'time']].head(5) Out[6]: total_bill tip smoker time 0 16.99 1.01 No Dinner 1 10.34 1.66 No Dinner 2 21.01 3.50 No Dinner 3 23.68 3.31 No Dinner 4 24.59 3.61 No Dinner
Calling the DataFrame without the list of column names would display all columns (akin to SQL’s *).
In SQL, you can add a calculated column:
SELECT *, tip/total_bill as tip_rate FROM tips LIMIT 5;
With pandas, you can use the DataFrame.assign() method of a DataFrame to append a new column:
DataFrame.assign()
In [7]: tips.assign(tip_rate=tips['tip'] / tips['total_bill']).head(5) Out[7]: total_bill tip sex smoker day time size tip_rate 0 16.99 1.01 Female No Sun Dinner 2 0.059447 1 10.34 1.66 Male No Sun Dinner 3 0.160542 2 21.01 3.50 Male No Sun Dinner 3 0.166587 3 23.68 3.31 Male No Sun Dinner 2 0.139780 4 24.59 3.61 Female No Sun Dinner 4 0.146808
Filtering in SQL is done via a WHERE clause.
SELECT * FROM tips WHERE time = 'Dinner' LIMIT 5;
DataFrames can be filtered in multiple ways; the most intuitive of which is using boolean indexing.
In [8]: tips[tips['time'] == 'Dinner'].head(5) Out[8]: total_bill tip sex smoker day time size 0 16.99 1.01 Female No Sun Dinner 2 1 10.34 1.66 Male No Sun Dinner 3 2 21.01 3.50 Male No Sun Dinner 3 3 23.68 3.31 Male No Sun Dinner 2 4 24.59 3.61 Female No Sun Dinner 4
The above statement is simply passing a Series of True/False objects to the DataFrame, returning all rows with True.
Series
In [9]: is_dinner = tips['time'] == 'Dinner' In [10]: is_dinner.value_counts() Out[10]: True 176 False 68 Name: time, dtype: int64 In [11]: tips[is_dinner].head(5) Out[11]: total_bill tip sex smoker day time size 0 16.99 1.01 Female No Sun Dinner 2 1 10.34 1.66 Male No Sun Dinner 3 2 21.01 3.50 Male No Sun Dinner 3 3 23.68 3.31 Male No Sun Dinner 2 4 24.59 3.61 Female No Sun Dinner 4
Just like SQL’s OR and AND, multiple conditions can be passed to a DataFrame using | (OR) and & (AND).
-- tips of more than $5.00 at Dinner meals SELECT * FROM tips WHERE time = 'Dinner' AND tip > 5.00;
# tips of more than $5.00 at Dinner meals In [12]: tips[(tips['time'] == 'Dinner') & (tips['tip'] > 5.00)] Out[12]: total_bill tip sex smoker day time size 23 39.42 7.58 Male No Sat Dinner 4 44 30.40 5.60 Male No Sun Dinner 4 47 32.40 6.00 Male No Sun Dinner 4 52 34.81 5.20 Female No Sun Dinner 4 59 48.27 6.73 Male No Sat Dinner 4 116 29.93 5.07 Male No Sun Dinner 4 155 29.85 5.14 Female No Sun Dinner 5 170 50.81 10.00 Male Yes Sat Dinner 3 172 7.25 5.15 Male Yes Sun Dinner 2 181 23.33 5.65 Male Yes Sun Dinner 2 183 23.17 6.50 Male Yes Sun Dinner 4 211 25.89 5.16 Male Yes Sat Dinner 4 212 48.33 9.00 Male No Sat Dinner 4 214 28.17 6.50 Female Yes Sat Dinner 3 239 29.03 5.92 Male No Sat Dinner 3
-- tips by parties of at least 5 diners OR bill total was more than $45 SELECT * FROM tips WHERE size >= 5 OR total_bill > 45;
# tips by parties of at least 5 diners OR bill total was more than $45 In [13]: tips[(tips['size'] >= 5) | (tips['total_bill'] > 45)] Out[13]: total_bill tip sex smoker day time size 59 48.27 6.73 Male No Sat Dinner 4 125 29.80 4.20 Female No Thur Lunch 6 141 34.30 6.70 Male No Thur Lunch 6 142 41.19 5.00 Male No Thur Lunch 5 143 27.05 5.00 Female No Thur Lunch 6 155 29.85 5.14 Female No Sun Dinner 5 156 48.17 5.00 Male No Sun Dinner 6 170 50.81 10.00 Male Yes Sat Dinner 3 182 45.35 3.50 Male Yes Sun Dinner 3 185 20.69 5.00 Male No Sun Dinner 5 187 30.46 2.00 Male Yes Sun Dinner 5 212 48.33 9.00 Male No Sat Dinner 4 216 28.15 3.00 Male Yes Sat Dinner 5
NULL checking is done using the notna() and isna() methods.
notna()
isna()
In [14]: frame = pd.DataFrame({'col1': ['A', 'B', np.NaN, 'C', 'D'], ....: 'col2': ['F', np.NaN, 'G', 'H', 'I']}) ....: In [15]: frame Out[15]: col1 col2 0 A F 1 B NaN 2 NaN G 3 C H 4 D I
Assume we have a table of the same structure as our DataFrame above. We can see only the records where col2 IS NULL with the following query:
col2
SELECT * FROM frame WHERE col2 IS NULL;
In [16]: frame[frame['col2'].isna()] Out[16]: col1 col2 1 B NaN
Getting items where col1 IS NOT NULL can be done with notna().
col1
SELECT * FROM frame WHERE col1 IS NOT NULL;
In [17]: frame[frame['col1'].notna()] Out[17]: col1 col2 0 A F 1 B NaN 3 C H 4 D I
In pandas, SQL’s GROUP BY operations are performed using the similarly named groupby() method. groupby() typically refers to a process where we’d like to split a dataset into groups, apply some function (typically aggregation) , and then combine the groups together.
groupby()
A common SQL operation would be getting the count of records in each group throughout a dataset. For instance, a query getting us the number of tips left by sex:
SELECT sex, count(*) FROM tips GROUP BY sex; /* Female 87 Male 157 */
The pandas equivalent would be:
In [18]: tips.groupby('sex').size() Out[18]: sex Female 87 Male 157 dtype: int64
Notice that in the pandas code we used size() and not count(). This is because count() applies the function to each column, returning the number of not null records within each.
size()
count()
not null
In [19]: tips.groupby('sex').count() Out[19]: total_bill tip smoker day time size sex Female 87 87 87 87 87 87 Male 157 157 157 157 157 157
Alternatively, we could have applied the count() method to an individual column:
In [20]: tips.groupby('sex')['total_bill'].count() Out[20]: sex Female 87 Male 157 Name: total_bill, dtype: int64
Multiple functions can also be applied at once. For instance, say we’d like to see how tip amount differs by day of the week - agg() allows you to pass a dictionary to your grouped DataFrame, indicating which functions to apply to specific columns.
agg()
SELECT day, AVG(tip), COUNT(*) FROM tips GROUP BY day; /* Fri 2.734737 19 Sat 2.993103 87 Sun 3.255132 76 Thur 2.771452 62 */
In [21]: tips.groupby('day').agg({'tip': np.mean, 'day': np.size}) Out[21]: tip day day Fri 2.734737 19 Sat 2.993103 87 Sun 3.255132 76 Thur 2.771452 62
Grouping by more than one column is done by passing a list of columns to the groupby() method.
SELECT smoker, day, COUNT(*), AVG(tip) FROM tips GROUP BY smoker, day; /* smoker day No Fri 4 2.812500 Sat 45 3.102889 Sun 57 3.167895 Thur 45 2.673778 Yes Fri 15 2.714000 Sat 42 2.875476 Sun 19 3.516842 Thur 17 3.030000 */
In [22]: tips.groupby(['smoker', 'day']).agg({'tip': [np.size, np.mean]}) Out[22]: tip size mean smoker day No Fri 4.0 2.812500 Sat 45.0 3.102889 Sun 57.0 3.167895 Thur 45.0 2.673778 Yes Fri 15.0 2.714000 Sat 42.0 2.875476 Sun 19.0 3.516842 Thur 17.0 3.030000
JOINs can be performed with join() or merge(). By default, join() will join the DataFrames on their indices. Each method has parameters allowing you to specify the type of join to perform (LEFT, RIGHT, INNER, FULL) or the columns to join on (column names or indices).
join()
merge()
In [23]: df1 = pd.DataFrame({'key': ['A', 'B', 'C', 'D'], ....: 'value': np.random.randn(4)}) ....: In [24]: df2 = pd.DataFrame({'key': ['B', 'D', 'D', 'E'], ....: 'value': np.random.randn(4)}) ....:
Assume we have two database tables of the same name and structure as our DataFrames.
Now let’s go over the various types of JOINs.
SELECT * FROM df1 INNER JOIN df2 ON df1.key = df2.key;
# merge performs an INNER JOIN by default In [25]: pd.merge(df1, df2, on='key') Out[25]: key value_x value_y 0 B -0.282863 1.212112 1 D -1.135632 -0.173215 2 D -1.135632 0.119209
merge() also offers parameters for cases when you’d like to join one DataFrame’s column with another DataFrame’s index.
In [26]: indexed_df2 = df2.set_index('key') In [27]: pd.merge(df1, indexed_df2, left_on='key', right_index=True) Out[27]: key value_x value_y 1 B -0.282863 1.212112 3 D -1.135632 -0.173215 3 D -1.135632 0.119209
-- show all records from df1 SELECT * FROM df1 LEFT OUTER JOIN df2 ON df1.key = df2.key;
# show all records from df1 In [28]: pd.merge(df1, df2, on='key', how='left') Out[28]: key value_x value_y 0 A 0.469112 NaN 1 B -0.282863 1.212112 2 C -1.509059 NaN 3 D -1.135632 -0.173215 4 D -1.135632 0.119209
-- show all records from df2 SELECT * FROM df1 RIGHT OUTER JOIN df2 ON df1.key = df2.key;
# show all records from df2 In [29]: pd.merge(df1, df2, on='key', how='right') Out[29]: key value_x value_y 0 B -0.282863 1.212112 1 D -1.135632 -0.173215 2 D -1.135632 0.119209 3 E NaN -1.044236
pandas also allows for FULL JOINs, which display both sides of the dataset, whether or not the joined columns find a match. As of writing, FULL JOINs are not supported in all RDBMS (MySQL).
-- show all records from both tables SELECT * FROM df1 FULL OUTER JOIN df2 ON df1.key = df2.key;
# show all records from both frames In [30]: pd.merge(df1, df2, on='key', how='outer') Out[30]: key value_x value_y 0 A 0.469112 NaN 1 B -0.282863 1.212112 2 C -1.509059 NaN 3 D -1.135632 -0.173215 4 D -1.135632 0.119209 5 E NaN -1.044236
UNION ALL can be performed using concat().
concat()
In [31]: df1 = pd.DataFrame({'city': ['Chicago', 'San Francisco', 'New York City'], ....: 'rank': range(1, 4)}) ....: In [32]: df2 = pd.DataFrame({'city': ['Chicago', 'Boston', 'Los Angeles'], ....: 'rank': [1, 4, 5]}) ....:
SELECT city, rank FROM df1 UNION ALL SELECT city, rank FROM df2; /* city rank Chicago 1 San Francisco 2 New York City 3 Chicago 1 Boston 4 Los Angeles 5 */
In [33]: pd.concat([df1, df2]) Out[33]: city rank 0 Chicago 1 1 San Francisco 2 2 New York City 3 0 Chicago 1 1 Boston 4 2 Los Angeles 5
SQL’s UNION is similar to UNION ALL, however UNION will remove duplicate rows.
SELECT city, rank FROM df1 UNION SELECT city, rank FROM df2; -- notice that there is only one Chicago record this time /* city rank Chicago 1 San Francisco 2 New York City 3 Boston 4 Los Angeles 5 */
In pandas, you can use concat() in conjunction with drop_duplicates().
drop_duplicates()
In [34]: pd.concat([df1, df2]).drop_duplicates() Out[34]: city rank 0 Chicago 1 1 San Francisco 2 2 New York City 3 1 Boston 4 2 Los Angeles 5
-- MySQL SELECT * FROM tips ORDER BY tip DESC LIMIT 10 OFFSET 5;
In [35]: tips.nlargest(10 + 5, columns='tip').tail(10) Out[35]: total_bill tip sex smoker day time size 183 23.17 6.50 Male Yes Sun Dinner 4 214 28.17 6.50 Female Yes Sat Dinner 3 47 32.40 6.00 Male No Sun Dinner 4 239 29.03 5.92 Male No Sat Dinner 3 88 24.71 5.85 Male No Thur Lunch 2 181 23.33 5.65 Male Yes Sun Dinner 2 44 30.40 5.60 Male No Sun Dinner 4 52 34.81 5.20 Female No Sun Dinner 4 85 34.83 5.17 Female No Thur Lunch 4 211 25.89 5.16 Male Yes Sat Dinner 4
-- Oracle's ROW_NUMBER() analytic function SELECT * FROM ( SELECT t.*, ROW_NUMBER() OVER(PARTITION BY day ORDER BY total_bill DESC) AS rn FROM tips t ) WHERE rn < 3 ORDER BY day, rn;
In [36]: (tips.assign(rn=tips.sort_values(['total_bill'], ascending=False) ....: .groupby(['day']) ....: .cumcount() + 1) ....: .query('rn < 3') ....: .sort_values(['day', 'rn'])) ....: Out[36]: total_bill tip sex smoker day time size rn 95 40.17 4.73 Male Yes Fri Dinner 4 1 90 28.97 3.00 Male Yes Fri Dinner 2 2 170 50.81 10.00 Male Yes Sat Dinner 3 1 212 48.33 9.00 Male No Sat Dinner 4 2 156 48.17 5.00 Male No Sun Dinner 6 1 182 45.35 3.50 Male Yes Sun Dinner 3 2 197 43.11 5.00 Female Yes Thur Lunch 4 1 142 41.19 5.00 Male No Thur Lunch 5 2
the same using rank(method=’first’) function
In [37]: (tips.assign(rnk=tips.groupby(['day'])['total_bill'] ....: .rank(method='first', ascending=False)) ....: .query('rnk < 3') ....: .sort_values(['day', 'rnk'])) ....: Out[37]: total_bill tip sex smoker day time size rnk 95 40.17 4.73 Male Yes Fri Dinner 4 1.0 90 28.97 3.00 Male Yes Fri Dinner 2 2.0 170 50.81 10.00 Male Yes Sat Dinner 3 1.0 212 48.33 9.00 Male No Sat Dinner 4 2.0 156 48.17 5.00 Male No Sun Dinner 6 1.0 182 45.35 3.50 Male Yes Sun Dinner 3 2.0 197 43.11 5.00 Female Yes Thur Lunch 4 1.0 142 41.19 5.00 Male No Thur Lunch 5 2.0
-- Oracle's RANK() analytic function SELECT * FROM ( SELECT t.*, RANK() OVER(PARTITION BY sex ORDER BY tip) AS rnk FROM tips t WHERE tip < 2 ) WHERE rnk < 3 ORDER BY sex, rnk;
Let’s find tips with (rank < 3) per gender group for (tips < 2). Notice that when using rank(method='min') function rnk_min remains the same for the same tip (as Oracle’s RANK() function)
rank(method='min')
In [38]: (tips[tips['tip'] < 2] ....: .assign(rnk_min=tips.groupby(['sex'])['tip'] ....: .rank(method='min')) ....: .query('rnk_min < 3') ....: .sort_values(['sex', 'rnk_min'])) ....: Out[38]: total_bill tip sex smoker day time size rnk_min 67 3.07 1.00 Female Yes Sat Dinner 1 1.0 92 5.75 1.00 Female Yes Fri Dinner 2 1.0 111 7.25 1.00 Female No Sat Dinner 1 1.0 236 12.60 1.00 Male Yes Sat Dinner 2 1.0 237 32.83 1.17 Male Yes Sat Dinner 2 2.0
UPDATE tips SET tip = tip*2 WHERE tip < 2;
In [39]: tips.loc[tips['tip'] < 2, 'tip'] *= 2
DELETE FROM tips WHERE tip > 9;
In pandas we select the rows that should remain, instead of deleting them
In [40]: tips = tips.loc[tips['tip'] <= 9]


There <% PL_V( 'was', $count ) %> <% $count %> new <% PL( 'entry', $count ) %> in the regions on your watch list over the past seven days.
If you want to remove regions from your watch list, you can edit it on your watch list page.
<%args> $vendors1 $user %args> <%init> my @vendors = $vendors1->all(); my $count = scalar @vendors; %init>


--- title: isalmostreal sidebar: function_reference_sidebar permalink: isalmostreal.html folder: function_reference toc: false ---
Check whether an array A is nearly real valued or not. Therefore an array with very small imaginary part is also seen as real valued.
| A | double |
| out | logical |
| precision | exponential treshold |
| norm | is used to compare the real and imaginary parts of A (default 1) |


Simple, transparent pricing
Only pay if we increase your revenue
Savy Lite
Made for merchants who are just getting started and want to collect data on how to best price their products.
Free
Savy Lite features
Customer emails
Price point data
24/7 support
.myshopify.com
Savy Sales
A powerful automated replacement for traditional clearance sales. Your success is our success - only pay if we increase your revenue.
Pay As You Go
40% of the difference between purchase price and merchant minimum for each Savy Sale product boughtSee an example
Savy Sales features
Customer emails
Price point data
24/7 support
Pay only for what you use
No setup or hidden fees
Whitelabeling
Real-time reporting
Automation
.myshopify.com
Different business model?
Do you want to implement flat rate pricing, have unusually large average order values, or operate with high refund rates? Get in touch to discuss custom pricing.
 Savy Sales is completely risk free to use, we don't have any setup or hidden fees, and if we don't make you money, you don't pay!
Savy gets a 40% cut of the revenue we add to your business. For example...
Savy gets a 40% cut of the revenue we add to your business. For example...
$ 60.00
-+
Let's say you've included this dress in your Savy Sale. In a regular clearance sale, you would have marked it down to $35, so you've set your merchant minimum at $35.
$ 50.00 $ 60.00
-+
A customer enters $50 for the dress and her price is accepted.
$ 50.00 $ 60.00
$ 50.00 $ 60.00
-+
She goes on to purchase the dress at $50. At this point, Savy would recieve 40% of the difference between purchase price and merchant minimum - so in this example we would be paid $6.
What forms of payment do you accept?
+
You can purchase Savy with any major credit card. Additionally, we can issue an invoice payable by bank transfer or check. Please contact us to arrange an invoice purchase.
How secure is Savy?
+
Protecting the data you trust to Savy is our first priority. Savy uses physical, procedural, and technical safeguards to preserve the integrity and security of your information. We regularly back up your data to prevent data loss and aid in recovery. Additionally, we host data in secure SSAE 16 / SOC1 certified data centers, implement firewalls and access restrictions on our servers to better protect your information, and work with third party security researchers to ensure our practices are secure.
Will I be charged sales tax?
+
Savy is not required to, and does not, collect your state’s sales or use tax. Purchases are not tax-exempt merely because a seller is not required to collect the tax. Your state’s law requires a purchaser to review untaxed purchases and, if any tax is owed, file a use tax return and pay any tax due.
How do refunds work?
+
If a Savy Sale item is returned, Savy will refund the 40% cut that would have been paid to us for said transaction.


{% include meta.html %}
{% include nav.html %}
Harbor's Features
Harbor increases your plan participation and assets while decreasing your operational burdens. Together, we can scale new markets.
Explore Below
{% include features.html %} {% include modules.html %} {% include footer.html %} {% include scripts.html %}


--- layout: post title: "Frequentism vs Bayesianism" date: 2016-04-11 comments: true archive: true notes: true tags: notes excerpt: Thoughts regarding the subtle differences between Frequentism and Bayesianism. ---
The disagreement between Frequentism and Bayesianism concerns the definition of probability.
Frequentism interprets probability as the limiting case of repeated measurements.
probabilites = fundamentally related frequency of any given value
Bayesianism interprets probability as degrees of certainty about statements.
probability = fundamentally related to our knowledge about an event
probability = statement of my knowledge of what the measurement result will be
Example: Given a measurements of a photon flux \(F\) from a given star estimate the true flux of the star.
| Different views | ||||
|---|---|---|---|---|
| Frequentist view: | Since by definition the probability of the true flux of a star is a fixed value then it is meaningless for the frequentist to talk about the probability. Talking about frequency distribution for a fixed value is nonsense. | |||
| Bayesian view: | Claims to measure the flux \(F\) of a star with some probability \(P(F)\). Although the probability can be estimated from frequencies of repeated experiments it is not fundamental. | |||
| Can meaningfully talk about the probability that the true flux of a star lies in a given range | ||||
| Probability codifies our knowledge of the value based on prior information and/or available data | ||||
| Philosophy difference leads to | different approaches to statistical analysis of data |
An example illustrating the different approaches:
Notation:
\(N\) = measurements
\(i^{th}\) = observation
\(F_{i}\) = photon flux
\(e_{i}\) = error related to \(F_{i}\)
Assumptions regarding \(e_{i}\): \(e_{i}\; \sim\; \cal{N}(\mu, \sigma^{2})\)
Frequentist:
\(e_{i} = \frac{\sigma}{std}\) of the results of a single measurement event w.r.t. the limit of the repetitions of that event.
Bayesian:
\(e_{i} = \frac{\sigma}{std}\) of (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.
Given \(D = \{F_{i}, e_{i}\}\) find the best estimate of the true Ftrue?
Since \(N \in \mathbb{N}\) thus a Poisson distribution is a good approximation to the measurement process.
\(F \;\sim\; \text{Pois}(\lambda_{i}), i = 1,\dots,n\) \(F \;\sim\; \text{Pois}(\lambda)\)
\(\{N_{i}, e_{i}\}\) is estimated from Poisson statistics using the standard square-root rule.
The true \(F_{true}\) flux is already known: \(F_{true} = 1000\), numbers of photons measured in 1 sec.
Frequentist approach:
Start classic maximum likelihood! Given \(D_{i} = \{F_{i}, e_{i}\}\), compute the probability distribution of the measurement given the true \(F_{true}\) flux, given also the assumption of Gaussian errors.
\[P(D_{i} | F_{true}) = \frac{1}{\sqrt{2\pi e^{2}_{i}}}\text{exp}[-\frac{(F_{i} - F_{true})^{2}}{2e^{2}_{i}}]\]
Construct likelihood function = product of probabilities for each data point.
\[\cal{L}(D | F_{true}) = \prod_{i=1}^{N} P(D | F_{true})\]
Since \(\cal{L}\) becomes very small to avoid underflow/overflow errors we compute the log-likelihood.
\[\text{log}(\cal{L}) = -\frac{1}{2}\sum^{N}_{i=1}[\text{log}(2\pi e^{2}_{i}) + \frac{(F_{i} - F_{true})^2}{e^{2}_{i}}]\]
Determine \(F_{true}\) susch that the likelihood is maximized.
Computed analytically by setting \([\frac{d\text{log}\cal{L}}{dF_{true}} = 0]\)
\(F_{est.} = \frac{\sum w_{i}F_{i}}{\sum w_{i}};\;\; w_{i} = \frac{1}{e^{2}_{i}}\;\; \text{if}\;\forall e_{i}\; \text{being equal}\)
this reduces to \(F_{est.} = \frac{1}{N}\sum_{i=1}^{N} F_{i} \rightarrow\; \text{simply the mean of the observed data when errors are equal}\)
What is the error of \(F_{est.}\)? Identifying the error in the two different approaches.
Accomplished by fitting a Gaussian approximation to the likelihood curve at the maximum.
For the simple case we can solve it analytically.
\(\sigma_{est.} = (\sum_{i=1}^{N} w_{i})^{-\frac{1}{2}}\rightarrow\; \text{std. of Gaussian approximation}\)
2. Bayesian approach:
Begins and ends with probabilities. We want to compute our knowledge of the parameters in question, i.e. \(P(F_{true}|D)\rightarrow\;\text{Bayesian} \neq P(D | F_{true})\rightarrow\;\text{Frequentist}\)
Formulation of the problem is fundamentally contrary to the frequentist philosophy.
It says that the probabilities have no meaning for model parameters like \(F_{true}\).
To compute $P(Ftrue | D) they apply Bayes Rule.$
\[P(F_{true} | D) = \frac{P(D | F_{true}) P(F_{true})}{P(D)}\]
What is controversial is not the Bayes law but instead the Bayesian interpretation of probability by the term \(P(F_{true} | D)\).
Setting \(P(F_{true})\;\propto\) 1 (a flat prior), we find \(P(F_{true} | D)\propto\;\cal{L}(D | F_{true})\). For a flat prior the Bayesian and the Frequentist become similar to each other.
The prior \(P(F_{true})\) allows inclusion of other information into the computation, useful in cases of combining multiple measurement strategies.
One of the most controversial pieces of Bayesian analysis is to specify the necessity of a prior.
Frequentist will point out that the prior is problematic when no true prior information is available. In many situations a trully non-informative prior doesn't exist.
Frequentists say the choice of prior necessarily biases your results and therefore has no place in statistical data analysis.
Bayesian would say that frequentism can be viewed as a simple case of the Bayesian approach for some (implicit) choice of the prior:
Bayesians argue that it would be better to make this implicit choice explicit, even if the choice might include some subjectivity.
How Bayesian results are computed in practice?
For a one parameter problem compute the posterior probability \(P(F_{true} | D)\) as a function of \(F_{true}\).
In other words compute the distribution reflecting our knowledge of the parameter \(F_{true}\).
The direct approach becomes increasingly intractable as the dimension of the model grows.
Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC).
The Goal is to generate a set of points draw from the posterior probability distribution and use them to determine the answer we seek.
In pure Bayesianism the answer to a question is not a single number with error bars; the answer is the posterior distribution over the model parameters.
Exploring a more sophisticated model: Adding a Dimension
Assume our observing object has some stochastic variation, i.e. it varies with time.
Propose a simple 2-parameter \(\cal{N}(\mu, \sigma), \theta = [\mu, \sigma]\) of the variability intrinsic to the object.
Model: \(F_{true}\;\sim\;\frac{1}{\sqrt{2\pi \sigma^{2}}}\text{exp}[\frac{(F - \mu)^{2}}{2\sigma^{2}}]\)
Frequentist approach:
\[\cal{L}(D | \theta) = \prod_{i=1}^{N}\frac{1}{\sqrt{2\pi(\sigma^{2} + e^{2}_{i})}}\text{exp}[-\frac{-(F_{i} - \mu)^{2}}{2(\sigma^{2} + e^{2}_{i})}]\]
Likelihood is the convolution of the intrinsic distribution with the error distribution.
Analytically maximize the above likelihood to find the best estimate for \(\mu\):
\(\mu_{est.} = \frac{\sum w_{i}F_{i}}{\sum w_{i}}\;; w_{i} = \frac{1}{\sigma^{2} + e^{2}_{i}}\)
Here we have a problem: The optimal value of \(\mu\) depends on the optimal value of \(\sigma\).
Results are correlated \(\rightarrow\) no longer possible to use analytic methods to arrive at the Frequentist result.
But we can use numerical optimization techniques to determine the maximum likelihood value.
Maximum likelihood gives best estimate of the parameters \(\mu\) and \(\sigma\) governing our model. This is only half the answer.
We need to compute Error Bars on \(\mu\) and \(\sigma\).
Several approaches to determine errors in a frequentist approach/paradigm.
In bootstrapping or sampling techniques there is a potential for errors to be correlated or even non-Gaussian, neither of which is reflected by simply finding the mean and std. of each model parameter.
Bayesian approach:
Almost exactly the same as it was in the previous problem.
The vast majority of commonly applied frequentist techniques make the explicit or implicit assumption of Gaussianity of the distribution.
Bayesian approaches generally don't require such assumptions
There are good arguements that a prior on \(\sigma\) subtley or subjectively biases the calculation in this case.
Conclusion:
Bayesianism and Frequentism are fundamentally different approaches to simple problems which can yield similar or even identical results.
Differences:
Frequentism considers probabilities related to frequencies of real or hypothetical events.
Bayesianism considers probabilities as measurements of degrees of knowledge
Frequentist analyses generally proceeds through the use of point estimates and maximum likelihood
Baysian analyses generally compute the posterior either directly or through some version of MCMC sampling
In simple problems, the two approaches yield similar results. As data and models grow in complexity the two approaches can diverge greatly. This is clea in 2 situations.
Credits to Jake VanderPlas


Morph-CSV Interface
CSVW Link
CSVW
Choose your CSVW file
Yarrrml Mapping Link
YARRRML
Choose your Yarrrml file
Sparql Query Link
Query
Choose your Sparql Query file
Submit


Login with your social account
...or login with your email
Forgot password?
Create Account
...or create an account
Forgot password?
Login
Forgot Password
Login
Create Account
Complete your Profile
Require to fill all mandatory fields.
Reset your Password


Command Prefix is currently set to % by default. To change a server prefix, look at %setprefix.
In order to check the prefix of your server, type %checkprefix.
In order to pull a photo of an idol, you can type an idol's stage name, full name, alias, their group name, or group alias name after the prefix.
examples: %irene, %blackpink, %red velvet. You may also put an idol's stage name, full name, or alias after a group such as %red velvet irene.
You can use this with many idols to get a random one. %red velvet irene joy seulgi
This way, you will pull any of the idols as long as they are in that group.
To find out more, look at the GroupMembers category.
♥ - Requires Bot Owner
★ - Requires Guild Permissions
😇 - Requires Bot Moderator
♥♥♥ -- Entire Section Requires Bot Owner
★★★ - Entire Section Requires Guild Permissions
😇😇😇 - Entire Section Requires Bot Moderator
This is just general command information. Further information can be found with %help (command).


[gecode-users] another question about stats
Mikael Zayenz Lagerkvist
zayenz at kth.se
Fri Feb 27 15:15:21 CET 2009
Hi,
The time is measured using the clock() POSIX function, which measures
the elapsed CPU-time of the process that calls it. Calling usleep does
not put the process into a busy-loop (and thus incurring CPU-time), it
just delays the scheduling of that process. Compare the reported
runtime with running your program using the time wrapper command to
see the wall-clock time, user time, and kernel time.
If the program is busy (i.e., not calling functions such as usleep)
and is running on an otherwise unloaded machine, wall-clock time and
CPU-time will be roughly equal.
Cheers,
Mikael
On Tue, Feb 24, 2009 at 5:23 PM, David Przybilla <paranoic.pum at gmail.com> wrote:
> I made a small testing sample, I made a simple propagator and some variables
> subscribing to it.
> Inside the propagate method there is a usleep line, which is supposed to
> delay the execution for some seconds.
> When I run the example, the delay is there, however when stats are shown,
> they say the example`s runtime was 0ms.
> what does "runtime" measure? am I doing anything wrong?
> Also, Im making performance comparisons between some programs on Gecode old
> versions and between the same ones on Gecode 2.1.1,
> I wonder whether I should measure the time on my own or just use the
> runtime thrown by gecode`s stats
>
> Thank you in advance!
>
> _______________________________________________
> Gecode users mailing list
> users at gecode.org
> https://www.gecode.org/mailman/listinfo/gecode-users
>
>
--
Mikael Zayenz Lagerkvist, http://www.ict.kth.se/~zayenz/
More information about the gecode-users
mailing list


ATTR_GEN_FILE_SELECTOR_CREATOR_MATCH, MSG_GEN_FILE_SELECTOR_SET_CREATOR, MSG_GEN_FILE_SELECTOR_GET_CREATOR
Most applications will store their documents in a special type of file; each of these files will include the token of the creator geode. (This is a function of the Document Control objects.)
As a result, you may limit the File Selector's searches to only those files created by a given geode. Set the token to be matched in the File Selector's
ATTR_GEN_FILE_SELECTOR_CREATOR_MATCH
variable data instance field. This search is employed in the same manner as a token search (above), so it is also subject to the FSFC_TOKEN_NO_ID attribute of
GFSI_fileCriteria
.
The
ATTR_GEN_FILE_SELECTOR_CREATOR_MATCH
field is a
GeodeToken
structure that contains both the match ID and the match characters of the creator token. The
GeodeToken
structure is shown in the previous section. You can set the creator information in the File Selector's Goc code, or you can set it at run-time with
MSG_GEN_FILE_SELECTOR_SET_CREATOR
. To retrieve the creator token, use
MSG_GEN_FILE_SELECTOR_GET_CREATOR
.
void MSG_GEN_FILE_SELECTOR_SET_CREATOR( dword tokenChars, ManufacturerID manufacturerID);
This message sets the
ATTR_GEN_FILE_SELECTOR_CREATOR_MATCH
field to the two passed values. This field is a vardata instance record of type
GeodeToken
. The token set with this message will be used to filter out certain files; directories and volumes are not affected by this filter. Only files which have their creator token the same as that of the File Selector will be displayed.
If the File Selector is visible when it receives this message, it will rescan the directory using the new token as a filter. Otherwise, the File Selector will store the passed token for later use.
Source: Unrestricted.
Destination: Any GenFileSelector object.
Parameters:
tokenChars
The token characters of the creator token to match files against. Use the macro SET_TOKEN_CHARS to set this dword value from the four individual characters. This macro is shown in the previous section.
manufacturerID
Return: Nothing.
Interception: Generally not intercepted.
See Also:
MSG_GEN_FILE_SELECTOR_SET_FILE_CRITERIA
,
MSG_GEN_FILE_SELECTOR_SET_TOKEN
void MSG_GEN_FILE_SELECTOR_GET_CREATOR( GetTokenCreatorParams *retValue);
This message returns the current creator match information set in the File Selector's
ATTR_GEN_FILE_SELECTOR_CREATOR_MATCH
field.
Source: Unrestricted.
Destination: Any GenFileSelector object.
Parameters:
retValue
A pointer to a
GetTokenCreatorParams
structure, shown below. The structure may be empty when passed.
Return: The
GetTokenCreatorParams
structure pointed to by
retValue
will contain the token in
ATTR_GEN_FILE_SELECTOR_CREATOR_MATCH
.
Structures:
The
GetTokenCreatorParams
structure consists of two elements: a
GeodeToken
structure containing the File Selector's token characters and manufacturer ID fields, and a reserved word. Its structure is shown below:
typedef struct { GeodeToken GTP_token; word GTP_unused; /* reserved */ } GetTokenCreatorParams;
Interception: Generally not intercepted.
See Also:
MSG_GEN_FILE_SELECTOR_SET_FILE_CRITERIA, MSG_GEN_FILE_SELECTOR_GET_TOKEN


Class: Rails::Engine
| Relationships & Source Files | |
| Namespace Children | |
|
Classes:
| |
| Extension / Inclusion / Inheritance Descendants | |
|
Subclasses:
|
|
| Super Chains via Extension / Inclusion / Inheritance | |
|
Class Chain:
self,
Railtie
|
|
|
Instance Chain:
self,
Railtie,
Initializable
|
|
| Inherits: |
Rails::Railtie
|
| Defined in: | railties/lib/rails/engine.rb, railties/lib/rails/engine/configuration.rb, railties/lib/rails/engine/railties.rb |
Overview
Engine
allows you to wrap a specific ::Rails application or subset of functionality and share it with other applications or within a larger packaged application. Since Rails 3.0, every Application is just an engine, which allows for simple feature and application sharing.
Any Engine
is also a Railtie, so the same methods (like rake_tasks
and generators
) and configuration options that are available in railties can also be used in engines.
Creating an Engine
In Rails versions prior to 3.0, your gems automatically behaved as engines, however, this coupled ::Rails to Rubygems. Since Rails 3.0, if you want a gem to automatically behave as an engine, you have to specify an Engine
for it somewhere inside your plugin's lib
folder (similar to how we specify a Railtie):
# lib/my_engine.rb
module MyEngine
class Engine < Rails::Engine
end
end
Then ensure that this file is loaded at the top of your config/application.rb
(or in your Gemfile
) and it will automatically load models, controllers and helpers inside #app, load routes at config/routes.rb
, load locales at config/locales/*
, and load tasks at lib/tasks/*
.
Configuration
Besides the Railtie configuration which is shared across the application, in a Engine
you can access autoload_paths
, eager_load_paths
and autoload_once_paths
, which, differently from a Railtie, are scoped to the current engine.
class MyEngine < Rails::Engine
# Add a load path for this specific Engine
config.autoload_paths << File. ("../lib/some/path", __FILE__)
initializer "my_engine.add_middleware" do |app|
app.middleware.use MyEngine::Middleware
end
end
Generators
You can set up generators for engines with config.generators
method:
class MyEngine < Rails::Engine
config.generators do |g|
g.orm :active_record
g.template_engine :erb
g.test_framework :test_unit
end
end
You can also set generators for an application by using config.app_generators
:
class MyEngine < Rails::Engine
# note that you can also pass block to app_generators in the same way you
# can pass it to generators method
config.app_generators.orm :datamapper
end
Paths
Since Rails 3.0, applications and engines have more flexible path configuration (as opposed to the previous hardcoded path configuration). This means that you are not required to place your controllers at app/controllers
, but in any place which you find convenient.
For example, let's suppose you want to place your controllers in lib/controllers
. You can set that as an option:
class MyEngine < Rails::Engine
paths["app/controllers"] = "lib/controllers"
end
You can also have your controllers loaded from both app/controllers
and lib/controllers
:
class MyEngine < Rails::Engine
paths["app/controllers"] << "lib/controllers"
end
The available paths in an engine are:
class MyEngine < Rails::Engine
paths["app"] # => ["app"]
paths["app/controllers"] # => ["app/controllers"]
paths["app/helpers"] # => ["app/helpers"]
paths["app/models"] # => ["app/models"]
paths["app/views"] # => ["app/views"]
paths["lib"] # => ["lib"]
paths["lib/tasks"] # => ["lib/tasks"]
paths["config"] # => ["config"]
paths["config/initializers"] # => ["config/initializers"]
paths["config/locales"] # => ["config/locales"]
paths["config/routes.rb"] # => ["config/routes.rb"]
end
The Application class adds a couple more paths to this set. And as in your Application, all folders under #app are automatically added to the load path. If you have an app/services
folder for example, it will be added by default.
Endpoint
An engine can also be a rack application. It can be useful if you have a rack application that you would like to wrap with Engine
and provide with some of the Engine
's features.
To do that, use the .endpoint method:
module MyEngine
class Engine < Rails::Engine
endpoint MyRackApplication
end
end
Now you can mount your engine in application's routes just like that:
Rails.application.routes.draw do
mount MyEngine::Engine => "/engine"
end
Middleware stack
As an engine can now be a rack endpoint, it can also have a middleware stack. The usage is exactly the same as in Application:
module MyEngine
class Engine < Rails::Engine
middleware.use SomeMiddleware
end
end
Routes
If you don't specify an endpoint, routes will be used as the default endpoint. You can use them just like you use an application's routes:
# ENGINE/config/routes.rb
MyEngine::Engine.routes.draw do
get "/" => "posts#index"
end
Mount priority
Note that now there can be more than one router in your application, and it's better to avoid passing requests through many routers. Consider this situation:
Rails.application.routes.draw do
mount MyEngine::Engine => "/blog"
get "/blog/omg" => "main#omg"
end
MyEngine
is mounted at /blog
, and /blog/omg
points to application's controller. In such a situation, requests to /blog/omg
will go through MyEngine
, and if there is no such route in Engine
's routes, it will be dispatched to main#omg
. It's much better to swap that:
Rails.application.routes.draw do
get "/blog/omg" => "main#omg"
mount MyEngine::Engine => "/blog"
end
Now, Engine
will get only requests that were not handled by Application.
Engine name
There are some places where an Engine's name is used:
-
routes: when you mount an
Engine
withmount(MyEngine::Engine => '/my_engine')
, it's used as default:as
option -
rake task for installing migrations
my_engine:install:migrations
Engine
name is set by default based on class name. For MyEngine::Engine
it will be my_engine_engine
. You can change it manually using the .engine_name method:
module MyEngine
class Engine < Rails::Engine
engine_name "my_engine"
end
end
Isolated Engine
Normally when you create controllers, helpers and models inside an engine, they are treated as if they were created inside the application itself. This means that all helpers and named routes from the application will be available to your engine's controllers as well.
However, sometimes you want to isolate your engine from the application, especially if your engine has its own router. To do that, you simply need to call .isolate_namespace. This method requires you to pass a module where all your controllers, helpers and models should be nested to:
module MyEngine
class Engine < Rails::Engine
isolate_namespace MyEngine
end
end
With such an engine, everything that is inside the MyEngine
module will be isolated from the application.
Consider such controller:
module MyEngine
class FooController < ActionController::Base
end
end
If an engine is marked as isolated, FooController
has access only to helpers from Engine
and url_helpers
from MyEngine::Engine.routes
.
The next thing that changes in isolated engines is the behavior of routes.  With an isolated engine, the namespace is applied by default, so you can ignore it in routes:
MyEngine::Engine.routes.draw do
resources :articles
end
The routes above will automatically point to MyEngine::ArticlesController
. Furthermore, you don't need to use longer url helpers like my_engine_articles_path
. Instead, you should simply use articles_path
as you would do with your application.
To make that behavior consistent with other parts of the framework, an isolated engine also has influence on ::ActiveModel::Naming. When you use a namespaced model, like MyEngine::Article
, it will normally use the prefix “my_engine”. In an isolated engine, the prefix will be omitted in url helpers and form fields for convenience.
polymorphic_url(MyEngine::Article.new) # => "articles_path"
form_for(MyEngine::Article.new) do
text_field :title # => <input type="text" name="article[title]" id="article_title" />
end
Additionally, an isolated engine will set its name according to namespace, so MyEngine::Engine.engine_name
will be “my_engine”. It will also set MyEngine.table_name_prefix
to “my_engine_”, changing the MyEngine::Article
model to use the my_engine_articles table.
Using Engine's routes outside Engine
Since you can now mount an engine inside application's routes, you do not have direct access to Engine
's url_helpers
inside Application. When you mount an engine in an application's routes, a special helper is created to allow you to do that. Consider such a scenario:
# config/routes.rb
Rails.application.routes.draw do
mount MyEngine::Engine => "/my_engine", as: "my_engine"
get "/foo" => "foo#index"
end
Now, you can use the my_engine
helper inside your application:
class FooController < ApplicationController
def index
my_engine.root_url # => /my_engine/
end
end
There is also a main_app
helper that gives you access to application's routes inside Engine
:
module MyEngine
class BarController
def index
main_app.foo_path # => /foo
end
end
end
Note that the :as
option given to mount takes the .engine_name as default, so most of the time you can simply omit it.
Finally, if you want to generate a url to an engine's route using polymorphic_url
, you also need to pass the engine helper. Let's say that you want to create a form pointing to one of the engine's routes. All you need to do is pass the helper as the first element in array with attributes for url:
form_for([my_engine, @user])
This code will use my_engine.user_path(@user)
to generate the proper route.
Isolated engine's helpers
Sometimes you may want to isolate engine, but use helpers that are defined for it. If you want to share just a few specific helpers you can add them to application's helpers in ApplicationController:
class ApplicationController < ActionController::Base
helper MyEngine::SharedEngineHelper
end
If you want to include all of the engine's helpers, you can use #helper
method on an engine's instance:
class ApplicationController < ActionController::Base
helper MyEngine::Engine.helpers
end
It will include all of the helpers from engine's directory. Take into account that this does not include helpers defined in controllers with helper_method or other similar solutions, only helpers defined in the helpers directory will be included.
Migrations & seed data
Engines can have their own migrations. The default path for migrations is exactly the same as in application: db/migrate
To use engine's migrations in application you can use rake task, which copies them to application's dir:
rake ENGINE_NAME:install:migrations
Note that some of the migrations may be skipped if a migration with the same name already exists in application. In such a situation you must decide whether to leave that migration or rename the migration in the application and rerun copying migrations.
If your engine has migrations, you may also want to prepare data for the database in the db/seeds.rb
file. You can load that data using the #load_seed method, e.g.
MyEngine::Engine.load_seed
Loading priority
In order to change engine's priority you can use config.railties_order
in main application. It will affect the priority of loading views, helpers, assets and all the other files related to engine or application.
# load Blog::Engine with highest priority, followed by application and other railties
config.railties_order = [Blog::Engine, :main_app, :all]
Constant Summary
Railtie - Inherited
Class Attribute Summary
Class Method Summary
- .endpoint(endpoint = nil)
- .engine_name
-
.find(path)
Finds engine with given path.
- .find_root(from)
- .inherited(base)
- .isolate_namespace(mod)
- .isolated (also: .isolated?) rw
- .new ⇒ Engine constructor
Railtie - Inherited
| .configure | Allows you to configure the railtie. |
| .console, .generators, .inherited, | |
| .instance | |
| .new, .railtie_name, .rake_tasks, .respond_to_missing?, .runner, .subclasses |
Instance Attribute Summary
- #engine_name readonly
- #middleware readonly
- #paths readonly
- #root readonly
-
#routes
readonly
Defines the routes for this engine.
Railtie - Inherited
Instance Method Summary
-
#app
(also: #build_middleware_stack)
Returns the underlying rack application for this engine.
-
#call(env)
Define the Rack API for this engine.
-
#config
Define the configuration object for the engine.
-
#eager_load!
Eager load the application by loading all ruby files inside eager_load paths.
-
#endpoint
Returns the endpoint for this engine.
-
#env_config
Defines additional Rack env configuration that is added on each call.
-
#helpers
Returns a module with all the helpers defined for the engine.
-
#helpers_paths
Returns all registered helpers paths.
- #isolated? ⇒ Boolean
-
#load_console(app = self)
Load console and invoke the registered hooks.
-
#load_generators(app = self)
Load Rails generators and invoke the registered hooks.
-
#load_runner(app = self)
Load Rails runner and invoke the registered hooks.
-
#load_seed
Load data from db/seeds.rb file.
-
#load_tasks(app = self)
Load Rake, railties tasks and invoke the registered hooks.
- #railties
Railtie - Inherited
Initializable - Included
Constructor Details
.new ⇒ Engine
# File 'railties/lib/rails/engine.rb', line 422
def initialize @_all_autoload_paths = nil @_all_load_paths = nil @app = nil @config = nil @env_config = nil @helpers = nil @routes = nil super end
Class Attribute Details
.called_from (rw)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 341
attr_accessor :called_from, :isolated
.isolated? (rw)
Alias for .isolated.
# File 'railties/lib/rails/engine.rb', line 343
alias :isolated? :isolated
Class Method Details
.endpoint(endpoint = nil)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 371
def endpoint(endpoint = nil) @endpoint ||= nil @endpoint = endpoint if endpoint @endpoint end
.engine_name
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 344
alias :engine_name :railtie_name
.find(path)
Finds engine with given path
# File 'railties/lib/rails/engine.rb', line 409
def find(path) = File. path Rails::Engine.subclasses.each do |klass| engine = klass.instance return engine if File. (engine.root) == end nil end
.find_root(from)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 367
def find_root(from) find_root_with_flag "lib", from end
.inherited(base)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 348
def inherited(base) unless base.abstract_railtie? Rails::Railtie::Configuration.eager_load_namespaces << base base.called_from = begin call_stack = if Kernel.respond_to?(:caller_locations) caller_locations.map { |l| l.absolute_path || l.path } else # Remove the line number from backtraces making sure we don't leave anything behind caller.map { |p| p.sub(/:\d+.*/, '') } end File.dirname(call_stack.detect { |p| p !~ %r[railties[\w.-]*/lib/rails|rack[\w.-]*/lib/rack] }) end end super end
.isolate_namespace(mod)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 377
def isolate_namespace(mod) engine_name(generate_railtie_name(mod.name)) self.routes.default_scope = { module: ActiveSupport::Inflector.underscore(mod.name) } self.isolated = true unless mod.respond_to?(:railtie_namespace) name, railtie = engine_name, self mod.singleton_class.instance_eval do define_method(:railtie_namespace) { railtie } unless mod.respond_to?(:table_name_prefix) define_method(:table_name_prefix) { "#{name}_" } end unless mod.respond_to?(:use_relative_model_naming?) class_eval "def use_relative_model_naming?; true; end", __FILE__, __LINE__ end unless mod.respond_to?(:railtie_helpers_paths) define_method(:railtie_helpers_paths) { railtie.helpers_paths } end unless mod.respond_to?(:railtie_routes_url_helpers) define_method(:railtie_routes_url_helpers) {|include_path_helpers = true| railtie.routes.url_helpers(include_path_helpers) } end end end end
.isolated (rw) Also known as: .isolated?
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 341
attr_accessor :called_from, :isolated
Instance Attribute Details
#engine_name (readonly)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 420
delegate :engine_name, :isolated?, to: :class
#middleware (readonly)
[ GitHub ]#paths (readonly)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 419
delegate :middleware, :root, :paths, to: :config
#root (readonly)
[ GitHub ]# File 'railties/lib/rails/engine.rb', line 419
delegate :middleware, :root, :paths, to: :config
#routes (readonly)
Defines the routes for this engine. If a block is given to routes, it is appended to the engine.
# File 'railties/lib/rails/engine.rb', line 530
def routes @routes ||= ActionDispatch::Routing::RouteSet.new @routes.append(&Proc.new) if block_given? @routes end
Instance Method Details
#app Also known as: #build_middleware_stack
Returns the underlying rack application for this engine.
# File 'railties/lib/rails/engine.rb', line 499
def app @app ||= begin config.middleware = config.middleware.merge_into(default_middleware_stack) config.middleware.build(endpoint) end end
#call(env)
Define the Rack API for this engine.
#config
Define the configuration object for the engine.
# File 'railties/lib/rails/engine.rb', line 537
def config @config ||= Engine::Configuration.new(self.class.find_root(self.class.called_from)) end
#eager_load!
Eager load the application by loading all ruby files inside eager_load paths.
# File 'railties/lib/rails/engine.rb', line 468
delegate :eager_load!, to: :instance
#endpoint
Returns the endpoint for this engine. If none is registered, defaults to an ::ActionDispatch::Routing::RouteSet.
# File 'railties/lib/rails/engine.rb', line 508
def endpoint self.class.endpoint || routes end
#env_config
Defines additional Rack env configuration that is added on each call.
# File 'railties/lib/rails/engine.rb', line 522
def env_config @env_config ||= { 'action_dispatch.routes' => routes } end
#helpers
Returns a module with all the helpers defined for the engine.
# File 'railties/lib/rails/engine.rb', line 482
def helpers @helpers ||= begin helpers = Module.new all = ActionController::Base.all_helpers_from_path(helpers_paths) ActionController::Base.modules_for_helpers(all).each do |mod| helpers.send(:include, mod) end helpers end end
#helpers_paths
Returns all registered helpers paths.
# File 'railties/lib/rails/engine.rb', line 494
def helpers_paths paths["app/helpers"].existent end
#isolated? ⇒ Boolean
# File 'railties/lib/rails/engine.rb', line 420
delegate :engine_name, :isolated?, to: :class
#load_console(app = self)
Load console and invoke the registered hooks. Check Railtie.console for more info.
#load_generators(app = self)
Load Rails generators and invoke the registered hooks. Check Railtie.generators for more info.
# File 'railties/lib/rails/engine.rb', line 459
def load_generators(app=self) require "rails/generators" run_generators_blocks(app) Rails::Generators.configure!(app.config.generators) self end
#load_runner(app = self)
Load Rails runner and invoke the registered hooks. Check Railtie.runner for more info.
#load_seed
Load data from db/seeds.rb file. It can be used in to load engines' seeds, e.g.:
Blog::Engine.load_seed
# File 'railties/lib/rails/engine.rb', line 545
def load_seed seed_file = paths["db/seeds.rb"].existent.first load(seed_file) if seed_file end
#load_tasks(app = self)
Load Rake, railties tasks and invoke the registered hooks. Check Railtie.rake_tasks for more info.


Data Discovery and Data Integration
Data discovery is the task of finding data sets relevant to their task at hand. Data integration is the task of combining those data sets.  Stonebraker et al. report [stonebraker2018dataintegration].
The semantic types of data sets are critical for both tasks.
Data discovery can be supported by information retrieval systems. Some queries like “find a file with the extension CSV that contains ‘url’” and “find a dataset that is under a public license matching ‘titanic’” are possible within current solutions based on full-text search and metadata standards 1, 2, 3. However, queries based on the semantics of the data sets are out of reach of such approaches. Semantic types could support systems, such as the knowledge-graph-based [aurum2018datadiscovery], to allow for queries as “find tables that contain a column with name ‘ID’ and have at least one column that looks like an input column”.
Providing types and their relations can provide additional support for common operations in data integration. Consider integrating two data sets, one coming from the office in New York, another from the same company but the one Amsterdam office. Without proper typing, the “wages” column in the first data set might be in dollar, while the second one in euro.
- stonebraker2018dataintegration
Stonebraker, M., & Ilyas, I. F. (2018). Data Integration: The Current Status and the Way Forward. IEEE Data Eng. Bull., 41(2), 3-9.
- aurum2018datadiscovery
Fernandez, R. C., Abedjan, Z., Koko, F., Yuan, G., Madden, S., & Stonebraker, M. (2018, April). Aurum: A data discovery system. In 2018 IEEE 34th International Conference on Data Engineering (ICDE) (pp. 1001-1012). IEEE.
Footnotes


Title
Link
Select Device Type:
desktop
Mobile
Target
None
New Tab
Pictures
Active
UPLOAD
please upload the 1000px X 500px and .JPG
@(Upload photos)


Demonstrates:
Implementing a data provider to create a totals row at the end of the grid.
This is a simplification of what the DataView does.
View Source:
View the source for this example on Github


Zidane leaves out Vinicius again for PSG encounter
by BeSoccerReal Madrid announced the 19 man squad to face Paris Saint Germain in the Champions League. Zidane has put out the same squad against Real Sociedad, so Vinicius Junior has once again been left out.
Zinedine Zidane has selected against PSG the same squad as the one he chose for the Real Sociedad match, so Vinicius Junior is out of the 19 man squad to face PSG at the Santiago Bernabeu.
The news means that Vinicius Junior is once again out of the squad. Far from continuing to prove himself after being key last year under under Solari, the Brazilian has fallen by the wayside under Zidane.
This is the third time that the winger has been in the stand in the last month, the second consecutive month. Vinicius was left out of the matches against Galatasaray and Real Sociedad and he has also played less. He has started just four times this season; the last time was on 19th October against Mallorca.
Vinicius' situation contrasts with that of Rodrygo, who has formed part of each of the last seven squad lists. The hat-trick against Galatasaray was a real turning point and he has started four of the last five matches.
This is the 19 man squad chosen by Zidane: Courtois, Areola and Altube; Carvajal, Militao, Ramos, Varane, Marcelo and Mendy; Kroos, Modric, Casemiro, Valverde and Isco: Hazard, Benzema, Bale, Jovi and Rodrygo.
Featured links
- The latest football transfers news and rumours
- Neymar or Mbappé? Zidane choose
- Kante explains why he turned down PSG
- What Cristiano said about Messi to his Man Utd teammates
One player will have to be left out come kick-off time.


All your files in one secure location,
accessible anywhere.
Fylo stores all your most important files in one secure location.
Access them wherever
you need, share and collaborate with
friends family, and co-workers.
Access your files, anywhere
The ability to use a smartphone, tablet, or computer
to access your account means your
files follow you
everywhere.
Security you can trust
2-factor authentication and user-controlled encryption are
just a couple of the security
features we allow to help
secure your files.
Real-time collaboration
Whether you're sharing holidays photos or work
documents, Fylo has you covered allowing for all
file
types to be securely stored and shared.
Store any type of file
The ability to use a smartphone, tablet, or computer
to access your account means your
files follow you
everywhere.
 Fylo has you
covered for all of your file
storage needs.
Securely share files and folders with friends, family and colleagues for live
collaboration. No email
attachments required.
Satish Patel
Founder & CEO, Huddle
Bruce McKenzie
Founder & CEO, Huddle
Iva Boyd
Founder & CEO, Huddle