cytopy.data.fcs¶
Classes
|
Represents a single cluster generated by a clustering experiment on a single file |
|
Defines the methodology and parameters of clustering to apply to an FCS File Group, or in the case of |
|
Cached index for population in an associated control |
|
Document representation of a single FCS file. |
|
Document representation of a file group; a selection of related fcs files (e.g. |
|
Cached populations; stores the index of events associated to a population for quick loading. |
-
class
cytopy.data.fcs.
Cluster
(*args, **kwargs)¶ Bases:
mongoengine.document.EmbeddedDocument
Represents a single cluster generated by a clustering experiment on a single file
- Parameters
cluster_id (str, required) – name associated to cluster
index (FileField) – index of cell events associated to cluster (very large array)
n_events (int, required) – number of events in cluster
prop_of_root (float, required) – proportion of events in cluster relative to root population
cluster_experiment (RefField) – reference to ClusteringDefinition
meta_cluster_id (str, optional) – associated meta-cluster
Methods
clean
()Hook for doing document level data cleaning before validation is run.
from_json
(json_data[, created])Converts json data to a Document instance
Get text score from text query
Load the index of data that corresponds to cells belonging to this cluster
save_index
(data)Save the index of data that corresponds to cells belonging to this cluster
to_json
(*args, **kwargs)Convert this document to JSON.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
Classes
Metaclass for all documents.
-
clean
()¶ Hook for doing document level data cleaning before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
classmethod
from_json
(json_data, created=False)¶ Converts json data to a Document instance
- Parameters
json_data (str) – The json data to load into the Document
created (bool) –
Boolean defining whether to consider the newly instantiated document as brand new or as persisted already: * If True, consider the document as brand new, no matter what data
it’s loaded with (i.e. even if an ID is loaded).
If False and an ID is NOT provided, consider the document as brand new.
If False and an ID is provided, assume that the object has already been persisted (this has an impact on the subsequent call to .save()).
Defaults to
False
.
-
get_text_score
()¶ Get text score from text query
-
load_index
() → numpy.array¶ Load the index of data that corresponds to cells belonging to this cluster
- Returns
Array of single cell events data
- Return type
np.array
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.DocumentMetaclass
-
save_index
(data: numpy.array) → None¶ Save the index of data that corresponds to cells belonging to this cluster
- Parameters
data (np.array, required) – Numpy array of single cell events data
- Returns
- Return type
None
-
to_json
(*args, **kwargs)¶ Convert this document to JSON.
- Parameters
use_db_field – Serialize field names as they appear in MongoDB (as opposed to attribute names on this document). Defaults to True.
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.
-
class
cytopy.data.fcs.
ClusteringDefinition
(*args, **values)¶ Bases:
mongoengine.document.Document
Defines the methodology and parameters of clustering to apply to an FCS File Group, or in the case of meta-clustering, a collection of FCS File Groups from the same FCS Experiment
- Parameters
clustering_uid (str, required) – unique identifier
method (str, required) – type of clustering performed, either PhenoGraph or FlowSOM
parameters (list, required) – parameters passed to clustering algorithm (list of tuples)
features (list, required) – list of channels/markers that clustering is performed on
transform_method (str, optional, (default:"logicle")) – type of transformation to be applied to data prior to clustering
root_population (str, required, (default:"root")) – population that clustering is performed on
cluster_prefix (str, optional, (default: "cluster")) – a prefix to add to the name of each resulting cluster
meta_method (str, required, (default: False)) – refers to whether the clustering is ‘meta-clustering’
meta_clustering_uid_target (str, optional) – clustering_uid for clustering definition that meta-clustering should target in each sample
Miscellaneous
Methods
cascade_save
(**kwargs)Recursively save any references and generic references on the document.
clean
()Hook for doing document level data cleaning before validation is run.
Compares the indexes defined in MongoEngine with the ones existing in the database.
create_index
(keys[, background])Creates the given indexes if required.
delete
([signal_kwargs])Delete the
Document
from the database.Drops the entire collection associated with this
Document
type from the database.ensure_index
(key_or_list[, drop_dups, …])Ensure that the given indexes are in place.
Checks the document meta data and ensures all the indexes exist.
from_json
(json_data[, created])Converts json data to a Document instance
Get text score from text query
Lists all of the indexes that should be created for given collection.
modify
([query])Perform an atomic update of the document in the database and reload the document object using updated version.
register_delete_rule
(document_cls, …)This method registers the delete rules to apply when removing this object.
reload
(*fields, **kwargs)Reloads all attributes from the database.
save
([force_insert, validate, clean, …])Save the
Document
to the database.select_related
([max_depth])Handles dereferencing of
DBRef
objects to a maximum depth in order to cut down the number queries to mongodb.switch_collection
(collection_name[, …])Temporarily switch the collection for a document instance.
switch_db
(db_alias[, keep_created])Temporarily switch the database for a document instance.
to_dbref
()Returns an instance of
DBRef
useful in __raw__ queries.to_json
(*args, **kwargs)Convert this document to JSON.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
update
(**kwargs)Performs an update on the
Document
A convenience wrapper toupdate()
.validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
Classes
Metaclass for top-level documents (i.e.
Attributes
Get the primary key.
-
exception
DoesNotExist
¶ Bases:
mongoengine.errors.DoesNotExist
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
exception
MultipleObjectsReturned
¶ Bases:
mongoengine.errors.MultipleObjectsReturned
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
cascade_save
(**kwargs)¶ Recursively save any references and generic references on the document.
-
clean
()¶ Hook for doing document level data cleaning before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
classmethod
compare_indexes
()¶ Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.
-
classmethod
create_index
(keys, background=False, **kwargs)¶ Creates the given indexes if required.
- Parameters
keys – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering
background – Allows index creation in the background
-
delete
(signal_kwargs=None, **write_concern)¶ Delete the
Document
from the database. This will only take effect if the document has been previously saved.- Parameters
signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.
write_concern – Extra keyword arguments are passed down which will be used as options for the resultant
getLastError
command. For example,save(..., w: 2, fsync: True)
will wait until at least two servers have recorded the write and will force an fsync on the primary server.
Changed in version 0.10.7: Add signal_kwargs argument
-
classmethod
drop_collection
()¶ Drops the entire collection associated with this
Document
type from the database.Raises
OperationError
if the document has no collection set (i.g. if it is abstract)Changed in version 0.10.7:
OperationError
exception raised if no collection available
-
classmethod
ensure_index
(key_or_list, drop_dups=False, background=False, **kwargs)¶ Ensure that the given indexes are in place. Deprecated in favour of create_index.
- Parameters
key_or_list – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering
background – Allows index creation in the background
drop_dups – Was removed/ignored with MongoDB >2.7.5. The value will be removed if PyMongo3+ is used
-
classmethod
ensure_indexes
()¶ Checks the document meta data and ensures all the indexes exist.
Global defaults can be set in the meta - see guide/defining-documents
Note
You can disable automatic index creation by setting auto_create_index to False in the documents meta data
-
classmethod
from_json
(json_data, created=False)¶ Converts json data to a Document instance
- Parameters
json_data (str) – The json data to load into the Document
created (bool) –
Boolean defining whether to consider the newly instantiated document as brand new or as persisted already: * If True, consider the document as brand new, no matter what data
it’s loaded with (i.e. even if an ID is loaded).
If False and an ID is NOT provided, consider the document as brand new.
If False and an ID is provided, assume that the object has already been persisted (this has an impact on the subsequent call to .save()).
Defaults to
False
.
-
get_text_score
()¶ Get text score from text query
-
classmethod
list_indexes
()¶ Lists all of the indexes that should be created for given collection. It includes all the indexes from super- and sub-classes.
-
modify
(query=None, **update)¶ Perform an atomic update of the document in the database and reload the document object using updated version.
Returns True if the document has been updated or False if the document in the database doesn’t match the query.
Note
All unsaved changes that have been made to the document are rejected if the method returns True.
- Parameters
query – the update will be performed only if the document in the database matches the query
update – Django-style update keyword arguments
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.TopLevelDocumentMetaclass
-
property
pk
¶ Get the primary key.
-
classmethod
register_delete_rule
(document_cls, field_name, rule)¶ This method registers the delete rules to apply when removing this object.
-
reload
(*fields, **kwargs)¶ Reloads all attributes from the database.
- Parameters
fields – (optional) args list of fields to reload
max_depth – (optional) depth of dereferencing to follow
New in version 0.1.2.
Changed in version 0.6: Now chainable
Changed in version 0.9: Can provide specific fields to reload
-
save
(force_insert=False, validate=True, clean=True, write_concern=None, cascade=None, cascade_kwargs=None, _refs=None, save_condition=None, signal_kwargs=None, **kwargs)¶ Save the
Document
to the database. If the document already exists, it will be updated, otherwise it will be created.- Parameters
force_insert – only try to create a new document, don’t allow updates of existing documents.
validate – validates the document; set to
False
to skip.clean – call the document clean method, requires validate to be True.
write_concern – Extra keyword arguments are passed down to
save()
ORinsert()
which will be used as options for the resultantgetLastError
command. For example,save(..., write_concern={w: 2, fsync: True}, ...)
will wait until at least two servers have recorded the write and will force an fsync on the primary server.cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies
cascade=True
._refs – A list of processed references used in cascading saves
save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises
OperationError
if the conditions are not satisfiedsignal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.
Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any
DBRef
objects that have changes are saved as well.Changed in version 0.6: Added cascading saves
Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.
Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.
Changed in version 0.10:
OperationError
exception raised if save_condition fails.Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError
Changed in version 0.10.7: Add signal_kwargs argument
Handles dereferencing of
DBRef
objects to a maximum depth in order to cut down the number queries to mongodb.New in version 0.5.
-
switch_collection
(collection_name, keep_created=True)¶ Temporarily switch the collection for a document instance.
Only really useful for archiving off data and calling save():
user = User.objects.get(id=user_id) user.switch_collection('old-users') user.save()
- Parameters
collection_name (str) – The database alias to use for saving the document
keep_created (bool) – keep self._created value after switching collection, else is reset to True
See also
Use
switch_db
if you need to read from another database
-
switch_db
(db_alias, keep_created=True)¶ Temporarily switch the database for a document instance.
Only really useful for archiving off data and calling save():
user = User.objects.get(id=user_id) user.switch_db('archive-db') user.save()
- Parameters
db_alias (str) – The database alias to use for saving the document
keep_created (bool) – keep self._created value after switching db, else is reset to True
See also
Use
switch_collection
if you need to read from another collection
-
to_dbref
()¶ Returns an instance of
DBRef
useful in __raw__ queries.
-
to_json
(*args, **kwargs)¶ Convert this document to JSON.
- Parameters
use_db_field – Serialize field names as they appear in MongoDB (as opposed to attribute names on this document). Defaults to True.
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
update
(**kwargs)¶ Performs an update on the
Document
A convenience wrapper toupdate()
.Raises
OperationError
if called on an object that has not yet been saved.
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.
-
class
cytopy.data.fcs.
ControlIndex
(*args, **kwargs)¶ Bases:
mongoengine.document.EmbeddedDocument
Cached index for population in an associated control
- Parameters
control_id (str) – Name of the control file
index (FileField) – numpy array storing index of events that belong to population
Methods
clean
()Hook for doing document level data cleaning before validation is run.
from_json
(json_data[, created])Converts json data to a Document instance
Get text score from text query
Retrieve the index values for the given population
save_index
(data)Given a new numpy array of index values, serialise and commit data to database
to_json
(*args, **kwargs)Convert this document to JSON.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
Classes
Metaclass for all documents.
-
clean
()¶ Hook for doing document level data cleaning before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
classmethod
from_json
(json_data, created=False)¶ Converts json data to a Document instance
- Parameters
json_data (str) – The json data to load into the Document
created (bool) –
Boolean defining whether to consider the newly instantiated document as brand new or as persisted already: * If True, consider the document as brand new, no matter what data
it’s loaded with (i.e. even if an ID is loaded).
If False and an ID is NOT provided, consider the document as brand new.
If False and an ID is provided, assume that the object has already been persisted (this has an impact on the subsequent call to .save()).
Defaults to
False
.
-
get_text_score
()¶ Get text score from text query
-
load_index
() → numpy.array¶ Retrieve the index values for the given population
- Returns
Array of index values
- Return type
np.array
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.DocumentMetaclass
-
save_index
(data: numpy.array) → None¶ Given a new numpy array of index values, serialise and commit data to database
- Parameters
data (np.array) – Array of index values
- Returns
- Return type
None
-
to_json
(*args, **kwargs)¶ Convert this document to JSON.
- Parameters
use_db_field – Serialize field names as they appear in MongoDB (as opposed to attribute names on this document). Defaults to True.
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.
-
class
cytopy.data.fcs.
File
(*args, **kwargs)¶ Bases:
mongoengine.document.EmbeddedDocument
Document representation of a single FCS file.
- Parameters
file_id (str, required) – Unique identifier for fcs file
file_type (str, required, (default='complete')) – One of either ‘complete’ or ‘control’; signifies the type of data stored
data (FileField) – Numpy array of fcs events data
compensated (bool, required, (default=False)) – Boolean value, if True then data have been compensated
channel_mappings (list) – List of standarised channel/marker mappings (corresponds to column names of underlying data)
Methods
clean
()Hook for doing document level data cleaning before validation is run.
from_json
(json_data[, created])Converts json data to a Document instance
Get text score from text query
pull
(sample)Retrieve single cell data from database
put
(data)Save single cell data to database
to_json
(*args, **kwargs)Convert this document to JSON.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
Classes
Metaclass for all documents.
-
clean
()¶ Hook for doing document level data cleaning before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
classmethod
from_json
(json_data, created=False)¶ Converts json data to a Document instance
- Parameters
json_data (str) – The json data to load into the Document
created (bool) –
Boolean defining whether to consider the newly instantiated document as brand new or as persisted already: * If True, consider the document as brand new, no matter what data
it’s loaded with (i.e. even if an ID is loaded).
If False and an ID is NOT provided, consider the document as brand new.
If False and an ID is provided, assume that the object has already been persisted (this has an impact on the subsequent call to .save()).
Defaults to
False
.
-
get_text_score
()¶ Get text score from text query
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.DocumentMetaclass
-
pull
(sample: int = None) → numpy.array¶ Retrieve single cell data from database
- Parameters
sample (int, optional) – If an integer value is given, a random sample of this size is returned
- Returns
Array of single cell data
- Return type
Numpy.array
-
put
(data: numpy.array) → None¶ Save single cell data to database
- Parameters
data (Numpy.array) – Single cell data (as a numpy array) to save to database
- Returns
- Return type
None
-
to_json
(*args, **kwargs)¶ Convert this document to JSON.
- Parameters
use_db_field – Serialize field names as they appear in MongoDB (as opposed to attribute names on this document). Defaults to True.
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.
-
class
cytopy.data.fcs.
FileGroup
(*args, **values)¶ Bases:
mongoengine.document.Document
Document representation of a file group; a selection of related fcs files (e.g. a sample and it’s associated controls)
- Parameters
primary_id (str, required) – Unique ID to associate to group
files (EmbeddedDocList) – List of File objects
flags (str, optional) – Warnings associated to file group
notes (str, optional) – Additional free text
populations (EmbeddedDocList) – Populations derived from this file group
gates (EmbeddedDocList) – Gate objects that have been applied to this file group
collection_datetime (DateTime, optional) – Date and time of sample collection
processing_datetime (DateTime, optional) – Date and time of sample processing
Miscellaneous
Methods
cascade_save
(**kwargs)Recursively save any references and generic references on the document.
clean
()Hook for doing document level data cleaning before validation is run.
Compares the indexes defined in MongoEngine with the ones existing in the database.
create_index
(keys[, background])Creates the given indexes if required.
delete
([signal_kwargs])Delete the
Document
from the database.delete_clusters
(clustering_uid, drop_all)Delete all cluster attaining to a given clustering UID
delete_gates
(gates)Delete one or many gates from FileGroup
delete_populations
(populations)Delete given populations
Drops the entire collection associated with this
Document
type from the database.ensure_index
(key_or_list[, drop_dups, …])Ensure that the given indexes are in place.
Checks the document meta data and ensures all the indexes exist.
from_json
(json_data[, created])Converts json data to a Document instance
get_population
(population_name)Given the name of a population associated to the FileGroup, returns the Population object
get_population_by_parent
(parent)Given the name of some parent population, return a list of Population object whom’s parent matches
Get text score from text query
Return a list of file IDs for associated control files
List ID of controls that have a cached index in each population of the saved population tree (i.e.
Yields names of saved gates
Lists all of the indexes that should be created for given collection.
Yields list of population names
modify
([query])Perform an atomic update of the document in the database and reload the document object using updated version.
register_delete_rule
(document_cls, …)This method registers the delete rules to apply when removing this object.
reload
(*fields, **kwargs)Reloads all attributes from the database.
save
([force_insert, validate, clean, …])Save the
Document
to the database.select_related
([max_depth])Handles dereferencing of
DBRef
objects to a maximum depth in order to cut down the number queries to mongodb.switch_collection
(collection_name[, …])Temporarily switch the collection for a document instance.
switch_db
(db_alias[, keep_created])Temporarily switch the database for a document instance.
to_dbref
()Returns an instance of
DBRef
useful in __raw__ queries.to_json
(*args, **kwargs)Convert this document to JSON.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
update
(**kwargs)Performs an update on the
Document
A convenience wrapper toupdate()
.update_population
(population_name, …)Given an existing population name, replace that population with the new population document
validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
validity
()Returns True if FileGroup is deemed ‘valid’; that is, the term ‘invalid’ is absent from the ‘flags’ attribute.
Classes
Metaclass for top-level documents (i.e.
Attributes
Get the primary key.
-
exception
DoesNotExist
¶ Bases:
mongoengine.errors.DoesNotExist
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
exception
MultipleObjectsReturned
¶ Bases:
mongoengine.errors.MultipleObjectsReturned
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
cascade_save
(**kwargs)¶ Recursively save any references and generic references on the document.
-
clean
()¶ Hook for doing document level data cleaning before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
classmethod
compare_indexes
()¶ Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.
-
classmethod
create_index
(keys, background=False, **kwargs)¶ Creates the given indexes if required.
- Parameters
keys – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering
background – Allows index creation in the background
-
delete
(signal_kwargs=None, **write_concern)¶ Delete the
Document
from the database. This will only take effect if the document has been previously saved.- Parameters
signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.
write_concern – Extra keyword arguments are passed down which will be used as options for the resultant
getLastError
command. For example,save(..., w: 2, fsync: True)
will wait until at least two servers have recorded the write and will force an fsync on the primary server.
Changed in version 0.10.7: Add signal_kwargs argument
-
delete_clusters
(clustering_uid: str = None, drop_all: bool = False)¶ Delete all cluster attaining to a given clustering UID
- Parameters
clustering_uid (str) – Unique identifier for clustering experiment that should have clusters deleted from file
drop_all (bool) – If True, all clusters for every population are dropped from database regardless of the clustering experiment they are associated too
- Returns
- Return type
None
-
delete_gates
(gates: list)¶ Delete one or many gates from FileGroup
- Parameters
gates (list or str) – Either a single gate name (str) or list of gate names (list of strings) of gates to remove
- Returns
- Return type
None
-
delete_populations
(populations: list) → None¶ Delete given populations
- Parameters
populations (list or str) – Either a list of populations (list of strings) to remove or a single population as a string
- Returns
- Return type
None
-
classmethod
drop_collection
()¶ Drops the entire collection associated with this
Document
type from the database.Raises
OperationError
if the document has no collection set (i.g. if it is abstract)Changed in version 0.10.7:
OperationError
exception raised if no collection available
-
classmethod
ensure_index
(key_or_list, drop_dups=False, background=False, **kwargs)¶ Ensure that the given indexes are in place. Deprecated in favour of create_index.
- Parameters
key_or_list – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering
background – Allows index creation in the background
drop_dups – Was removed/ignored with MongoDB >2.7.5. The value will be removed if PyMongo3+ is used
-
classmethod
ensure_indexes
()¶ Checks the document meta data and ensures all the indexes exist.
Global defaults can be set in the meta - see guide/defining-documents
Note
You can disable automatic index creation by setting auto_create_index to False in the documents meta data
-
classmethod
from_json
(json_data, created=False)¶ Converts json data to a Document instance
- Parameters
json_data (str) – The json data to load into the Document
created (bool) –
Boolean defining whether to consider the newly instantiated document as brand new or as persisted already: * If True, consider the document as brand new, no matter what data
it’s loaded with (i.e. even if an ID is loaded).
If False and an ID is NOT provided, consider the document as brand new.
If False and an ID is provided, assume that the object has already been persisted (this has an impact on the subsequent call to .save()).
Defaults to
False
.
-
get_population
(population_name: str) → cytopy.data.fcs.Population¶ Given the name of a population associated to the FileGroup, returns the Population object
- Parameters
population_name (str) – Name of population to retrieve from database
- Returns
- Return type
-
get_population_by_parent
(parent: str)¶ Given the name of some parent population, return a list of Population object whom’s parent matches
- Parameters
parent (str) – Name of the parent population to search for
- Returns
List of Populations
- Return type
List
-
get_text_score
()¶ Get text score from text query
-
list_controls
() → list¶ Return a list of file IDs for associated control files
- Returns
- Return type
list
-
list_gated_controls
() → list¶ List ID of controls that have a cached index in each population of the saved population tree (i.e. they have been gated)
- Returns
List of control IDs for gated controls
- Return type
list
-
list_gates
() → iter¶ Yields names of saved gates
- Returns
- Return type
Generator
-
classmethod
list_indexes
()¶ Lists all of the indexes that should be created for given collection. It includes all the indexes from super- and sub-classes.
-
list_populations
() → iter¶ Yields list of population names :returns: :rtype: Generator
-
modify
(query=None, **update)¶ Perform an atomic update of the document in the database and reload the document object using updated version.
Returns True if the document has been updated or False if the document in the database doesn’t match the query.
Note
All unsaved changes that have been made to the document are rejected if the method returns True.
- Parameters
query – the update will be performed only if the document in the database matches the query
update – Django-style update keyword arguments
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.TopLevelDocumentMetaclass
-
property
pk
¶ Get the primary key.
-
classmethod
register_delete_rule
(document_cls, field_name, rule)¶ This method registers the delete rules to apply when removing this object.
-
reload
(*fields, **kwargs)¶ Reloads all attributes from the database.
- Parameters
fields – (optional) args list of fields to reload
max_depth – (optional) depth of dereferencing to follow
New in version 0.1.2.
Changed in version 0.6: Now chainable
Changed in version 0.9: Can provide specific fields to reload
-
save
(force_insert=False, validate=True, clean=True, write_concern=None, cascade=None, cascade_kwargs=None, _refs=None, save_condition=None, signal_kwargs=None, **kwargs)¶ Save the
Document
to the database. If the document already exists, it will be updated, otherwise it will be created.- Parameters
force_insert – only try to create a new document, don’t allow updates of existing documents.
validate – validates the document; set to
False
to skip.clean – call the document clean method, requires validate to be True.
write_concern – Extra keyword arguments are passed down to
save()
ORinsert()
which will be used as options for the resultantgetLastError
command. For example,save(..., write_concern={w: 2, fsync: True}, ...)
will wait until at least two servers have recorded the write and will force an fsync on the primary server.cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies
cascade=True
._refs – A list of processed references used in cascading saves
save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises
OperationError
if the conditions are not satisfiedsignal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.
Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any
DBRef
objects that have changes are saved as well.Changed in version 0.6: Added cascading saves
Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.
Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.
Changed in version 0.10:
OperationError
exception raised if save_condition fails.Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError
Changed in version 0.10.7: Add signal_kwargs argument
Handles dereferencing of
DBRef
objects to a maximum depth in order to cut down the number queries to mongodb.New in version 0.5.
-
switch_collection
(collection_name, keep_created=True)¶ Temporarily switch the collection for a document instance.
Only really useful for archiving off data and calling save():
user = User.objects.get(id=user_id) user.switch_collection('old-users') user.save()
- Parameters
collection_name (str) – The database alias to use for saving the document
keep_created (bool) – keep self._created value after switching collection, else is reset to True
See also
Use
switch_db
if you need to read from another database
-
switch_db
(db_alias, keep_created=True)¶ Temporarily switch the database for a document instance.
Only really useful for archiving off data and calling save():
user = User.objects.get(id=user_id) user.switch_db('archive-db') user.save()
- Parameters
db_alias (str) – The database alias to use for saving the document
keep_created (bool) – keep self._created value after switching db, else is reset to True
See also
Use
switch_collection
if you need to read from another collection
-
to_dbref
()¶ Returns an instance of
DBRef
useful in __raw__ queries.
-
to_json
(*args, **kwargs)¶ Convert this document to JSON.
- Parameters
use_db_field – Serialize field names as they appear in MongoDB (as opposed to attribute names on this document). Defaults to True.
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
update
(**kwargs)¶ Performs an update on the
Document
A convenience wrapper toupdate()
.Raises
OperationError
if called on an object that has not yet been saved.
-
update_population
(population_name: str, new_population: cytopy.data.fcs.Population)¶ Given an existing population name, replace that population with the new population document
- Parameters
population_name (str) – Name of population to be replaced
new_population (Population) – Updated/new Population document
- Returns
- Return type
None
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.
-
validity
() → bool¶ Returns True if FileGroup is deemed ‘valid’; that is, the term ‘invalid’ is absent from the ‘flags’ attribute.
- Returns
True if valid, else False
- Return type
bool
-
class
cytopy.data.fcs.
Population
(*args, **kwargs)¶ Bases:
mongoengine.document.EmbeddedDocument
Cached populations; stores the index of events associated to a population for quick loading.
- Parameters
population_name (str, required) – name of population
index (FileField) – numpy array storing index of events that belong to population
prop_of_parent (float, required) – proportion of events as a percentage of parent population
prop_of_total (float, required) – proportion of events as a percentage of all events
warnings (list, optional) – list of warnings associated to population
parent (str, required, (default: "root")) – name of parent population
children (list, optional) – list of child populations (list of strings)
geom (list, required) – list of key value pairs (tuples; (key, value)) for defining geom of population e.g. the definition for an ellipse that ‘captures’ the population
clusters (EmbeddedDocListField) – list of associated Cluster documents
Methods
clean
()Hook for doing document level data cleaning before validation is run.
ctrl_search
(ctrl_id)If specified control exists in control_idx return True, otherwise False
delete_clusters
(clustering_uid, drop_all)Given a clustering UID, remove associated clusters
from_json
(json_data[, created])Converts json data to a Document instance
get_cluster
(cluster_id, meta)Given a cluster ID return the Cluster document and array of index values
get_many_clusters
(clustering_uid)Given a clustering UID return the associated clusters
Get text score from text query
Generate a list of clustering experiment UIDs
list_clusters
(meta)Returns a set of all existing clusters.
Retrieve the index values for the given population
replace_cluster_experiment
(current_uid, …)Given a clustering UID and new clustering definition, replace the clustering definition
save_control_idx
(control_idx)Save index for control files.
save_index
(data)Given a new numpy array of index values, serialise and commit data to database
to_json
(*args, **kwargs)Convert this document to JSON.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
Generate a python dictionary object for this population
update_cluster
(cluster_id, new_cluster)Given the ID for a specific cluster, replace the cluster with a new Cluster document
validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
Classes
Metaclass for all documents.
-
clean
()¶ Hook for doing document level data cleaning before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
ctrl_search
(ctrl_id) → bool¶ If specified control exists in control_idx return True, otherwise False
- Returns
- Return type
bool
-
delete_clusters
(clustering_uid: str = None, drop_all: bool = False) → None¶ Given a clustering UID, remove associated clusters
- Parameters
clustering_uid (str) – UID for clusters to be removed
drop_all (bool) – If True, all clusters are removed regardless of clustering experiment UID
- Returns
- Return type
None
-
classmethod
from_json
(json_data, created=False)¶ Converts json data to a Document instance
- Parameters
json_data (str) – The json data to load into the Document
created (bool) –
Boolean defining whether to consider the newly instantiated document as brand new or as persisted already: * If True, consider the document as brand new, no matter what data
it’s loaded with (i.e. even if an ID is loaded).
If False and an ID is NOT provided, consider the document as brand new.
If False and an ID is provided, assume that the object has already been persisted (this has an impact on the subsequent call to .save()).
Defaults to
False
.
-
get_cluster
(cluster_id: str, meta: bool = True) -> (<class 'cytopy.data.fcs.Cluster'>, <built-in function array>)¶ Given a cluster ID return the Cluster document and array of index values
- Parameters
cluster_id (str) – ID for cluster to pull from database
meta (bool) – If True, search will be isolated to clusters associated to a meta cluster ID
- Returns
Cluster Document, Array of index values
- Return type
Cluster, np.array
-
get_many_clusters
(clustering_uid: str) → list¶ Given a clustering UID return the associated clusters
- Parameters
clustering_uid (str) – UID for clusters of interest
- Returns
List of cluster documents
- Return type
list
-
get_text_score
()¶ Get text score from text query
-
list_clustering_experiments
() → set¶ Generate a list of clustering experiment UIDs
- Returns
Clustering experiment UIDs
- Return type
set
-
list_clusters
(meta: bool = True) → set¶ Returns a set of all existing clusters.
- Parameters
meta (bool) – If True, search is isolated to meta-clusters
- Returns
Cluster IDs
- Return type
set
-
load_index
() → numpy.array¶ Retrieve the index values for the given population
- Returns
Array of index values
- Return type
np.array
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.DocumentMetaclass
-
replace_cluster_experiment
(current_uid: str, new_cluster_definition: cytopy.data.fcs.ClusteringDefinition) → None¶ Given a clustering UID and new clustering definition, replace the clustering definition for all associated clusters
- Parameters
current_uid (str) – UID of clusters to be updated
new_cluster_definition (ClusteringDefinition) – New clustering definition
- Returns
- Return type
None
-
save_control_idx
(control_idx: dict)¶ Save index for control files. Takes a dictionary of values where the key corresponds to the control ID and the value is a numpy array of index values
- Parameters
control_idx (dict) – Dictionary of values where the key corresponds to the control ID and the value is a numpy array of index values
- Returns
- Return type
None
-
save_index
(data: numpy.array) → None¶ Given a new numpy array of index values, serialise and commit data to database
- Parameters
data (np.array) – Array of index values
- Returns
- Return type
None
-
to_json
(*args, **kwargs)¶ Convert this document to JSON.
- Parameters
use_db_field – Serialize field names as they appear in MongoDB (as opposed to attribute names on this document). Defaults to True.
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
to_python
() → dict¶ Generate a python dictionary object for this population
- Returns
Dictionary representation of population document
- Return type
dict
-
update_cluster
(cluster_id: str, new_cluster: cytopy.data.fcs.Cluster) → None¶ Given the ID for a specific cluster, replace the cluster with a new Cluster document
- Parameters
cluster_id (str) – Cluster ID for cluster to replace
new_cluster (Cluster) – Cluster document to use for updating cluster
- Returns
- Return type
None
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.


Use the checkbox below to ignore switch statements which have a default branch.
Powered by InspectionGadgets


Projects
Tangible Virtual-Reality
Orange #futureself
Medical 3D Viewer
City 3D
Lego Thief Raid
Vivo Avatar - World Cup 2014
Kotobee Ebook
Oimo.js - Physics Engine
Ammo Lab
Loth Portfolio
Olympe
Kuk-a-droid


In addition to setting up system-level services for each user the configuration import subsystem sets up a whole load of service bundles for each user, to be managed by that user's personal instance of service-manager
.
These service bundles live in the user's service bundles directory in the user's home directory.
The services fall into one of three main groups: logging, non-Desktop Bus, and Desktop Bus.
There are three major logging services.
simple-servers-log
/var/log/user/
username/simple-servers/
.
socket-servers-log
/var/log/user/
username/socket-servers/
.
dbus-servers-log
/var/log/user/
username/dbus-servers/
.
Non-Desktop Bus services are either straightforward simple services or services that listen on sockets. They do not require the per-user Desktop Bus broker to be running. Examples are:
emacs
--fg-daemon
mode only).
urxvt
urxvtd
dæmon.
gpg-agent
/run/user/
username/gnupg/S.gpg-agent
.
These services are not set up with a DBUS_SESSION_BUS_ADDRESS
environment variable.
Desktop Bus services require the per-user Desktop Bus broker to be running.
This can either be running as the system-level user-dbus-daemon@
username service or as a user-level dbus-daemon
service.
The latter requires that the full per-user service manager be running; and cannot be used until the per-user service manager is started.
They also need to know how to talk to it.
So the configuration import subsystem chain loads them from the userenv
program with the --set-dbus
option, setting an environment variable that gives the address (in path=
form because no-one understands the runtime=
form yet) of the /run/user/
name/bus
socket.
By convention, a Desktop Bus service is named by a common name by which users refer to it directly (such as gnome-terminal-server
), and has an alias pointing to it from its formal Desktop Bus "server" name (such as org.gnome.Terminal
).
The alias is used by the Desktop Bus broker when demand-starting the service, because the Desktop Bus broker only knows the formal names of services.
Processes run under per-user service management execute in a dæmon-style context.
They don't have controlling terminals, and (initially) they only have the environment variables set up by the user-services@
username service itself, which just runs userenv
.
In particular:
They do not have the environment variables that are set up in the users' rc/profile/login scripts for shells. Shells, and thus their rc script mechanisms, are not involved in the running of the per-user service manager instance.
They do not have the environment variables that are set up via the login class records in the users' ~/.login_conf
and the /etc/login.conf
databases.
Per-user service management does not, conceptually, run within a login session.
Every user-level service is afforded an environment directory in its service definition, in the conventional location that is accessble via the likes of system-control --user print-service-env org.example.Sheila
.
Environment variable for services should be configured there.
It is the wrong thing to do to explicitly set up per-user services with TTY
or DISPLAY
environment variables, and this is the sign of a faulty service program.
Service programs serve a single user that can be logged in multiple times in multiple forms of login session.
They should not require terminal access, and should be capable of connecting to multiple X displays simultaneously and not need an X display in order to start up.
Programs that are known to get this wrong include GNOME Terminal, KDE Notify, GNOME Weather, GNOME Maps, and MATE Notification.
These fail to work if they are not given a DISPLAY
environment variable in order to start up in their initial not-a-client-of-any-displays-yet state, where no D-BUS clients have connected to tell them about X displays to make windows on.
This is accounted for in the configuration import subsystem, which sets up a DISPLAY
environment variable for each user's gnome-terminal-server, knotify
, org.gnome.Weather.Application
, org.gnome.Maps
and mate-notification-daemon
service.
It just sets up DISPLAY
as :0
for the service.
You might need to tweak this for unusual X setups.


After installing the Hudson Claim Plugin, you have to configure each job by manually checking the "allow broken build claiming" option.
Obviously, it can be a bit cumbersome. But here comes the great groovy script console again.
With the following simple script, you can activate the option on every jobs of your server in just one go.
Don't hesitate to give me feedback (and backup your config before playing with those scripts, as always).
import hudson.model.* import hudson.maven.* import hudson.tasks.* for(item in Hudson.instance.items) { println("job $item.name") hasClaim = false; for(p in item.publishers ) { if(p instanceof hudson.plugins.claim.ClaimPublisher) { hasClaim = true; } } if(!hasClaim) { println(">>>>>>>> Adding claim right to $item.name") item.getPublishersList().add(new hudson.plugins.claim.ClaimPublisher() ); item.save() } }
Known quirk:
- Funnily, if I run this script many times, I see the same item coming back. But when I check inside the job configuration, the checkbox was correctly modified. I don't know where it comes from yet. If you have any idea, just let me know.
- See the shorter version in the comment below - assuming the plugin has a good hashCode/equals you can just use "replace" instead of searching and adding. This is essentially the same as the Chuck Norris example.


He hates going for long walks but doesn't mind a long flight or two. He's got a bit of a tmeper and only responds to "dracaris". Still haven't gotten him fire trained yet and was disappointed to find he burned a young boy while going for his morining hunt. Had to be strict with hime and sent him to the dungeon but that only made things worse. He stopped growing and became quite a handful for the feeders. I'm still deciding whether or not to let him and his sibling out but I think I'll wait for their older brother to get back before I make my choice.
Here's a few tips you might want to know about keeping Dragons:
Still learning but don't let that stop you from going out and getting youself this Lovely Lizard! Will update as I learn more!
For now, you can head to this link to learn more about dragons.


episcanpy.api¶
API¶
Import epiScanpy’s high-level API as:
import episcanpy.api as epi
Count Matrices: CT¶
Loading data, loading annotations, building count matrices, filtering of lowly covered methylation variables. Filtering of lowly covered cells.
Load features¶
In order to build a count matrix for either methylation or open chromatin data, loading the segmentation of the genome of interest or the set of features of interest is a prerequirement.
|
The function load features is here to transform a bed file into a usable set of units to measure methylation levels. |
|
Generate windows/bins of the given size for the appropriate genome (default choice is human). |
|
If the features loaded are too smalls or of different sizes, it is possible to normalise them to a unique given size by extending the feature coordinate in both directions. |
|
Plot the different feature sizes in an histogram. |
|
Extract the names of the loaded features, specifying the chromosome they originated from. |
Reading methylation file¶
Functions to read methylation files, extract methylation and buildthe count matrices:
|
Build methylation count matrix for a given annotation. |
|
Read file from which you want to extract the methylation level and (assuming it is like the Ecker/Methylpy format) extract the number of methylated read and the total number of read for the cytosines covered and in the right genomic context (CG or CH) :param sample_name: name of the file to read to extract key information. |
|
read the raw count matrix and convert it into an AnnData object. |
Reading open chromatin(ATAC) file¶
ATAC-seq specific functions to build count matrices and load data:
|
Build a count matrix one set of features at a time. |
|
Convert regular atac matrix into a sparse Anndata: |
Preprocessing: PP¶
Imputing missing data (methylation), filtering lowly covered cells or variables, correction for batch effect.
|
Histogram of the number of open features (in the case of ATAC-seq data) per cell. |
|
convert the count matrix into a binary matrix. |
|
Automatically computes PCA coordinates, loadings and variance decomposition, a neighborhood graph of observations, t-distributed stochastic neighborhood embedding (tSNE) Uniform Manifold Approximation and Projection (UMAP) |
|
Load observational metadata in adata.obs. |
|
Load sparse matrix (including matrices corresponding to 10x data) as AnnData objects. |
Methylation matrices¶
Methylation specific count matrices.
|
Impute missing values in methyaltion level matrices. |
|
read the raw count matrix and convert it into an AnnData object. |
|
Temporary function to load and impute methyaltion count matrix into an AnnData object |
Tools: TL¶
|
It is a wrap-up function of scanpy sc.tl.rank_genes_groups function. |
|
Compute silhouette scores. |
|
Automatically computes PCA coordinates, loadings and variance decomposition, a neighborhood graph of observations, t-distributed stochastic neighborhood embedding (tSNE) Uniform Manifold Approximation and Projection (UMAP) |
|
Convert list of known cell type markers from literature to a dictionary Input list of known marker genes First row is considered the header |
|
Use markers of a given cell type to plot peak openness for peaks in promoters of the given markers Input cell type, cell type markers, peak promoter intersections |
Plotting: PL¶
The plotting module episcanpy.plotting
largely parallels the tl.*
and a few of the pp.*
functions.
For most tools and for some preprocessing functions, you’ll find a plotting function with the same name.


Focus Manager for AngularJS and the Return Key
And here's a
contentEditable
element.
Enter
should create a new paragraph.
And now a bullet list
Pressing enter should add a new bullet...


Before Change
def sanity_checks(self):
if not self.get_rule_param("anvi_get_sequences_for_hmm_hits", "--gene-names"):
raise ConfigError("You must provide a list of genes to use for the phylogenomics. \
To do so, please use the "--gene-names" parameter of rule anvi_get_sequences_for_hmm_hits.")
if not self.get_rule_param("anvi_get_sequences_for_hmm_hits", "--return-best-hit"):
run.warning("You changed the value for --return-best-hit for the rule anvi_get_sequences_for_hmm_hits \
to something other than the default value, which is "true", while we allow you to do it \
After Change
def sanity_checks(self):
if not self.get_rule_param("anvi_get_sequences_for_hmm_hits", "--gene-names"):
run.warning("You did not provide a list of genes to use for phylogenomics. This is ok and things might work \
but we wanted to make sure this is intended. If you change your mind, you can provide a list of genes \
by using the "--gene-names" parameter of rule anvi_get_sequences_for_hmm_hits.")
if not self.get_rule_param("anvi_get_sequences_for_hmm_hits", "--return-best-hit"):
run.warning("You changed the value for --return-best-hit for the rule anvi_get_sequences_for_hmm_hits \
to something other than the default value, which is "true", while we allow you to do it \


One Mile Matt Playground Site
Hello! This is the One Mile Matt project playground.
Map
Draggable Marker Location
WGS84:
,
OSGB36:


Apps
-
1.SmartCodes QR BETA
-
2.MathNinja
-
3.DeskRock
Life Developing
Some say that we haven’t come out with enough apps, well, the truth is is that it takes a while to have an app that functions properly and looks elegant. But don’t worry, More apps are on their way.


Map Options: Browse map Query map
Select Layers to Display: ASOS Network AWOS Network RWIS Network KCCI Network DCP Network COOP Network Rivers Counties
Set your zoom option: Zoom in 10 times Zoom in 8 times Zoom in 4 times Zoom in 3 times Zoom in 2 times Recenter Map Zoom out 2 times Zoom out 3 times Zoom out 4 times Zoom out 8 times Zoom out 10 times


Go To Sleep
Start Game
About
Disable effects
(improves performance)
Back to Home
Count sheep until you fall asleep!
Made with ❤ by
Andrés Villarreal
Back to Home
Back to Home


Automation of preprocessing often requires computationally costly preprocessing combinations. This package helps to find near-best combinations faster. Metaheuristics supported are taboo search, simulated annealing, reheating and late acceptance. Start conditions include random and grid starts. End conditions include all iteration rounds completed, objective threshold reached and convergence. Metaheuristics, start and end conditions can be hybridized and hyperparameters optimized. Parallel computations are supported. The package is intended to be used with package 'preprocomb' and takes its 'GridClass' object as input.
| Version: | 0.2.0 |
| Depends: | R (≥ 2.10) |
| Imports: | utils, methods, stats, reshape2, ggplot2, preprocomb (≥ 0.3.0), doParallel, foreach |
| Suggests: | testthat, knitr, rmarkdown, rpart |
| Published: | 2016-06-30 |
| Author: | Markus Vattulainen |
| Maintainer: | Markus Vattulainen <markus.vattulainen at gmail.com> |
| License: | GPL-2 |
| NeedsCompilation: | no |
| CRAN checks: | metaheur results |
| Reference manual: | metaheur.pdf |
| Vignettes: |
Metaheur |
| Package source: | metaheur_0.2.0.tar.gz |
| Windows binaries: | r-devel: metaheur_0.2.0.zip, r-release: metaheur_0.2.0.zip, r-oldrel: metaheur_0.2.0.zip |
| OS X binaries: | r-release: metaheur_0.2.0.tgz, r-oldrel: metaheur_0.2.0.tgz |
| Old sources: | metaheur archive |
Please use the canonical form https://CRAN.R-project.org/package=metaheur to link to this page.


|
Overview
Contents
Players
Layers
Examples
Configuration
InFAQ
Changes
Licence
Back to Goodies
|
The Status Monitor's History Page
The History Page is a plug-in page for the Status
Monitor that displays a histogram of JNIPort activity.
The graph is updated at intervals controlled by the 'View⇒Update interval⇒'
sub-menu (which is initially set from the JVM's #statusMonitorSettings
configuration settings). The traces are labelled by the “legend” (which can be turned off
from the View menu), they are:
- Local refs (solid blue)
The number of “local” references to Java objects that were wrapped by JNIPort
objects at the sampling time.
- Global refs (solid dark green)
The number of “global” references to Java objects that were wrapped by JNIPort
objects at the sampling time.
The distinction between local and global is an internal matter (reflecting
one of the odd features of JNI). Local references
are not used at all if JNIPort has been configured to support
callbacks.
- Refs created per sec (light green line)
The rate at which JNIPort wrapped references to Java objects over the period since the
previous sampling time.
- Refs released per sec (pink line)
The rate at which JNIPort discarded and released references to Java objects over the
period since the previous sampling time. (Since references are released by finalisation,
this is a measure of how much work JNIPort is giving the finaliser to do.)
- Callbacks per sec (yellow line)
The rate at which JNIPort serviced callbacks over the
period since the previous sampling time.
|


|
Modjy is a WSGI server, so as such it must attempt to comply with the WSGI specification as much as possible. Ideally, this means that it should be possible to take any python WSGI middleware component and run it on modjy without modification. The purpose of this page is list the details of modjy's WSGI compliance, and any considerations which might affect application portability.
A - WSGI variables
Modjy's WSGI environment contains all of the required WSGI environment variables. They are listed here, along with modjy-specific notes where appropriate.
| Variable | Description | Modjy Default | Modjy notes |
|---|---|---|---|
| wsgi.version | The version of the WSGI standard with which modjy complies. | (1,0) | Modjy is designed against version 1.0 of the WSGI standard. |
| wsgi.url_scheme | A string representing the scheme portion of the URL. | e.g. http or https | |
| wsgi.input | The input stream for this request. | The supplied input stream is WSGI-compliant | See below for notes about the WSGI input stream. |
| wsgi.errors | The error stream for this request. | An output stream to which applications can write diagnostic output. | See below for notes about the destination of diagnostic output. |
| wsgi.multithread | A boolean value indicating whether or not an application object may be called simultaneously from multiple threads. | 1 | The default behaviour of modjy is to call application objects from multiple threads. The multithread configuration parameter can be used to force modjy to only call application objects from one thread at a time. Also, if caching of applications is disabled through use of the cache_callables parameter, new application objects are created for each new HTTP request and called once only for that request. Therefore, wsgi.multithread will also be false when cache_callables is false. |
| wsgi.multiprocess | A boolean value indicating whether or not similar application objects will be called from multiple operating system processes. | 0 | This value will always be false on modjy. |
| wsgi.run_once | A boolean value indicating if the server expects to call the application object only once. | 0 | Under modjy, this value depends solely on the value of the cache_callables parameter. If caching of application objects is disabled, wsgi.run_once will be true, otherwise it will be false. |
| wsgi.file_wrapper | TBD | TBD | TBD |
B - Required CGI environment variables
All of the following variables appear in the WSGI dictionary as string values.
| Variable | Description | Example values | Modjy notes |
|---|---|---|---|
| REQUEST_METHOD | The HTTP request method for this request. | GET, PUT, POST, etc. | |
| SCRIPT_NAME | The initial portion of the request URL's path that corresponds to the application object. | ||
| PATH_INFO | The remainder of the request URL's path, after the SCRIPT_NAME has been subtracted. | ||
| QUERY_STRING | A string containing the query portion of the URL request. | "name1=value&name2=value2", etc. | |
| CONTENT_TYPE | The mime type of the body content being uploaded for this request. | "application/octet-stream", etc. | May be the empty string, e.g. if there is no body being uploaded. |
| CONTENT_LENGTH | The length of the body content being uploaded for this request. | May be the empty string, e.g. if there is no body being uploaded. | |
| SERVER_NAME | The name of the network host on which this request was received. | "www.myserver.com", etc. | This value is that returned by the HttpServletRequest method getLocalName(), which was introduced in the servlet 2.4 specification. For a discussion of the difference between the getLocalName() and getServerName() methods, see the Sun document J2EE 1.4 Compatibility Issues and the CGI specification, section 4.1.14. |
| SERVER_PORT | The number of the network port on which this request was received. | "8080", "80", etc. | This value is that returned by the HttpServletRequest method getLocalPort(). See the notes above under SERVER_NAME for a discussion. |
C - Optional CGI environment variables.
In addition to the CGI variables required by WSGI, modjy also supplies values for the following CGI environment variables, both standard and non-standard.
All of the following variables appear in the WSGI dictionary as string values.
| Variable | Description | Example values | Modjy notes |
|---|---|---|---|
| PATH_TRANSLATED | |||
| SERVER_PROTOCOL | The HTTP version in use for this request. | HTTP/1.1, HTTP/1.0, etc. | |
| REMOTE_HOST | The host name for the remote machine making this request. | machine.domain.com | This value will only be set to a dns name if your J2EE container is configured to do reverse-DNS lookups on request IP addresses. If your container is not so configured, then this will contain the numeric IP address of the remote machine, i.e. this variable will contain the same value as REMOTE_ADDR, described below. With Tomcat 6, reverse-dns lookups are configured through the enableLookups attribute of <Connector> elements, e.g. HTTP Connectors for standalone Tomcats or AJP Connectors if you're hosting your Tomcat inside an Apache server. |
| REMOTE_ADDR | The IP address of the remote machine making this request. | "<IP_ADDRESS>", "<IP_ADDRESS>", etc. | This will contain a string representation of the numeric IP address of the remote machine. |
| REMOTE_PORT | The TCP port number through which this request was made from the client machine. | "1234", "2345", etc. | This will contain a string representation of the port number of the socket from the remote machine which provided the request. |
| HTTPS | A string indicating if the request was received through a secure HTTPS connection. | either "on", "off", etc. | |
| AUTH_TYPE | A string indicating the type of authentication credentials included in this request. | "BASIC", "DIGEST", etc. | If there was no authentication information supplied with this request, this variable will not appear in the WSGI environment. |
| REMOTE_USER | A string indicating the login name of the user making this request. | If there was no authentication information supplied with this request, this variable will not appear in the WSGI environment. |
The following is a table of the modjy-specific environment variables that appear in the WSGI environment.
| Variable | Description | Value | Notes |
|---|---|---|---|
| modjy.version | The version of modjy currently running. | (0, 22, 3) | The value will always be a 3-tuple, containing the (major, minor, micro) release numbers. |
| modjy.params.* | The parameters with which this modjy servlet instance was configured. This allows the WSGI application to inspect the configuration of the container. | For example, the modjy configuration variable cache_callables will appear in the WSGI namespace under the name modjy.params.cache_callables. The values of each variable/parameter will be the original string from the servlet configuration file. |
As well as setting WSGI environment variables for its own container specific things, modjy also creates environment variables to provide access to the original J2EE objects for a HTTP request and for the modjy servlet. Those variables are listed in the table below.
NB: The usage of these variables is a breach of the WSGI specification! Reliance on being able to bypass WSGI middleware and access the original request directly will make your application non-portable, possibly broken and may even cause middleware meltdown!
NB: These objects and their contents are only guaranteed to be valid for the duration of the current request/response cycle. Should it be desired to retain any information from any of these objects, a copy of the object or its values should be taken, or of corresponding values from the WSGI environment.
| Variable | Value |
|---|---|
| j2ee.request | The javax.servlet.http.HttpRequest object corresponding to this WSGI request. |
| j2ee.response | The javax.servlet.http.HttpResponse object corresponding to this WSGI request. |
| j2ee.servlet | The javax.servlet.http.HttpServlet under which this WSGI request is running, i.e. the modjy servlet. |
| j2ee.servlet_context | The javax.servlet.ServletContext object for the j2ee.servlet defined above. |
| j2ee.servlet_config | The javax.servlet.ServletConfig object for the j2ee.servlet defined above. |
A - Reading beyond content-length on wsgi.input
According to the WSGI 1.0 specification: "The server is not required to read past the client's specified Content-Length, and is allowed to simulate an end-of-file condition if the application attempts to read past that point. The application should not attempt to read more data than is specified by the CONTENT_LENGTH variable".
Currently, modjy's behaviour in this situation is defined by the J2EE container. This is because the standard J2EE javax.servlet.ServletInputStream (on which the modjy wsgi.input is layered) is container-defined, and will thus have container-specific behaviour.
This means that applications can potentially expect any of the following behaviours, depending on the J2EE container
In the future, I may implement a wrapper for wsgi.input that provides uniform, and perhaps configurable, behaviour across J2EE containers in this scenario.
B - 100 Continue
TBD.
What happens to diagnostic output is dependent on the configuration of the J2EE servlet container in which modjy is running. Under Tomcat 6, for example, this is configured through the swallowOutput attribute of org.apache.catalina.core.StandardContext objects. A true setting for this attribute will cause java System.err and System.out output, and thus wsgi.errors, to appear in the servlet context log.
The WSGI spec describes optional support for bulk file transfers. Modjy currently does not support this feature, but may in the future.


HOME
ME
INTERESTS
BIRDS
Ostriches have the largest eyes of any land animal.
Some ducks sleep with one eye open.
Budgies are susceptible to contagious yawning.
About two-thirds of all bird species live in tropical rainforests.
Kiwis have nostrils on the end of their beaks.
The Wandering Albatross has the greatest wingspan at up to 3.7m.
Birds have three fingers on each wing.
Ostriches have the largest eyes of any land animal.
Some ducks sleep with one eye open.
Budgies are susceptible to contagious yawning.
About two-thirds of all bird species live in tropical rainforests.
Kiwis have nostrils on the end of their beaks.
The Wandering Albatross has the greatest wingspan at up to 3.7m.
Birds have three fingers on each wing.
Penguins are cute, Albatrosses are cooler.
BIRD FACTS!
A group of flamingos is called a flamboyance.
An albatross can soar for as long as six hours without moving its wings.
A Bald Eagle is “bald” because it is piebald (black and white)
"Anatidaephobia" is the phobia that somewhere, somehow a duck or goose is constantly watching them.
A group of ravens is called an “unkindness” or a "conspiracy".
Penguins can jump as high as 6 feet in the air.
Bassian thrush are known to dislodge prey out of leaves by directing their farts at them.
A group of flamingos is called a flamboyance.
An albatross can soar for as long as six hours without moving its wings.
A Bald Eagle is “bald” because it is piebald (black and white)
"Anatidaephobia" is the phobia that somewhere, somehow a duck or goose is constantly watching them.
A group of ravens is called an “unkindness” or a "conspiracy".
Penguins can jump as high as 6 feet in the air.
Bassian thrush are known to dislodge prey out of leaves by directing their farts at them.


Joe Song
Computer Science and Engineering Major at OSU Media and Recruitment Coordinator of ACM-W Central Ohio Community Representative of OCWIC


TFrameViewer and TFrameBrowser components
property fvOptions: set of (fvMetaRefresh, fvNoBorder, fvOverLinksActive, fvNoLinkUnderline,
fvPrintTableBackground, fvPrintBackground, fvShowVScroll, fvNoFocusRect,
fvNoWheelMouse);
The fvOptions property currently has the following items:
fvMetaRefresh
Turns on the response to the <META Http-Equiv="refresh" ...> tag allowing the document to be replaced as specified by the Contents attribute.
fvNoBorder
Turns off TFrameViewer's extra outer border. Use in design phase only.
fvNoFocusRect
Turns off the black outer rectangle which appears when a frame has the focus. Note that this rectangle never shows if fvNoBorder is active.
fvNoLinkUnderline
Turns off underlines on links.
When set, link color will change when the mouse is over the link. Color is determined by the special olink attribute of the <body> tag, if present. If this attribute is absent, the DefOverLinkColor property determines the color.
fvPrintMonchromeBlack
If set, monochrome printers will always print black. This is set by default since some monochrome printers apparently will not print anything but black.
fvPrintBackground
If set, allows all background images and colors to be printed. This is off by default.
fvPrintTableBackground
If set, allows Table background colors to be printed. This is set by default.
fvShowVScroll
htShowVScroll alters the way the vertical scrollbar is displayed. If fvShowVScroll is not set, no vertical scrollbar is shown if none is needed. If fvShowVScroll is set, the vertical scrollbar will always be shown but may be shown as disabled.
fvNoWheelMouse
fvNoWheelMouse disables the mouse wheel so that the middle button may be used for other purposes.
fvShowDummyCaret
Shows the current character position when text is not otherwise selected.
fvNoLinkHilite
If set, prevents the selection or highlighting of portions of links. This can make clicking a link more reliable by avoiding the possibility of making a selection instead.


Defines the balancing mode used for variable composition calculations.
int jmpSetBalancing (int modeFlag, char *balanceElement);
modeFlag [input]
Integer that specifies the type of balancing to perform when running a concentration stepping calculation. This parameter can take one of the following values:
JMP_BALANCE_ONE_ELEMENT |
To compensate the change in concentration of the varying element by applying a reverse change to one of the other elements in the alloy. |
JMP_BALANCE_ALL_ELEMENTS |
To compensate the change in concentration of the varying element by distributing the required balance over all other elements in the alloy. |
balanceElement [input, optional]
Pointer to a null-terminated sequence of characters that specifies the element to which to apply the balancing. When specifying the mode flag JMP_BALANCE_ALL_ELEMENTS
, this parameter is ignored and may be set to NULL
.
JMP_STATUS_OK |
Request has been successfully completed. |
JMP_STATUS_ERROR |
An error has occurred. |
| Headers | jmpcore.h jmpsolver.h |
| Library | jmpsolver.lib /jmpsolver_x64.lib |
| DLL | jmpsolver.dll /jmpsolver_x64.dll |


In this Blog Post I go over getting started with Rustlang
Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.
Installing Rust in Mac OS X / Linux
To install Rust in Linux and Mac OS X run the following command:
curl https://sh.rustup.rs -sSf | sh
For other installation methods please read the following documentation
Add the cargo binaries to your path by adding the following entry in ~/.profile, ~/.bash_profile ~/.bashrc, ~/.zshrc:
echo "export PATH="$HOME/.cargo/bin:$PATH"" >> ~/.zshrc
Note here that I appended this using >>
to ~/.zshrc but you may choose ~/.bashrc or another file.
Installing Rust in Windows
On Windows, go to install and follow the instructions for installing Rust.
You will need to install C++ build tools for Visual Studio 2013 or later here but it is best to just install Visual Studio Build Tools
The tools are in the Other Tools and Frameworks section which looks like this:
Check Rust Installation
If Rust is installed then you should be able to run the following command in your terminal windows:
rustc --version
rustc 1.26.2 (594fb253c 2018-06-01)
Updating Rustlang
In order to update the rust binaries you need to run the following command:
rustup update
Now when we run the --version
options with rustc a new version should be reported if the update found a new version:
rustc --version
rustc 1.28.0 (9634041f0 2018-07-30)
Viewing Local Documentation
In order to see local documentation you can run the following command:
rustup doc
Cargo Package Manager
Rust comes with a powerful package manager called cargo.
You can create new rust packages by using the cargo package manager.
The command you use is: cargo new
with options and provide a name
Create a new binary template package
If you want to create a new rust binary template application then run the following command:
cargo new PACKAGE_NAME --bin
Create a new library template package
If you want to create a new rust library template application then run the following command:
cargo new PACKAGE_NAME --lib
The contents of a new binary template package
When you create a new binary application cargo will create the following files:
Notice that the following files were created:
- Cargo.lock
- This file locks your dependencies
- Cargo.toml
- This file will have basic metadata for the rust package and dependencies
- src directory
- main.rs
- This will be main program for the application
- main.rs
Contents of Cargo.toml:
[package]
name = "hello-world"
version = "0.1.0"
authors = ["Jean-Marcel Belmont<EMAIL_ADDRESS>[dependencies]
Contents of src/main.rs:
fn main() {
println!("Hello, world!");
}
Build Rust package
In order to build the rust package you can run the following command:
This command generates the target directory with the following content:
Running Rust package
You could run rust packages by issuing the following command:
cargo run
This will build and run the rust package.
Check the Rust package for build issues
You can use the cargo check
command to see if your package will build but it will not create an executable.
Running tests with Cargo
You can easily run tests by using the cargo test
command in the root of your project.
Whenever you create a new library it will stub out a test for you in src/lib.rs:
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
Notice here that there is an annotation of #[cfg(test)]
for the tests module and #[test]
for the test case.
Resources to learn Rustlang
Arguably the best resource to learn Rust is the Official Rust Book.
The Rust Book will walk you through each and every aspect of the Rust Language so please read it!
Read the Standard Library
Another way to learn Rust well is to read the Standard Library Here
You should also read A tour of the standard library which will list all the primitive types, modules, and macros in Rust.
Using the Rust Playground
Much like Golang has a playground so does Rust.
Here is a link to the Rust Playground that uses Rust nightly and has debugging mode enabled Here
Community Links
Check out all the community related information Here
If you like this Blog Post please follow me at jbelmont at Github


France introduces baguette vending machines (but they're not a hit with everyone)
The decline of the bakery in France has led to a gap in the market that is being filled by the baguette machine - but what's it like to get your bread from a machine?
by Milo Boyd, https://www.mirror.co.uk/authors/milo-boyd/The French national institution of the boulangerie has fallen on hard times - and the high-tech solution has split a country.
The installation of vending machines to distribute the famous baguettes is symbolic of the 'tragic' death of French bakeries, some fear.
As with high street businesses across the world, changing shopping habits powered by online retailers and out of town supermarkets have left independent bakeries in France desperate for customers.
As a result many have closed, dooming villages and towns to life without easy-to-access delicious fresh bread.
Rushing in to fill the gap in the market they've left is an invention seemingly at odds with the long and proud tradition of French bread - the baguette vending machine.
Rather than a steaming loaf of pain fresh from the oven, many people in France are now slotting a Euro into large metal machines with names such as 'Le Bread Xpress'.
The machines are stocked with about 100 par-baked loaves of bread produced by local wholesale bakeries.
Once the customer has made their choice, the machine finishes baking the selected loaf.
Those lucky enough to be using a machine made by Le Bread Xpress - a brand so French its official logo replaces the 'A' in 'bread' for an Eiffel Tower - can be snacking on a loaf 30 seconds after making the purchase.
The machines are increasingly common in France, with the brand alone having reached 120 locations as of March this year.
Unsurprisingly, reaction to the automated bread vendors is mixed.
"Nice hot crispy crust on on the outside soft on the inside," one person enthused of their Le Bread Xpress loaf.
"The way bread should be. I bought one on the way out of the mall and nearly at the entire thing before I got home."
Another pain purchaser was less impresed.
"Don't waste your money," they wrote online. "Basically microwaved to finish cooking. We threw it out after a few bites."
Many are equally as unhappy with the decline of the traditional bakery that has led to the machines' rise.
"Our little village is dying,” said Hélène Collard, whose family has lived in La Chapelle-en-Juger in Normandy for four generations, told The New York Times about the closure of the village's last bakery.
“We’re no longer in contact with the other inhabitants. It was the only meeting point left.”
Two years ago La Chapelle-en-Juger lost its last bakery in what one newspaper called "un drame" or "a tragedy".
Others have gone further in describing how things are post-boulangerie.
“Without bread, there is no more life,” said Gérard Vigot as he stood across the street from the now empty bakery
"This is a dead village.”


Kubeconfig zip file
Add kubeconfig
Fabric network spec yaml file
Add network spec file
Create Network
Submitted Fabric Network Requests
Network ID
Status
Logs
Submitted At
Operation
{{ range .nets }}
{{ . }}
Pending
null
Add Organization
Add New Organization Into Existing Channels
Organization name
Organization Certificates
Add new org certs tar gzip
Specify channel list (string separated by commas,like channel1,channel2,channel3)
Cancel
Add Organization
Delete
{{ end}}


Katib GRPC API v1beta1
HP or NAS algorithm settings.
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
HP or NAS algorithm specification.
| Field | Type | Label | Description |
| algorithm_name | string |
|
|
| algorithm_settings | AlgorithmSetting | repeated |
|
| Field | Type | Label | Description |
| trial_name | string |
|
EarlyStoppingRule represents single early stopping rule.
| Field | Type | Label | Description |
| name | string | Name of the rule. Usually, metric name. |
|
| value | string | Value of the metric. |
|
| comparison | ComparisonType | Correlation between name and value, one of equal, less or greater |
|
| start_step | int32 | Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric. |
Early stopping algorithm settings.
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
Early stopping algorithm specification.
| Field | Type | Label | Description |
| algorithm_name | string |
|
|
| algorithm_settings | EarlyStoppingSetting | repeated |
|
Structure for a single Experiment.
| Field | Type | Label | Description |
| name | string | Name for the Experiment. |
|
| spec | ExperimentSpec | Experiment specification. |
Specification of an Experiment. Experiment represents a single optimization run over a feasible space.
Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.
It is assumed that objective function f(x) does not change in the course of an Experiment.
| Field | Type | Label | Description |
| parameter_specs | ExperimentSpec.ParameterSpecs |
|
|
| objective | ObjectiveSpec | Objective specification for the Experiment. |
|
| algorithm | AlgorithmSpec | HP or NAS algorithm specification for the Experiment. |
|
| early_stopping | EarlyStoppingSpec | Early stopping specification for the Experiment. |
|
| parallel_trial_count | int32 | How many Trials can be processed in parallel. |
|
| max_trial_count | int32 | Max completed Trials to mark Experiment as succeeded. |
|
| nas_config | NasConfig | NAS configuration for the Experiment. |
List of ParameterSpec.
| Field | Type | Label | Description |
| parameters | ParameterSpec | repeated |
|
Feasible space for optimization.
Int and Double type use Max/Min.
Discrete and Categorical type use List.
| Field | Type | Label | Description |
| max | string | Max Value |
|
| min | string | Minimum Value |
|
| list | string | repeated | List of Values. |
| step | string | Step for double or int parameter |
| Field | Type | Label | Description |
| early_stopping_rules | EarlyStoppingRule | repeated |
|
| Field | Type | Label | Description |
| experiment | Experiment |
|
|
| trials | Trial | repeated |
|
| db_manager_address | string |
|
| Field | Type | Label | Description |
| observation_log | ObservationLog |
|
| Field | Type | Label | Description |
| trial_name | string |
|
|
| metric_name | string |
|
|
| start_time | string | The start of the time range. RFC3339 format |
|
| end_time | string | The end of the time range. RFC3339 format |
| Field | Type | Label | Description |
| parameter_assignments | GetSuggestionsReply.ParameterAssignments | repeated |
|
| algorithm | AlgorithmSpec |
|
|
| early_stopping_rules | EarlyStoppingRule | repeated |
|
| Field | Type | Label | Description |
| assignments | ParameterAssignment | repeated |
|
| trial_name | string | Optional field to override the trial name |
|
| labels | GetSuggestionsReply.ParameterAssignments.LabelsEntry | repeated | Optional field to add labels to the generated Trials |
| Field | Type | Label | Description |
| key | string |
|
|
| value | string |
|
| Field | Type | Label | Description |
| experiment | Experiment |
|
|
| trials | Trial | repeated | All completed trials owned by the experiment. |
| current_request_number | int32 | The number of Suggestions requested at one time. When you set 3 to current_request_number, you get three Suggestions at one time. |
|
| total_request_number | int32 | The number of Suggestions requested till now. |
GraphConfig contains a config of DAG
| Field | Type | Label | Description |
| num_layers | int32 | Number of layers |
|
| input_sizes | int32 | repeated | Dimensions of input size |
| output_sizes | int32 | repeated | Dimensions of output size |
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
| Field | Type | Label | Description |
| time_stamp | string | RFC3339 format |
|
| metric | Metric |
|
NasConfig contains a config of NAS job
| Field | Type | Label | Description |
| graph_config | GraphConfig | Config of DAG |
|
| operations | NasConfig.Operations | List of Operation |
| Field | Type | Label | Description |
| operation | Operation | repeated |
|
Objective specification.
| Field | Type | Label | Description |
| type | ObjectiveType | Type of optimization. |
|
| goal | double | Goal of optimization, can be empty. |
|
| objective_metric_name | string | Primary metric name for the optimization. |
|
| additional_metric_names | string | repeated | List of additional metrics to record from Trial. This can be empty if we only care about the objective metric. |
| Field | Type | Label | Description |
| metrics | Metric | repeated |
|
| Field | Type | Label | Description |
| metric_logs | MetricLog | repeated |
|
Config for operations in DAG
| Field | Type | Label | Description |
| operation_type | string | Type of operation in DAG |
|
| parameter_specs | Operation.ParameterSpecs |
|
List of ParameterSpec
| Field | Type | Label | Description |
| parameters | ParameterSpec | repeated |
|
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
Config for a hyperparameter.
Katib will create each Hyper parameter from this config.
| Field | Type | Label | Description |
| name | string | Name of the parameter. |
|
| parameter_type | ParameterType | Type of the parameter. |
|
| feasible_space | FeasibleSpace | FeasibleSpace for the parameter. |
| Field | Type | Label | Description |
| trial_name | string |
|
|
| observation_log | ObservationLog |
|
| Field | Type | Label | Description |
| trial_name | string |
|
Structure for a single Trial.
| Field | Type | Label | Description |
| name | string | Name for the Trial. |
|
| spec | TrialSpec | Trial specification. |
|
| status | TrialStatus | Trial status. |
Specification of a Trial. It represents Trial's parameter assignments and objective.
| Field | Type | Label | Description |
| objective | ObjectiveSpec | Objective specification for the Trial. |
|
| parameter_assignments | TrialSpec.ParameterAssignments | List of assignments generated for the Trial. |
|
| labels | TrialSpec.LabelsEntry | repeated | Map of labels assigned to the Trial |
| Field | Type | Label | Description |
| key | string |
|
|
| value | string |
|
List of ParameterAssignment
| Field | Type | Label | Description |
| assignments | ParameterAssignment | repeated |
|
Current Trial status. It contains Trial's latest condition, start time, completion time, observation.
| Field | Type | Label | Description |
| start_time | string | Trial start time in RFC3339 format |
|
| completion_time | string | Trial completion time in RFC3339 format |
|
| condition | TrialStatus.TrialConditionType | Trial current condition. It is equal to the latest Trial CR condition. |
|
| observation | Observation | The best Trial observation in logs. |
Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid
| Field | Type | Label | Description |
| experiment | Experiment |
|
Return INVALID_ARGUMENT Error if Early Stopping Settings are not Valid
| Field | Type | Label | Description |
| early_stopping | EarlyStoppingSpec |
|
| Name | Number | Description |
| UNKNOWN_COMPARISON | 0 | Unknown comparison, not used |
| EQUAL | 1 | Equal comparison, e.g. accuracy = 0.7 |
| LESS | 2 | Less comparison, e.g. accuracy < 0.7 |
| GREATER | 3 | Greater comparison, e.g. accuracy > 0.7 |
Direction of optimization. Minimize or Maximize.
| Name | Number | Description |
| UNKNOWN | 0 | Undefined type and not used. |
| MINIMIZE | 1 | Minimize |
| MAXIMIZE | 2 | Maximize |
Types of value for HyperParameter.
| Name | Number | Description |
| UNKNOWN_TYPE | 0 | Undefined type and not used. |
| DOUBLE | 1 | Double float type. Use "Max/Min". |
| INT | 2 | Int type. Use "Max/Min". |
| DISCRETE | 3 | Discrete number type. Use "List" as float. |
| CATEGORICAL | 4 | Categorical type. Use "List" as string. |
Trial can be in one of 8 conditions.
TODO (andreyvelich): Remove unused conditions.
| Name | Number | Description |
| CREATED | 0 | |
| RUNNING | 1 | |
| SUCCEEDED | 2 | |
| KILLED | 3 | |
| FAILED | 4 | |
| METRICSUNAVAILABLE | 5 | |
| EARLYSTOPPED | 6 | |
| UNKNOWN | 7 |
DBManager service defines APIs to manage Katib database.
| Method Name | Request Type | Response Type | Description |
| ReportObservationLog | ReportObservationLogRequest | ReportObservationLogReply | Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI. |
| GetObservationLog | GetObservationLogRequest | GetObservationLogReply | Get all log of Observations for a Trial. |
| DeleteObservationLog | DeleteObservationLogRequest | DeleteObservationLogReply | Delete all log of Observations for a Trial. |
EarlyStopping service defines APIs to manage Katib Early Stopping algorithms
| Method Name | Request Type | Response Type | Description |
| GetEarlyStoppingRules | GetEarlyStoppingRulesRequest | GetEarlyStoppingRulesReply | |
| SetTrialStatus | SetTrialStatusRequest | SetTrialStatusReply | |
| ValidateEarlyStoppingSettings | ValidateEarlyStoppingSettingsRequest | ValidateEarlyStoppingSettingsReply |
Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms
| Method Name | Request Type | Response Type | Description |
| GetSuggestions | GetSuggestionsRequest | GetSuggestionsReply | |
| ValidateAlgorithmSettings | ValidateAlgorithmSettingsRequest | ValidateAlgorithmSettingsReply |
| .proto Type | Notes | C++ | Java | Python | Go | C# | PHP | Ruby |
| double | double | double | float | float64 | double | float | Float | |
| float | float | float | float | float32 | float | float | Float | |
| int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| uint32 | Uses variable-length encoding. | uint32 | int | int/long | uint32 | uint | integer | Bignum or Fixnum (as required) |
| uint64 | Uses variable-length encoding. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum or Fixnum (as required) |
| sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int | uint32 | uint | integer | Bignum or Fixnum (as required) |
| fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum |
| sfixed32 | Always four bytes. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sfixed64 | Always eight bytes. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| bool | bool | boolean | boolean | bool | bool | boolean | TrueClass/FalseClass | |
| string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode | string | string | string | String (UTF-8) |
| bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str | []byte | ByteString | string | String (ASCII-8BIT) |


@* * Copyright 2021 HM Revenue & Customs * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *@ @(formItem:Form[_], fieldName: String, errorArgs: Seq[Any], args: (Symbol,Any)*)(implicit request: Request[_], messages: Messages) @import uk.gov.hmrc.play.views.html.helpers.input @fieldSetClasses = @{if(formItem.hasErrors){ if(formItem(fieldName).hasErrors){ "form-group-error" } else { formItem.errors.map { error => error.args.nonEmpty match { case true => if(errorArgs.filter(_ == error.args(0)).nonEmpty){ "form-group-error" } case false => " " } } } }}
@args.toMap.get('_inputHint).map { hint =>@hint
}


H.B. Woodrose & Melatonin
Lately, I have been quite curious to try out some new things, preferably in the natural range. A close friend of mine suggested I try out a few things and Hawaiian Baby Woodrose was one of those recommendations. Through my history with psychedelic substances, I have tried psilocybin, LSD, Salvia Divinorum, and MDMA (if that counts). I love them all, but since the quality and quantity is horrible where I live, I have learned to use other resources in order to enlighten myself when I have the time to do it.
I ordered a total of 50 seeds from an online herb/seed store and got them in on a Monday. That night I told myself it would probably be better to wait until tomorrow since I was so tired. Tuesday rolls around, and I get off work at around 12:00am.  I came to a conclusion: eat two seeds before I go to bed and take my melatonin.
Well, I ate the two seeds, felt a little rumble in my stomach, and took my daily herbs and vitamins (vitamin C, Ginseng, Spurlina, St. Johns Wart), and then my melatonin. I always alternate with my melatonin after every night. I go from 6mg to 9mg to 6mg and so forth. Before I passed out, I noticed things were getting a little stranger than usual, but nothing really dramatic. My white room seemed a bit brighter, and the TV was a little louder than normal. I slept for about seven hours straight without and disturbances. I woke up and noticed things were a bit more vivid, my mood was positive, and I was ready to get going for work.
Work was excellent that day. I was in fun loving mood, my surroundings were rich in color, and I was wide awake without any stomach troubles whatsoever. After work I ventured into my room and smoked a bowl of salvia divinorum to calm myself, and went a long playing my videogame for about an hour. I pondered to myself: if I repeated what I did last night, will it happen again? I ate three more seeds that night. My stomach acted up a little more than what it did the previous night, but nothing really serious. I woke up the next morning in almost the same manner. Work that day went along the same way it did yesterday. I enjoyed my time at work and loved being around the people there.
For one whole week now I have been taking about 2-4 seeds every night before I've gone to bed, and I have been waking up in a great mood. I'm not exactly sure whether prolonged use of HBW is dangerous to the internal organs, but I have not noticed any side effects whatsoever. I have yet to try it while awake, so I'm a bit curious on what will happen if I do that.
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


Notebooks
Story: Springbok bones
Springbok bones
Title
Springbok bones
Collection
Summary
A description of the springboks' bones and the |xam names given to them.
Comments
1) Date on p.7267v: 4 August 1878, 2) See also !nanna-sse, 3) This story is found in Book VIII-14
Contributors
|han≠kass'o (Klein Jantje) (VIII)
Date
4 August 1878 (given on or about)
Categories
Keywords
springbok (the names of their bones) , names (|xam) , names (for springboks' bones) , bones (|xam names for the springboks') , bones (a description of the springboks')
Story Pages
7266v-7268v