|
||||||||||
| PREV NEXT | FRAMES NO FRAMES |
| Packages that use Expression | |
|---|---|
| org.apache.xalan.xsltc.compiler |
| Uses of Expression in org.apache.xalan.xsltc.compiler |
|---|
| Subclasses of Expression in org.apache.xalan.xsltc.compiler | |
|---|---|
(package private) class |
AbsoluteLocationPath
|
(package private) class |
AbsolutePathPattern
|
(package private) class |
AlternativePattern
|
(package private) class |
AncestorPattern
|
(package private) class |
AttributeValue
|
(package private) class |
AttributeValueTemplate
|
(package private) class |
BinOpExpr
|
(package private) class |
BooleanCall
|
(package private) class |
BooleanExpr
This class implements inlined calls to the XSLT standard functions true() and false(). |
(package private) class |
CastCall
|
(package private) class |
CastExpr
|
(package private) class |
CeilingCall
|
(package private) class |
ConcatCall
|
(package private) class |
ContainsCall
|
(package private) class |
CurrentCall
|
(package private) class |
DocumentCall
|
(package private) class |
ElementAvailableCall
|
(package private) class |
EqualityExpr
|
(package private) class |
FilteredAbsoluteLocationPath
|
(package private) class |
FilterExpr
|
(package private) class |
FilterParentPath
|
(package private) class |
FloorCall
|
(package private) class |
FormatNumberCall
|
(package private) class |
FunctionAvailableCall
|
(package private) class |
FunctionCall
|
(package private) class |
GenerateIdCall
|
(package private) class |
IdKeyPattern
|
(package private) class |
IdPattern
|
(package private) class |
IntExpr
|
(package private) class |
KeyCall
|
(package private) class |
KeyPattern
|
(package private) class |
LangCall
|
(package private) class |
LastCall
|
(package private) class |
LiteralExpr
|
(package private) class |
LocalNameCall
|
class |
LocationPathPattern
|
(package private) class |
LogicalExpr
|
(package private) class |
NameBase
|
(package private) class |
NameCall
|
(package private) class |
NamespaceUriCall
|
(package private) class |
NotCall
|
(package private) class |
NumberCall
|
(package private) class |
ParameterRef
|
(package private) class |
ParentLocationPath
|
(package private) class |
ParentPattern
|
class |
Pattern
|
(package private) class |
PositionCall
|
(package private) class |
Predicate
|
(package private) class |
ProcessingInstructionPattern
|
(package private) class |
RealExpr
|
(package private) class |
RelationalExpr
|
(package private) class |
RelativeLocationPath
|
(package private) class |
RelativePathPattern
|
(package private) class |
RoundCall
|
(package private) class |
SimpleAttributeValue
|
(package private) class |
StartsWithCall
|
(package private) class |
Step
|
(package private) class |
StepPattern
|
(package private) class |
StringCall
|
(package private) class |
StringLengthCall
|
(package private) class |
UnaryOpExpr
|
(package private) class |
UnionPathExpr
|
(package private) class |
UnparsedEntityUriCall
|
(package private) class |
UnresolvedRef
|
(package private) class |
VariableRef
|
(package private) class |
VariableRefBase
|
| Fields in org.apache.xalan.xsltc.compiler declared as Expression | |
|---|---|
private Expression |
FunctionAvailableCall._arg
|
private Expression |
BooleanCall._arg
|
private Expression |
ArgumentList._arg
|
private Expression |
DocumentCall._arg1
|
private Expression |
DocumentCall._arg2
|
private Expression |
StartsWithCall._base
|
private Expression |
ContainsCall._base
|
private Expression[] |
UnionPathExpr._components
|
private Expression |
UnparsedEntityUriCall._entity
|
private Expression |
Predicate._exp
The predicate's expression. |
private Expression |
TransletOutput._filename
|
private Expression |
FilterParentPath._filterExpr
|
private Expression |
FormatNumberCall._format
|
private Expression |
LangCall._lang
|
private Expression |
UnaryOpExpr._left
|
private Expression |
RelationalExpr._left
|
private Expression |
LogicalExpr._left
|
private Expression |
EqualityExpr._left
|
private Expression |
CastExpr._left
|
private Expression |
BinOpExpr._left
|
private Expression |
KeyCall._name
The name of the key. |
private Expression |
FormatNumberCall._name
|
private Expression |
NameBase._param
|
private Expression |
FilterParentPath._path
|
private Expression |
FilteredAbsoluteLocationPath._path
|
private Expression |
AbsoluteLocationPath._path
|
private Expression |
UnionPathExpr._pathExpr
|
private Expression |
FilterExpr._primary
Primary expression of this filter. |
private Expression |
UnionPathExpr._rest
|
private Expression |
RelationalExpr._right
|
private Expression |
LogicalExpr._right
|
private Expression |
EqualityExpr._right
|
private Expression |
CastCall._right
A reference to the expression being casted. |
private Expression |
BinOpExpr._right
|
private Expression |
WithParam._select
Parameter's default value. |
protected Expression |
VariableBase._select
|
private Expression |
ValueOf._select
|
private Expression |
Sort._select
|
private Expression |
ForEach._select
|
private Expression |
CopyOf._select
|
private Expression |
ApplyTemplates._select
|
private Expression |
ParentLocationPath._step
|
private Expression |
When._test
|
private Expression |
If._test
|
(package private) Expression |
FunctionCall._thisArgument
Stores reference to object for non-static Java calls |
private Expression |
StartsWithCall._token
|
private Expression |
ContainsCall._token
|
private Expression |
Key._use
The expression that generates the values for this key. |
private Expression |
Predicate._value
Cached value of method getCompareValue() . |
private Expression |
Number._value
|
private Expression |
KeyCall._value
The value to look up in the key/index. |
private Expression |
FormatNumberCall._value
|
| Methods in org.apache.xalan.xsltc.compiler that return Expression | |
|---|---|
protected Expression |
FunctionCall.argument()
|
protected Expression |
FunctionCall.argument(int i)
|
Expression |
Predicate.getCompareValue()
Returns the value in an expression of the form 'step = value'. |
Expression |
Predicate.getExpr()
|
protected Expression |
FilterExpr.getExpr()
|
Expression |
CastExpr.getExpr()
|
Expression |
VariableBase.getExpression()
Returns the expression from this variable's select attribute (if any) |
Expression |
EqualityExpr.getLeft()
|
Expression |
FilteredAbsoluteLocationPath.getPath()
|
Expression |
AbsoluteLocationPath.getPath()
|
Expression |
EqualityExpr.getRight()
|
Expression |
ParentLocationPath.getStep()
|
Expression |
When.getTest()
|
Expression |
Parser.parseExpression(SyntaxTreeNode parent,
java.lang.String exp)
Parse an XPath expression: |
Expression |
Parser.parseExpression(SyntaxTreeNode parent,
java.lang.String attr,
java.lang.String def)
Parse an XPath expression: |
| Methods in org.apache.xalan.xsltc.compiler with parameters of type Expression | |
|---|---|
protected void |
FunctionCall.setArgument(int i,
Expression exp)
|
static void |
Sort.translateSortIterator(ClassGenerator classGen,
MethodGenerator methodGen,
Expression nodeSet,
java.util.Vector sortObjects)
Compiles code that instantiates a SortingIterator object. |
| Constructors in org.apache.xalan.xsltc.compiler with parameters of type Expression | |
|---|---|
AbsoluteLocationPath(Expression path)
|
|
ArgumentList(Expression arg,
ArgumentList rest)
|
|
BinOpExpr(int op,
Expression left,
Expression right)
|
|
CastExpr(Expression left,
Type type)
Construct a cast expression and check that the conversion is valid by calling typeCheck(). |
|
EqualityExpr(int op,
Expression left,
Expression right)
|
|
FilteredAbsoluteLocationPath(Expression path)
|
|
FilterExpr(Expression primary,
java.util.Vector predicates)
|
|
FilterParentPath(Expression filterExpr,
Expression path)
|
|
LogicalExpr(int op,
Expression left,
Expression right)
Creates a new logical expression - either OR or AND. |
|
ParentLocationPath(RelativeLocationPath path,
Expression step)
|
|
Predicate(Expression exp)
Initializes a predicate. |
|
RelationalExpr(int op,
Expression left,
Expression right)
|
|
UnaryOpExpr(Expression left)
|
|
UnionPathExpr(Expression pathExpr,
Expression rest)
|
|
||||||||||
| PREV NEXT | FRAMES NO FRAMES |


--- layout: post tags: [Miscellaneous] date: 2009-03-23 19:03:38 UTC title: "Joining Safewhere" --- {% include JB/setup %}
Today I spent my first working day at Safewhere, where I'll be working as a Senior Software Engineer. In the last couple of years, Safewhere and I have crossed paths a couple of times, and each time they always left me with the impression of a very professional and congenial ISV, so I'm super-excited to be joining!
Expect the occasional Geneva or Federation-related blog post to intermingle with the usual TDD stuff in the future :)
Comments


-
Real Estate data miner cli tool
Python MIT Updated17 minutes ago -
Introduction to the File Searcher App from Talk Python to me
Python MIT Updated7 days ago -
dotworkflow is vim and dot files used for development
Vim script MIT Updated10 days ago -
a Dungeons and Dragons / MUD style text-based role playing game involving wizards and creatures text game
Python MIT Updated16 days ago -
30 Day Challenge from http://wesbos.com/javascript30/
HTML Updated27 days ago -
Birthday countdown app from Python Jumpstart by Building 10 Apps
Python MIT Updatedon Jul 23 -
a basic weather client app
Python MIT Updatedon Jul 23 -
Introducing the LOLCat Factory App from TPTM course
Python MIT Updatedon Jul 23 -
collection of python algorithm and mini projects
Python Updatedon Jul 17 -
journey to C programming language. the outcome who knows ;)
C MIT Updatedon Jul 17 -
basic journal app in python
Python MIT Updatedon Jun 30 -
GuessNumber from Python Jumpstart by Building 10 Apps
Python Updatedon May 13 -
Init Python Project from Python Jumpstart by Building 10 Apps
Python Updatedon May 11 -
server-side code to store a list of your favorite movies, including box art imagery and a movie trailer URL.
-
My Road Trip with javascript theory, algorithm and functions
JavaScript LGPL-3.0 Updatedon May 7 -
Portfolio Web and Blog Site
JavaScript MIT Updatedon May 5 -
CSS-Grid tutorial from wesbos course
CSS Updatedon May 3 -
A Collection of professional front-end template for building fast, robust, and adaptable web apps or sites
HTML Updatedon Apr 22 -
Portfolio SIte
CSS Updatedon Apr 12 -
PDF-file replicate that is designed in HTML and CSS for developing a responsive website
-
a simple news paper like blogger application
HTML Updatedon Apr 3 -
An cat command line tool with super powers
Go GPL-3.0 Updatedon Mar 6 -
open-source collect of tutorials, algorithms, function and theories of the golang
Go GPL-2.0 Updatedon Mar 6 -
PSAS (Python System Admin Scripts)
LGPL-3.0 Updatedon Jan 16 -
python-syntax
Forked from hdima/python-syntaxPython syntax highlighting script for Vim
-
This is the code for the FrontEndNews site built during some screencasts from codeschool.
HTML Updatedon Dec 19, 2017 -
Project written in JavaScript to powers an online resume. from Udacity coures project
JavaScript Updatedon Nov 25, 2017 -
A Classic frogger like from udacity course
JavaScript Updatedon Nov 25, 2017 -
My Journey into Programming with different languages
BSD-3-Clause Updatedon Nov 2, 2017 -
My 1st project of React
CSS Updatedon Oct 17, 2017


The recommended Ponymail ID generator is the DKIM-ID generator. It simplifies a message using an algorithm based on DKIM relaxed/simple canonicalisation, hashes it with an SHA-256 HMAC, and then encodes the truncated digest using base32 with the custom alphabet 0-9 b-d f-h j-t v-z and the padding stripped.
As well as plain Python doctests, we also use the hypothesis package to check properties of the DKIM-ID generator algorithm. This has the advantage of providing a kind of partial specification as well as testing the code. The suite can be run using:
PYTHONPATH=../tools python3 dkim_id_test.py
And exported to HTML using docutils and the command:
HTML=1 PYTHONPATH=../tools \ python3 dkim_id_test.py > dkim_id_test.html
The first step of generating a DKIM-ID is to convert all line endings of the input to CRLF by upgrading bare CR and LF characters.
If the message is submitted to the Signer with any local encoding that will be modified before transmission, that modification to canonical [RFC5322] form MUST be done before signing. In particular, bare CR or LF characters (used by some systems as a local line separator convention) MUST be converted to the SMTP-standard CRLF sequence before the message is signed.
We follow the algorithm used in dkim_header in dkim.c in version 2.10 of libopendkim, the implementation of which is this, reformatted for brevity:
for (p = hdr; p < q && *p != '\0'; p++) { if (*p == '\n' && prev != '\r') { /* bare LF */ dkim_dstring_catn(tmphdr, CRLF, 2); } else if (prev == '\r' && *p != '\n') { /* bare CR */ dkim_dstring_cat1(tmphdr, '\n'); dkim_dstring_cat1(tmphdr, *p); } else { /* other */ dkim_dstring_cat1(tmphdr, *p); } prev = *p; } if (prev == '\r') { /* end CR */ dkim_dstring_cat1(tmphdr, '\n'); }
Our version of this algorithm is called rfc5322_endings.
>>> from dkim_id import rfc5322_endings
It works on bytes and produces bytes.
We test properties of the DKIM-ID related functions not by formally proving them, as there are no mainstream frameworks for formal verification of Python (though Nagini may be worth trying), but instead by fuzzing with hypothesis as a property checker.
>>> from hypothesis import given >>> from hypothesis.strategies import from_regex as regex, text
The regex producer outputs str instances, and we use it because hypothesis does not allow us to use patterns or other smart generation with only bytes. Therefore we use the smart str generators and then convert the output to bytes using cp1252 or utf-8 encoding as necessary.
>>> def cp1252(text: str) -> bytes: ... return bytes(text, "cp1252") >>> def utf8(text: str): ... return bytes(text, "utf-8")
We'll also use our own decorator to make tests run automatically.
>>> def thesis(hypo, *args): ... def decorator(func): ... func = hypo(*args)(func) ... func() ... return func ... return decorator
Since rfc5322_endings only converts endings, sequences containing neither CR nor LF are unaffected.
>>> @thesis(given, regex(r"\A[^\r\n]*\Z")) ... def non_cr_lf_unaffected(text: str) -> None: ... data: bytes = utf8(text) ... assert data == rfc5322_endings(data), repr(data)
The algorithm is that any LF not preceded with CR will have one inserted before it, and likewise for CR not followed by LF. Therefore we expect the result to always have the same number of CR and LFs.
>>> @thesis(given, text(alphabet="\r\n.")) ... def cr_lf_same_cardinality(text: str) -> None: ... data: bytes = rfc5322_endings(utf8(text)) ... crs = data.count(b"\r") ... lfs = data.count(b"\n") ... assert crs == lfs, repr(data)
That the number of CRs or LFs will never be reduced.
>>> @thesis(given, text(alphabet="\r\n.")) ... def cr_lf_no_reduce(text: str) -> None: ... a: bytes = utf8(text) ... b: bytes = rfc5322_endings(a) ... assert b.count(b"\r") >= a.count(b"\r"), repr(data) ... assert b.count(b"\n") >= a.count(b"\n"), repr(data)
That if we delete all CRLF subsequences, there will be no CR or LFs remaining in the sequence.
>>> @thesis(given, text(alphabet="\r\n.")) ... def only_crlf_subsequences(text: str) -> None: ... data: bytes = rfc5322_endings(utf8(text)) ... data = data.replace(b"\r\n", b".") ... assert data.count(b"\r") == 0, repr(data) ... assert data.count(b"\n") == 0, repr(data)
That if we split on CR or LF sequences, the input and output will be the same.
>>> @thesis(given, text(alphabet="\r\nabc. ")) ... def non_crlf_subsequences(text: str) -> None: ... def split(data: bytes): ... data = data.replace(b"\r", b"\n") ... while b"\n\n" in data: ... data = data.replace(b"\n\n", b"\n") ... return data.strip(b"\n").split(b"\n") ... data: bytes = utf8(text) ... expected = split(data) ... normed: bytes = rfc5322_endings(data) ... assert split(normed) == expected, repr(data)
And that all of this is equivalent to saying that every CR is now followed by LF and every LF is preceded by CR.
>>> @thesis(given, text(alphabet="\r\n.")) ... def cr_and_lf_pairs(text: str) -> None: ... data: bytes = rfc5322_endings(utf8(text)) ... if b"\r" in data: ... datum: bytes ... for datum in data.split(b"\r")[1:]: ... assert datum.startswith(b"\n"), repr(data) ... if b"\n" in data: ... datum: bytes ... for datum in data.split(b"\n")[:-1]: ... assert datum.endswith(b"\r"), repr(data)
Most importantly, the number of CRLFs in the output must be equal to the number of CRLFs in the input, plus the number of individual CRs and LFs once the CRLFs have been removed.
>>> @thesis(given, text(alphabet="\r\n.")) ... def crlf_count(text: str) -> None: ... nocrlf = text.replace("\r\n", "") ... expected = text.count("\r\n") ... expected += nocrlf.count("\r") ... expected += nocrlf.count("\n") ... data: bytes = rfc5322_endings(utf8(text)) ... assert data.count(b"\r\n") == expected, repr(text)
We'll now give a few examples. First, with no CR or LF.
>>> rfc5322_endings(b"") b'' >>> rfc5322_endings(b"abc") b'abc'
All of the following are equivalent to CRLF.
>>> rfc5322_endings(b"\r") b'\r\n' >>> rfc5322_endings(b"\n") b'\r\n' >>> rfc5322_endings(b"\r\n") b'\r\n'
And the following are equivalent to CRLF CRLF.
>>> rfc5322_endings(b"\r\r") b'\r\n\r\n' >>> rfc5322_endings(b"\n\n") b'\r\n\r\n' >>> rfc5322_endings(b"\n\r") b'\r\n\r\n'
The next important component of DKIM-ID generation is DKIM head canonicalisation using the relaxed canonicalisation algorithm. The algorithm is not trivial, consisting of five separate steps:
- Convert all header field names (not the header field values) to lowercase. For example, convert "SUBJect: AbC" to "subject: AbC".
- Unfold all header field continuation lines as described in [RFC5322]; in particular, lines with terminators embedded in continued header field values (that is, CRLF sequences followed by WSP) MUST be interpreted without the CRLF. Implementations MUST NOT remove the CRLF at the end of the header field value.
- Convert all sequences of one or more WSP characters to a single SP character. WSP characters here include those before and after a line folding boundary.
- Delete all WSP characters at the end of each unfolded header field value.
- Delete any WSP characters remaining before and after the colon separating the header field name from the header field value. The colon separator MUST be retained.
We'll use hypothesis to check each of these properties in turn. The canonicalisation function is called rfc6376_relaxed_head.
>>> from dkim_id import rfc6376_relaxed_head
And to test it, we'll need the lists producer from hypothesis.
>>> from hypothesis.strategies import lists >>> chars = text(alphabet="\x00\t\r\n\f .ABCabc\xc0").map(cp1252) >>> headers = lists(lists(chars, min_size=2, max_size=2))
Step one is to convert header field names only to lowercase. Since other normalisation steps will occur, to test it we need to take only the alphabetical octets.
>>> def alphabetical(data: bytes) -> bytes: ... from typing import Set ... upper: bytes = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ" ... alpha: Set[int] = set(upper + upper.lower()) ... return bytes([b for b in data if b in alpha])
Then we can make a direct comparison.
>>> @thesis(given, headers) ... def step_1_field_names_lower(headers) -> None: ... ks = [alphabetical(kv[0]) for kv in headers] ... for i, (k, v) in enumerate(rfc6376_relaxed_head(headers)): ... assert ks[i].lower() == alphabetical(k), repr(headers)
Including that values use the same case.
>>> @thesis(given, headers) ... def step_1_field_values_case(headers) -> None: ... vs = [kv[1] for kv in headers] ... alpha = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" ... cases = set(alpha + alpha.lower()) ... for i, (k, v) in enumerate(rfc6376_relaxed_head(headers)): ... assert (set(vs[i]) & cases) == (set(v) & cases), repr(headers)
Step two is to unfold continuations by removing CRLF except at the end. This would only produce consistent results if the value is in rfc5322_endings normal form, so we extend the step to remove all CR or LF, except for a trailing CRLF in the header field value.
>>> rfc6376_relaxed_head([[b"", b"\r"]]) [[b'', b'']] >>> rfc6376_relaxed_head([[b"", b"\n"]]) [[b'', b'']] >>> rfc6376_relaxed_head([[b"", b"\r\n"]]) [[b'', b'\r\n']] >>> rfc6376_relaxed_head([[b"", b"...\r"]]) [[b'', b'...']] >>> rfc6376_relaxed_head([[b"", b"...\n"]]) [[b'', b'...']] >>> rfc6376_relaxed_head([[b"", b"...\r\n"]]) [[b'', b'...\r\n']] >>> rfc6376_relaxed_head([[b"", b"a\rb\r\n"]]) [[b'', b'ab\r\n']] >>> rfc6376_relaxed_head([[b"", b"a\nb\r\n"]]) [[b'', b'ab\r\n']] >>> rfc6376_relaxed_head([[b"", b"a\r\nb\r\n"]]) [[b'', b'ab\r\n']]
We do this even though, for example, b"a\r\nb\r\n" is not a possible header field value because the first CRLF is not followed by a space or a tab, meaning that it is not a continuation.
We apply the CR and LF removal to header field names too, following libopendkim, although rfc6376_relaxed_head should never encounter CR or LF in a header field name during DKIM-ID generation. The removal of CR and LF in header names includes CRLF at the end of a header field name, unlike in a header field value where trailing CRLF is retained.
>>> rfc6376_relaxed_head([[b"...\r\n", b""]]) [[b'...', b'']]
>>> header_text = (text(alphabet="\x00\t\r\n\f .ABCabc\xc0") ... .map(cp1252) ... .map(rfc5322_endings)) >>> wild_headers = lists(lists(header_text, min_size=2, max_size=2))
The wild_headers producer gives us headers which have not been normalised, and can therefore be used to test the extended step, e.g. for CR and LF deletion.
>>> @thesis(given, wild_headers) ... def step_2_cr_lf_deletion(headers) -> None: ... for (k, v) in rfc6376_relaxed_head(headers): ... assert b"\r" not in k, repr(headers) ... assert b"\n" not in k, repr(headers) ... if v.endswith(b"\r\n"): ... v = v[:-2] ... assert b"\r" not in v, repr(headers) ... assert b"\n" not in v, repr(headers)
We can also test that any trailing CRLF in a header field value is retained.
>>> @thesis(given, wild_headers) ... def step_2_field_values_trailing_crlf(headers) -> None: ... vs = [kv[1] for kv in headers] ... for i, (k, v) in enumerate(rfc6376_relaxed_head(headers)): ... a = vs[i].endswith(b"\r\n") ... b = v.endswith(b"\r\n") ... assert a == b, repr(headers)
Step three is to reduce all sequences of spaces or tabs to a single space, i.e. all sequences that match [ \t]+ must be replaced with " ". The RFC sounds like it's saying that step three should be applied to both names and values, but may regard the issue as moot since WSP is not allowed in header names according to RFC 5322:
[...] A field name MUST be composed of printable US-ASCII characters (i.e., characters that have values between 33 and 126, inclusive), except colon.
Since RFC 6376 says to convert to RFC 5322 normal form first, that implies removing all characters outside of the range 33 to 126. It is not clear that ignoring characters out of this range, e.g. converting "T\x00o" to "To", has no detrimental security properties. Neither RFC 4409 section 8 nor RFC 6376 section 3.8 and 8 discuss this issue. The latter simply says that "Signers and Verifiers SHOULD take reasonable steps to ensure that the messages they are processing are valid".
In any case, libopendkim also doesn't delete all characters outside the range 33 to 126 in header field names. Instead, it deletes only tab, CR, LF, and space. But RFC 6376 also says in step five to delete "any WSP characters remaining before and after the colon", with "remaining" being the operative word here. This suggests that it did consider the earlier step three to apply to headers too, otherwise the WSP characters would not be "remaining" ones. But if it considered the earlier step three to apply to header field names, then it must also consider that there may be spaces and tabs inside header field names even after RFC 5322 normalisation. Hence, we consider that RFC 6376 is primarily suggesting to apply RFC 5322 line ending normalisation, which notably it introduces by saying "in particular" in section 5.3. We also consider that it suggests reducing spaces and tabs to a single space in step three, answering the question of what to do with "T o" (it remains "T o") and "T\x00o" (it remains "T\x00o").
In summary, we follow RFC 6376 as literally as possible, contrary to libopendkim in this case, and apply step three to header field names.
>>> rfc6376_relaxed_head([[b"Spaced \t \t\tKey", b"Value\r\n"]]) [[b'spaced key', b'Value\r\n']]
With this, rfc6376_relaxed_head accepts arbitrary bytes for names and values, and deals with them in a consistent and considered way, including tab and space other values outside 33 to 126. This also includes retaining colon and semicolon, even though they are problematic in DKIM signing.
>>> rfc6376_relaxed_head([[b":", b"Value\r\n"]]) [[b':', b'Value\r\n']] >>> rfc6376_relaxed_head([[b";", b"Value\r\n"]]) [[b';', b'Value\r\n']]
In the component of the DKIM-ID generator which uses header canonicalisation it's impossible for it to have colon in the header name, but it is possible for it to have semicolon. Such a header could not be signed using DKIM as it uses semicolon as the separator in the list of headers which have been signed, but it will be ignored in DKIM-ID generation as long as the defaults are followed or ";" is not manually specified as a subset header to keep. Another problematic header which is possible is the empty header. The case of a header name starting with WSP also doesn't arise, because such lines are continuation lines.
Overall, there should never be a tab in canonicalised header field names and values, and there should never be a double space in canonicalised header field names and values.
>>> @thesis(given, wild_headers) ... def step_3_field_values(headers) -> None: ... for (k, v) in rfc6376_relaxed_head(headers): ... assert b"\t" not in k, repr(headers) ... assert b"\t" not in v, repr(headers) ... assert b" " not in k, repr(headers) ... assert b" " not in v, repr(headers)
Internally, the function that performs this step is called rfc6376_shrink_head.
>>> from dkim_id import rfc6376_shrink_head
And it should work like a more efficient version of iteratively removing double spaces, except that it also strips leading and trailing whitespace, which is for steps four and five.
>>> @thesis(given, wild_headers) ... def step_3_reduce_iterative(headers) -> None: ... for (k, v) in headers: ... kk = k.replace(b"\t", b" ") ... vv = v.replace(b"\t", b" ") ... while b" " in kk: ... kk = kk.replace(b" ", b" ") ... kk = kk.strip(b" ") ... while b" " in vv: ... vv = vv.replace(b" ", b" ") ... vv = vv.strip(b" ") ... assert rfc6376_shrink_head(k) == kk, repr(k) ... assert rfc6376_shrink_head(v) == vv, repr(v)
This also means that leading whitespace is removed from the beginnings of header names. Again this is not a case which could occur during DKIM-ID generation, in this case because such a name would have been regarded as a continuation, even at the beginning of a message where it is regarded as the continuation of the empty name.
>>> rfc6376_relaxed_head([[b" Key", b"Value\r\n"]]) [[b'key', b'Value\r\n']]
Step four says that spaces and tabs at the end of a header field value are removed.
It is possible to give a header field value without a trailing CRLF to rfc6376_relaxed_head, and so any trailing tabs or spaces there must be removed.
>>> rfc6376_relaxed_head([[b"", b"Value\t "]]) [[b'', b'Value']]
But the RFC 5322 message grammar states that all headers shall end with CRLF. An overly literal reading of RFC 6376 therefore implies that spaces and tabs are never removed from the end of a field value, because the value must always end with CRLF according to RFC 5322. But if they were never removed then there would be no need for the step, so the implication is that the "end" for the purposes of this step is before the trailing CRLF.
A reading of dkim_canon_header_string in libopendkim suggests that it could leave a header ending with space CRLF, but this hasn't been tested. We remove the space correctly.
>>> rfc6376_relaxed_head([[b"Key", b"Value \r\n"]]) [[b'key', b'Value\r\n']]
Indeed, a header field value must never end with space or tab.
>>> @thesis(given, wild_headers) ... def step_4_field_values_ends(headers) -> None: ... for (k, v) in rfc6376_relaxed_head(headers): ... assert not v.endswith(b" "), repr(headers) ... assert not v.endswith(b"\t"), repr(headers)
And must never end with space CRLF or tab CRLF.
>>> @thesis(given, wild_headers) ... def step_4_field_values_ends_2(headers) -> None: ... for (k, v) in rfc6376_relaxed_head(headers): ... assert not v.endswith(b" \r\n"), repr(headers) ... assert not v.endswith(b"\t\r\n"), repr(headers)
Indeed, it should never be possible to contain, let alone end, with a tab anyway after step three since that replaces all sequences of spaces and tabs with a single space, leaving no tabs at all in the output before it reaches step four.
Step five is to remove spaces and tabs from the end of header names, and from the start of header values. Again, all tabs should have been removed anyway in step three, so this step could have specified only removing spaces.
>>> @thesis(given, wild_headers) ... def step_5_wsp_around_colon(headers) -> None: ... for (k, v) in rfc6376_relaxed_head(headers): ... assert not k.endswith(b" "), repr(headers) ... assert not k.endswith(b"\t"), repr(headers) ... assert not v.startswith(b" "), repr(headers) ... assert not v.startswith(b"\t"), repr(headers)
We can combine headers in order to check their size.
>>> from dkim_id import rfc6376_join
This can be used to test one of the general properties of rfc6376_relaxed_head, that it never enlarges the data given to it.
>>> @thesis(given, wild_headers) ... def head_never_enlarged(headers) -> None: ... a: bytes = rfc6376_join(headers) ... h: List[List[bytes]] = rfc6376_relaxed_head(headers) ... b: bytes = rfc6376_join(h) ... assert len(a) >= len(b), repr(headers)
Perhaps the most important general property of canonicalisation is that once canonicalised, attempting to canonicalise again produces the same data. In other words canonicalisation is absolute, and data cannot be canonicalised further.
>>> @thesis(given, wild_headers) ... def recanonicalisation_is_identity(headers) -> None: ... a = rfc6376_relaxed_head(headers) ... b = rfc6376_relaxed_head(a) ... assert a == b, repr(headers)
The body canonicalisation function is called rfc6376_simple_body.
>>> from dkim_id import rfc6376_simple_body
It maps an empty body to CRLF, and then ensures that there is at most one CRLF at the end of the body. Therefore, a consequence is that it ensures that the output is never empty.
>>> @thesis(given, chars) ... def body_not_empty(body) -> None: ... body_c = rfc6376_simple_body(body) ... assert len(body_c) > 0, repr(body)
And that the output never ends CRLF CRLF.
>>> @thesis(given, chars) ... def body_no_trailing_crlfcrlf(body) -> None: ... body_c = rfc6376_simple_body(body) ... assert not body_c.endswith(b"\r\n\r\n") > 0, repr(body)
But it could end non-CR LF CRLF, or CR CRLF if the input were not RFC 5322 ending normalised.
>>> rfc6376_simple_body(b"Non-CR\n\r\n") b'Non-CR\n\r\n' >>> rfc6376_simple_body(b"CR\r\r\n") b'CR\r\r\n'
The function enlarges data only when its input is empty.
>>> @thesis(given, chars.filter(lambda b: b != b"")) ... def body_enlarging_edge(body) -> None: ... body_c = rfc6376_simple_body(body) ... assert len(body_c) <= len(body), repr(body)
The prefix of the output up to any trailing CRLF the shared by the input.
>>> @thesis(given, chars) ... def body_same_prefix(body) -> None: ... body_c = rfc6376_simple_body(body) ... size_c = len(body_c) ... if body_c.endswith(b"\r\n"): ... size_c -= 2 ... assert body[:size_c] == body_c[:size_c], repr(body)
And any remainder must consist solely of CRLFs in both input and output.
>>> @thesis(given, chars) ... def body_suffix_crlfs(body) -> None: ... body_c = rfc6376_simple_body(body) ... size_c = len(body_c) ... if body_c.endswith(b"\r\n"): ... size_c -= 2 ... assert not body[size_c:].replace(b"\r\n", b""), repr(body) ... assert not body_c[size_c:].replace(b"\r\n", b""), repr(body)
The main parser is called rfc6376_split.
>>> from dkim_id import rfc6376_split
It does not perform canonicalisation. If there is no CRLF header and body boundary separator, then it returns None for the body.
Each header field is defined by RFC 5322 as ending with CRLF which is inclusive to that header field. Any CRLF following that indicates the start of a body, which may be empty. Therefore, in the case of the empty document there are no headers and no body.
>>> rfc6376_split(b"") ([], None)
In the case of just CRLF there are no headers, since they must contain at least one character before their CRLF. RFC 5322 section 2.2 says that header fields "are lines beginning with a field name, followed by a colon", which implies at least the presence of a colon, and section 3.6.8 says "field-name = 1*ftext" which means the name must include at least one printable character. As there is nothing after the CRLF in the case of just a CRLF, there is an empty body.
>>> rfc6376_split(b"\r\n") ([], b'')
In the case of CRLF CRLF there are no headers, and there is a body which is CRLF.
>>> rfc6376_split(b"\r\n\r\n") ([], b'\r\n')
And then this pattern repeats.
>>> rfc6376_split(b"\r\n\r\n\r\n") ([], b'\r\n\r\n') >>> rfc6376_split(b"\r\n\r\n\r\n\r\n") ([], b'\r\n\r\n\r\n')
When we have a header, a single trailing CRLF is regarded as part of that header. This means that there is no body.
>>> rfc6376_split(b"Key:Value\r\n") ([[b'Key', b'Value\r\n']], None)
But appending another CRLF to that gives an empty body.
>>> rfc6376_split(b"Key:Value\r\n\r\n") ([[b'Key', b'Value\r\n']], b'')
As rfc6376_split does not perform canonicalisation, we have the edge cases of isolated CRs and LFs. There should never be isolated CRs and LFs in DKIM-ID generation because RFC 5322 ending normalisation is applied before splitting, but in such cases where the function is called with isolated CRs and LFs they are considered as header field name or header field value data.
>>> rfc6376_split(b"\r") ([[b'\r', b'']], None) >>> rfc6376_split(b"\n") ([[b'\n', b'']], None) >>> rfc6376_split(b"\n\r\n") ([[b'\n', b'\r\n']], None) >>> rfc6376_split(b"\r\r\n") ([[b'\r', b'\r\n']], None) >>> rfc6376_split(b"\r...\r\n") ([[b'\r...', b'\r\n']], None) >>> rfc6376_split(b"\n...\r\n") ([[b'\n...', b'\r\n']], None) >>> rfc6376_split(b"\n:\n\r\n") ([[b'\n', b'\n\r\n']], None) >>> rfc6376_split(b"\n...:\n...\r\n") ([[b'\n...', b'\n...\r\n']], None)
A header field name without any header field value is just regarded as being the same as one with an empty value.
>>> rfc6376_split(b"Key\r\n\r\n") ([[b'Key', b'\r\n']], b'') >>> rfc6376_split(b"Key:\r\n\r\n") ([[b'Key', b'\r\n']], b'')
For greater consistency with how bodies are handled, the former could have been interpreted as [b'Key', None], but this would increase the complexity of the code, and lead to the question of where the trailing CRLF ought to be stored.
In some cases, one of the mbox formats may accidentally be passed to rfc6376_split, containing a line like this in its headers, usually at the start but potentially later in the headers too:
"From MAILER-DAEMON Fri Jul 8 12:08:34 2011"
Which would be interpreted as a header field whose name is:
"From MAILER-DAEMON Fri Jul 8 12"
And which could also collect any following continuation line.
>>> rfc6376_split(b"To:You\r\nFrom Me\r\n More\r\n") ([[b'To', b'You\r\n'], [b'From Me', b'\r\n More\r\n']], None)
This is safe because even after canonicalisation it is not possible to confuse a "From " line with a "From:" header field, unless no text follows the "From " and it is followed by a continuation. If no text follows the "From " then it is not in one of the mbox formats anyway. And if it is followed by a continuation, then interpreting it as a From header field is reasonable.
Similarly to a name without a value, a continuation value without a preceding line is treated as though the header field name is empty.
>>> rfc6376_split(b" More\r\n") ([[b'', b' More\r\n']], None)
An alternative to this would be to treat the line itself as a header field name, but then that creates the issue of whether to remove the leading whitespace, and whether to parse a colon in it. It would also make it inconsistent with all other field names, which must not start with a space.
The type of the body, the second element of the tuple returned from rfc6376_split, directly correlates to whether the input starts with CRLF or whether CRLF CRLF occurs in the input. If it does so, then we say that the input message contains a header and body boundary.
>>> def contains_boundary(data: bytes) -> bool: ... return data.startswith(b"\r\n") or (b"\r\n\r\n" in data)
We use a simple subset of all possible inputs to check this correlation.
>>> text_message = (text(alphabet="\x00\t\r\n\f .:ABCabc\xc0") ... .map(cp1252))
Although rfc6376_split should always take input in RFC 5322 ending normal form, we test without that normal form.
>>> @thesis(given, text_message) ... def body_type_correlation(data) -> None: ... headers, body = rfc6376_split(data) ... body_not_none = (body is not None) ... assert contains_boundary(data) is body_not_none, repr(data)
If the input is not RFC 5322 normalised, then CR and LF can appear in header field names, as already demonstrated. Colon, however, should never appear in a header field name.
>>> @thesis(given, text_message) ... def no_split_colon(data) -> None: ... headers, body = rfc6376_split(data) ... for (k, v) in headers: ... assert b":" not in k, repr(data)
And if the input is RFC 5322 normalised, then colon, CR, and LF should never appear in header field names.
>>> @thesis(given, text_message) ... def no_normal_split_chars(data) -> None: ... data = rfc5322_endings(data) ... headers, body = rfc6376_split(data) ... for (k, v) in headers: ... assert b":" not in k, repr(data) ... assert b"\r" not in k, repr(data) ... assert b"\n" not in k, repr(data)
The version of the main parser which performs canonicalisation is called rfc6376_split_canon.
>>> from dkim_id import rfc6376_split_canon
It takes head_subset, head_canon, and body_canon arguments. The first is a set of bytes, lower case header field names to keep when parsing the headers. If head_subset is None, all headers are retained, which is useful for testing. The second is a boolean of whether to apply rfc6376_relaxed_head, and the third is a boolean of whether to apply rfc6376_simple_body and potentially modify the headers too for consistency.
If there was no body, i.e. no header body boundary CRLF in the message, then the returned body should be None rather than b"".
>>> @thesis(given, text_message) ... def body_none(message) -> None: ... boundary = contains_boundary(rfc5322_endings(message)) ... headers, body = rfc6376_split_canon(message) ... assert boundary is (body is not None), repr(message)
We can perform the canonicalisation steps ourselves. We need to import rfc6376_simple_holistic, which ensures that headers are augmented with CRLF if necessary when there is either no body or an empty body but body canonicalisation synthesizes one.
>>> from dkim_id import rfc6376_simple_holistic
And then DKIM relaxed/simple can be applied consistently.
>>> @thesis(given, text_message) ... def manual_canon(message) -> None: ... # uc = uncanonicalised, ec = expected canon, ac = actual canon ... headers_uc, body_uc = rfc6376_split_canon(message) ... headers_ec, body_ec = rfc6376_split_canon(message, ... head_canon=True, body_canon=True) ... headers_ac = rfc6376_relaxed_head(headers_uc) ... headers_ac, body_ac = rfc6376_simple_holistic(headers_ac, body_uc) ... assert headers_ac == headers_ec, repr(message) ... assert body_ac == body_ec, repr(message)
The header and body canonicalisation steps are optional. Even when retaining all headers (which is the default) and performing neither kind of canonicalisation (which is also the default), the input message is not necessarily the same as the output message, whether RFC 5322 normalisation were performed or not. This is because, for example, the construction of broken headers, i.e. those without colons, is fixed in the process.
>>> rfc6376_split_canon(b"Key") ([[b'Key', b'']], None) >>> rfc6376_join(*rfc6376_split_canon(b"Key")) b'Key:'
We call the process of splitting and then joining "reforming". There is a function called rfc6376_reformed that performs this.
>>> from dkim_id import rfc6376_reformed
Then rfc6376_reformed should be exactly equivalent to using rfc6376_split and then rfc6376_join.
>>> @thesis(given, text_message) ... def normal(message) -> None: ... a = rfc6376_join(*rfc6376_split(message)) ... b = rfc6376_reformed(message) ... assert a == b, repr(message)
We can use rfc6376_reformed_canon to canonicalise a message whilst reforming it. This function accepts an additional lid parameter to specify a list ID, in the RFC 2919 sense, and returns a list ID and the canonicalised message. The output list ID will be an empty bytes object if the input list ID was in any List-Id header in the input message.
>>> from dkim_id import rfc6376_reformed_canon
Then if we make our own headers, canonicalise them, and then join them, we should always get a canonicalised message.
>>> @thesis(given, headers) ... def more_manual_canon(headers) -> None: ... headers_c = rfc6376_relaxed_head(headers) ... message_c = rfc6376_join(headers_c) ... assert message_c == rfc6376_reformed_canon(message_c, ... head_canon=True, body_canon=False)[1], repr(message_c)
DKIM-ID generation uses the standard rfc6376_reformed_canon call with rfc4871_subset headers and both head and body canonicalised. We refer to this combination as reformed and relaxed/simple canonicalisation, or just "rascal" for short. The function that performs this is called rfc6376_rascal. Like rfc6376_reformed_canon, this function accepts an additional lid parameter to specify a list ID, in the RFC 2919 sense, and returns a list ID and the canonicalised message.
>>> from dkim_id import rfc6376_rascal
A missing or empty body is encoded, per RFC 6376 simple body canonicalisation, as CRLF. We always perform body canonicalisation if body_canon is True, which means that even if there is no body (i.e. there was no header and body boundary in the original) there will always be body canonicalisation, which means that the body will always be non-empty, and will always be appended by rfc6376_join after the header and body separator CRLF. This means that there will always be a header and body boundary in the rascal output.
>>> @thesis(given, text_message) ... def rascal_contains_boundary(data) -> None: ... rascal = rfc6376_rascal(data)[1] ... assert contains_boundary(rascal), repr(data)
In particular, it means that the empty input document will become CRLF CRLF, which is the header and body separator CRLF followed by the canonicalised empty body CRLF. Two CRLFs, but with completely different roles.
>>> rfc6376_rascal(b"") (b'', b'\r\n\r\n')
And, because trailing CRs or LFs are RFC 5322 ending normalised and then canonicalised to a single CRLF, it means that any sequence of CRs or LFs will be rascaled to CRLF CRLF too.
>>> @thesis(given, text(alphabet="\r\n").map(utf8)) ... def normal_crlfs_to_crlf2(data) -> None: ... rascal = rfc6376_rascal(data)[1] ... assert rascal == b"\r\n\r\n", repr(data)
Since the input is considered to be a message, arbitrary text without metacharacters will usually be regarded as a discardable header field.
>>> rfc6376_rascal(b"Text") (b'', b'\r\n\r\n')
This is true even when colon is included, as long as the prefix is not one of the standard header field names in rfc4871_subset.
>>> rfc6376_rascal(b"Discarded: Value") (b'', b'\r\n\r\n')
But if the header is in the subset, it will indeed be retained. In this case, holistic canonicalisation ensures that CRLF is appended to the header too.
>>> rfc6376_rascal(b"To: Recipient") (b'', b'to:Recipient\r\n\r\n\r\n')
In other words this is a header field b'to:Recipient\r\n', followed by a CRLF header and body boundary, followed by the CRLF of the canonicalised missing body.
If there is no header value for a subset header, then it is treated as if the header value were empty.
>>> rfc6376_rascal(b"To") (b'', b'to:\r\n\r\n\r\n') >>> rfc6376_rascal(b"To:") (b'', b'to:\r\n\r\n\r\n')
RFC 6376 says that canonicalisation should, obviously, come before signing.
Canonicalization simply prepares the email for presentation to the signing or verification algorithm.
But a more subtle consequence of this is that subsetting headers also comes after canonicalisation, because subsetting is not part of canonicalisation - it's part of signing.
This is important in our expansion of the RFC 6376 algorithm to cover all inputs because e.g. it means that header field names with trailing whitespace are treated the same as without that whitespace.
>>> rfc6376_rascal(b"To \n") (b'', b'to:\r\n\r\n\r\n')
But a header name with whitespace inside it is not, unlike in the libopendkim algorithm, treated the same as one without whitespace inside it, for reasons already discussed in the documentation of RFC 6376 header canonicalisation step three.
>>> rfc6376_rascal(b"T o\n") (b'', b'\r\n\r\n')
We use a subset of headers specified in RFC 4871. We use RFC 4871 even though it was obsoleted by RFC 6376 because the earlier RFC has a more extensive list of headers, and the later RFC says anyway that the choice of which headers to include is a matter of choice dependent on the signing environment. Since DKIM-ID generation does not even include signing, our requirements are somewhat different anyway.
>>> from dkim_id import rfc4871_subset
Whenever the rfc4871_subset headers are specified as the subset to be retained, they should indeed be retained in the output of rfc6376_rascal.
>>> for k in rfc4871_subset: ... minimal = k + b":\r\n\r\n\r\n" ... assert minimal == rfc6376_rascal(minimal)[1], repr(minimal)
Though the subset is loosely called the "RFC 4871 subset", there is one header in rfc4871_subset which RFC 4871 doesn't recommend: DKIM-Signature itself.
>>> b"dkim-signature" in rfc4871_subset True
We include the DKIM-Signature header field in the subset of retained headers because then if the sender has signed their message it ought to be reflected in the identifier for that message. It would not have made sense for RFC 4817 to recommend that header field for signing input, because it is itself the signing output! But if, for example, there were an widely implemented RFC specifying a precursor to DKIM which was later superseded by DKIM, it is reasonable to assume that RFC 4817 would have recommended including the output of the precursor in the headers to sign, combining the two approaches. Similarly, since DKIM is a precursor to DKIM-ID, DKIM-ID is able to include its output as an input.
When we have a canonicalised message with subsetted headers, we take the SHA-256 HMAC digest of that message and then encode a truncated version of it using pibble32, which is base32 with the alphabet 0-9 b-d f-h j-t v-z, and remove the padding.
>>> from dkim_id import pibble32
The alphabet used means that the pibble32 output is always lowercase, and never contains the letters a, e, i, or u.
We need the binary producer from hypothesis.
>>> from hypothesis.strategies import binary
And then we can test these general properties.
>>> @thesis(given, binary()) ... def pibble32_general(data) -> None: ... encoded = pibble32(data) ... assert encoded == encoded.lower(), repr(data) ... encoded_set = set(encoded) ... assert not (encoded_set & {"a", "e", "i", "u"}), repr(data)
There may be padding, but only when the data length is not divisible by five.
>>> @thesis(given, binary()) ... def pibble32_padding(data) -> None: ... encoded = pibble32(data) ... no_padding = not encoded.endswith("=") ... divisible_by_five = not (len(data) % 5) ... assert no_padding is divisible_by_five, repr(data)
We strip the padding on the DKIM-ID since it is fixed at a width of 128 bits, and the pibble32 output is byte aligned anyway, i.e. the decoder accepts no other padding than "======".
The length of the pibble32 output will always be the same as when base32 encoding it.
>>> @thesis(given, binary()) ... def pibble32_length(data) -> None: ... from base64 import b32encode ... assert len(pibble32(data)) == len(b32encode(data)), repr(data)
Here are a some specific examples:
>>> pibble32(b"") '' >>> pibble32(b"\x00") '00======' >>> pibble32(b"\x01") '04======' >>> pibble32(b"\x02") '08======' >>> pibble32(b"\xff") 'zw======' >>> pibble32(b"\x00\x00\x00\x00\x00") '00000000' >>> pibble32(b"\x00\x00\x01\x00\x00") '00002000' >>> pibble32(b"\x00\x00\x02\x00\x00") '00004000' >>> pibble32(b"\x00\x00\xff\x00\x00") '000hy000' >>> pibble32(b"\x00\x00\xff\xff\x00") '000hzzr0' >>> pibble32(b"\xff\xff\xff\xff\xff") 'zzzzzzzz'
When the input length is divisible by five, the output length is always 8 / 5 of that length.
>>> @thesis(given, binary()) ... def pibble32_eight_fifths(data) -> None: ... size = len(data) ... resized = size - (size % 5) ... fives = data[:resized] ... assert len(pibble32(fives)) == (resized * 8 / 5), repr(data)
And when it's not divisible by five, the length is rounded up to the next number divisible by five.
This means that 160 bits of input is multiplied by 8 / 5, which gives 256 bits, or 32 bytes, of output.
>>> 160 * 8 // 5 256 >>> 256 // 8 32
Once the rascaled version of the message is obtained, it it hashed and then pibble32 encoded to form the DKIM-ID. We want to check that the output is pibble32 encoded, at least in that its length is correct and its alphabet is a subset of what is expected.
>>> digit = "0123456789" >>> lower = "abcdefghijklmnopqrstuvwxyz" >>> pibble32_alphabet = (set(digit) | set(lower)) - {"a", "e", "i", "u"}
We guard against typos in the alphabet by testing expected properties, first by checking the digits.
>>> assert len(digit) == 10 >>> assert len(set(digit)) == 10 >>> assert list(digit) == sorted(list(digit)) >>> assert digit.isdigit()
Then the lowercase letters.
>>> assert len(lower) == 26 >>> assert len(set(lower)) == 26 >>> assert list(lower) == sorted(list(lower)) >>> assert lower.isalpha()
And then the whole alphabet.
>>> assert len(pibble32_alphabet) == 32
Now we can test the DKIM-ID output, from function dkim_id.
>>> from dkim_id import dkim_id
By checking that its output is consistent with the pibble32 encoding.
>>> @thesis(given, text_message) ... def consistent_output(data) -> None: ... dkimid: str = dkim_id(data) ... assert len(dkimid) == 32, repr(data) ... assert not (set(dkimid) - pibble32_alphabet), repr(data)
We can also check that the unpibbled output is the same as the SHA-256 HMAC of the rascal.
>>> from dkim_id import unpibble32 >>> from hmac import digest as hmac_digest >>> @thesis(given, text_message) ... def check_hash_digest(data) -> None: ... rascal: bytes = rfc6376_rascal(data)[1] ... digest_e: bytes = hmac_digest(b"", rascal, "sha256")[:160 // 8] ... dkimid: str = dkim_id(data) ... digest_a: bytes = unpibble32(dkimid) ... assert digest_a == digest_e, repr(data)
And here are some example outputs for some simple messages.
>>> dkim_id(b"") '8fgp2do75oqo6qd08vs4p7dpp1gj4vjn' >>> dkim_id(b"To: You") 'wowc4vvd0ftwm0q24106mldg67komfl0' >>> dkim_id(b"To: You\r\n") 'wowc4vvd0ftwm0q24106mldg67komfl0' >>> dkim_id(b"To: You\r\nFrom: Me") 'kf7f6zxt7w7k1h1lhxmg9mxngkl5vbcm' >>> dkim_id(b"To: You\r\nFrom: Me\r\n\r\nBody") 'xx5nf02ptvv92tt73kg7n7o9o5t4ngvd' >>> dkim_id(b"To: You\r\nFrom: Me\r\n\r\nBody\r\n") 'b752nf3njqs9r5qwmrkh3n2s24y7y33g'


| View source on GitHub |
Function builder for a dnn logit_fn.
tf.compat.v1.estimator.experimental.dnn_logit_fn_builder( units, hidden_units, feature_columns, activation_fn, dropout, input_layer_partitioner, batch_norm )
units
: An int indicating the dimension of the logit layer. In the
MultiHead case, this should be the sum of all component Heads' logit
dimensions.hidden_units
: Iterable of integer number of hidden units per layer.feature_columns
: Iterable of feature_column._FeatureColumn
model inputs.activation_fn
: Activation function applied to each layer.dropout
: When not None
, the probability we will drop out a given
coordinate.input_layer_partitioner
: Partitioner for input layer.batch_norm
: Whether to use batch normalization after each hidden layer.A logit_fn (see below).
ValueError
: If units is not an int.


Avoid accidentally moving files, editor tabs, tool window buttons, and other UI components. When enabled, hold the Alt key to move things around.
By default, this option is disabled and you can move everything without any extra keys.


Place of Supply : {{invoice.place_of_supply}}
| # | Item&Description | HSN/SAC | Qty | Rate | CGST | SGST | Amount |
|---|---|---|---|---|---|---|---|
| {{ forloop.counter }} | {{transaction.item_details}} |
998314 |
{{transaction.quantity}} |
{{transaction.rate}} |
{{transaction.tax}} 9% |
450.00 9% |
{{transaction.amount}} |
1. Deduct Tax at source wherever applicable-PAN AASCSAQ668F
2. To be settled within one week from date of invoice
3. Payment may also be made through New Banking to the following account
| Bank Name | : | ICIC Bank | |
| Account Name | : | Aadolf Enterprises Private Limited | |
| Account No | : |<PHONE_NUMBER>26 | |
| Branch | : | CHENNAI-T.NAGAR | |
| IFSC Code | : | ICIC0006026 |


A Powershell Todo Manager
For those always at the command line
October 16th, 2019
I spend a lot of time on the command line, monitoring builds and processes. I'm able to get a lot done in one window, especially with tmux. I also use a running list of todo items to help me manage my work and personal life. I've used products from Google, Microsoft, and others to manage todo lists. Those products are great, but I don't like leaving my terminal to manage them.
I've been learning powershell recently. One of the powershell legends is Jeff Hicks and has developed a lot of powershell tools including a todo list manager called MyTasks. So since I am always at the command line and I'm learning powershell anyway it made since to try it out.
MyTasks is in the powershell gallery so it makes installation very easy.
Install-Module MyTasks
The first time you run it you need to set the path for store all your task data (everything is stored as xml on disc).
New-Item -Path "~/Users/Taylor/Documents/" -Name "MyTasks" -ItemType "directory"
Set-MyTaskHome -Path "~/Users/Taylor/Documents/MyTasks"
Now its installed and setup. The primary commands you'll use are New-MyTask, Get-MyTask, and Set-MyTask. Lets go over these.
New-MyTask
New-MyTask -Name "Run Report" -Description "Generate report for finance" `
-Category Work -DueDate "10/20/2019"
New-MyTask -Name "Book Trip" -Category Personal
Get-MyTask
Get-MyTask
ID Name Description DueDate OverDue Category Progress
-- ---- ----------- ------- ------- -------- --------
1 Run Report Generate report for 10/20/19 False Work 0
finance
2 Book Trip 10/23/19 False Personal 0
Get-MyTask -Name "Book Trip"
ID Name Description DueDate OverDue Category Progress
-- ---- ----------- ------- ------- -------- --------
2 Book Trip 10/23/19 False Personal 0
Set-MyTask
Set-MyTask -Name "Book Trip" -Progress 40
Set-MyTask -Name "Run Report" -NewName "Run Finance Report" -DueDate "10/30/2019"
Benefits
Along with being available in the command line, I really like having my todo list "powershell ready" so I can manipulate it with the tools I already know. I could generate an HTML report and have my todo list be available for others to see if I so wish with:
Get-MyTask | ConvertTo-HTML
Happy scripting!


08 Apr 2015
Cyclades, part I
Disclaimer: This is where the really nice pictures of beaches and islands start. If you already hear the old hobo demon whispering in your ear you should tread lightly. I’m looking at you, Sofia!
Leaving Athens
On Sunday 05/04/2015, Holger and I left Athens to do some relaxed island-hopping in the Cyclades. Named after its round shape, this group of islands is what people imagine when they hear “Greece”. Clear turcoise water, We had decided to make the best use of Holger’s limited vacation time by visiting two islands: Milos (not very touristy) and Santorini (very, very touristy). The best of both worlds!
So on said Sunday we agreed to meet up at Athen’s ferry terminal (yes, yes: it’s technically in Pireas, but you really can’t tell it’s a different city). Since I dislike being late or missing flights, boats or coaches, I arrived at the port way too early. So after finding the correct gate, I had plenty of time to sprawl out in the shade and watch a number of ferries enter the port, dock, offload, load and depart. Sure, airports have that atmosphere of travel and adventure, but they often feel so rushed. Ships, now that’s a different matter!
Anyway, before I get too philosophical: Ships. We took one of those. For around 50 euros per person you are allowed to get on one of the shiny ones. Hence we got on a shiny, blue catamaran one which would carry us to the island of Milos in around 4 hours.
The ferry was quite neat and well-equipped but we spent most of the trip on deck, looking at the fantastically blue water or the islands we passed.
Include file ‘image.html img=”pics/IMG_20150405_120653_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150405_120653.jpg”’ contains invalid characters or sequences Your intrepid protagonists!
Milos
We reached Milos in the late afternoon and found our hotel pretty much immediately. Truth of the matter is that the town of Adamantas is rather small. We saw several hotels, hostels and rooms for rent, but it was clear that tourist season was yet to start. That might also be the reason why we managed to get a room with two beds for 10 euro per person and night. As we reached the place indicated on the booking confirmation, this guy walked up to us, ushered us into this spacious room on the ground floor and I gave him 20 euros. Simple.
Include file ‘image.html img=”pics/IMG_20150405_183952_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150405_183952.jpg”’ contains invalid characters or sequences Even the clouds look amazing here! I’m currently using this shot as a wallpaper.
Include file ‘image.html img=”pics/IMG_20150405_184303_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150405_184303.jpg”’ contains invalid characters or sequences Meeting the locals
After a hearty meal in an almost completely empty restaurant and a few shots of raki, we wandered around the deserted town until after dark. Amongst other things, we came upon the island’s mining museum. Turns out that Milos actually earns more revenue from mining rare earths and obsidian than from tourism. That at least explains the relatively low-key tourism infrastructure in the town. I was left wondering who in their right minds would mine for obsidian though. I knew that they were historically used as blades and arrow points, because these stones tend to have sharp, glass-like edges. Amusingly enough, that seems to be one modern application, too: Obsidian is used for experimental scalpel blades.
Include file ‘image.html img=”pics/IMG_20150405_193154_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150405_193154.jpg”’ contains invalid characters or sequences Sunset on Milos
The next day we got up early (Hahaha…no. No, we didn’t) and decided to do a quick hike along the northern-most cliffs before leaving the island with a ferry in the afternoon.
Include file ‘image.html img=”pics/IMG_20150406_111507_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150406_111507.jpg”’ contains invalid characters or sequences Walking along the coastline.
Include file ‘image.html img=”pics/IMG_20150406_112219_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150406_112219.jpg”’ contains invalid characters or sequences Up on the cliff. Near the edge. Thanks for the shock, Holger!
Just as we sat down on the beach behind the cliff, it started drizzling. Glancing up the ominuous-looking clouds above us we decided to leg it and scuttle back to the hotel before we got properly drenched. This turned out to be one of the better decisions we made, because just as we got close to the hostel, it started raining in earnest. We waited for the rain to die down, grabbed our bags, left the key in the door (apparently that’s how you do it on Milos) and headed for the ferry terminal. The ferry arrived on Greek time (50 minutes late).
Looking at a map of the islands and thinking of the first leg of our journey, we figured that our trip to Santorini would also be about 4 hours long. Nope. Paying only 20 euros for a ticket, we got the less shiny ship this time. 7 hours on a ferry which was as deserted as the town of Adamantas had been (for similar reasons, I gather). At first both of us passed out on the sofas in the ferry’s bar area. The warmth and the gentle rocking of the waves just put us right to sleep. Fast forward a few hours, we were rather confused that we were still at sea. According to our GPS we were moving…slowly. Should have shelled out for the shiny ship!
Also, the wind had picked up significantly and the ship was lurching up and down a lot. The crew apparently had some problems with the heating: It heated well enough, but you couldn’t turn it off. Something smelled vaguely burned. The heat, the smell and the rocking motions caused some queasiness amongst the passengers. At some point I chose to step outside, too. For the view, of course! Observe:
Include file ‘image.html img=”pics/IMG_20150406_182907_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150406_182907.jpg”’ contains invalid characters or sequences Some island. Could be Ios, I think. It’s late here as I’m writing this. Can you tell?

Include file ‘image.html img=”pics/IMG_20150406_194259_scaled.jpg” title=”Image” caption=”” url=”http://www.escapingsloth.com/pics/fullsize/IMG_20150406_194259.jpg”’ contains invalid characters or sequences I’m an artist!
Much, much later than anticipated, we arrived at Santorini and slipped inside the horseshoe-shaped island towards the port. In the setting sun the view of the towns up top on the cliffs was spectacular: We had both read that most of Santorini’s urban area perched on those cliffs but we hadn’t anticipated just how HIGH those things were. As these sights unfolded before us in dusk, I have no nice pictures to show for it (my smartphone camera can’t handle lack of light too well). But trust me, there will be plenty of awe-inspiring pictures in a later post!
We docked at Santorini’s new port and let ourselves be talked into paying an outrageous sum to be transported from the tourist information to our hostel. It was late, there were no visible buses or cabs, but I’m still too ashamed to mention how much we paid for the 8km car ride.
Lesson 004: Tourist informations can be great. Some of those places have been incredibly helpful. In some cases they are little more than travel agencies shuffling people to hotels and getting commissions on bookings and transport fares. The guy in Santorini tried to justify the outrageous prices by saying we would also receive a map of the island. The same basic map you get for free at every hostel. Screw that. We should have just called a cab and waited for a bit. Don’t get talked into paying for a service you don’t need.
Santorini really deserves its own post, since Holger and I stayed there for 2 days together, after which I stayed for another 4, which was not planned but very lucky anyway.
Until next time,
Arne


Ben Kinney Tech is hiring Rails Developers (and some .NET)! ONSITE FULL TIME Bellingham, WA Possible VISA
We’re looking for someone with a passion for programming and for writing beautiful code. You will join a team of exceptional developers building custom tools and web applications for the real estate industry. We are committed to simple, functional, and powerful user interfaces, API’s, and information storage systems.
Requirements: 1+ years of direct experience in Ruby on Rails, along with other common libraries Good understanding of front-end technologies, such as JavaScript, HTML5, and CSS3 Solid understanding of the syntax of Ruby and its nuances A knack for writing clean, readable Ruby code Able to implement automated testing platforms and unit tests Great to have: Experience with JavaScript frameworks Familiarity with continuous integration Experience with version control systems (e.g. Git)
We offer: Open, collaborative workspaces A fully stocked kitchen Professional experience working on a team as well as experience using Git, Visual Studio Online, and other development tools
Our software products: -Blossor: Blossor.com -ActiveRain: http://activerain.com/ -Brivity: https://www.brivity.com/ -KWKLY: http://kwkly.com/


§712-1222 Promoting gambling in the second degree. (1) A person commits the offense of promoting gambling in the second degree if the person knowingly advances or profits from gambling activity.
(2) Promoting gambling in the second degree is a misdemeanor. [L 1972, c 9, pt of §1; am L 1973, c 201, pt of §1; am L 1987, c 83, §2; gen ch 1992]


Millcreek Canyon
- Scenic Overlook / Thaynes Canyon / Desolation Trail
- Porter Fork
- Churck Fork
- Winter Road
- Trails around the Yurt: Big Water Trail, Little Water Trail, Old Red Pine Road Trail
Big Cottonwood Canyon
- Days Fork
- Spruces Campground
- Brighton: Silver Lake


Sections of this page below are all controlled by separate AngularJS applications
According to the Reader's Choice
{{ city.text }}


The background color of this box should be animating through a variety of colors. This won't work in IE 9 or earlier though.


“People expect to be bored by eLearning—let’s show them it doesn’t have to be like that!” - Cammy Bean


Analysis: Latitude vs. Temperature
In this plot we clearly see that the average temperature as plotted against lattitude is colder in the far northern and southernly regions and warmer near the Equator. We also notice that the average temperature in the norther hemisphere is higher than that of the southern hemisphere. This is expected and consistent with the date of observation, which occurs in the late spring of the northern hemisphere. I would expect that if we were to plot the mathematical model of solar energy per square foot on a globe oriented 23.5 degrees from its axis of rotation, that we would see a curve that almost perfectly matches this one.
The difference in temperature between day and night likely accounts for some of the broadness of the line. Note that the broadness of the figure is narrower deep in the southern hemisphere where there is less landmass on opposite sides of the globe


Installation
The Easy Way
1. Run the install program provided.
Using it with a Program
2. Have the program run "GssScan (directory) (filename)" to scan a document into the specified directory and give it the specified filename.
© 2007-2013 GssEziSoft.com. All Rights Reserved.


{% extends "base.html" %} {% load subdomainurls %} {% block title %}Pricing{% endblock %} {% block content %}
No. StackMachine is only free during the beta period. Pricing will be announced when the beta is over.
StackMachine currently only supports the LÖVE framework. We'll be adding support for other popular frameworks (Unity, LibGDX, etc) in the near future. If you'd like early access, sign up using this form.
0%. All sale revenue (after processing fees) goes directly to you. Please note that this percentage may change in the future.
Nope. All games receive an unlimited number of downloads.


{% extends "base.html" %} {% block content %}
You can view datasets and tags from our bucket when you are logged-out. The first step to fully use and take advantage of our bucket and auditor is: you must register and login. After logged-in, use the form "Analyse Data" to enter the Kaggle URL:
To enjoy full functionalities, you may need to install Kaggle in your computer:
Open a Kaggle account (if you don't have one yet), and follow these steps to download kaggle.json files, which helps to run the API: (1) Go to the Account tab in your Kaggle profile and scroll to the API section. (2) Click Create new API Token to download the kaggle.json file, and add it to the /.kaggle path. For more complete and detailed instructions on how to use the Kaggle API, visit Kaggle's documentation.
Use the form-fields in "Add Data Set" and "Edit Data Set" to enter, classify and update your data entry:
The forms contain the fields: "tag(s)" (multi-choice to pick from gender, ethinicity, location and more to represent your dataset ), "name" (your dataset title), "description" (a short summary of your dataset report), "report" (upload your analysis report pdf from the bias auditor), "date-picker" (select today's date to show when the data was added or last updated), "to do" (toogle "on" if you are still working and updating the data).
Use the form "Add Data Tag" to enter new tags for your data entry.
Location: If your data has a Location tag, we look to see if all locations are equally represented. String and number formats accepted.
Gender and/or Sex: If your data has a Gender or Sex tag, we look to see if all genders are equally represented. For example, if you only have two genders in your data, we will recommend that you consider adding data to include other genders. Only string inputs are accepted. If values are numbers, they will be treated as strings.
Ethnicity: If your data has a Ethnicity tag, we look to see if all ethnicities are equally represented. The distribution of your data should be proportional to the distribution in the population the data is from. Only string inputs are accepted.
Religion: If your data has a Religion tag, we look to see if all religions are equally represented. The distribution of your data should be proportional to the distribution in the population the data is from. Only string inputs are accepted.
Income: If your data has an Income tag, we look to see if all Income are proportionally represented. The distribution of your data should be proportional to the distribution in the population the data is from. Only number inputs are accepted.
Age: If your data has an Age tag, we look to see if all ages are proportionally represented. The distribution of your data should be proportional to the distribution in the population the data is from. Only number inputs are accepted.
Education: If your data has an Education tag, we look to see if all education levels are proportionally represented. The distribution of your data should be proportional to the distribution in the population the data is from. Only string inputs are accepted.
Defined as: prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair
Sample bias occurs when a dataset does not reflect the realities of the environment in which a model will run.
This type of bias occurs when the data collected for training differs from that collected in the real world, or when faulty measurements result in data distortion.
Though not data bias in the traditional sense, this still warrants mentioning due to its prevalence in AI technology of late. Racial bias occurs when data skews in favor of particular demographics. This can also be extended to gender bias.


--- layout: default ---
Thanks for reaching out! If I've sent you to this page, it's so that we can schedule a time together!
My preferences are for meetings that are
I also have a preference for meeting near my office building (Gates) if we're meeting during business hours.
To schedule a meeting with me, please send me an invite via calendly.com/motiwari (preferred).
If you're having trouble with Calendly, please use motiwari.com/book.
Please include the location to meet and/or your phone number/Skype/Google Hangout/email.


Loading
—
checks
in the last
—
Last check:
—
There is a big gap of time where no checkups were performed, so some graphs may look distorted.


Hospital and Health Service boundaries - QueenslandHospital and Health Service boundaries - Queensland
Identification
Title: Hospital and Health Service boundaries - Queensland
Alternative title: Hospital and Health Service boundaries
Date:
2020-01-06
(publication)
Presentation form code: mapDigital
Related resource:Identifier: PROP.QLD_HOSPITAL_HEALTHSERV_BDY
Relationship to related resource: source
Description
Abstract / Description: This dataset shows the 16 Queensland Health Department, Hospital and Health Service area boundaries across Queensland
Purpose: To provide boundary information for the 16 Queensland Health, Hospital and Health Service areas.
Additional Information: The Children's Health Queensland area is located at the Queensland's Children's Hospital, South Brisbane and has no graphical extent. It is represented in the dataset as a small area feature above the hospital.
Attributes:
HHS - Name of the Hospital and Health Service Region
For further information on Queensland Hospital and Health Boards, refer to:
https://www.legislation.qld.gov.au/view/html/inforce/current/sl-2012-0024#sch.1
https://www.legislation.qld.gov.au/view/html/inforce/current/sl-2012-0024#sch.6
Language: eng
Character set: unknown
Topic
ISO topic category codes: boundaries
Spatial Data Information
Spatial representation type: vector
Coordinate Reference System EPSG code: EPSG:4283
Title: EPSG Geodetic Parameter
Dataset
Date:
2007-07-16
(revision)
Edition: Version 6.13
Contacts
Point of contact: Organisation: Department of Natural Resources, Mines and Energy
Address:
Delivery point: 275 George Street
City: Brisbane
State: QLD
Postcode: 4000
Country: Australia
Email<EMAIL_ADDRESS>Owner: Organisation: Department of Health
Status
Progress status: completed
Maintenance and update frequency: asNeeded
Data / Resource Constraints
Resource Access Level:
-->
Security classification (ISO 19115): unclassified
Copyright: (as use constraint)
© State of Queensland (Department of Natural Resources, Mines and Energy) 2020
Licence: (as use constraint)
This material is licensed under a
Creative Commons - Attribution 4.0 International licence.
The Department of Natural Resources, Mines and Energy requests attribution in the following manner:
© State of Queensland (Department of Natural Resources, Mines and Energy) 2020.
Temporal and Spatial extents:
Bounding box: West bounding longitude: 138.000
East bounding longitude: 155.000
South bounding latitude: -29.500
North bounding latitude: -9.000
Temporal extent:
Start: 2019-01-06 End: 2020-01-06
Data Quality
Data quality scope:
Level: dataset
History:
Lineage statement:The boundaries are as per the original gazetted description which was based on Local Government Areas (with modifications) at the time (ASGC 2010), to best capture the same geography due to part LGAs being used in some instances, the ASGC Statistical Local Areas (SLA) and meshblock combinations where used to create the digital boundaries. Over time as the standard geography changed to ASGS some modifications to the description have been applied. The latest description of the HHS boundaries is listed in Schedule 1 of the Hospital and Health Boards Regulation 2012. In Schedule 6, further detailed description can be found with regard to the HHS where whole LGA are not used, ASGS versions are also listed that apply.
Distribution
Format(s):
Name/identifier: Queensland Spatial Catalogue
Specification: Download Service
Online resource:
Ordering information:Instructions: Open the Queensland Spatial Catalogue (QSpatial). Select the large SEARCH button and all available records are displayed. Select one of the four filter options provided and then the further options within the filter. The resultant search is then displayed. Select your record and complete the order.
Format(s):
Name/identifier: downloadableData
Version: 10.0
Specification: ESRI ArcGIS Server Geoportal Extension Content Type
Distributor:
Format(s):
Name/identifier: From the Queensland Spatial Catalogue available as ESRI Shape file SHP, MapInfo Tab file TAB, ESRI file geodatabase FGDB, Google file KMZ, and Geopackage version 1.0, 1.1, or 1.2 file GPKG
Transfer size (megabytes): 1
Online resource:
Transfer options:
Online resource:Website:
Function: download
Web Service Finder:Instructions: Use the URL to search QSpatial for web services displaying or using this dataset.
Metadata Reference Information
Hierarchy level (scope of the metadata): dataset
Identifier: {793947A0-C7D2-4154-AC45-4CCA7C16946A}
Metadata date stamp:
2020-01-06
Language: eng
Character set: utf8
Metadata standard name: ISO 19115:2005, ANZLIC ISO 1.1
Metadata version name: QGIS


{% include 'navbar.html' %}
{% include 'flash.html'%}
Home
About Us
Contact US
TIM
TIM Isn't Moode (TIM) is an online engagement platform for students and teachers.
Login
Register


This tool allows to maintain the database in total, or only a selected table, or a selected index.
Maintenance comes in three flavors:
VACUUM will scan the database or table for rows, that are not in use any more. If a row is updated or deleted, the previous content isn't replaced, but rather marked invalid. The new data is inserted freshly into the database.
Please press the Help button to see the PostgreSQL help about the VACUUM command to learn more about the options.
The output of the database server is displayed in the messages page as they arrive. If Verbose is selected, the server will send very detailed info about what it did.
While this tool is very handy for ad-hoc maintenance purposes, you are encouraged to install an automatic job, that performs a VACUUM job regularly to keep your database in a neat state.
ANALYZE investigates statistical values about the selected database or table. This enables the query optimizer to select the fastest query plan, to give optimal performance. Every time your data is changing radically, you should perform this task. It can be included in a VACUUM run, using the appropriate option.
REINDEX rebuilds the indexes in case these have degenerated caused by unusual data patterns inserted. This can happen for example if you insert many rows with increasing index values, and delete low index values.
The RECREATE option doesn't call the REINDEX SQL command internally, instead it drops the existing table and recreates it according to the current index definition.This doesn't lock the table exclusively, as REINDEX does, but will lock write access only.


‘Welcome to Nevada’ sign coming to I-11 at Arizona border
LAS VEGAS (AP) - Nevada transportation officials have announced plans to install a “Welcome to Nevada” monument sign along Interstate 11 at the Arizona border.
The Nevada Department of Transportation has said the $396,090 sign features an extruded state silhouette constructed from stainless steel, concrete and other materials and remain lit at night.
Officials say the sign would stand about 20-feet (6-meters) tall and 8-feet (2-meters) wide.
Transportation officials say sign construction started Monday and is expected to continue weekdays from 5 a.m. to 5 p.m. until Jan. 5 with a holiday break between Nov. 22 and Dec. 2.
Officials have urged motorists to use caution while traveling.
Officials say the special monument signs are being placed at a few major gateway locations, with the first two up in Northern Nevada and Crystal Bay.
Sign up for Daily Newsletters
Copyright © 2019 The Washington Times, LLC.
Please read our comment policy before commenting.


|
Adbase
Adinf C++ base library V2
|
#include <Interface.hpp>
Definition at line 125 of file Interface.hpp.
| adbase::mc::Interface::Interface | ( | ) |
Definition at line 7 of file Interface.cpp.
| adbase::mc::Interface::~Interface | ( | ) |
Definition at line 14 of file Interface.cpp.
| const AddHandler & adbase::mc::Interface::getAddHandler | ( | ) |
Definition at line 147 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const AppendHandler & adbase::mc::Interface::getAppendHandler | ( | ) |
Definition at line 154 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const DecrementHandler & adbase::mc::Interface::getDecrementHandler | ( | ) |
Definition at line 161 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const DeleteHandler & adbase::mc::Interface::getDeleteHandler | ( | ) |
Definition at line 168 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const FlushHandler & adbase::mc::Interface::getFlushHandler | ( | ) |
Definition at line 175 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const GetHandler & adbase::mc::Interface::getGetHandler | ( | ) |
Definition at line 182 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const IncrementHandler & adbase::mc::Interface::getIncrementHandler | ( | ) |
Definition at line 189 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const NoopHandler & adbase::mc::Interface::getNoopHandler | ( | ) |
Definition at line 203 of file Interface.cpp.
Referenced by adbase::mc::Binary::processData().
| const PostExecute & adbase::mc::Interface::getPostExecute | ( | ) |
Definition at line 259 of file Interface.cpp.
| const PreExecute & adbase::mc::Interface::getPreExecute | ( | ) |
Definition at line 252 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData().
| const PrependHandler & adbase::mc::Interface::getPrependHandler | ( | ) |
Definition at line 196 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const QuitHandler & adbase::mc::Interface::getQuitHandler | ( | ) |
Definition at line 210 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const ReplaceHander & adbase::mc::Interface::getReplaceHandler | ( | ) |
Definition at line 217 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const SetHandler & adbase::mc::Interface::getSetHandler | ( | ) |
Definition at line 224 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| const StatHandler & adbase::mc::Interface::getStatHandler | ( | ) |
Definition at line 245 of file Interface.cpp.
Referenced by adbase::mc::Binary::processData().
| const UnknownExecute & adbase::mc::Interface::getUnknownExecute | ( | ) |
Definition at line 266 of file Interface.cpp.
| const VerbosityHandler & adbase::mc::Interface::getVerbosityHandler | ( | ) |
Definition at line 238 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData().
| const VersionHandler & adbase::mc::Interface::getVersionHandler | ( | ) |
Definition at line 231 of file Interface.cpp.
Referenced by adbase::mc::Ascii::processData(), and adbase::mc::Binary::processData().
| void adbase::mc::Interface::setAddHandler | ( | const AddHandler & | addHandler | ) |
Definition at line 21 of file Interface.cpp.
| void adbase::mc::Interface::setAppendHandler | ( | const AppendHandler & | appendHandler | ) |
Definition at line 28 of file Interface.cpp.
| void adbase::mc::Interface::setDecrementHandler | ( | const DecrementHandler & | decrementHandler | ) |
Definition at line 35 of file Interface.cpp.
| void adbase::mc::Interface::setDeleteHandler | ( | const DeleteHandler & | deleteHandler | ) |
Definition at line 42 of file Interface.cpp.
| void adbase::mc::Interface::setFlushHandler | ( | const FlushHandler & | flushHandler | ) |
Definition at line 49 of file Interface.cpp.
| void adbase::mc::Interface::setGetHandler | ( | const GetHandler & | getHandler | ) |
Definition at line 56 of file Interface.cpp.
| void adbase::mc::Interface::setIncrementHandler | ( | const IncrementHandler & | incrementHandler | ) |
Definition at line 63 of file Interface.cpp.
| void adbase::mc::Interface::setNoopHandler | ( | const NoopHandler & | noopHandler | ) |
Definition at line 77 of file Interface.cpp.
| void adbase::mc::Interface::setPostExecute | ( | const PostExecute & | postExecute | ) |
Definition at line 133 of file Interface.cpp.
| void adbase::mc::Interface::setPreExecute | ( | const PreExecute & | preExecute | ) |
Definition at line 126 of file Interface.cpp.
| void adbase::mc::Interface::setPrependHandler | ( | const PrependHandler & | prependHandler | ) |
Definition at line 70 of file Interface.cpp.
| void adbase::mc::Interface::setQuitHandler | ( | const QuitHandler & | quitHandler | ) |
Definition at line 84 of file Interface.cpp.
| void adbase::mc::Interface::setReplaceHandler | ( | const ReplaceHander & | replaceHandler | ) |
Definition at line 91 of file Interface.cpp.
| void adbase::mc::Interface::setSetHandler | ( | const SetHandler & | setHandler | ) |
Definition at line 98 of file Interface.cpp.
| void adbase::mc::Interface::setStatHandler | ( | const StatHandler & | statHandler | ) |
Definition at line 119 of file Interface.cpp.
| void adbase::mc::Interface::setUnknownExecute | ( | const UnknownExecute & | unknownExecute | ) |
Definition at line 140 of file Interface.cpp.
| void adbase::mc::Interface::setVerbosityHandler | ( | const VerbosityHandler & | verbosityHandler | ) |
Definition at line 112 of file Interface.cpp.
| void adbase::mc::Interface::setVersionHandler | ( | const VersionHandler & | versionHandler | ) |
Definition at line 105 of file Interface.cpp.


Classification Problem Evaluation Metrics
When building a classification model, in order to check how efficiently the model performs we can evaluate it with metrics . The evaluation metrics compares how accurately the predicted classes match the actual classes for classification problems. As we know that there are several types of datasets and classification can also be binary or multiclass, based on that different evaluation matrix which are listed below can be applied in different scenarios.
- Confusion Matrix
It can be generally used while dealing with multiclass classification. It is not like any other type of metrics in which we will get an integer to decide how efficient the model is, rather we deal with four set of values which are True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN).
| Positive | Negative | |
| Positive | True Positive (TP) | False Positive (FP) |
| Neagtive | False Negative (FN) | True Neagtive (TN) |
Let us take an example so as to not confuse.
Consider a dataset of pregnancy tests of a woman in which there are two classes : pregnant or not pregnant
TP - It indicates that the woman is actually pregnant and the model also predicts it
correctly.
FP - It indicates that the woman is actually not pregnant but the model predicts as pregnant.
FN - It indicates that the woman is acutally pregnant but the model predicts as not pregnant.
TN - It indicates that the woman is not pregnant and the model predicts it correctly as not
pregnant.
Sometimes the cost of FN can be higher and sometimes the cost of FP can be higher.
Imagine that if we predict COVID-19 residents as healthy patients and they do not need to quarantine, there would be a
massive number of COVID-19 infections. Hence the cost of false negatives is much higher than the cost of false
positives.
Well, since missing important emails will clearly be more of a problem than receiving spam, we can say that in this
case, FP will have a higher cost than FN.
Since FN and FP leads to false predictions we need to minimize it.
It is one of the most simple and common classification metrics. It is determined by the total number of correct
predictions divided by total number of predictions made for a dataset.
Accuracy = (total number of correct predictions) / (total number of predictions)
In terms of confusion matrix, accuracy can be defined as
Accuracy = (TP + TN ) / (TP + FP + FN + TN)
But this metric has a disadvantage which is it cannot be used for imbalanced class datasets.
Note : Balanced class datasets are those in which classes are equally distributed whereas in
imbalanced dataset there is always a majority present of a particular class.
For example, if fraud detection dataset consists of 95% of non-fraud classes and only 5% of fraud classes then it is
called an imbalanced dataset.
In this case the classification model will be showing 95% accuracy as it trained with the majority non-fraud class only.
And when given a fraud case to predict, it will predict it as non-fraud class based on training.
And hence accuracy sometimes can be misleading in a way which would increase our hopes about model or also it may
cost life of a person if predicted for a particular disease with imbalanced dataset.
3. Precision
In terms of confusion matrix, precision is defined as the ratio of True Positives to all the positives predicted by the model.
Precision = TP / (TP + FP)
The more False Positive the model predicts, the lower is it's precision value.
4. Recall (Senstivity) This is defined as the ratio of True Positives to all the positives in your dataset.
Recall = TP / (TP + FN)
The more False Negatives the model predicts the lower the recall.
The idea of recall and precision seems to be abstract. Let me illustrate the difference with the help of an example.
Let's say that we have dataset of bank loans with the confusion matrix given below
| Positive | Negative | |
| Positive: | TP-599 | FP-0 |
| Negative: | FN-33 | TN-22 |
Precision : Out of the loan that is predicted aas a bad loan, how many did we classify correctly:question:
Precision = 599 /(599 + 0) = 100%
Recall : Out of the actual bad loan, how many did we correctly predict as bad loan:question:
Recall = 599 / (599 + 33) = 94.5%
Precision vs Recall
For a particular model, instead of using either precision or recall we can combine both of them to obtain F1-score.
- F1-score
For some models, sometimes the recall would be high and sometimes precision would be high. We cannot even take mean of these two values as that also would be insignificant measure for evaluating model. Hence we use harmonic mean of precision and recall which is termed as F1-score. It balances out the both the precision and recall in one integer. An F1 score is considered perfect when it’s 1, while the model is a total failure when it’s 0.
F1-score = 2 * 1 / (1/precision + 1/recall)= (2 * precision * recall)/(precision + recall)
In the above example of bank loans, F1-score will be
F1-score = (2 * 1 * 0.945) / (1 + 0.945) = 0.971
- Specificity
It is also derived from confusion matrix which defines the ability of a model to predict correctly the True Negative cases.
Specificity =TN /(TN + FP)
For example, in cancer detection test specificity will be defined as the person not having cancer and the model
correctly rejecting the patients who test negative.
7. Loss
It can be used for multiclass classification where it assigns a probability to each class for all examples. It is used
to compare models not only on the basis of outputs but their probabilistic outcomes.
y_ij, indicates whether sample i belongs to class j or not
p_ij, indicates the probability of sample i belonging to class j
In this graph we can see that as the probability of the model increases, the logarithmic
value decreases and vice versa. Hence lower the log value higher accurate will be the model.
8. Area Under Curve It is used for binary classification. In order to understand area under curve we will first define two terms. True Positive Rate (Sensitivity) : It is defined as the ratio of True Positive to all the other positives in dataset.
TPR = TP/ (TP + FN)
False Positive Rate : It is defined as the ratio of False Positive to all negatives in dataset.
FPR =FP/ (FP + TN)
The probabilistic interpretation of AUC score is that if you randomly choose a positive case and a negative case, the probability that the positive case outranks the negative case according to the classifier is given by the AUC. It is defined as the area under the curve which is plotted by FPR vs TPR. Basically, for every threshold, we calculate TPR and FPR and plot it on one chart. The value can range from 0 to 1. However AUC score of a random classifier for balanced data is 0.5.
So basically we have 8 evaluation metrics for dealing with classification problems and we can choose any of them depending on how our dataset and classification model is.
You can learn more about this topic fromhere.


Class MixedRealityCameraSystem
The Mixed Reality Toolkit's default implementation of the IMixedRealityCameraSystem.
Inherited Members
Namespace: XRTK.Services.CameraSystem
Assembly: XRTK.dll
Syntax
[Guid("5C656EE3-FE7C-4FB3-B3EE-DF3FC0D0973D")]
public class MixedRealityCameraSystem : BaseSystem, IMixedRealityCameraSystem, IMixedRealitySystem, IMixedRealityService, IDisposable
Constructors
| Improve this Doc View SourceMixedRealityCameraSystem(MixedRealityCameraSystemProfile)
Declaration
public MixedRealityCameraSystem(MixedRealityCameraSystemProfile profile)
Parameters
| Type | Name | Description |
|---|---|---|
| MixedRealityCameraSystemProfile | profile |
Properties
| Improve this Doc View SourceCameraDataProviders
The list of IMixedRealityCameraDataProviders registered and running with the system.
Declaration
public IReadOnlyCollection<IMixedRealityCameraDataProvider> CameraDataProviders { get; }
Property Value
| Type | Description |
|---|---|
| IReadOnlyCollection<IMixedRealityCameraDataProvider> |
MainCameraRig
The reference to the IMixedRealityCameraRig attached to the Main Camera (typically this is the player's camera).
Declaration
public IMixedRealityCameraRig MainCameraRig { get; }
Property Value
| Type | Description |
|---|---|
| IMixedRealityCameraRig |
Priority
Optional Priority to reorder registered managers based on their respective priority, reduces the risk of race conditions by prioritizing the order in which services are evaluated.
Declaration
public override uint Priority { get; }
Property Value
| Type | Description |
|---|---|
| UInt32 |
Overrides
Methods
| Improve this Doc View SourceDestroy()
Optional Destroy function to perform cleanup of the service before the Mixed Reality Toolkit is destroyed.
Declaration
public override void Destroy()
Overrides
| Improve this Doc View SourceRegisterCameraDataProvider(IMixedRealityCameraDataProvider)
Registers the IMixedRealityCameraDataProvider with the IMixedRealityCameraSystem.
Declaration
public void RegisterCameraDataProvider(IMixedRealityCameraDataProvider dataProvider)
Parameters
| Type | Name | Description |
|---|---|---|
| IMixedRealityCameraDataProvider | dataProvider |
SetHeadHeight(Single, Boolean)
Sets the HeadHeight. If
Declaration
public void SetHeadHeight(float value, bool setForAllCameraProviders = false)
Parameters
| Type | Name | Description |
|---|---|---|
| Single | value | The height value to set. |
| Boolean | setForAllCameraProviders | If |
UnRegisterCameraDataProvider(IMixedRealityCameraDataProvider)
UnRegisters the IMixedRealityCameraDataProvider with the IMixedRealityCameraSystem.
Declaration
public void UnRegisterCameraDataProvider(IMixedRealityCameraDataProvider dataProvider)
Parameters
| Type | Name | Description |
|---|---|---|
| IMixedRealityCameraDataProvider | dataProvider |


{% load static %}
|                                                         Add Lab Attendence | |
|---|---|
|                                  Select Subject | |
|                                  Select Class | |
|                                  Select Division | |
|                                  Select Date | |
|                                                  |


#1 Material Design Admin Template.
Materialize - A unique truly material design admin template has minimal, sleek, clean and intuitive design which makes your next project look awesome & user friendly, Have look Materialize today!
We did most of the heavy lifting for you to provide a default stylings that incorporate our custom components. Additionally, we refined animations and transitions to provide a smoother experience for developers.
By utilizing elements and principles of Material Design, we were able to create a framework that incorporates components and animations that provide more feedback to users. Additionally, a single underlying responsive system across all platforms allow for a more unified user experience.
We have provided detailed documentation as well as specific code examples to help new users get started. We are also always open to feedback and can answer any questions a user may have about Materialize.
We did most of the heavy lifting for you to provide a default stylings that incorporate our custom components. Additionally, we refined animations and transitions to provide a smoother experience for developers.
By utilizing elements and principles of Material Design, we were able to create a framework that incorporates components and animations that provide more feedback to users. Additionally, a single underlying responsive system across all platforms allow for a more unified user experience.
We have provided detailed documentation as well as specific code examples to help new users get started. We are also always open to feedback and can answer any questions a user may have about Materialize.
DEMO
I am a very simple card. I am good at containing small bits of information. I am convenient because I require little markup to use effectively.
I am a very simple card. I am good at containing small bits of information. I am convenient because I require little markup to use effectively.
I am a very simple card. I am good at containing small bits of information. I am convenient because I require little markup to use effectively.


AudioNetworkReborn - single channel
radio
audio control
loading...
I'd like to be RECEIVER
I'd like to be TRANSMITTER
https://github.com/robertrypula/audio-network-reborn
-
waiting for first frame


| Name | Symbol | Location | Date of Purchase | Quantity | Cost | Purchase Price | Change | Percentage Change | Price | Value | Detail Gain/Loss | {{eachStock.company}} | {{eachStock.symbol}} | {{eachStock.location}} | {{eachStock.date_in | date}} | {{eachStock.qty}} | {{eachStock.cost| currency: "€"}} | {{eachStock.cost/eachStock.qty| currency: "€"}} | {{eachStock.change}} | {{eachStock.pChg}} | {{eachStock.price | currency: "€"}} | {{eachStock.value | currency: "€"}} | {{eachStock.sumgl| currency: "€"}}
|---|